[clang][ubsan] Switch UBSAN optimization to `llvm.allow.{runtime,ubsan}.check()` #84858

vitalybuka · 2024-03-12T00:13:17Z

Intrinsic introduced with #84850. Intrinsics improves performance
by 3% comparing to removing traps (on
"test-suite/MultiSource/Benchmarks" with PGO+ThinLTO).

The pass will be renamed with #84853.

RFC: https://discourse.llvm.org/t/rfc-add-llvm-experimental-hot-intrinsic-or-llvm-hot/77641

llvmbot · 2024-03-12T00:13:46Z

@llvm/pr-subscribers-backend-risc-v
@llvm/pr-subscribers-llvm-transforms
@llvm/pr-subscribers-clang-codegen

@llvm/pr-subscribers-clang

Author: Vitaly Buka (vitalybuka)

Changes

Intrinsic introduced with #84850. Using intrinsic improves performance
by 3% comparing to removing traps (on
"test-suite/MultiSource/Benchmarks" with PGO+ThinLTO).

The pass will be renamed with #84852.

Patch is 20.23 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/84858.diff

5 Files Affected:

(modified) clang/lib/CodeGen/BackendUtil.cpp (+3-3)
(modified) clang/lib/CodeGen/CGExpr.cpp (+14-1)
(modified) clang/test/CodeGen/remote-traps.c (+20-3)
(modified) llvm/lib/Transforms/Instrumentation/RemoveTrapsPass.cpp (+15-11)
(modified) llvm/test/Transforms/RemoveTraps/remove-traps.ll (+64-25)

diff --git a/clang/lib/CodeGen/BackendUtil.cpp b/clang/lib/CodeGen/BackendUtil.cpp
index 82b30b8d815629..7e53469a48d42c 100644
--- a/clang/lib/CodeGen/BackendUtil.cpp
+++ b/clang/lib/CodeGen/BackendUtil.cpp
@@ -100,9 +100,9 @@ using namespace llvm;
 namespace llvm {
 extern cl::opt<bool> PrintPipelinePasses;
 
-cl::opt<bool> ClRemoveTraps("clang-remove-traps", cl::Optional,
-                            cl::desc("Insert remove-traps pass."),
-                            cl::init(false));
+static cl::opt<bool> ClRemoveTraps("clang-remove-traps", cl::Optional,
+                                   cl::desc("Insert remove-traps pass."),
+                                   cl::init(false));
 
 // Experiment to move sanitizers earlier.
 static cl::opt<bool> ClSanitizeOnOptimizerEarlyEP(
diff --git a/clang/lib/CodeGen/CGExpr.cpp b/clang/lib/CodeGen/CGExpr.cpp
index 59a7fe8925001c..3a27622f165995 100644
--- a/clang/lib/CodeGen/CGExpr.cpp
+++ b/clang/lib/CodeGen/CGExpr.cpp
@@ -56,7 +56,14 @@ using namespace CodeGen;
 // Experiment to make sanitizers easier to debug
 static llvm::cl::opt<bool> ClSanitizeDebugDeoptimization(
     "ubsan-unique-traps", llvm::cl::Optional,
-    llvm::cl::desc("Deoptimize traps for UBSAN so there is 1 trap per check"),
+    llvm::cl::desc("Deoptimize traps for UBSAN so there is 1 trap per check."),
+    llvm::cl::init(false));
+
+// TODO: Introduce frontend options to enabled per sanitizers, similar to
+// `fsanitize-trap`.
+static llvm::cl::opt<bool> ClSanitizeExpHot(
+    "ubsan-exp-hot", llvm::cl::Optional,
+    llvm::cl::desc("Pass UBSAN checks if `llvm.experimental.hot()` is true."),
     llvm::cl::init(false));
 
 //===--------------------------------------------------------------------===//
@@ -3805,6 +3812,12 @@ void CodeGenFunction::EmitTrapCheck(llvm::Value *Checked,
                                     SanitizerHandler CheckHandlerID) {
   llvm::BasicBlock *Cont = createBasicBlock("cont");
 
+  if (ClSanitizeExpHot) {
+    Checked =
+        Builder.CreateOr(Checked, Builder.CreateCall(CGM.getIntrinsic(
+                                      llvm::Intrinsic::experimental_hot)));
+  }
+
   // If we're optimizing, collapse all calls to trap down to just one per
   // check-type per function to save on code size.
   if ((int)TrapBBs.size() <= CheckHandlerID)
diff --git a/clang/test/CodeGen/remote-traps.c b/clang/test/CodeGen/remote-traps.c
index 6751afb96d25f2..934b76fae0d2a5 100644
--- a/clang/test/CodeGen/remote-traps.c
+++ b/clang/test/CodeGen/remote-traps.c
@@ -1,5 +1,7 @@
-// RUN: %clang_cc1 -O1 -emit-llvm -fsanitize=signed-integer-overflow -fsanitize-trap=signed-integer-overflow %s -o - | FileCheck %s 
-// RUN: %clang_cc1 -O1 -emit-llvm -fsanitize=signed-integer-overflow -fsanitize-trap=signed-integer-overflow -mllvm -clang-remove-traps -mllvm -remove-traps-random-rate=1 %s -o - | FileCheck %s --implicit-check-not="call void @llvm.ubsantrap" --check-prefixes=REMOVE
+// RUN: %clang_cc1 -O1 %s -o - -emit-llvm -fsanitize=signed-integer-overflow -fsanitize-trap=signed-integer-overflow -mllvm -ubsan-exp-hot | FileCheck %s
+// RUN: %clang_cc1 -O1 %s -o - -emit-llvm -fsanitize=signed-integer-overflow -fsanitize-trap=signed-integer-overflow -mllvm -ubsan-exp-hot -mllvm -clang-remove-traps -mllvm -remove-traps-random-rate=1 %s -o - | FileCheck %s --check-prefixes=REMOVE
+
+#include <stdbool.h>
 
 int test(int x) {
   return x + 123;
@@ -12,4 +14,19 @@ int test(int x) {
 // CHECK-NEXT: unreachable
 
 // REMOVE-LABEL: define {{.*}}i32 @test(
-// REMOVE: call { i32, i1 } @llvm.sadd.with.overflow.i32(
+// REMOVE: add i32 %x, 123
+// REMOVE-NEXT: ret i32
+
+
+bool experimental_hot() __asm("llvm.experimental.hot");
+
+bool test_asm() {
+  return experimental_hot();
+}
+
+// CHECK-LABEL: define {{.*}}i1 @test_asm(
+// CHECK: [[R:%.*]] = tail call zeroext i1 @llvm.experimental.hot()
+// CHECK: ret i1 [[R]]
+
+// REMOVE-LABEL: define {{.*}}i1 @test_asm(
+// REMOVE: ret i1 true
diff --git a/llvm/lib/Transforms/Instrumentation/RemoveTrapsPass.cpp b/llvm/lib/Transforms/Instrumentation/RemoveTrapsPass.cpp
index d87f7482a21d25..fa4716f2e7a403 100644
--- a/llvm/lib/Transforms/Instrumentation/RemoveTrapsPass.cpp
+++ b/llvm/lib/Transforms/Instrumentation/RemoveTrapsPass.cpp
@@ -11,6 +11,7 @@
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/Analysis/ProfileSummaryInfo.h"
+#include "llvm/IR/Constant.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/Intrinsics.h"
@@ -37,7 +38,7 @@ STATISTIC(NumChecksRemoved, "Number of removed checks");
 
 static bool removeUbsanTraps(Function &F, const BlockFrequencyInfo &BFI,
                              const ProfileSummaryInfo *PSI) {
-  SmallVector<IntrinsicInst *, 16> Remove;
+  SmallVector<std::pair<IntrinsicInst *, Value *>, 16> ReplaceWithValue;
   std::unique_ptr<RandomNumberGenerator> Rng;
 
   auto ShouldRemove = [&](bool IsHot) {
@@ -56,15 +57,12 @@ static bool removeUbsanTraps(Function &F, const BlockFrequencyInfo &BFI,
         continue;
       auto ID = II->getIntrinsicID();
       switch (ID) {
-      case Intrinsic::ubsantrap: {
+      case Intrinsic::experimental_hot: {
         ++NumChecksTotal;
 
         bool IsHot = false;
         if (PSI) {
-          uint64_t Count = 0;
-          for (const auto *PR : predecessors(&BB))
-            Count += BFI.getBlockProfileCount(PR).value_or(0);
-
+          uint64_t Count = BFI.getBlockProfileCount(&BB).value_or(0);
           IsHot =
               HotPercentileCutoff.getNumOccurrences()
                   ? (HotPercentileCutoff > 0 &&
@@ -72,10 +70,14 @@ static bool removeUbsanTraps(Function &F, const BlockFrequencyInfo &BFI,
                   : PSI->isHotCount(Count);
         }
 
-        if (ShouldRemove(IsHot)) {
-          Remove.push_back(II);
+        bool ToRemove = ShouldRemove(IsHot);
+        ReplaceWithValue.push_back({
+            II,
+            ToRemove ? Constant::getAllOnesValue(II->getType())
+                     : (Constant::getNullValue(II->getType())),
+        });
+        if (ToRemove)
           ++NumChecksRemoved;
-        }
         break;
       }
       default:
@@ -84,10 +86,12 @@ static bool removeUbsanTraps(Function &F, const BlockFrequencyInfo &BFI,
     }
   }
 
-  for (IntrinsicInst *I : Remove)
+  for (auto [I, V] : ReplaceWithValue) {
+    I->replaceAllUsesWith(V);
     I->eraseFromParent();
+  }
 
-  return !Remove.empty();
+  return !ReplaceWithValue.empty();
 }
 
 PreservedAnalyses RemoveTrapsPass::run(Function &F,
diff --git a/llvm/test/Transforms/RemoveTraps/remove-traps.ll b/llvm/test/Transforms/RemoveTraps/remove-traps.ll
index 71549e7d9b4122..80b86e0f7455aa 100644
--- a/llvm/test/Transforms/RemoveTraps/remove-traps.ll
+++ b/llvm/test/Transforms/RemoveTraps/remove-traps.ll
@@ -12,7 +12,8 @@ define dso_local noundef i32 @simple(ptr noundef readonly %0) {
 ; NOPROFILE-LABEL: define dso_local noundef i32 @simple(
 ; NOPROFILE-SAME: ptr noundef readonly [[TMP0:%.*]]) {
 ; NOPROFILE-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; NOPROFILE-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; NOPROFILE-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], false
+; NOPROFILE-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; NOPROFILE:       3:
 ; NOPROFILE-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; NOPROFILE-NEXT:    unreachable
@@ -23,8 +24,10 @@ define dso_local noundef i32 @simple(ptr noundef readonly %0) {
 ; ALL-LABEL: define dso_local noundef i32 @simple(
 ; ALL-SAME: ptr noundef readonly [[TMP0:%.*]]) {
 ; ALL-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; ALL-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; ALL-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], true
+; ALL-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; ALL:       3:
+; ALL-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; ALL-NEXT:    unreachable
 ; ALL:       4:
 ; ALL-NEXT:    [[TMP5:%.*]] = load i32, ptr [[TMP0]], align 4
@@ -33,7 +36,8 @@ define dso_local noundef i32 @simple(ptr noundef readonly %0) {
 ; HOT-LABEL: define dso_local noundef i32 @simple(
 ; HOT-SAME: ptr noundef readonly [[TMP0:%.*]]) {
 ; HOT-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; HOT-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; HOT-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], false
+; HOT-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; HOT:       3:
 ; HOT-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; HOT-NEXT:    unreachable
@@ -44,7 +48,8 @@ define dso_local noundef i32 @simple(ptr noundef readonly %0) {
 ; HOT70-LABEL: define dso_local noundef i32 @simple(
 ; HOT70-SAME: ptr noundef readonly [[TMP0:%.*]]) {
 ; HOT70-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; HOT70-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; HOT70-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], false
+; HOT70-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; HOT70:       3:
 ; HOT70-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; HOT70-NEXT:    unreachable
@@ -52,7 +57,9 @@ define dso_local noundef i32 @simple(ptr noundef readonly %0) {
 ; HOT70-NEXT:    [[TMP5:%.*]] = load i32, ptr [[TMP0]], align 4
 ; HOT70-NEXT:    ret i32 [[TMP5]]
 ;
-  %2 = icmp eq ptr %0, null
+  %chk = icmp eq ptr %0, null
+  %hot = call i1 @llvm.experimental.hot()
+  %2 = or i1 %chk, %hot
   br i1 %2, label %3, label %4
 
 3:
@@ -69,7 +76,8 @@ define dso_local noundef i32 @hot(ptr noundef readonly %0) !prof !36 {
 ; NOPROFILE-LABEL: define dso_local noundef i32 @hot(
 ; NOPROFILE-SAME: ptr noundef readonly [[TMP0:%.*]]) !prof [[PROF16:![0-9]+]] {
 ; NOPROFILE-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; NOPROFILE-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; NOPROFILE-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], false
+; NOPROFILE-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; NOPROFILE:       3:
 ; NOPROFILE-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; NOPROFILE-NEXT:    unreachable
@@ -80,8 +88,10 @@ define dso_local noundef i32 @hot(ptr noundef readonly %0) !prof !36 {
 ; ALL-LABEL: define dso_local noundef i32 @hot(
 ; ALL-SAME: ptr noundef readonly [[TMP0:%.*]]) !prof [[PROF16:![0-9]+]] {
 ; ALL-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; ALL-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; ALL-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], true
+; ALL-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; ALL:       3:
+; ALL-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; ALL-NEXT:    unreachable
 ; ALL:       4:
 ; ALL-NEXT:    [[TMP5:%.*]] = load i32, ptr [[TMP0]], align 4
@@ -90,8 +100,10 @@ define dso_local noundef i32 @hot(ptr noundef readonly %0) !prof !36 {
 ; HOT-LABEL: define dso_local noundef i32 @hot(
 ; HOT-SAME: ptr noundef readonly [[TMP0:%.*]]) !prof [[PROF16:![0-9]+]] {
 ; HOT-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; HOT-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; HOT-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], true
+; HOT-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; HOT:       3:
+; HOT-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; HOT-NEXT:    unreachable
 ; HOT:       4:
 ; HOT-NEXT:    [[TMP5:%.*]] = load i32, ptr [[TMP0]], align 4
@@ -100,7 +112,8 @@ define dso_local noundef i32 @hot(ptr noundef readonly %0) !prof !36 {
 ; HOT70-LABEL: define dso_local noundef i32 @hot(
 ; HOT70-SAME: ptr noundef readonly [[TMP0:%.*]]) !prof [[PROF16:![0-9]+]] {
 ; HOT70-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; HOT70-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; HOT70-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], false
+; HOT70-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; HOT70:       3:
 ; HOT70-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; HOT70-NEXT:    unreachable
@@ -108,7 +121,9 @@ define dso_local noundef i32 @hot(ptr noundef readonly %0) !prof !36 {
 ; HOT70-NEXT:    [[TMP5:%.*]] = load i32, ptr [[TMP0]], align 4
 ; HOT70-NEXT:    ret i32 [[TMP5]]
 ;
-  %2 = icmp eq ptr %0, null
+  %chk = icmp eq ptr %0, null
+  %hot = call i1 @llvm.experimental.hot()
+  %2 = or i1 %chk, %hot
   br i1 %2, label %3, label %4
 
 3:
@@ -124,7 +139,8 @@ define dso_local noundef i32 @veryHot(ptr noundef readonly %0) !prof !39 {
 ; NOPROFILE-LABEL: define dso_local noundef i32 @veryHot(
 ; NOPROFILE-SAME: ptr noundef readonly [[TMP0:%.*]]) !prof [[PROF17:![0-9]+]] {
 ; NOPROFILE-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; NOPROFILE-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; NOPROFILE-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], false
+; NOPROFILE-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; NOPROFILE:       3:
 ; NOPROFILE-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; NOPROFILE-NEXT:    unreachable
@@ -135,8 +151,10 @@ define dso_local noundef i32 @veryHot(ptr noundef readonly %0) !prof !39 {
 ; ALL-LABEL: define dso_local noundef i32 @veryHot(
 ; ALL-SAME: ptr noundef readonly [[TMP0:%.*]]) !prof [[PROF17:![0-9]+]] {
 ; ALL-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; ALL-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; ALL-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], true
+; ALL-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; ALL:       3:
+; ALL-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; ALL-NEXT:    unreachable
 ; ALL:       4:
 ; ALL-NEXT:    [[TMP5:%.*]] = load i32, ptr [[TMP0]], align 4
@@ -145,8 +163,10 @@ define dso_local noundef i32 @veryHot(ptr noundef readonly %0) !prof !39 {
 ; HOT-LABEL: define dso_local noundef i32 @veryHot(
 ; HOT-SAME: ptr noundef readonly [[TMP0:%.*]]) !prof [[PROF17:![0-9]+]] {
 ; HOT-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; HOT-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; HOT-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], true
+; HOT-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; HOT:       3:
+; HOT-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; HOT-NEXT:    unreachable
 ; HOT:       4:
 ; HOT-NEXT:    [[TMP5:%.*]] = load i32, ptr [[TMP0]], align 4
@@ -155,14 +175,18 @@ define dso_local noundef i32 @veryHot(ptr noundef readonly %0) !prof !39 {
 ; HOT70-LABEL: define dso_local noundef i32 @veryHot(
 ; HOT70-SAME: ptr noundef readonly [[TMP0:%.*]]) !prof [[PROF17:![0-9]+]] {
 ; HOT70-NEXT:    [[TMP2:%.*]] = icmp eq ptr [[TMP0]], null
-; HOT70-NEXT:    br i1 [[TMP2]], label [[TMP3:%.*]], label [[TMP4:%.*]]
+; HOT70-NEXT:    [[TMP6:%.*]] = or i1 [[TMP2]], true
+; HOT70-NEXT:    br i1 [[TMP6]], label [[TMP3:%.*]], label [[TMP4:%.*]]
 ; HOT70:       3:
+; HOT70-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; HOT70-NEXT:    unreachable
 ; HOT70:       4:
 ; HOT70-NEXT:    [[TMP5:%.*]] = load i32, ptr [[TMP0]], align 4
 ; HOT70-NEXT:    ret i32 [[TMP5]]
 ;
-  %2 = icmp eq ptr %0, null
+  %chk = icmp eq ptr %0, null
+  %hot = call i1 @llvm.experimental.hot()
+  %2 = or i1 %chk, %hot
   br i1 %2, label %3, label %4
 
 3:
@@ -182,7 +206,8 @@ define dso_local noundef i32 @branchColdFnHot(i32 noundef %0, ptr noundef readon
 ; NOPROFILE-NEXT:    br i1 [[TMP3]], label [[TMP9:%.*]], label [[TMP4:%.*]], !prof [[PROF18:![0-9]+]]
 ; NOPROFILE:       4:
 ; NOPROFILE-NEXT:    [[TMP5:%.*]] = icmp eq ptr [[TMP1]], null
-; NOPROFILE-NEXT:    br i1 [[TMP5]], label [[TMP6:%.*]], label [[TMP7:%.*]]
+; NOPROFILE-NEXT:    [[TMP11:%.*]] = or i1 [[TMP5]], false
+; NOPROFILE-NEXT:    br i1 [[TMP11]], label [[TMP6:%.*]], label [[TMP7:%.*]]
 ; NOPROFILE:       6:
 ; NOPROFILE-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; NOPROFILE-NEXT:    unreachable
@@ -199,8 +224,10 @@ define dso_local noundef i32 @branchColdFnHot(i32 noundef %0, ptr noundef readon
 ; ALL-NEXT:    br i1 [[TMP3]], label [[TMP9:%.*]], label [[TMP4:%.*]], !prof [[PROF18:![0-9]+]]
 ; ALL:       4:
 ; ALL-NEXT:    [[TMP5:%.*]] = icmp eq ptr [[TMP1]], null
-; ALL-NEXT:    br i1 [[TMP5]], label [[TMP6:%.*]], label [[TMP7:%.*]]
+; ALL-NEXT:    [[TMP11:%.*]] = or i1 [[TMP5]], true
+; ALL-NEXT:    br i1 [[TMP11]], label [[TMP6:%.*]], label [[TMP7:%.*]]
 ; ALL:       6:
+; ALL-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; ALL-NEXT:    unreachable
 ; ALL:       7:
 ; ALL-NEXT:    [[TMP8:%.*]] = load i32, ptr [[TMP1]], align 4
@@ -215,7 +242,8 @@ define dso_local noundef i32 @branchColdFnHot(i32 noundef %0, ptr noundef readon
 ; HOT-NEXT:    br i1 [[TMP3]], label [[TMP9:%.*]], label [[TMP4:%.*]], !prof [[PROF18:![0-9]+]]
 ; HOT:       4:
 ; HOT-NEXT:    [[TMP5:%.*]] = icmp eq ptr [[TMP1]], null
-; HOT-NEXT:    br i1 [[TMP5]], label [[TMP6:%.*]], label [[TMP7:%.*]]
+; HOT-NEXT:    [[TMP11:%.*]] = or i1 [[TMP5]], false
+; HOT-NEXT:    br i1 [[TMP11]], label [[TMP6:%.*]], label [[TMP7:%.*]]
 ; HOT:       6:
 ; HOT-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; HOT-NEXT:    unreachable
@@ -232,7 +260,8 @@ define dso_local noundef i32 @branchColdFnHot(i32 noundef %0, ptr noundef readon
 ; HOT70-NEXT:    br i1 [[TMP3]], label [[TMP9:%.*]], label [[TMP4:%.*]], !prof [[PROF18:![0-9]+]]
 ; HOT70:       4:
 ; HOT70-NEXT:    [[TMP5:%.*]] = icmp eq ptr [[TMP1]], null
-; HOT70-NEXT:    br i1 [[TMP5]], label [[TMP6:%.*]], label [[TMP7:%.*]]
+; HOT70-NEXT:    [[TMP11:%.*]] = or i1 [[TMP5]], false
+; HOT70-NEXT:    br i1 [[TMP11]], label [[TMP6:%.*]], label [[TMP7:%.*]]
 ; HOT70:       6:
 ; HOT70-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; HOT70-NEXT:    unreachable
@@ -247,7 +276,9 @@ define dso_local noundef i32 @branchColdFnHot(i32 noundef %0, ptr noundef readon
   br i1 %3, label %9, label %4, !prof !38
 
 4:
-  %5 = icmp eq ptr %1, null
+  %chk = icmp eq ptr %1, null
+  %hot = call i1 @llvm.experimental.hot()
+  %5 = or i1 %chk, %hot
   br i1 %5, label %6, label %7
 
 6:
@@ -270,7 +301,8 @@ define dso_local noundef i32 @branchHotFnCold(i32 noundef %0, ptr noundef readon
 ; NOPROFILE-NEXT:    br i1 [[TMP3]], label [[TMP9:%.*]], label [[TMP4:%.*]], !prof [[PROF19:![0-9]+]]
 ; NOPROFILE:       4:
 ; NOPROFILE-NEXT:    [[TMP5:%.*]] = icmp eq ptr [[TMP1]], null
-; NOPROFILE-NEXT:    br i1 [[TMP5]], label [[TMP6:%.*]], label [[TMP7:%.*]]
+; NOPROFILE-NEXT:    [[TMP11:%.*]] = or i1 [[TMP5]], false
+; NOPROFILE-NEXT:    br i1 [[TMP11]], label [[TMP6:%.*]], label [[TMP7:%.*]]
 ; NOPROFILE:       6:
 ; NOPROFILE-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; NOPROFILE-NEXT:    unreachable
@@ -287,8 +319,10 @@ define dso_local noundef i32 @branchHotFnCold(i32 noundef %0, ptr noundef readon
 ; ALL-NEXT:    br i1 [[TMP3]], label [[TMP9:%.*]], label [[TMP4:%.*]], !prof [[PROF19:![0-9]+]]
 ; ALL:       4:
 ; ALL-NEXT:    [[TMP5:%.*]] = icmp eq ptr [[TMP1]], null
-; ALL-NEXT:    br i1 [[TMP5]], label [[TMP6:%.*]], label [[TMP7:%.*]]
+; ALL-NEXT:    [[TMP11:%.*]] = or i1 [[TMP5]], true
+; ALL-NEXT:    br i1 [[TMP11]], label [[TMP6:%.*]], label [[TMP7:%.*]]
 ; ALL:       6:
+; ALL-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; ALL-NEXT:    unreachable
 ; ALL:       7:
 ; ALL-NEXT:    [[TMP8:%.*]] = load i32, ptr [[TMP1]], align 4
@@ -303,8 +337,10 @@ define dso_local noundef i32 @branchHotFnCold(i32 noundef %0, ptr noundef readon
 ; HOT-NEXT:    br i1 [[TMP3]], label [[TMP9:%.*]], label [[TMP4:%.*]], !prof [[PROF19:![0-9]+]]
 ; HOT:       4:
 ; HOT-NEXT:    [[TMP5:%.*]] = icmp eq ptr [[TMP1]], null
-; HOT-NEXT:    br i1 [[TMP5]], label [[TMP6:%.*]], label [[TMP7:%.*]]
+; HOT-NEXT:    [[TMP11:%.*]] = or i1 [[TMP5]], true
+; HOT-NEXT:    br i1 [[TMP11]], label [[TMP6:%.*]], label [[TMP7:%.*]]
 ; HOT:       6:
+; HOT-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; HOT-NEXT:    unreachable
 ; HOT:       7:
 ; HOT-NEXT:    [[TMP8:%.*]] = load i32, ptr [[TMP1]], align 4
@@ -319,7 +355,8 @@ define dso_local noundef i32 @branchHotFnCold(i32 noundef %0, ptr noundef readon
 ; HOT70-NEXT:    br i1 [[TMP3]], label [[TMP9:%.*]], label [[TMP4:%.*]], !prof [[PROF19:![0-9]+]]
 ; HOT70:       4:
 ; HOT70-NEXT:    [[TMP5:%.*]] = icmp eq ptr [[TMP1]], null
-; HOT70-NEXT:    br i1 [[TMP5]], label [[TMP6:%.*]], label [[TMP7:%.*]]
+; HOT70-NEXT:    [[TMP11:%.*]] = or i1 [[TMP5]], false
+; HOT70-NEXT:    br i1 [[TMP11]], label [[TMP6:%.*]], label [[TMP7:%.*]]
 ; HOT70:       6:
 ; HOT70-NEXT:    tail call void @llvm.ubsantrap(i8 22)
 ; HOT70-NEXT:    unreachable
@@ -334,7 +371,9 @@ define dso_local noundef i32 @branchHotFnCold(i32 nounde...
[truncated]

clang/lib/CodeGen/CGExpr.cpp

clang/lib/CodeGen/BackendUtil.cpp

Extracted from #84858

…n}.check()` Intrinsic introduced with #84850. Intrinsics improves performance by 3% comparing to removing traps (on "test-suite/MultiSource/Benchmarks" with PGO+ThinLTO). The pass will be renamed with #84853. RFC: https://discourse.llvm.org/t/rfc-add-llvm-experimental-hot-intrinsic-or-llvm-hot/77641 Reviewers: aeubanks, nikic, kstoimenov, dvyukov Reviewed By: kstoimenov Pull Request: #84858

llvmbot added clang Clang issues not falling into any other category clang:codegen IR generation bugs: mangling, exceptions, etc. llvm:transforms labels Mar 12, 2024

vitalybuka requested review from nikic, dvyukov, jroelofs, aeubanks and kstoimenov March 12, 2024 00:15

vitalybuka mentioned this pull request Mar 12, 2024

[InstCombiner] Remove trivially dead llvm.allow.{runtime,ubsan}.check() #84851

Merged

jroelofs reviewed Mar 12, 2024

View reviewed changes

clang/lib/CodeGen/CGExpr.cpp Outdated Show resolved Hide resolved

kstoimenov approved these changes Mar 12, 2024

View reviewed changes

MaskRay reviewed Mar 13, 2024

View reviewed changes

clang/lib/CodeGen/BackendUtil.cpp Outdated Show resolved Hide resolved

vitalybuka changed the title ~~[clang][ubsan] Switch UBSAN optimization to llvm.experimental.hot~~ [clang][ubsan] Switch UBSAN optimization to llvm.allow.{runtime,ubsan}.check() Apr 1, 2024

vitalybuka added a commit that referenced this pull request Apr 1, 2024

[ubsan][NFC] Remove recently added cl::init(false)

e93b5f5

Extracted from #84858

vitalybuka changed the base branch from users/vitalybuka/spr/main.clangubsan-switch-ubsan-optimization-to-llvmexperimentalhot to main April 2, 2024 21:24

llvmbot added the clang:frontend Language frontend issues, e.g. anything involving "Sema" label Apr 4, 2024

vitalybuka changed the base branch from main to users/vitalybuka/spr/main.clangbuiltin-implement-__builtin_allow_runtime_check April 4, 2024 01:22

MaskRay approved these changes Apr 4, 2024

View reviewed changes

vitalybuka force-pushed the users/vitalybuka/spr/clangubsan-switch-ubsan-optimization-to-llvmexperimentalhot branch from 908b395 to 634a277 Compare April 5, 2024 00:36

llvmbot added the backend:RISC-V label Apr 5, 2024

vitalybuka changed the base branch from users/vitalybuka/spr/main.clangbuiltin-implement-__builtin_allow_runtime_check to main April 5, 2024 00:36

vitalybuka force-pushed the users/vitalybuka/spr/clangubsan-switch-ubsan-optimization-to-llvmexperimentalhot branch from 634a277 to 0146448 Compare April 5, 2024 00:38

vitalybuka merged commit a9d9387 into main Apr 5, 2024

vitalybuka deleted the users/vitalybuka/spr/clangubsan-switch-ubsan-optimization-to-llvmexperimentalhot branch April 5, 2024 00:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[clang][ubsan] Switch UBSAN optimization to `llvm.allow.{runtime,ubsan}.check()` #84858

[clang][ubsan] Switch UBSAN optimization to `llvm.allow.{runtime,ubsan}.check()` #84858

Uh oh!

vitalybuka commented Mar 12, 2024 •

edited

Loading

Uh oh!

llvmbot commented Mar 12, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

[clang][ubsan] Switch UBSAN optimization to llvm.allow.{runtime,ubsan}.check() #84858

[clang][ubsan] Switch UBSAN optimization to llvm.allow.{runtime,ubsan}.check() #84858

Uh oh!

Conversation

vitalybuka commented Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

[clang][ubsan] Switch UBSAN optimization to `llvm.allow.{runtime,ubsan}.check()` #84858

[clang][ubsan] Switch UBSAN optimization to `llvm.allow.{runtime,ubsan}.check()` #84858

vitalybuka commented Mar 12, 2024 •

edited

Loading

llvmbot commented Mar 12, 2024 •

edited

Loading