[GlobalISel] Introduce `G_POISON` #127825

mtsokol · 2025-02-19T16:23:13Z

Here's a WIP PR for visibility, so far I managed to introduce G_POISON and duplicate/mimic some of the G_IMPLICIT_DEF behavior (draft changes for sure). One test already runs with llc call up to irtranslator stage in calllowering-nocrashret.ll test.

I'm still missing legalization, as my tests fail with unable to legalize instruction: G_POISON, so I need a bit more time to get a better grasp of it.

llvmbot · 2025-02-20T02:59:08Z

@llvm/pr-subscribers-llvm-globalisel

Author: Mateusz Sokół (mtsokol)

Changes

Addresses #127486

Hi @arsenm,

Here's a WIP PR for visibility, so far I managed to introduce G_POISON and duplicate/mimic some of the G_IMPLICIT_DEF behavior (draft changes for sure). One test already runs with llc call up to irtranslator stage in calllowering-nocrashret.ll test.

I'm still missing legalization, as my tests fail with unable to legalize instruction: G_POISON, so I need a bit more time to get a better grasp of it.

Patch is 28.12 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/127825.diff

18 Files Affected:

(modified) llvm/docs/GlobalISel/GenericOpcode.rst (+9)
(modified) llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h (+17)
(modified) llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h (+8)
(modified) llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h (+4-1)
(modified) llvm/include/llvm/Support/TargetOpcodes.def (+4-1)
(modified) llvm/include/llvm/Target/GenericOpcodes.td (+6)
(modified) llvm/include/llvm/Target/GlobalISel/Combine.td (+70-23)
(modified) llvm/lib/CodeGen/GlobalISel/CSEInfo.cpp (+2-1)
(modified) llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp (+41-2)
(modified) llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp (+2)
(modified) llvm/lib/CodeGen/GlobalISel/LegacyLegalizerInfo.cpp (+2)
(modified) llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp (+6-1)
(modified) llvm/lib/CodeGen/GlobalISel/LostDebugLocObserver.cpp (+1)
(modified) llvm/lib/CodeGen/GlobalISel/MachineIRBuilder.cpp (+4)
(modified) llvm/lib/CodeGen/GlobalISel/Utils.cpp (+8-2)
(modified) llvm/lib/CodeGen/MachineSSAContext.cpp (+2-1)
(modified) llvm/test/CodeGen/X86/GlobalISel/calllowering-nocrashret.ll (+13-2)
(modified) llvm/test/TableGen/GlobalISelEmitter/GlobalISelEmitter.td (+1-1)

diff --git a/llvm/docs/GlobalISel/GenericOpcode.rst b/llvm/docs/GlobalISel/GenericOpcode.rst
index 8b76a407c34f8..fcffb78c8fc07 100644
--- a/llvm/docs/GlobalISel/GenericOpcode.rst
+++ b/llvm/docs/GlobalISel/GenericOpcode.rst
@@ -24,6 +24,15 @@ An undefined value.
 
   %0:_(s32) = G_IMPLICIT_DEF
 
+G_POISON
+^^^^^^^^
+
+A poison value.
+
+.. code-block:: none
+
+  %0:_(s32) = G_POISON
+
 G_CONSTANT
 ^^^^^^^^^^
 
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h b/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h
index 9b78342c8fc39..1f0f30d1e5e85 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h
@@ -431,16 +431,27 @@ class CombinerHelper {
   /// G_IMPLICIT_DEF.
   bool matchAnyExplicitUseIsUndef(MachineInstr &MI) const;
 
+  /// Return true if any explicit use operand on \p MI is defined by a
+  /// G_POISON.
+  bool matchAnyExplicitUseIsPoison(MachineInstr &MI) const;
+
   /// Return true if all register explicit use operands on \p MI are defined by
   /// a G_IMPLICIT_DEF.
   bool matchAllExplicitUsesAreUndef(MachineInstr &MI) const;
 
+  /// Return true if all register explicit use operands on \p MI are defined by
+  /// a G_POISON.
+  bool matchAllExplicitUsesArePoison(MachineInstr &MI) const;
+
   /// Return true if a G_SHUFFLE_VECTOR instruction \p MI has an undef mask.
   bool matchUndefShuffleVectorMask(MachineInstr &MI) const;
 
   /// Return true if a G_STORE instruction \p MI is storing an undef value.
   bool matchUndefStore(MachineInstr &MI) const;
 
+  /// Return true if a G_STORE instruction \p MI is storing a poison value.
+  bool matchPoisonStore(MachineInstr &MI) const;
+
   /// Return true if a G_SELECT instruction \p MI has an undef comparison.
   bool matchUndefSelectCmp(MachineInstr &MI) const;
 
@@ -466,6 +477,9 @@ class CombinerHelper {
   /// Replace an instruction with a G_IMPLICIT_DEF.
   void replaceInstWithUndef(MachineInstr &MI) const;
 
+  /// Replace an instruction with a G_POISON.
+  void replaceInstWithPoison(MachineInstr &MI) const;
+
   /// Delete \p MI and replace all of its uses with its \p OpIdx-th operand.
   void replaceSingleDefInstWithOperand(MachineInstr &MI, unsigned OpIdx) const;
 
@@ -506,6 +520,9 @@ class CombinerHelper {
   /// Check if operand \p OpIdx is undef.
   bool matchOperandIsUndef(MachineInstr &MI, unsigned OpIdx) const;
 
+  /// Check if operand \p OpIdx is poison.
+  bool matchOperandIsPoison(MachineInstr &MI, unsigned OpIdx) const;
+
   /// Check if operand \p OpIdx is known to be a power of 2.
   bool matchOperandIsKnownToBeAPowerOfTwo(MachineInstr &MI,
                                           unsigned OpIdx) const;
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h b/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h
index 9e5d4d34f24d2..ddcff441cea3b 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h
@@ -343,6 +343,14 @@ class GImplicitDef : public GenericMachineInstr {
   }
 };
 
+/// Represents a G_POISON.
+class GPoison : public GenericMachineInstr {
+public:
+  static bool classof(const MachineInstr *MI) {
+    return MI->getOpcode() == TargetOpcode::G_POISON;
+  }
+};
+
 /// Represents a G_SELECT.
 class GSelect : public GenericMachineInstr {
 public:
diff --git a/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h b/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h
index 7b0475ac2481d..68399c93a9364 100644
--- a/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h
+++ b/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h
@@ -1017,9 +1017,12 @@ class MachineIRBuilder {
   /// \return a MachineInstrBuilder for the newly created instruction.
   MachineInstrBuilder buildExtract(const DstOp &Res, const SrcOp &Src, uint64_t Index);
 
-  /// Build and insert \p Res = IMPLICIT_DEF.
+  /// Build and insert \p Res = G_IMPLICIT_DEF.
   MachineInstrBuilder buildUndef(const DstOp &Res);
 
+  /// Build and insert \p Res = G_POISON.
+  MachineInstrBuilder buildPoison(const DstOp &Res);
+
   /// Build and insert \p Res = G_MERGE_VALUES \p Op0, ...
   ///
   /// G_MERGE_VALUES combines the input elements contiguously into a larger
diff --git a/llvm/include/llvm/Support/TargetOpcodes.def b/llvm/include/llvm/Support/TargetOpcodes.def
index 5ef3707b81fe9..0170ac327c855 100644
--- a/llvm/include/llvm/Support/TargetOpcodes.def
+++ b/llvm/include/llvm/Support/TargetOpcodes.def
@@ -295,9 +295,12 @@ HANDLE_TARGET_OPCODE(G_ABDS)
 /// Generic absolute difference unsigned instruction.
 HANDLE_TARGET_OPCODE(G_ABDU)
 
-
+// Generic implicit definition.
 HANDLE_TARGET_OPCODE(G_IMPLICIT_DEF)
 
+// Generic poison value.
+HANDLE_TARGET_OPCODE(G_POISON)
+
 /// Generic PHI instruction with types.
 HANDLE_TARGET_OPCODE(G_PHI)
 
diff --git a/llvm/include/llvm/Target/GenericOpcodes.td b/llvm/include/llvm/Target/GenericOpcodes.td
index e134bab61bf63..5b71641854eac 100644
--- a/llvm/include/llvm/Target/GenericOpcodes.td
+++ b/llvm/include/llvm/Target/GenericOpcodes.td
@@ -92,6 +92,12 @@ def G_IMPLICIT_DEF : GenericInstruction {
   let hasSideEffects = false;
 }
 
+def G_POISON : GenericInstruction {
+  let OutOperandList = (outs type0:$dst);
+  let InOperandList = (ins);
+  let hasSideEffects = false;
+}
+
 def G_PHI : GenericInstruction {
   let OutOperandList = (outs type0:$dst);
   let InOperandList = (ins variable_ops);
diff --git a/llvm/include/llvm/Target/GlobalISel/Combine.td b/llvm/include/llvm/Target/GlobalISel/Combine.td
index 3590ab221ad44..cb7ba26ce57ca 100644
--- a/llvm/include/llvm/Target/GlobalISel/Combine.td
+++ b/llvm/include/llvm/Target/GlobalISel/Combine.td
@@ -430,6 +430,12 @@ def binop_right_undef_to_undef: GICombineRule<
          [{ return Helper.matchOperandIsUndef(*${root}, 2); }]),
   (apply [{ Helper.replaceInstWithUndef(*${root}); }])>;
 
+def binop_right_poison_to_poison
+    : GICombineRule<(defs root:$root),
+                    (match(wip_match_opcode G_SHL, G_ASHR, G_LSHR):$root,
+                        [{ return Helper.matchOperandIsPoison(*${root}, 2); }]),
+                    (apply [{ Helper.replaceInstWithPoison(*${root}); }])>;
+
 def unary_undef_to_zero: GICombineRule<
   (defs root:$root),
   (match (wip_match_opcode G_ABS):$root,
@@ -447,6 +453,17 @@ def unary_undef_to_undef : GICombineRule<
   (match (unary_undef_to_undef_frags $dst)),
   (apply [{ Helper.replaceInstWithUndef(*${dst}.getParent()); }])>;
 
+def unary_poison_to_poison_frags
+    : GICombinePatFrag<(outs root:$dst), (ins),
+                       !foreach(op,
+                                [G_TRUNC, G_BITCAST, G_ANYEXT, G_PTRTOINT,
+                                 G_INTTOPTR, G_FPTOSI, G_FPTOUI],
+                                (pattern(op $dst, $x), (G_POISON $x)))>;
+def unary_poison_to_poison
+    : GICombineRule<
+          (defs root:$dst), (match(unary_poison_to_poison_frags $dst)),
+          (apply [{ Helper.replaceInstWithPoison(*${dst}.getParent()); }])>;
+
 // Instructions where if any source operand is undef, the instruction can be
 // replaced with undef.
 def propagate_undef_any_op: GICombineRule<
@@ -455,6 +472,15 @@ def propagate_undef_any_op: GICombineRule<
          [{ return Helper.matchAnyExplicitUseIsUndef(*${root}); }]),
   (apply [{ Helper.replaceInstWithUndef(*${root}); }])>;
 
+// Instructions where if any source operand is poison, the instruction can be
+// replaced with poison.
+def propagate_poison_any_op
+    : GICombineRule<
+          (defs root:$root),
+          (match(wip_match_opcode G_ADD, G_SUB, G_XOR):$root,
+              [{ return Helper.matchAnyExplicitUseIsPoison(*${root}); }]),
+          (apply [{ Helper.replaceInstWithPoison(*${root}); }])>;
+
 // Instructions where if all source operands are undef, the instruction can be
 // replaced with undef.
 def propagate_undef_all_ops: GICombineRule<
@@ -463,6 +489,15 @@ def propagate_undef_all_ops: GICombineRule<
           [{ return Helper.matchAllExplicitUsesAreUndef(*${root}); }]),
   (apply [{ Helper.replaceInstWithUndef(*${root}); }])>;
 
+// Instructions where if all source operands are poison, the instruction can be
+// replaced with poison.
+def propagate_poison_all_ops
+    : GICombineRule<
+          (defs root:$root),
+          (match(wip_match_opcode G_SHUFFLE_VECTOR, G_BUILD_VECTOR):$root,
+              [{ return Helper.matchAllExplicitUsesArePoison(*${root}); }]),
+          (apply [{ Helper.replaceInstWithPoison(*${root}); }])>;
+
 // Replace a G_SHUFFLE_VECTOR with an undef mask with a G_IMPLICIT_DEF.
 def propagate_undef_shuffle_mask: GICombineRule<
   (defs root:$root),
@@ -654,6 +689,13 @@ def erase_undef_store : GICombineRule<
   (apply [{ Helper.eraseInst(*${root}); }])
 >;
 
+// Erase stores of poison values.
+def erase_poison_store
+    : GICombineRule<(defs root:$root),
+                    (match(wip_match_opcode G_STORE):$root,
+                        [{ return Helper.matchPoisonStore(*${root}); }]),
+                    (apply [{ Helper.eraseInst(*${root}); }])>;
+
 def simplify_add_to_sub_matchinfo: GIDefMatchData<"std::tuple<Register, Register>">;
 def simplify_add_to_sub: GICombineRule <
   (defs root:$root, simplify_add_to_sub_matchinfo:$info),
@@ -1955,6 +1997,11 @@ def undef_combines : GICombineGroup<[undef_to_fp_zero, undef_to_int_zero,
                                      erase_undef_store,
                                      insert_extract_vec_elt_out_of_bounds]>;
 
+def poison_combines
+    : GICombineGroup<[binop_right_poison_to_poison, unary_poison_to_poison,
+                      propagate_poison_any_op, propagate_poison_all_ops,
+                      erase_poison_store]>;
+
 def identity_combines : GICombineGroup<[select_same_val, right_identity_zero,
                                         binop_same_val, binop_left_to_zero,
                                         binop_right_to_zero, p2i_to_i2p,
@@ -2006,29 +2053,29 @@ def shuffle_combines : GICombineGroup<[combine_shuffle_concat,
                                        combine_shuffle_undef_rhs,
                                        combine_shuffle_disjoint_mask]>;
 
-def all_combines : GICombineGroup<[integer_reassoc_combines, trivial_combines,
-    vector_ops_combines, freeze_combines, cast_combines,
-    insert_vec_elt_combines, extract_vec_elt_combines, combines_for_extload,
-    combine_extracted_vector_load,
-    undef_combines, identity_combines, phi_combines,
-    simplify_add_to_sub, hoist_logic_op_with_same_opcode_hands, shifts_too_big,
-    reassocs, ptr_add_immed_chain, cmp_combines,
-    shl_ashr_to_sext_inreg, sext_inreg_of_load,
-    width_reduction_combines, select_combines,
-    known_bits_simplifications, trunc_shift,
-    not_cmp_fold, opt_brcond_by_inverting_cond,
-    const_combines, xor_of_and_with_same_reg, ptr_add_with_zero,
-    shift_immed_chain, shift_of_shifted_logic_chain, load_or_combine,
-    div_rem_to_divrem, funnel_shift_combines, bitreverse_shift, commute_shift,
-    form_bitfield_extract, constant_fold_binops, constant_fold_fma,
-    constant_fold_cast_op, fabs_fneg_fold,
-    intdiv_combines, mulh_combines, redundant_neg_operands,
-    and_or_disjoint_mask, fma_combines, fold_binop_into_select,
-    sub_add_reg, select_to_minmax,
-    fsub_to_fneg, commute_constant_to_rhs, match_ands, match_ors,
-    simplify_neg_minmax, combine_concat_vector,
-    sext_trunc, zext_trunc, prefer_sign_combines, shuffle_combines,
-    combine_use_vector_truncate, merge_combines, overflow_combines]>;
+def all_combines
+    : GICombineGroup<
+          [integer_reassoc_combines, trivial_combines, vector_ops_combines,
+           freeze_combines, cast_combines, insert_vec_elt_combines,
+           extract_vec_elt_combines, combines_for_extload,
+           combine_extracted_vector_load, undef_combines, poison_combines,
+           identity_combines, phi_combines, simplify_add_to_sub,
+           hoist_logic_op_with_same_opcode_hands, shifts_too_big, reassocs,
+           ptr_add_immed_chain, cmp_combines, shl_ashr_to_sext_inreg,
+           sext_inreg_of_load, width_reduction_combines, select_combines,
+           known_bits_simplifications, trunc_shift, not_cmp_fold,
+           opt_brcond_by_inverting_cond, const_combines,
+           xor_of_and_with_same_reg, ptr_add_with_zero, shift_immed_chain,
+           shift_of_shifted_logic_chain, load_or_combine, div_rem_to_divrem,
+           funnel_shift_combines, bitreverse_shift, commute_shift,
+           form_bitfield_extract, constant_fold_binops, constant_fold_fma,
+           constant_fold_cast_op, fabs_fneg_fold, intdiv_combines,
+           mulh_combines, redundant_neg_operands, and_or_disjoint_mask,
+           fma_combines, fold_binop_into_select, sub_add_reg, select_to_minmax,
+           fsub_to_fneg, commute_constant_to_rhs, match_ands, match_ors,
+           simplify_neg_minmax, combine_concat_vector, sext_trunc, zext_trunc,
+           prefer_sign_combines, shuffle_combines, combine_use_vector_truncate,
+           merge_combines, overflow_combines]>;
 
 // A combine group used to for prelegalizer combiners at -O0. The combines in
 // this group have been selected based on experiments to balance code size and
diff --git a/llvm/lib/CodeGen/GlobalISel/CSEInfo.cpp b/llvm/lib/CodeGen/GlobalISel/CSEInfo.cpp
index 0222069cfc576..1e92856e7b647 100644
--- a/llvm/lib/CodeGen/GlobalISel/CSEInfo.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/CSEInfo.cpp
@@ -54,6 +54,7 @@ bool CSEConfigFull::shouldCSEOpc(unsigned Opc) {
   case TargetOpcode::G_CONSTANT:
   case TargetOpcode::G_FCONSTANT:
   case TargetOpcode::G_IMPLICIT_DEF:
+  case TargetOpcode::G_POISON:
   case TargetOpcode::G_ZEXT:
   case TargetOpcode::G_SEXT:
   case TargetOpcode::G_ANYEXT:
@@ -82,7 +83,7 @@ bool CSEConfigFull::shouldCSEOpc(unsigned Opc) {
 
 bool CSEConfigConstantOnly::shouldCSEOpc(unsigned Opc) {
   return Opc == TargetOpcode::G_CONSTANT || Opc == TargetOpcode::G_FCONSTANT ||
-         Opc == TargetOpcode::G_IMPLICIT_DEF;
+         Opc == TargetOpcode::G_IMPLICIT_DEF || Opc == TargetOpcode::G_POISON;
 }
 
 std::unique_ptr<CSEConfigBase>
diff --git a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
index 0dfbb91f2ac54..73d7a572dabe6 100644
--- a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
@@ -328,6 +328,7 @@ bool CombinerHelper::matchCombineConcatVectors(
       for (const MachineOperand &BuildVecMO : Def->uses())
         Ops.push_back(BuildVecMO.getReg());
       break;
+    case TargetOpcode::G_POISON:
     case TargetOpcode::G_IMPLICIT_DEF: {
       LLT OpType = MRI.getType(Reg);
       // Keep one undef value for all the undef operands.
@@ -2695,6 +2696,12 @@ bool CombinerHelper::matchAnyExplicitUseIsUndef(MachineInstr &MI) const {
   });
 }
 
+bool CombinerHelper::matchAnyExplicitUseIsPoison(MachineInstr &MI) const {
+  return any_of(MI.explicit_uses(), [this](const MachineOperand &MO) {
+    return MO.isReg() && getOpcodeDef(TargetOpcode::G_POISON, MO.getReg(), MRI);
+  });
+}
+
 bool CombinerHelper::matchAllExplicitUsesAreUndef(MachineInstr &MI) const {
   return all_of(MI.explicit_uses(), [this](const MachineOperand &MO) {
     return !MO.isReg() ||
@@ -2702,6 +2709,13 @@ bool CombinerHelper::matchAllExplicitUsesAreUndef(MachineInstr &MI) const {
   });
 }
 
+bool CombinerHelper::matchAllExplicitUsesArePoison(MachineInstr &MI) const {
+  return all_of(MI.explicit_uses(), [this](const MachineOperand &MO) {
+    return !MO.isReg() ||
+           getOpcodeDef(TargetOpcode::G_POISON, MO.getReg(), MRI);
+  });
+}
+
 bool CombinerHelper::matchUndefShuffleVectorMask(MachineInstr &MI) const {
   assert(MI.getOpcode() == TargetOpcode::G_SHUFFLE_VECTOR);
   ArrayRef<int> Mask = MI.getOperand(3).getShuffleMask();
@@ -2714,6 +2728,11 @@ bool CombinerHelper::matchUndefStore(MachineInstr &MI) const {
                       MRI);
 }
 
+bool CombinerHelper::matchPoisonStore(MachineInstr &MI) const {
+  assert(MI.getOpcode() == TargetOpcode::G_STORE);
+  return getOpcodeDef(TargetOpcode::G_POISON, MI.getOperand(0).getReg(), MRI);
+}
+
 bool CombinerHelper::matchUndefSelectCmp(MachineInstr &MI) const {
   assert(MI.getOpcode() == TargetOpcode::G_SELECT);
   return getOpcodeDef(TargetOpcode::G_IMPLICIT_DEF, MI.getOperand(1).getReg(),
@@ -2953,6 +2972,12 @@ bool CombinerHelper::matchOperandIsUndef(MachineInstr &MI,
          getOpcodeDef(TargetOpcode::G_IMPLICIT_DEF, MO.getReg(), MRI);
 }
 
+bool CombinerHelper::matchOperandIsPoison(MachineInstr &MI,
+                                          unsigned OpIdx) const {
+  MachineOperand &MO = MI.getOperand(OpIdx);
+  return MO.isReg() && getOpcodeDef(TargetOpcode::G_POISON, MO.getReg(), MRI);
+}
+
 bool CombinerHelper::matchOperandIsKnownToBeAPowerOfTwo(MachineInstr &MI,
                                                         unsigned OpIdx) const {
   MachineOperand &MO = MI.getOperand(OpIdx);
@@ -2992,6 +3017,12 @@ void CombinerHelper::replaceInstWithUndef(MachineInstr &MI) const {
   MI.eraseFromParent();
 }
 
+void CombinerHelper::replaceInstWithPoison(MachineInstr &MI) const {
+  assert(MI.getNumDefs() == 1 && "Expected only one def?");
+  Builder.buildPoison(MI.getOperand(0));
+  MI.eraseFromParent();
+}
+
 bool CombinerHelper::matchSimplifyAddToSub(
     MachineInstr &MI, std::tuple<Register, Register> &MatchInfo) const {
   Register LHS = MI.getOperand(1).getReg();
@@ -3056,6 +3087,7 @@ bool CombinerHelper::matchCombineInsertVecElts(
   // If we didn't end in a G_IMPLICIT_DEF and the source is not fully
   // overwritten, bail out.
   return TmpInst->getOpcode() == TargetOpcode::G_IMPLICIT_DEF ||
+         TmpInst->getOpcode() == TargetOpcode::G_POISON ||
          all_of(MatchInfo, [](Register Reg) { return !!Reg; });
 }
 
@@ -3426,12 +3458,13 @@ bool CombinerHelper::matchUseVectorTruncate(MachineInstr &MI,
   if (I < 2)
     return false;
 
-  // Check the remaining source elements are only G_IMPLICIT_DEF
+  // Check the remaining source elements are only G_IMPLICIT_DEF or G_POISON
   for (; I < NumOperands; ++I) {
     auto SrcMI = MRI.getVRegDef(BuildMI->getSourceReg(I));
     auto SrcMIOpc = SrcMI->getOpcode();
 
-    if (SrcMIOpc != TargetOpcode::G_IMPLICIT_DEF)
+    if (SrcMIOpc != TargetOpcode::G_IMPLICIT_DEF &&
+        SrcMIOpc != TargetOpcode::G_POISON)
       return false;
   }
 
@@ -7892,6 +7925,12 @@ bool CombinerHelper::matchShuffleDisjointMask(MachineInstr &MI,
   if (getOpcodeDef(TargetOpcode::G_IMPLICIT_DEF, Shuffle.getSrc2Reg(), MRI))
     return false;
 
+  if (getOpcodeDef(TargetOpcode::G_POISON, Shuffle.getSrc1Reg(), MRI))
+    return false;
+
+  if (getOpcodeDef(TargetOpcode::G_POISON, Shuffle.getSrc2Reg(), MRI))
+    return false;
+
   const LLT DstTy = MRI.getType(Shuffle.getReg(0));
   const LLT Src1Ty = MRI.getType(Shuffle.getSrc1Reg());
   if (!isLegalOrBeforeLegalizer(
diff --git a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
index d01de29826cad..95eae9c8c4f3f 100644
--- a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
@@ -3660,6 +3660,8 @@ bool IRTranslator::translate(const Constant &C, Register Reg) {
     EntryBuilder->buildConstant(Reg, *CI);
   else if (auto CF = dyn_cast<ConstantFP>(&C))
     EntryBuilder->buildFConstant(Reg, *CF);
+  else if (isa<PoisonValue>(C))
+    EntryBuilder->buildPoison(Reg);
   else if (isa<UndefValue>(C))
     EntryBuilder->buildUndef(Reg);
   else if (isa<ConstantPointerNull>(C))
diff --git a/llvm/lib/CodeGen/GlobalISel/LegacyLegalizerInfo.cpp b/llvm/lib/CodeGen/GlobalISel/LegacyLegalizerInfo.cpp
index f338f66997657..bcc0760c7794e 100644
--- a/llvm/lib/CodeGen/GlobalISel/LegacyLegalizerInfo.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/LegacyLegalizerInfo.cpp
@@ -82,6 +82,8 @@ LegacyLegalizerInfo::LegacyLegalizerInfo() {
 
   setLegalizeScalarToDifferentSizeStrategy(
       TargetOpcode::G_IMPLICIT_DEF, 0, narrowToSmallerAndUnsupportedIfTooSmall);
+  setLegalizeScalarToDifferentSizeStrategy(
+      TargetOpcode::G_POISON, 0, narrowToSmallerAndUnsupportedIfTooSmall);
   setLegalizeScalarToDifferentSizeStrategy(
       TargetOpcode::G_ADD, 0, widenToLargerTypesAndNarrowToLargest);
   setLegalizeScalarToDifferentSizeStrategy(
diff --git a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp b/llvm/lib/Cod...
[truncated]

arsenm · 2025-02-20T02:59:41Z

llvm/include/llvm/Target/GlobalISel/Combine.td

+def binop_right_poison_to_poison
+    : GICombineRule<(defs root:$root),
+                    (match(wip_match_opcode G_SHL, G_ASHR, G_LSHR):$root,
+                        [{ return Helper.matchOperandIsPoison(*${root}, 2); }]),
+                    (apply [{ Helper.replaceInstWithPoison(*${root}); }])>;


I would drop all of the combine handling in the initial patch. There needs to be more fusion with undef

Sure, I removed all of it from this PR. I'll place it in the follow-up.

arsenm · 2025-02-20T03:00:10Z

llvm/test/CodeGen/X86/GlobalISel/calllowering-nocrashret.ll

+  ; CHECK:   $xmm0 = COPY [[ANYEXT]](<4 x s32>)
+  ; CHECK:   RET 0, implicit $xmm0
+entry:
+  ret <4 x i1> poison ;


don't need comment ;

arsenm · 2025-02-20T03:01:13Z

llvm/test/CodeGen/X86/GlobalISel/calllowering-nocrashret.ll

@@ -11,3 +11,14 @@ define <4 x i1> @foo() {
 entry:
  ret <4 x i1> undef ;
 }
+
+define <4 x i1> @foo_poison() {


Better to move this to a more obviously related test file, either an irtranslator-poison.ll or test/CodeGen/GlobalISel/arm64-irtranslator.ll is used as the catch-all

arsenm · 2025-02-20T03:10:40Z

llvm/lib/CodeGen/GlobalISel/CSEInfo.cpp

@@ -82,7 +83,7 @@ bool CSEConfigFull::shouldCSEOpc(unsigned Opc) {

 bool CSEConfigConstantOnly::shouldCSEOpc(unsigned Opc) {
  return Opc == TargetOpcode::G_CONSTANT || Opc == TargetOpcode::G_FCONSTANT ||
-         Opc == TargetOpcode::G_IMPLICIT_DEF;
+         Opc == TargetOpcode::G_IMPLICIT_DEF || Opc == TargetOpcode::G_POISON;


Beyond the scope of this patch, but we probably should stop CSEing G_IMPLICIT_DEF, and only do it for G_POISON

arsenm · 2025-02-20T03:12:11Z

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

+  if (getOpcodeDef(TargetOpcode::G_POISON, Shuffle.getSrc1Reg(), MRI))
+    return false;


In the combine patch, we should use something smarter for matching. I don't particularly like getOpcodeDef, and this should probably use the mi_match on an UndefValue similar to how the IR does it

github-actions · 2025-04-20T17:17:12Z

✅ With the latest revision this PR passed the C/C++ code formatter.

arsenm · 2025-04-21T11:05:41Z

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

+    return MO.isReg() && getOpcodeDef(TargetOpcode::G_POISON, MO.getReg(), MRI);
+  });
+}
+
 bool CombinerHelper::matchAllExplicitUsesAreUndef(MachineInstr &MI) const {
  return all_of(MI.explicit_uses(), [this](const MachineOperand &MO) {
    return !MO.isReg() ||
           getOpcodeDef(TargetOpcode::G_IMPLICIT_DEF, MO.getReg(), MRI);


This should also catch the poison case (separate patch though, getOpcodeDef is a bad interface)

arsenm · 2025-04-21T11:06:03Z

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

 bool CombinerHelper::matchAllExplicitUsesAreUndef(MachineInstr &MI) const {
  return all_of(MI.explicit_uses(), [this](const MachineOperand &MO) {
    return !MO.isReg() ||
           getOpcodeDef(TargetOpcode::G_IMPLICIT_DEF, MO.getReg(), MRI);
  });
 }

+bool CombinerHelper::matchAllExplicitUsesArePoison(MachineInstr &MI) const {
+  return all_of(MI.explicit_uses(), [this](const MachineOperand &MO) {
+    return !MO.isReg() ||


This shouldn't need to check if the operand is a register, but this is consistent with the other cases

arsenm · 2025-04-21T11:06:45Z

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

+      MRI.getVRegDef(Passthru)->getOpcode() != TargetOpcode::G_IMPLICIT_DEF &&
+      MRI.getVRegDef(Passthru)->getOpcode() != TargetOpcode::G_POISON;


Avoid 2 x getVRegDef

arsenm · 2025-04-21T11:08:27Z

llvm/lib/CodeGen/GlobalISel/Utils.cpp

@@ -1566,6 +1568,7 @@ bool llvm::isAllOnesOrAllOnesSplat(const MachineInstr &MI,
                                   const MachineRegisterInfo &MRI,
                                   bool AllowUndefs) {
  switch (MI.getOpcode()) {
+  case TargetOpcode::G_POISON:


This will need revisiting, poison could be more aggressive here

arsenm · 2025-04-21T11:10:16Z

llvm/lib/Target/AArch64/AArch64Combine.td

+def fullrevpoison: GICombineRule <
+  (defs root:$root, shuffle_matchdata:$matchinfo),
+  (match (G_POISON $src2),
+         (G_SHUFFLE_VECTOR $src, $src1, $src2, $mask):$root,
+         [{ return ShuffleVectorInst::isReverseMask(${mask}.getShuffleMask(),
+                                                    ${mask}.getShuffleMask().size()); }]),
+  (apply [{ applyFullRev(*${root}, MRI); }])
+>;


You can repeat the existing pattern for the opcodes with foreach, e.g. see idempotent_prop_frags in the generic combines

Thanks for the tip! Done!

mtsokol · 2025-04-22T14:18:00Z

Now I'm left with ~9 AMDGPU test failures which I need to inspect. I still have pending comment on separating the simple test.

arsenm · 2025-04-22T17:17:18Z

llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp

+           (getOpcodeDef(TargetOpcode::G_IMPLICIT_DEF, MO.getReg(), MRI) ||
+            getOpcodeDef(TargetOpcode::G_POISON, MO.getReg(), MRI));


Can you add a fixme? We shouldn't look up copy chains twice just to check 2 opcodes. We should delete getOpcodeDef as a helper, it's not good

Done! In the next patch I can come up with some helpers where getOpcodeDef accepts a set of opcodes.

mtsokol · 2025-04-22T21:23:09Z

@arsenm So right now I adjusted all places in GlobalISel's include/lib directories and Target/{AMDGPU,RISCV,Aarch64,X86} directories where G_IMPLICIT_DEF was used and considered G_POISON the same way.

Unfortunately I'm still left with 4 GlobalISel/AMDGPU - for those I couldn't come up with the places where G_IMPLICIT_DEF is still handled and G_POISON isn't. I guess I need more time to get a better grasp of what is happening in the lowering from GlobalISel to AMDGPU to try something new - I ran out of trial&error ideas.

EDIT: There are also 4 Global/Aarch64 failures but those are back after removing combine.td changes.

mtsokol mentioned this pull request Feb 19, 2025

GlobalISel should have G_POISON #127486

Open

arsenm added the llvm:globalisel label Feb 20, 2025

arsenm reviewed Feb 20, 2025

View reviewed changes

arsenm requested review from aemerson and davemgreen February 20, 2025 03:01

arsenm reviewed Feb 20, 2025

View reviewed changes

[GlobalISel] Introduce G_POISON

0d07595

mtsokol force-pushed the globalisel_poison branch from 48adc07 to 0d07595 Compare April 19, 2025 16:49

mtsokol force-pushed the globalisel_poison branch from cbb1d91 to 958a5c3 Compare April 20, 2025 17:37

Adjust targets for G_POISON

a8569f2

mtsokol force-pushed the globalisel_poison branch from 958a5c3 to a8569f2 Compare April 20, 2025 17:43

mtsokol added 3 commits April 20, 2025 19:00

First patch for failing tests

59a800d

Remove combine logic

a8cc507

another patch

c6b0e4e

mtsokol force-pushed the globalisel_poison branch from 18ab129 to c6b0e4e Compare April 20, 2025 20:29

Trial - update AMDGPU tests

b62284a

arsenm reviewed Apr 21, 2025

View reviewed changes

Further improvements

2af2004

arsenm reviewed Apr 22, 2025

View reviewed changes

Further work

386e424

mtsokol force-pushed the globalisel_poison branch from 3702e4b to 386e424 Compare April 22, 2025 22:16

petar-avramovic mentioned this pull request May 22, 2025

AMDGPU/GlobalISel: add RegBankLegalize rules for extends and trunc #132383

Merged

		if (getOpcodeDef(TargetOpcode::G_POISON, Shuffle.getSrc1Reg(), MRI))
		return false;

		MRI.getVRegDef(Passthru)->getOpcode() != TargetOpcode::G_IMPLICIT_DEF &&
		MRI.getVRegDef(Passthru)->getOpcode() != TargetOpcode::G_POISON;

		(getOpcodeDef(TargetOpcode::G_IMPLICIT_DEF, MO.getReg(), MRI) \|\|
		getOpcodeDef(TargetOpcode::G_POISON, MO.getReg(), MRI));

[GlobalISel] Introduce G_POISON #127825

Are you sure you want to change the base?

[GlobalISel] Introduce G_POISON #127825

Uh oh!

Conversation

mtsokol commented Feb 19, 2025

Uh oh!

llvmbot commented Feb 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mtsokol commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mtsokol commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

[GlobalISel] Introduce `G_POISON` #127825

[GlobalISel] Introduce `G_POISON` #127825

github-actions bot commented Apr 20, 2025 •

edited

Loading

mtsokol commented Apr 22, 2025 •

edited

Loading

mtsokol commented Apr 22, 2025 •

edited

Loading