[TableGen][GlobalISel] Add MIR Pattern Builtins

Adds a new feature to MIR patterns: builtin instructions.
They offer some additional capabilities that currently cannot be expressed without falling back to C++ code.
There are two builtins added with this patch, but more can be added later as new needs arise:
 - GIReplaceReg
 - GIEraseRoot

Depends on D158714, D158713

Reviewed By: arsenm, aemerson

Differential Revision: https://reviews.llvm.org/D158975

Author: Pierre-vh

llvm/docs/GlobalISel/MIRPatterns.rst

@@ -101,6 +101,46 @@ pattern, you can try naming your patterns to see exactly where the issue is.
   // using $x again here copies operand 1 from G_AND into the new inst.
   (apply (COPY $root, $x))
 
+Builtin Operations
+------------------
+
+MIR Patterns also offer builtin operations, also called "builtin instructions".
+They offer some powerful features that would otherwise require use of C++ code.
+
+GIReplaceReg
+~~~~~~~~~~~~
+
+.. code-block:: text
+  :caption: Usage
+
+  (apply (GIReplaceReg $old, $new))
+
+Operands:
+
+* ``$old`` (out) register defined by a matched instruction
+* ``$new`` (in)  register
+
+Semantics:
+
+* Can only appear in an 'apply' pattern.
+* If both old/new are operands of matched instructions,
+  ``canReplaceReg`` is checked before applying the rule.
+
+
+GIEraseRoot
+~~~~~~~~~~~
+
+.. code-block:: text
+  :caption: Usage
+
+  (apply (GIEraseRoot))
+
+Semantics:
+
+* Can only appear as the only pattern of an 'apply' pattern list.
+* The root cannot have any output operands.
+* The root must be a CodeGenInstruction
+
 
 Limitations
 -----------
@@ -114,10 +154,6 @@ This a non-exhaustive list of known issues with MIR patterns at this time.
 * Instructions with multiple defs cannot be the root of a ``GICombinePatFrag``.
 * Using ``GICombinePatFrag`` in the ``apply`` pattern of a ``GICombineRule``
   is not supported.
-* Deleting the matched pattern in a ``GICombineRule`` needs to be done using
-  ``G_IMPLICIT_DEF`` or C++.
-* Replacing the root of a pattern with another instruction needs to be done
-  using COPY.
 * We cannot rewrite a matched instruction other than the root.
 * Matching/creating a (CImm) immediate >64 bits is not supported
   (see comment in ``GIM_CheckConstantInt``)
@@ -179,33 +215,41 @@ The following expansions are available for MIR patterns:
 Common Pattern #1: Replace a Register with Another
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The 'apply' pattern must always redefine its root.
-It cannot just replace it with something else directly.
-A simple workaround is to just use a COPY that'll be eliminated later.
+The 'apply' pattern must always redefine all operands defined by the match root.
+Sometimes, we do not need to create instructions, simply replace a def with
+another matched register. The ``GIReplaceReg`` builtin can do just that.
 
 .. code-block:: text
 
   def Foo : GICombineRule<
     (defs root:$dst),
     (match (G_FNEG $tmp, $src), (G_FNEG $dst, $tmp)),
-    (apply (COPY $dst, $src))>;
+    (apply (GIReplaceReg $dst, $src))>;
 
-Common Pattern #2: Erasing a Pattern
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+This also works if the replacement register is a temporary register from the
+``apply`` pattern.
 
-As said before, we must always emit something in the 'apply' pattern.
-If we wish to delete the matched instruction, we can simply replace its
-definition with a ``G_IMPLICIT_DEF``.
+.. code-block:: text
+
+  def ReplaceTemp : GICombineRule<
+    (defs root:$a),
+    (match    (G_BUILD_VECTOR $tmp, $x, $y),
+              (G_UNMERGE_VALUES $a, $b, $tmp)),
+    (apply  (G_UNMERGE_VALUES $a, i32:$new, $y),
+            (GIReplaceReg $b, $new))>
+
+Common Pattern #2: Erasing a Def-less Root
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If we simply want to erase a def-less match root, we can use the
+``GIEraseRoot`` builtin.
 
 .. code-block:: text
 
   def Foo : GICombineRule<
-    (defs root:$dst),
-    (match (G_FOO $tmp, $src), (G_BAR $dst, $tmp)),
-    (apply (G_IMPLICIT_DEF $dst))>;
-
-If the instruction has no definition, like ``G_STORE``, we cannot use
-an instruction pattern in 'apply' - C++ has to be used.
+    (defs root:$mi),
+    (match (G_STORE $a, $b):$mi),
+    (apply (GIEraseRoot))>;
 
 Common Pattern #3: Emitting a Constant Value
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutor.h

@@ -258,6 +258,13 @@ enum {
   GIM_CheckIsSameOperand,
   GIM_CheckIsSameOperandIgnoreCopies,
 
+  /// Check we can replace all uses of a register with another.
+  /// - OldInsnID
+  /// - OldOpIdx
+  /// - NewInsnID
+  /// - NewOpIdx
+  GIM_CheckCanReplaceReg,
+
   /// Predicates with 'let PredicateCodeUsesOperands = 1' need to examine some
   /// named operands that will be recorded in RecordedOperands. Names of these
   /// operands are referenced in predicate argument list. Emitter determines
@@ -431,6 +438,20 @@ enum {
   /// - Expected type
   GIR_MakeTempReg,
 
+  /// Replaces all references to a register from an instruction
+  /// with another register from another instruction.
+  /// - OldInsnID
+  /// - OldOpIdx
+  /// - NewInsnID
+  /// - NewOpIdx
+  GIR_ReplaceReg,
+
+  /// Replaces all references to a register with a temporary register.
+  /// - OldInsnID
+  /// - OldOpIdx
+  /// - TempRegIdx
+  GIR_ReplaceRegWithTempReg,
+
   /// A successful emission
   GIR_Done,
 

llvm/include/llvm/CodeGen/GlobalISel/GIMatchTableExecutorImpl.h

@@ -866,6 +866,25 @@ bool GIMatchTableExecutor::executeMatchTable(
       }
       break;
     }
+    case GIM_CheckCanReplaceReg: {
+      int64_t OldInsnID = MatchTable[CurrentIdx++];
+      int64_t OldOpIdx = MatchTable[CurrentIdx++];
+      int64_t NewInsnID = MatchTable[CurrentIdx++];
+      int64_t NewOpIdx = MatchTable[CurrentIdx++];
+
+      DEBUG_WITH_TYPE(TgtExecutor::getName(),
+                      dbgs() << CurrentIdx << ": GIM_CheckCanReplaceReg(MIs["
+                             << OldInsnID << "][" << OldOpIdx << "] = MIs["
+                             << NewInsnID << "][" << NewOpIdx << "])\n");
+
+      Register Old = State.MIs[OldInsnID]->getOperand(OldOpIdx).getReg();
+      Register New = State.MIs[NewInsnID]->getOperand(NewOpIdx).getReg();
+      if (!canReplaceReg(Old, New, MRI)) {
+        if (handleReject() == RejectAndGiveUp)
+          return false;
+      }
+      break;
+    }
     case GIM_Reject:
       DEBUG_WITH_TYPE(TgtExecutor::getName(),
                       dbgs() << CurrentIdx << ": GIM_Reject\n");
@@ -1237,7 +1256,45 @@ bool GIMatchTableExecutor::executeMatchTable(
                              << "] = GIR_MakeTempReg(" << TypeID << ")\n");
       break;
     }
+    case GIR_ReplaceReg: {
+      int64_t OldInsnID = MatchTable[CurrentIdx++];
+      int64_t OldOpIdx = MatchTable[CurrentIdx++];
+      int64_t NewInsnID = MatchTable[CurrentIdx++];
+      int64_t NewOpIdx = MatchTable[CurrentIdx++];
+
+      DEBUG_WITH_TYPE(TgtExecutor::getName(),
+                      dbgs() << CurrentIdx << ": GIR_ReplaceReg(MIs["
+                             << OldInsnID << "][" << OldOpIdx << "] = MIs["
+                             << NewInsnID << "][" << NewOpIdx << "])\n");
 
+      Register Old = State.MIs[OldInsnID]->getOperand(OldOpIdx).getReg();
+      Register New = State.MIs[NewInsnID]->getOperand(NewOpIdx).getReg();
+      if (Observer)
+        Observer->changingAllUsesOfReg(MRI, Old);
+      MRI.replaceRegWith(Old, New);
+      if (Observer)
+        Observer->finishedChangingAllUsesOfReg();
+      break;
+    }
+    case GIR_ReplaceRegWithTempReg: {
+      int64_t OldInsnID = MatchTable[CurrentIdx++];
+      int64_t OldOpIdx = MatchTable[CurrentIdx++];
+      int64_t TempRegID = MatchTable[CurrentIdx++];
+
+      DEBUG_WITH_TYPE(TgtExecutor::getName(),
+                      dbgs() << CurrentIdx << ": GIR_ReplaceRegWithTempReg(MIs["
+                             << OldInsnID << "][" << OldOpIdx << "] = TempRegs["
+                             << TempRegID << "])\n");
+
+      Register Old = State.MIs[OldInsnID]->getOperand(OldOpIdx).getReg();
+      Register New = State.TempRegisters[TempRegID];
+      if (Observer)
+        Observer->changingAllUsesOfReg(MRI, Old);
+      MRI.replaceRegWith(Old, New);
+      if (Observer)
+        Observer->finishedChangingAllUsesOfReg();
+      break;
+    }
     case GIR_Coverage: {
       int64_t RuleID = MatchTable[CurrentIdx++];
       assert(CoverageInfo);

llvm/include/llvm/Target/GlobalISel/Combine.td

@@ -110,6 +110,42 @@ class GICombinePatFrag<dag outs, dag ins, list<dag> alts> {
   list<dag> Alternatives = alts;
 }
 
+//===----------------------------------------------------------------------===//
+// Pattern Builtins
+//===----------------------------------------------------------------------===//
+
+// "Magic" Builtin instructions for MIR patterns.
+// The definitions that implement
+class GIBuiltinInst;
+
+// Replace all references to a register with another one.
+//
+// Usage:
+//    (apply (GIReplaceReg $old, $new))
+//
+// Operands:
+// - $old (out) register defined by a matched instruction
+// - $new (in)  register
+//
+// Semantics:
+// - Can only appear in an 'apply' pattern.
+// - If both old/new are operands of matched instructions,
+//   "canReplaceReg" is checked before applying the rule.
+def GIReplaceReg : GIBuiltinInst;
+
+// Apply action that erases the match root.
+//
+// Usage:
+//    (apply (GIEraseRoot))
+//
+// Semantics:
+// - Can only appear as the only pattern of an 'apply' pattern list.
+// - The root cannot have any output operands.
+// - The root must be a CodeGenInstruction
+//
+// TODO: Allow using this directly, like (apply GIEraseRoot)
+def GIEraseRoot : GIBuiltinInst;
+
 //===----------------------------------------------------------------------===//
 
 def extending_load_matchdata : GIDefMatchData<"PreferredTuple">;
@@ -141,7 +177,7 @@ def idempotent_prop_frags : GICombinePatFrag<
 def idempotent_prop : GICombineRule<
    (defs root:$dst),
    (match (idempotent_prop_frags $dst, $src)),
-   (apply (COPY $dst, $src))>;
+   (apply (GIReplaceReg $dst, $src))>;
 
 
 def extending_loads : GICombineRule<
@@ -337,7 +373,7 @@ def select_undef_cmp: GICombineRule<
   (defs root:$dst),
   (match (G_IMPLICIT_DEF $undef),
          (G_SELECT $dst, $undef, $x, $y)),
-  (apply (COPY $dst, $y))
+  (apply (GIReplaceReg $dst, $y))
 >;
 
 // Fold (true ? x : y) -> x
@@ -384,14 +420,14 @@ def right_identity_zero_frags : GICombinePatFrag<
 def right_identity_zero: GICombineRule<
   (defs root:$dst),
   (match (right_identity_zero_frags $dst, $lhs)),
-  (apply (COPY $dst, $lhs))
+  (apply (GIReplaceReg $dst, $lhs))
 >;
 
 // Fold x op 1 -> x
 def right_identity_one: GICombineRule<
   (defs root:$dst),
   (match (G_MUL $dst, $x, 1)),
-  (apply (COPY $dst, $x))
+  (apply (GIReplaceReg $dst, $x))
 >;
 
 // Fold (x op x) - > x
@@ -405,7 +441,7 @@ def binop_same_val_frags : GICombinePatFrag<
 def binop_same_val: GICombineRule<
   (defs root:$dst),
   (match (binop_same_val_frags $dst, $src)),
-  (apply (COPY $dst, $src))
+  (apply (GIReplaceReg $dst, $src))
 >;
 
 // Fold (0 op x) - > 0
@@ -456,7 +492,7 @@ def div_rem_to_divrem : GICombineRule<
 def binop_right_to_zero: GICombineRule<
   (defs root:$dst),
   (match (G_MUL $dst, $lhs, 0:$zero)),
-  (apply (COPY $dst, $zero))
+  (apply (GIReplaceReg $dst, $zero))
 >;
 
 // Erase stores of undef values.
@@ -623,7 +659,7 @@ def fneg_fneg_fold: GICombineRule <
   (defs root:$dst),
   (match (G_FNEG $t, $src),
          (G_FNEG $dst, $t)),
-  (apply (COPY $dst, $src))
+  (apply (GIReplaceReg $dst, $src))
 >;
 
 // Fold (unmerge(merge x, y, z)) -> z, y, z.
@@ -1033,7 +1069,7 @@ def add_sub_reg_frags : GICombinePatFrag<
 def add_sub_reg: GICombineRule <
   (defs root:$dst),
   (match (add_sub_reg_frags $dst, $src)),
-  (apply (COPY $dst, $src))>;
+  (apply (GIReplaceReg $dst, $src))>;
 
 def buildvector_identity_fold : GICombineRule<
   (defs root:$build_vector, register_matchinfo:$matchinfo),

llvm/test/CodeGen/AArch64/GlobalISel/prelegalizercombiner-trivial-arith.mir

@@ -74,8 +74,9 @@ body:             |
     ; CHECK-LABEL: name: mul_0_cant_replace
     ; CHECK: liveins: $w0
     ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: %x:_(s32) = COPY $w0
     ; CHECK-NEXT: %cst:_(s32) = G_CONSTANT i32 0
-    ; CHECK-NEXT: %op:gpr(s32) = COPY %cst(s32)
+    ; CHECK-NEXT: %op:gpr(s32) = G_MUL %x, %cst
     ; CHECK-NEXT: $w0 = COPY %op(s32)
     ; CHECK-NEXT: RET_ReallyLR implicit $w0
     %x:_(s32) = COPY $w0