Skip to content

Commit 54cb405

Browse files
vntkmrVineet Kumar
and
Vineet Kumar
authored
[X86][ISel][FMA] Get a handle on operand nodes when negating FMA (#130176)
When negating an FMA opcode, a new node created for a negated FMA operand may be deleted while recursively negating another FMA operand. This causes the following assertion to fail: ``` llc: /root/llvm-project/llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:7686: llvm::SDValue llvm::SelectionDAG::getNode(unsigned int, const llvm::SDLoc&, llvm::EVT, llvm::SDValue, llvm::SDValue, llvm::SDValue, llvm::SDNodeFlags): Assertion `N1.getOpcode() != ISD::DELETED_NODE && N2.getOpcode() != ISD::DELETED_NODE && N3.getOpcode() != ISD::DELETED_NODE && "Operand is DELETED_NODE!"' failed. ``` This patch adds a temporary handle on the new negated nodes to prevent them from being deleted. For eg. see https://godbolt.org/z/Tq4PvnKM4 . Co-authored-by: Vineet Kumar <[email protected]>
1 parent b00ad36 commit 54cb405

File tree

2 files changed

+30
-0
lines changed

2 files changed

+30
-0
lines changed

llvm/lib/Target/X86/X86ISelLowering.cpp

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55694,7 +55694,12 @@ static SDValue combineFMA(SDNode *N, SelectionDAG &DAG,
5569455694
// Do not convert the passthru input of scalar intrinsics.
5569555695
// FIXME: We could allow negations of the lower element only.
5569655696
bool NegA = invertIfNegative(A);
55697+
// Create a dummy use for A so that in the process of negating B or C
55698+
// recursively, it is not deleted.
55699+
HandleSDNode NegAHandle(A);
5569755700
bool NegB = invertIfNegative(B);
55701+
// Similar to A, get a handle on B.
55702+
HandleSDNode NegBHandle(B);
5569855703
bool NegC = invertIfNegative(C);
5569955704

5570055705
if (!NegA && !NegB && !NegC)
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
2+
; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -mattr=+avx512f -mattr=+fma | FileCheck %s
3+
4+
define void @fma_neg(<8 x i1> %r280, ptr %pp1, ptr %pp2) {
5+
; CHECK-LABEL: fma_neg:
6+
; CHECK: # %bb.0:
7+
; CHECK-NEXT: vpmovsxwq %xmm0, %zmm0
8+
; CHECK-NEXT: vpsllq $63, %zmm0, %zmm0
9+
; CHECK-NEXT: vptestmq %zmm0, %zmm0, %k1
10+
; CHECK-NEXT: vmovdqu64 (%rdi), %zmm0
11+
; CHECK-NEXT: vpxorq {{\.?LCPI[0-9]+_[0-9]+}}(%rip){1to8}, %zmm0, %zmm1 {%k1} {z}
12+
; CHECK-NEXT: vxorpd %xmm2, %xmm2, %xmm2
13+
; CHECK-NEXT: vfnmadd213pd {{.*#+}} zmm2 = -(zmm0 * zmm2) + zmm1
14+
; CHECK-NEXT: vmovupd %zmm2, (%rsi)
15+
; CHECK-NEXT: vzeroupper
16+
; CHECK-NEXT: retq
17+
%r290 = load <8 x double>, ptr %pp1, align 8
18+
%r307 = fneg <8 x double> %r290
19+
%r309 = select <8 x i1> %r280, <8 x double> %r307, <8 x double> zeroinitializer
20+
%r311 = tail call <8 x double> @llvm.x86.avx512.vfmadd.pd.512(<8 x double> %r307, <8 x double> zeroinitializer, <8 x double> %r309, i32 4)
21+
store <8 x double> %r311, ptr %pp2, align 8
22+
ret void
23+
}
24+
25+
declare <8 x double> @llvm.x86.avx512.vfmadd.pd.512(<8 x double>, <8 x double>, <8 x double>, i32 immarg)

0 commit comments

Comments
 (0)