[CodeGen] Provide MachineFunction::hasUnsafeFPMath
#127488
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
paperchalice
force-pushed
the
rm-unsafe-fp-math
branch
5 times, most recently
from
5cf693c to
25a2cb2
Compare
paperchalice
requested review from
justinfargnoli,
arsenm and
Artem-B
and removed request for
justinfargnoli,
arsenm and
Artem-B
llvmbot
added
backend:ARM
backend:AArch64
backend:AMDGPU
backend:PowerPC
backend:X86
llvm:globalisel
backend:NVPTX
llvm:SelectionDAG
|
@llvm/pr-subscribers-llvm-globalisel @llvm/pr-subscribers-backend-aarch64 Author: None (paperchalice) ChangesAddress #27735 incrementally. New code should use Patch is 31.58 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/127488.diff 21 Files Affected:
diff --git a/llvm/include/llvm/CodeGen/MachineFunction.h b/llvm/include/llvm/CodeGen/MachineFunction.h
index c3eb27b946287..86f1fa9fed952 100644
--- a/llvm/include/llvm/CodeGen/MachineFunction.h
+++ b/llvm/include/llvm/CodeGen/MachineFunction.h
@@ -820,6 +820,10 @@ class LLVM_ABI MachineFunction {
/// True if this function needs frame moves for debug or exceptions.
bool needsFrameMoves() const;
+ /// True if function attribute unsafe-fp-math is true or
+ /// --enable-unsafe-fp-math is passed.
+ bool hasUnsafeFPMath() const;
+
/// Get the function properties
const MachineFunctionProperties &getProperties() const { return Properties; }
MachineFunctionProperties &getProperties() { return Properties; }
diff --git a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
index b193d8bb0aa18..75b1d3715ec6c 100644
--- a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
@@ -5774,7 +5774,7 @@ bool CombinerHelper::canCombineFMadOrFMA(MachineInstr &MI,
LLT DstType = MRI.getType(MI.getOperand(0).getReg());
if (CanReassociate &&
- !(Options.UnsafeFPMath || MI.getFlag(MachineInstr::MIFlag::FmReassoc)))
+ !(MF->hasUnsafeFPMath() || MI.getFlag(MachineInstr::MIFlag::FmReassoc)))
return false;
// Floating-point multiply-add with intermediate rounding.
@@ -5787,7 +5787,7 @@ bool CombinerHelper::canCombineFMadOrFMA(MachineInstr &MI,
return false;
AllowFusionGlobally = Options.AllowFPOpFusion == FPOpFusion::Fast ||
- Options.UnsafeFPMath || HasFMAD;
+ MF->hasUnsafeFPMath() || HasFMAD;
// If the addition is not contractable, do not combine.
if (!AllowFusionGlobally && !MI.getFlag(MachineInstr::MIFlag::FmContract))
return false;
diff --git a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
index d0a62340a5f32..4490bb4f83b49 100644
--- a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
@@ -7879,7 +7879,7 @@ LegalizerHelper::lowerFPTRUNC_F64_TO_F16(MachineInstr &MI) {
if (MRI.getType(Src).isVector()) // TODO: Handle vectors directly.
return UnableToLegalize;
- if (MIRBuilder.getMF().getTarget().Options.UnsafeFPMath) {
+ if (MIRBuilder.getMF().hasUnsafeFPMath()) {
unsigned Flags = MI.getFlags();
auto Src32 = MIRBuilder.buildFPTrunc(S32, Src, Flags);
MIRBuilder.buildFPTrunc(Dst, Src32, Flags);
diff --git a/llvm/lib/CodeGen/MachineFunction.cpp b/llvm/lib/CodeGen/MachineFunction.cpp
index 7d504ef5a0482..d29610bfeb1b4 100644
--- a/llvm/lib/CodeGen/MachineFunction.cpp
+++ b/llvm/lib/CodeGen/MachineFunction.cpp
@@ -699,6 +699,11 @@ bool MachineFunction::needsFrameMoves() const {
!F.getParent()->debug_compile_units().empty();
}
+/// True if function attribute unsafe-fp-math is true.
+bool MachineFunction::hasUnsafeFPMath() const {
+ return F.getFnAttribute("unsafe-fp-math").getValueAsBool();
+}
+
namespace llvm {
template<>
diff --git a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
index f4caaf426de6a..2dc827a8040f5 100644
--- a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
@@ -16280,15 +16280,6 @@ ConstantFoldBITCASTofBUILD_VECTOR(SDNode *BV, EVT DstEltVT) {
return DAG.getBuildVector(VT, DL, Ops);
}
-// Returns true if floating point contraction is allowed on the FMUL-SDValue
-// `N`
-static bool isContractableFMUL(const TargetOptions &Options, SDValue N) {
- assert(N.getOpcode() == ISD::FMUL);
-
- return Options.AllowFPOpFusion == FPOpFusion::Fast || Options.UnsafeFPMath ||
- N->getFlags().hasAllowContract();
-}
-
// Returns true if `N` can assume no infinities involved in its computation.
static bool hasNoInfs(const TargetOptions &Options, SDValue N) {
return Options.NoInfsFPMath || N->getFlags().hasNoInfs();
@@ -16320,8 +16311,9 @@ SDValue DAGCombiner::visitFADDForFMACombine(SDNode *N) {
if (!HasFMAD && !HasFMA)
return SDValue();
- bool AllowFusionGlobally = (Options.AllowFPOpFusion == FPOpFusion::Fast ||
- Options.UnsafeFPMath || HasFMAD);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
+ bool AllowFusionGlobally =
+ (Options.AllowFPOpFusion == FPOpFusion::Fast || UnsafeFPMath || HasFMAD);
// If the addition is not contractable, do not combine.
if (!AllowFusionGlobally && !N->getFlags().hasAllowContract())
return SDValue();
@@ -16379,8 +16371,7 @@ SDValue DAGCombiner::visitFADDForFMACombine(SDNode *N) {
// fadd (G, (fma A, B, (fma (C, D, (fmul (E, F)))))) -->
// fma A, B, (fma C, D, fma (E, F, G)).
// This requires reassociation because it changes the order of operations.
- bool CanReassociate =
- Options.UnsafeFPMath || N->getFlags().hasAllowReassociation();
+ bool CanReassociate = UnsafeFPMath || N->getFlags().hasAllowReassociation();
if (CanReassociate) {
SDValue FMA, E;
if (isFusedOp(N0) && N0.hasOneUse()) {
@@ -16558,8 +16549,9 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
return SDValue();
const SDNodeFlags Flags = N->getFlags();
- bool AllowFusionGlobally = (Options.AllowFPOpFusion == FPOpFusion::Fast ||
- Options.UnsafeFPMath || HasFMAD);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
+ bool AllowFusionGlobally =
+ (Options.AllowFPOpFusion == FPOpFusion::Fast || UnsafeFPMath || HasFMAD);
// If the subtraction is not contractable, do not combine.
if (!AllowFusionGlobally && !N->getFlags().hasAllowContract())
@@ -16714,8 +16706,8 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
}
}
- auto isReassociable = [&Options](SDNode *N) {
- return Options.UnsafeFPMath || N->getFlags().hasAllowReassociation();
+ auto isReassociable = [UnsafeFPMath](SDNode *N) {
+ return UnsafeFPMath || N->getFlags().hasAllowReassociation();
};
auto isContractableAndReassociableFMUL = [&isContractableFMUL,
@@ -16729,7 +16721,7 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
// More folding opportunities when target permits.
if (Aggressive && isReassociable(N)) {
- bool CanFuse = Options.UnsafeFPMath || N->getFlags().hasAllowContract();
+ bool CanFuse = UnsafeFPMath || N->getFlags().hasAllowContract();
// fold (fsub (fma x, y, (fmul u, v)), z)
// -> (fma x, y (fma u, v, (fneg z)))
if (CanFuse && isFusedOp(N0) &&
@@ -16878,6 +16870,16 @@ SDValue DAGCombiner::visitFMULForFMADistributiveCombine(SDNode *N) {
if (!hasNoInfs(Options, FAdd))
return SDValue();
+ // Returns true if floating point contraction is allowed on the FMUL-SDValue
+ // `N`
+ auto isContractableFMUL = [this](const TargetOptions &Options, SDValue N) {
+ assert(N.getOpcode() == ISD::FMUL);
+
+ return Options.AllowFPOpFusion == FPOpFusion::Fast ||
+ DAG.getMachineFunction().hasUnsafeFPMath() ||
+ N->getFlags().hasAllowContract();
+ };
+
// Floating-point multiply-add without intermediate rounding.
bool HasFMA =
isContractableFMUL(Options, SDValue(N, 0)) &&
@@ -16886,7 +16888,7 @@ SDValue DAGCombiner::visitFMULForFMADistributiveCombine(SDNode *N) {
// Floating-point multiply-add with intermediate rounding. This can result
// in a less precise result due to the changed rounding order.
- bool HasFMAD = Options.UnsafeFPMath &&
+ bool HasFMAD = DAG.getMachineFunction().hasUnsafeFPMath() &&
(LegalOperations && TLI.isFMADLegal(DAG, N));
// No valid opcode, do not combine.
@@ -16971,6 +16973,7 @@ SDValue DAGCombiner::visitFADD(SDNode *N) {
SDValue N1 = N->getOperand(1);
bool N0CFP = DAG.isConstantFPBuildVectorOrConstantFP(N0);
bool N1CFP = DAG.isConstantFPBuildVectorOrConstantFP(N1);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
EVT VT = N->getValueType(0);
SDLoc DL(N);
const TargetOptions &Options = DAG.getTarget().Options;
@@ -17052,7 +17055,7 @@ SDValue DAGCombiner::visitFADD(SDNode *N) {
// If 'unsafe math' or reassoc and nsz, fold lots of things.
// TODO: break out portions of the transformations below for which Unsafe is
// considered and which do not require both nsz and reassoc
- if (((Options.UnsafeFPMath && Options.NoSignedZerosFPMath) ||
+ if (((UnsafeFPMath && Options.NoSignedZerosFPMath) ||
(Flags.hasAllowReassociation() && Flags.hasNoSignedZeros())) &&
AllowNewConst) {
// fadd (fadd x, c1), c2 -> fadd x, c1 + c2
@@ -17190,6 +17193,7 @@ SDValue DAGCombiner::visitFSUB(SDNode *N) {
const TargetOptions &Options = DAG.getTarget().Options;
const SDNodeFlags Flags = N->getFlags();
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
if (SDValue R = DAG.simplifyFPBinop(N->getOpcode(), N0, N1, Flags))
return R;
@@ -17239,7 +17243,7 @@ SDValue DAGCombiner::visitFSUB(SDNode *N) {
}
}
- if (((Options.UnsafeFPMath && Options.NoSignedZerosFPMath) ||
+ if (((UnsafeFPMath && Options.NoSignedZerosFPMath) ||
(Flags.hasAllowReassociation() && Flags.hasNoSignedZeros())) &&
N1.getOpcode() == ISD::FADD) {
// X - (X + Y) -> -Y
@@ -17376,7 +17380,6 @@ SDValue DAGCombiner::visitFMUL(SDNode *N) {
ConstantFPSDNode *N1CFP = isConstOrConstSplatFP(N1, true);
EVT VT = N->getValueType(0);
SDLoc DL(N);
- const TargetOptions &Options = DAG.getTarget().Options;
const SDNodeFlags Flags = N->getFlags();
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
@@ -17400,7 +17403,8 @@ SDValue DAGCombiner::visitFMUL(SDNode *N) {
if (SDValue NewSel = foldBinOpIntoSelect(N))
return NewSel;
- if (Options.UnsafeFPMath || Flags.hasAllowReassociation()) {
+ if (DAG.getMachineFunction().hasUnsafeFPMath() ||
+ Flags.hasAllowReassociation()) {
// fmul (fmul X, C1), C2 -> fmul X, C1 * C2
if (DAG.isConstantFPBuildVectorOrConstantFP(N1) &&
N0.getOpcode() == ISD::FMUL) {
@@ -17526,10 +17530,10 @@ template <class MatchContextClass> SDValue DAGCombiner::visitFMA(SDNode *N) {
ConstantFPSDNode *N1CFP = dyn_cast<ConstantFPSDNode>(N1);
EVT VT = N->getValueType(0);
SDLoc DL(N);
- const TargetOptions &Options = DAG.getTarget().Options;
// FMA nodes have flags that propagate to the created nodes.
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
MatchContextClass matcher(DAG, TLI, N);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
// Constant fold FMA.
if (SDValue C =
@@ -17552,8 +17556,8 @@ template <class MatchContextClass> SDValue DAGCombiner::visitFMA(SDNode *N) {
return matcher.getNode(ISD::FMA, DL, VT, NegN0, NegN1, N2);
}
- // FIXME: use fast math flags instead of Options.UnsafeFPMath
- if (Options.UnsafeFPMath) {
+ // FIXME: use fast math flags instead of MachineFunction::hasUnsafeFPMath()
+ if (UnsafeFPMath) {
if (N0CFP && N0CFP->isZero())
return N2;
if (N1CFP && N1CFP->isZero())
@@ -17571,8 +17575,7 @@ template <class MatchContextClass> SDValue DAGCombiner::visitFMA(SDNode *N) {
!DAG.isConstantFPBuildVectorOrConstantFP(N1))
return matcher.getNode(ISD::FMA, DL, VT, N1, N0, N2);
- bool CanReassociate =
- Options.UnsafeFPMath || N->getFlags().hasAllowReassociation();
+ bool CanReassociate = UnsafeFPMath || N->getFlags().hasAllowReassociation();
if (CanReassociate) {
// (fma x, c1, (fmul x, c2)) -> (fmul x, c1+c2)
if (matcher.match(N2, ISD::FMUL) && N0 == N2.getOperand(0) &&
@@ -17667,7 +17670,7 @@ SDValue DAGCombiner::combineRepeatedFPDivisors(SDNode *N) {
// TODO: Limit this transform based on optsize/minsize - it always creates at
// least 1 extra instruction. But the perf win may be substantial enough
// that only minsize should restrict this.
- bool UnsafeMath = DAG.getTarget().Options.UnsafeFPMath;
+ bool UnsafeMath = DAG.getMachineFunction().hasUnsafeFPMath();
const SDNodeFlags Flags = N->getFlags();
if (LegalDAG || (!UnsafeMath && !Flags.hasAllowReciprocal()))
return SDValue();
@@ -17744,6 +17747,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
const TargetOptions &Options = DAG.getTarget().Options;
SDNodeFlags Flags = N->getFlags();
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
if (SDValue R = DAG.simplifyFPBinop(N->getOpcode(), N0, N1, Flags))
return R;
@@ -17774,7 +17778,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
// isn't too nasty (eg NaN, denormal, ...).
if (((st == APFloat::opOK && !Recip.isDenormal()) ||
(st == APFloat::opInexact &&
- (Options.UnsafeFPMath || Flags.hasAllowReciprocal()))) &&
+ (UnsafeFPMath || Flags.hasAllowReciprocal()))) &&
(!LegalOperations ||
// FIXME: custom lowering of ConstantFP might fail (see e.g. ARM
// backend)... we should handle this gracefully after Legalize.
@@ -17785,7 +17789,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
DAG.getConstantFP(Recip, DL, VT));
}
- if (Options.UnsafeFPMath || Flags.hasAllowReciprocal()) {
+ if (UnsafeFPMath || Flags.hasAllowReciprocal()) {
// If this FDIV is part of a reciprocal square root, it may be folded
// into a target-specific square root estimate instruction.
if (N1.getOpcode() == ISD::FSQRT) {
@@ -17860,7 +17864,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
// Fold X/Sqrt(X) -> Sqrt(X)
if ((Options.NoSignedZerosFPMath || Flags.hasNoSignedZeros()) &&
- (Options.UnsafeFPMath || Flags.hasAllowReassociation()))
+ (UnsafeFPMath || Flags.hasAllowReassociation()))
if (N1.getOpcode() == ISD::FSQRT && N0 == N1.getOperand(0))
return N1;
@@ -18356,7 +18360,7 @@ SDValue DAGCombiner::visitFP_ROUND(SDNode *N) {
// single-step fp_round we want to fold to.
// In other words, double rounding isn't the same as rounding.
// Also, this is a value preserving truncation iff both fp_round's are.
- if (DAG.getTarget().Options.UnsafeFPMath || N0IsTrunc)
+ if (DAG.getMachineFunction().hasUnsafeFPMath() || N0IsTrunc)
return DAG.getNode(
ISD::FP_ROUND, DL, VT, N0.getOperand(0),
DAG.getIntPtrConstant(NIsTrunc && N0IsTrunc, DL, /*isTarget=*/true));
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
index c6475f0219903..1470410d0e1ec 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
@@ -3643,7 +3643,7 @@ bool SelectionDAGLegalize::ExpandNode(SDNode *Node) {
break;
case ISD::FP_TO_FP16:
LLVM_DEBUG(dbgs() << "Legalizing FP_TO_FP16\n");
- if (!TLI.useSoftFloat() && TM.Options.UnsafeFPMath) {
+ if (!TLI.useSoftFloat() && DAG.getMachineFunction().hasUnsafeFPMath()) {
SDValue Op = Node->getOperand(0);
MVT SVT = Op.getSimpleValueType();
if ((SVT == MVT::f64 || SVT == MVT::f80) &&
diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
index bd9994bcb669c..e65078decebfe 100644
--- a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
@@ -11327,7 +11327,7 @@ SDValue AArch64TargetLowering::LowerSELECT_CC(ISD::CondCode CC, SDValue LHS,
AArch64CC::CondCode CC1, CC2;
changeFPCCToAArch64CC(CC, CC1, CC2);
- if (DAG.getTarget().Options.UnsafeFPMath) {
+ if (DAG.getMachineFunction().hasUnsafeFPMath()) {
// Transform "a == 0.0 ? 0.0 : x" to "a == 0.0 ? a : x" and
// "a != 0.0 ? x : 0.0" to "a != 0.0 ? x : a" to avoid materializing 0.0.
ConstantFPSDNode *RHSVal = dyn_cast<ConstantFPSDNode>(RHS);
diff --git a/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp b/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
index 17dd8a073eff0..a993dbb493903 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
+++ b/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
@@ -6351,7 +6351,7 @@ static bool isCombineInstrCandidateFP(const MachineInstr &Inst) {
TargetOptions Options = Inst.getParent()->getParent()->getTarget().Options;
// We can fuse FADD/FSUB with FMUL, if fusion is either allowed globally by
// the target options or if FADD/FSUB has the contract fast-math flag.
- return Options.UnsafeFPMath ||
+ return Inst.getParent()->getParent()->hasUnsafeFPMath() ||
Options.AllowFPOpFusion == FPOpFusion::Fast ||
Inst.getFlag(MachineInstr::FmContract);
return true;
@@ -6457,7 +6457,7 @@ bool AArch64InstrInfo::isAssociativeAndCommutative(const MachineInstr &Inst,
case AArch64::FMUL_ZZZ_H:
case AArch64::FMUL_ZZZ_S:
case AArch64::FMUL_ZZZ_D:
- return Inst.getParent()->getParent()->getTarget().Options.UnsafeFPMath ||
+ return Inst.getParent()->getParent()->hasUnsafeFPMath() ||
(Inst.getFlag(MachineInstr::MIFlag::FmReassoc) &&
Inst.getFlag(MachineInstr::MIFlag::FmNsz));
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp b/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
index cca9fa72d0ca5..9bb25bd02141a 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
@@ -2623,7 +2623,7 @@ bool AMDGPUTargetLowering::allowApproxFunc(const SelectionDAG &DAG,
if (Flags.hasApproximateFuncs())
return true;
auto &Options = DAG.getTarget().Options;
- return Options.UnsafeFPMath || Options.ApproxFuncFPMath;
+ return DAG.getMachineFunction().hasUnsafeFPMath() || Options.ApproxFuncFPMath;
}
bool AMDGPUTargetLowering::needsDenormHandlingF32(const SelectionDAG &DAG,
@@ -2746,7 +2746,7 @@ SDValue AMDGPUTargetLowering::LowerFLOGCommon(SDValue Op,
const auto &Options = getTargetMachine().Options;
if (VT == MVT::f16 || Flags.hasApproximateFuncs() ||
- Options.ApproxFuncFPMath || Options.UnsafeFPMath) {
+ Options.ApproxFuncFPMath || DAG.getMachineFunction().hasUnsafeFPMath()) {
if (VT == MVT::f16 && !Subtarget->has16BitInsts()) {
// Log and multiply in f32 is good enough for f16.
@@ -3573,7 +3573,7 @@ SDValue AMDGPUTargetLowering::LowerFP_TO_FP16(SDValue Op, SelectionDAG &DAG) con
if (N0.getValueType() == MVT::f32)
return DAG.getNode(AMDGPUISD::FP_TO_FP16, DL, Op.getValueType(), N0);
- if (getTargetMachine().Options.UnsafeFPMath) {
+ if (DAG.getMachineFunction().hasUnsafeFPMath()) {
// There is a generic expand for FP_TO_FP16 with unsafe fast math.
return SDValue();
}
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td b/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td
index 6a5065cd4a0e8..10db60a4e890e 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td
+++ b/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td
@@ -94,7 +94,7 @@ def FP64Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().F
def NoFP16Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().FP64FP16Denormals == DenormalMode::getPreserveSign()">;
def NoFP32Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().FP32Denormals == DenormalMode::getPreserveSign()">;
def NoFP64Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().FP64FP16Denormals == DenormalMode::getPreserveSign()">;
-def UnsafeFPMath : Predicate<"TM.Options.UnsafeFPMath">;
+def UnsafeFPMath : Predicate<"MF->hasUnsafeFPMath()">;
}
def FMA : Predicate<"Subtarget->hasFMA()">;
diff --git a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
index e9e47eaadd557..f9eea24d0b9a1 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
@@ -3270,7 +3270,7 @@ static bool allowApproxFunc(const MachineFunction &MF, unsigned Flags) {
if (Flags & MachineInstr::FmAfn)
return true;
const auto &Options = MF.getTarget().Options;
- return Options.UnsafeFPMath || Options.ApproxFuncFPMath;
+ return MF.hasUnsafeFPMath() || Options.ApproxFuncFPMath;
}
static bool needsDenormHandlingF32(const MachineFunction &MF, Register Src,
@@ -3376,7 +3376,7 @@ bool AMDGPULegalizerInfo::legalizeFlogCommon(MachineInstr &MI,
static_cast<const AMDGPUTargetMachine &>(MF.getTarg...
[truncated]
|
|
@llvm/pr-subscribers-backend-arm Author: None (paperchalice) ChangesAddress #27735 incrementally. New code should use Patch is 31.58 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/127488.diff 21 Files Affected:
diff --git a/llvm/include/llvm/CodeGen/MachineFunction.h b/llvm/include/llvm/CodeGen/MachineFunction.h
index c3eb27b946287..86f1fa9fed952 100644
--- a/llvm/include/llvm/CodeGen/MachineFunction.h
+++ b/llvm/include/llvm/CodeGen/MachineFunction.h
@@ -820,6 +820,10 @@ class LLVM_ABI MachineFunction {
/// True if this function needs frame moves for debug or exceptions.
bool needsFrameMoves() const;
+ /// True if function attribute unsafe-fp-math is true or
+ /// --enable-unsafe-fp-math is passed.
+ bool hasUnsafeFPMath() const;
+
/// Get the function properties
const MachineFunctionProperties &getProperties() const { return Properties; }
MachineFunctionProperties &getProperties() { return Properties; }
diff --git a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
index b193d8bb0aa18..75b1d3715ec6c 100644
--- a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
@@ -5774,7 +5774,7 @@ bool CombinerHelper::canCombineFMadOrFMA(MachineInstr &MI,
LLT DstType = MRI.getType(MI.getOperand(0).getReg());
if (CanReassociate &&
- !(Options.UnsafeFPMath || MI.getFlag(MachineInstr::MIFlag::FmReassoc)))
+ !(MF->hasUnsafeFPMath() || MI.getFlag(MachineInstr::MIFlag::FmReassoc)))
return false;
// Floating-point multiply-add with intermediate rounding.
@@ -5787,7 +5787,7 @@ bool CombinerHelper::canCombineFMadOrFMA(MachineInstr &MI,
return false;
AllowFusionGlobally = Options.AllowFPOpFusion == FPOpFusion::Fast ||
- Options.UnsafeFPMath || HasFMAD;
+ MF->hasUnsafeFPMath() || HasFMAD;
// If the addition is not contractable, do not combine.
if (!AllowFusionGlobally && !MI.getFlag(MachineInstr::MIFlag::FmContract))
return false;
diff --git a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
index d0a62340a5f32..4490bb4f83b49 100644
--- a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
@@ -7879,7 +7879,7 @@ LegalizerHelper::lowerFPTRUNC_F64_TO_F16(MachineInstr &MI) {
if (MRI.getType(Src).isVector()) // TODO: Handle vectors directly.
return UnableToLegalize;
- if (MIRBuilder.getMF().getTarget().Options.UnsafeFPMath) {
+ if (MIRBuilder.getMF().hasUnsafeFPMath()) {
unsigned Flags = MI.getFlags();
auto Src32 = MIRBuilder.buildFPTrunc(S32, Src, Flags);
MIRBuilder.buildFPTrunc(Dst, Src32, Flags);
diff --git a/llvm/lib/CodeGen/MachineFunction.cpp b/llvm/lib/CodeGen/MachineFunction.cpp
index 7d504ef5a0482..d29610bfeb1b4 100644
--- a/llvm/lib/CodeGen/MachineFunction.cpp
+++ b/llvm/lib/CodeGen/MachineFunction.cpp
@@ -699,6 +699,11 @@ bool MachineFunction::needsFrameMoves() const {
!F.getParent()->debug_compile_units().empty();
}
+/// True if function attribute unsafe-fp-math is true.
+bool MachineFunction::hasUnsafeFPMath() const {
+ return F.getFnAttribute("unsafe-fp-math").getValueAsBool();
+}
+
namespace llvm {
template<>
diff --git a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
index f4caaf426de6a..2dc827a8040f5 100644
--- a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
@@ -16280,15 +16280,6 @@ ConstantFoldBITCASTofBUILD_VECTOR(SDNode *BV, EVT DstEltVT) {
return DAG.getBuildVector(VT, DL, Ops);
}
-// Returns true if floating point contraction is allowed on the FMUL-SDValue
-// `N`
-static bool isContractableFMUL(const TargetOptions &Options, SDValue N) {
- assert(N.getOpcode() == ISD::FMUL);
-
- return Options.AllowFPOpFusion == FPOpFusion::Fast || Options.UnsafeFPMath ||
- N->getFlags().hasAllowContract();
-}
-
// Returns true if `N` can assume no infinities involved in its computation.
static bool hasNoInfs(const TargetOptions &Options, SDValue N) {
return Options.NoInfsFPMath || N->getFlags().hasNoInfs();
@@ -16320,8 +16311,9 @@ SDValue DAGCombiner::visitFADDForFMACombine(SDNode *N) {
if (!HasFMAD && !HasFMA)
return SDValue();
- bool AllowFusionGlobally = (Options.AllowFPOpFusion == FPOpFusion::Fast ||
- Options.UnsafeFPMath || HasFMAD);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
+ bool AllowFusionGlobally =
+ (Options.AllowFPOpFusion == FPOpFusion::Fast || UnsafeFPMath || HasFMAD);
// If the addition is not contractable, do not combine.
if (!AllowFusionGlobally && !N->getFlags().hasAllowContract())
return SDValue();
@@ -16379,8 +16371,7 @@ SDValue DAGCombiner::visitFADDForFMACombine(SDNode *N) {
// fadd (G, (fma A, B, (fma (C, D, (fmul (E, F)))))) -->
// fma A, B, (fma C, D, fma (E, F, G)).
// This requires reassociation because it changes the order of operations.
- bool CanReassociate =
- Options.UnsafeFPMath || N->getFlags().hasAllowReassociation();
+ bool CanReassociate = UnsafeFPMath || N->getFlags().hasAllowReassociation();
if (CanReassociate) {
SDValue FMA, E;
if (isFusedOp(N0) && N0.hasOneUse()) {
@@ -16558,8 +16549,9 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
return SDValue();
const SDNodeFlags Flags = N->getFlags();
- bool AllowFusionGlobally = (Options.AllowFPOpFusion == FPOpFusion::Fast ||
- Options.UnsafeFPMath || HasFMAD);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
+ bool AllowFusionGlobally =
+ (Options.AllowFPOpFusion == FPOpFusion::Fast || UnsafeFPMath || HasFMAD);
// If the subtraction is not contractable, do not combine.
if (!AllowFusionGlobally && !N->getFlags().hasAllowContract())
@@ -16714,8 +16706,8 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
}
}
- auto isReassociable = [&Options](SDNode *N) {
- return Options.UnsafeFPMath || N->getFlags().hasAllowReassociation();
+ auto isReassociable = [UnsafeFPMath](SDNode *N) {
+ return UnsafeFPMath || N->getFlags().hasAllowReassociation();
};
auto isContractableAndReassociableFMUL = [&isContractableFMUL,
@@ -16729,7 +16721,7 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
// More folding opportunities when target permits.
if (Aggressive && isReassociable(N)) {
- bool CanFuse = Options.UnsafeFPMath || N->getFlags().hasAllowContract();
+ bool CanFuse = UnsafeFPMath || N->getFlags().hasAllowContract();
// fold (fsub (fma x, y, (fmul u, v)), z)
// -> (fma x, y (fma u, v, (fneg z)))
if (CanFuse && isFusedOp(N0) &&
@@ -16878,6 +16870,16 @@ SDValue DAGCombiner::visitFMULForFMADistributiveCombine(SDNode *N) {
if (!hasNoInfs(Options, FAdd))
return SDValue();
+ // Returns true if floating point contraction is allowed on the FMUL-SDValue
+ // `N`
+ auto isContractableFMUL = [this](const TargetOptions &Options, SDValue N) {
+ assert(N.getOpcode() == ISD::FMUL);
+
+ return Options.AllowFPOpFusion == FPOpFusion::Fast ||
+ DAG.getMachineFunction().hasUnsafeFPMath() ||
+ N->getFlags().hasAllowContract();
+ };
+
// Floating-point multiply-add without intermediate rounding.
bool HasFMA =
isContractableFMUL(Options, SDValue(N, 0)) &&
@@ -16886,7 +16888,7 @@ SDValue DAGCombiner::visitFMULForFMADistributiveCombine(SDNode *N) {
// Floating-point multiply-add with intermediate rounding. This can result
// in a less precise result due to the changed rounding order.
- bool HasFMAD = Options.UnsafeFPMath &&
+ bool HasFMAD = DAG.getMachineFunction().hasUnsafeFPMath() &&
(LegalOperations && TLI.isFMADLegal(DAG, N));
// No valid opcode, do not combine.
@@ -16971,6 +16973,7 @@ SDValue DAGCombiner::visitFADD(SDNode *N) {
SDValue N1 = N->getOperand(1);
bool N0CFP = DAG.isConstantFPBuildVectorOrConstantFP(N0);
bool N1CFP = DAG.isConstantFPBuildVectorOrConstantFP(N1);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
EVT VT = N->getValueType(0);
SDLoc DL(N);
const TargetOptions &Options = DAG.getTarget().Options;
@@ -17052,7 +17055,7 @@ SDValue DAGCombiner::visitFADD(SDNode *N) {
// If 'unsafe math' or reassoc and nsz, fold lots of things.
// TODO: break out portions of the transformations below for which Unsafe is
// considered and which do not require both nsz and reassoc
- if (((Options.UnsafeFPMath && Options.NoSignedZerosFPMath) ||
+ if (((UnsafeFPMath && Options.NoSignedZerosFPMath) ||
(Flags.hasAllowReassociation() && Flags.hasNoSignedZeros())) &&
AllowNewConst) {
// fadd (fadd x, c1), c2 -> fadd x, c1 + c2
@@ -17190,6 +17193,7 @@ SDValue DAGCombiner::visitFSUB(SDNode *N) {
const TargetOptions &Options = DAG.getTarget().Options;
const SDNodeFlags Flags = N->getFlags();
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
if (SDValue R = DAG.simplifyFPBinop(N->getOpcode(), N0, N1, Flags))
return R;
@@ -17239,7 +17243,7 @@ SDValue DAGCombiner::visitFSUB(SDNode *N) {
}
}
- if (((Options.UnsafeFPMath && Options.NoSignedZerosFPMath) ||
+ if (((UnsafeFPMath && Options.NoSignedZerosFPMath) ||
(Flags.hasAllowReassociation() && Flags.hasNoSignedZeros())) &&
N1.getOpcode() == ISD::FADD) {
// X - (X + Y) -> -Y
@@ -17376,7 +17380,6 @@ SDValue DAGCombiner::visitFMUL(SDNode *N) {
ConstantFPSDNode *N1CFP = isConstOrConstSplatFP(N1, true);
EVT VT = N->getValueType(0);
SDLoc DL(N);
- const TargetOptions &Options = DAG.getTarget().Options;
const SDNodeFlags Flags = N->getFlags();
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
@@ -17400,7 +17403,8 @@ SDValue DAGCombiner::visitFMUL(SDNode *N) {
if (SDValue NewSel = foldBinOpIntoSelect(N))
return NewSel;
- if (Options.UnsafeFPMath || Flags.hasAllowReassociation()) {
+ if (DAG.getMachineFunction().hasUnsafeFPMath() ||
+ Flags.hasAllowReassociation()) {
// fmul (fmul X, C1), C2 -> fmul X, C1 * C2
if (DAG.isConstantFPBuildVectorOrConstantFP(N1) &&
N0.getOpcode() == ISD::FMUL) {
@@ -17526,10 +17530,10 @@ template <class MatchContextClass> SDValue DAGCombiner::visitFMA(SDNode *N) {
ConstantFPSDNode *N1CFP = dyn_cast<ConstantFPSDNode>(N1);
EVT VT = N->getValueType(0);
SDLoc DL(N);
- const TargetOptions &Options = DAG.getTarget().Options;
// FMA nodes have flags that propagate to the created nodes.
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
MatchContextClass matcher(DAG, TLI, N);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
// Constant fold FMA.
if (SDValue C =
@@ -17552,8 +17556,8 @@ template <class MatchContextClass> SDValue DAGCombiner::visitFMA(SDNode *N) {
return matcher.getNode(ISD::FMA, DL, VT, NegN0, NegN1, N2);
}
- // FIXME: use fast math flags instead of Options.UnsafeFPMath
- if (Options.UnsafeFPMath) {
+ // FIXME: use fast math flags instead of MachineFunction::hasUnsafeFPMath()
+ if (UnsafeFPMath) {
if (N0CFP && N0CFP->isZero())
return N2;
if (N1CFP && N1CFP->isZero())
@@ -17571,8 +17575,7 @@ template <class MatchContextClass> SDValue DAGCombiner::visitFMA(SDNode *N) {
!DAG.isConstantFPBuildVectorOrConstantFP(N1))
return matcher.getNode(ISD::FMA, DL, VT, N1, N0, N2);
- bool CanReassociate =
- Options.UnsafeFPMath || N->getFlags().hasAllowReassociation();
+ bool CanReassociate = UnsafeFPMath || N->getFlags().hasAllowReassociation();
if (CanReassociate) {
// (fma x, c1, (fmul x, c2)) -> (fmul x, c1+c2)
if (matcher.match(N2, ISD::FMUL) && N0 == N2.getOperand(0) &&
@@ -17667,7 +17670,7 @@ SDValue DAGCombiner::combineRepeatedFPDivisors(SDNode *N) {
// TODO: Limit this transform based on optsize/minsize - it always creates at
// least 1 extra instruction. But the perf win may be substantial enough
// that only minsize should restrict this.
- bool UnsafeMath = DAG.getTarget().Options.UnsafeFPMath;
+ bool UnsafeMath = DAG.getMachineFunction().hasUnsafeFPMath();
const SDNodeFlags Flags = N->getFlags();
if (LegalDAG || (!UnsafeMath && !Flags.hasAllowReciprocal()))
return SDValue();
@@ -17744,6 +17747,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
const TargetOptions &Options = DAG.getTarget().Options;
SDNodeFlags Flags = N->getFlags();
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
if (SDValue R = DAG.simplifyFPBinop(N->getOpcode(), N0, N1, Flags))
return R;
@@ -17774,7 +17778,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
// isn't too nasty (eg NaN, denormal, ...).
if (((st == APFloat::opOK && !Recip.isDenormal()) ||
(st == APFloat::opInexact &&
- (Options.UnsafeFPMath || Flags.hasAllowReciprocal()))) &&
+ (UnsafeFPMath || Flags.hasAllowReciprocal()))) &&
(!LegalOperations ||
// FIXME: custom lowering of ConstantFP might fail (see e.g. ARM
// backend)... we should handle this gracefully after Legalize.
@@ -17785,7 +17789,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
DAG.getConstantFP(Recip, DL, VT));
}
- if (Options.UnsafeFPMath || Flags.hasAllowReciprocal()) {
+ if (UnsafeFPMath || Flags.hasAllowReciprocal()) {
// If this FDIV is part of a reciprocal square root, it may be folded
// into a target-specific square root estimate instruction.
if (N1.getOpcode() == ISD::FSQRT) {
@@ -17860,7 +17864,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
// Fold X/Sqrt(X) -> Sqrt(X)
if ((Options.NoSignedZerosFPMath || Flags.hasNoSignedZeros()) &&
- (Options.UnsafeFPMath || Flags.hasAllowReassociation()))
+ (UnsafeFPMath || Flags.hasAllowReassociation()))
if (N1.getOpcode() == ISD::FSQRT && N0 == N1.getOperand(0))
return N1;
@@ -18356,7 +18360,7 @@ SDValue DAGCombiner::visitFP_ROUND(SDNode *N) {
// single-step fp_round we want to fold to.
// In other words, double rounding isn't the same as rounding.
// Also, this is a value preserving truncation iff both fp_round's are.
- if (DAG.getTarget().Options.UnsafeFPMath || N0IsTrunc)
+ if (DAG.getMachineFunction().hasUnsafeFPMath() || N0IsTrunc)
return DAG.getNode(
ISD::FP_ROUND, DL, VT, N0.getOperand(0),
DAG.getIntPtrConstant(NIsTrunc && N0IsTrunc, DL, /*isTarget=*/true));
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
index c6475f0219903..1470410d0e1ec 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
@@ -3643,7 +3643,7 @@ bool SelectionDAGLegalize::ExpandNode(SDNode *Node) {
break;
case ISD::FP_TO_FP16:
LLVM_DEBUG(dbgs() << "Legalizing FP_TO_FP16\n");
- if (!TLI.useSoftFloat() && TM.Options.UnsafeFPMath) {
+ if (!TLI.useSoftFloat() && DAG.getMachineFunction().hasUnsafeFPMath()) {
SDValue Op = Node->getOperand(0);
MVT SVT = Op.getSimpleValueType();
if ((SVT == MVT::f64 || SVT == MVT::f80) &&
diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
index bd9994bcb669c..e65078decebfe 100644
--- a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
@@ -11327,7 +11327,7 @@ SDValue AArch64TargetLowering::LowerSELECT_CC(ISD::CondCode CC, SDValue LHS,
AArch64CC::CondCode CC1, CC2;
changeFPCCToAArch64CC(CC, CC1, CC2);
- if (DAG.getTarget().Options.UnsafeFPMath) {
+ if (DAG.getMachineFunction().hasUnsafeFPMath()) {
// Transform "a == 0.0 ? 0.0 : x" to "a == 0.0 ? a : x" and
// "a != 0.0 ? x : 0.0" to "a != 0.0 ? x : a" to avoid materializing 0.0.
ConstantFPSDNode *RHSVal = dyn_cast<ConstantFPSDNode>(RHS);
diff --git a/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp b/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
index 17dd8a073eff0..a993dbb493903 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
+++ b/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
@@ -6351,7 +6351,7 @@ static bool isCombineInstrCandidateFP(const MachineInstr &Inst) {
TargetOptions Options = Inst.getParent()->getParent()->getTarget().Options;
// We can fuse FADD/FSUB with FMUL, if fusion is either allowed globally by
// the target options or if FADD/FSUB has the contract fast-math flag.
- return Options.UnsafeFPMath ||
+ return Inst.getParent()->getParent()->hasUnsafeFPMath() ||
Options.AllowFPOpFusion == FPOpFusion::Fast ||
Inst.getFlag(MachineInstr::FmContract);
return true;
@@ -6457,7 +6457,7 @@ bool AArch64InstrInfo::isAssociativeAndCommutative(const MachineInstr &Inst,
case AArch64::FMUL_ZZZ_H:
case AArch64::FMUL_ZZZ_S:
case AArch64::FMUL_ZZZ_D:
- return Inst.getParent()->getParent()->getTarget().Options.UnsafeFPMath ||
+ return Inst.getParent()->getParent()->hasUnsafeFPMath() ||
(Inst.getFlag(MachineInstr::MIFlag::FmReassoc) &&
Inst.getFlag(MachineInstr::MIFlag::FmNsz));
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp b/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
index cca9fa72d0ca5..9bb25bd02141a 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
@@ -2623,7 +2623,7 @@ bool AMDGPUTargetLowering::allowApproxFunc(const SelectionDAG &DAG,
if (Flags.hasApproximateFuncs())
return true;
auto &Options = DAG.getTarget().Options;
- return Options.UnsafeFPMath || Options.ApproxFuncFPMath;
+ return DAG.getMachineFunction().hasUnsafeFPMath() || Options.ApproxFuncFPMath;
}
bool AMDGPUTargetLowering::needsDenormHandlingF32(const SelectionDAG &DAG,
@@ -2746,7 +2746,7 @@ SDValue AMDGPUTargetLowering::LowerFLOGCommon(SDValue Op,
const auto &Options = getTargetMachine().Options;
if (VT == MVT::f16 || Flags.hasApproximateFuncs() ||
- Options.ApproxFuncFPMath || Options.UnsafeFPMath) {
+ Options.ApproxFuncFPMath || DAG.getMachineFunction().hasUnsafeFPMath()) {
if (VT == MVT::f16 && !Subtarget->has16BitInsts()) {
// Log and multiply in f32 is good enough for f16.
@@ -3573,7 +3573,7 @@ SDValue AMDGPUTargetLowering::LowerFP_TO_FP16(SDValue Op, SelectionDAG &DAG) con
if (N0.getValueType() == MVT::f32)
return DAG.getNode(AMDGPUISD::FP_TO_FP16, DL, Op.getValueType(), N0);
- if (getTargetMachine().Options.UnsafeFPMath) {
+ if (DAG.getMachineFunction().hasUnsafeFPMath()) {
// There is a generic expand for FP_TO_FP16 with unsafe fast math.
return SDValue();
}
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td b/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td
index 6a5065cd4a0e8..10db60a4e890e 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td
+++ b/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td
@@ -94,7 +94,7 @@ def FP64Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().F
def NoFP16Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().FP64FP16Denormals == DenormalMode::getPreserveSign()">;
def NoFP32Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().FP32Denormals == DenormalMode::getPreserveSign()">;
def NoFP64Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().FP64FP16Denormals == DenormalMode::getPreserveSign()">;
-def UnsafeFPMath : Predicate<"TM.Options.UnsafeFPMath">;
+def UnsafeFPMath : Predicate<"MF->hasUnsafeFPMath()">;
}
def FMA : Predicate<"Subtarget->hasFMA()">;
diff --git a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
index e9e47eaadd557..f9eea24d0b9a1 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
@@ -3270,7 +3270,7 @@ static bool allowApproxFunc(const MachineFunction &MF, unsigned Flags) {
if (Flags & MachineInstr::FmAfn)
return true;
const auto &Options = MF.getTarget().Options;
- return Options.UnsafeFPMath || Options.ApproxFuncFPMath;
+ return MF.hasUnsafeFPMath() || Options.ApproxFuncFPMath;
}
static bool needsDenormHandlingF32(const MachineFunction &MF, Register Src,
@@ -3376,7 +3376,7 @@ bool AMDGPULegalizerInfo::legalizeFlogCommon(MachineInstr &MI,
static_cast<const AMDGPUTargetMachine &>(MF.getTarg...
[truncated]
|
|
@llvm/pr-subscribers-backend-x86 Author: None (paperchalice) ChangesAddress #27735 incrementally. New code should use Patch is 31.58 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/127488.diff 21 Files Affected:
diff --git a/llvm/include/llvm/CodeGen/MachineFunction.h b/llvm/include/llvm/CodeGen/MachineFunction.h
index c3eb27b946287..86f1fa9fed952 100644
--- a/llvm/include/llvm/CodeGen/MachineFunction.h
+++ b/llvm/include/llvm/CodeGen/MachineFunction.h
@@ -820,6 +820,10 @@ class LLVM_ABI MachineFunction {
/// True if this function needs frame moves for debug or exceptions.
bool needsFrameMoves() const;
+ /// True if function attribute unsafe-fp-math is true or
+ /// --enable-unsafe-fp-math is passed.
+ bool hasUnsafeFPMath() const;
+
/// Get the function properties
const MachineFunctionProperties &getProperties() const { return Properties; }
MachineFunctionProperties &getProperties() { return Properties; }
diff --git a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
index b193d8bb0aa18..75b1d3715ec6c 100644
--- a/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp
@@ -5774,7 +5774,7 @@ bool CombinerHelper::canCombineFMadOrFMA(MachineInstr &MI,
LLT DstType = MRI.getType(MI.getOperand(0).getReg());
if (CanReassociate &&
- !(Options.UnsafeFPMath || MI.getFlag(MachineInstr::MIFlag::FmReassoc)))
+ !(MF->hasUnsafeFPMath() || MI.getFlag(MachineInstr::MIFlag::FmReassoc)))
return false;
// Floating-point multiply-add with intermediate rounding.
@@ -5787,7 +5787,7 @@ bool CombinerHelper::canCombineFMadOrFMA(MachineInstr &MI,
return false;
AllowFusionGlobally = Options.AllowFPOpFusion == FPOpFusion::Fast ||
- Options.UnsafeFPMath || HasFMAD;
+ MF->hasUnsafeFPMath() || HasFMAD;
// If the addition is not contractable, do not combine.
if (!AllowFusionGlobally && !MI.getFlag(MachineInstr::MIFlag::FmContract))
return false;
diff --git a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
index d0a62340a5f32..4490bb4f83b49 100644
--- a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
@@ -7879,7 +7879,7 @@ LegalizerHelper::lowerFPTRUNC_F64_TO_F16(MachineInstr &MI) {
if (MRI.getType(Src).isVector()) // TODO: Handle vectors directly.
return UnableToLegalize;
- if (MIRBuilder.getMF().getTarget().Options.UnsafeFPMath) {
+ if (MIRBuilder.getMF().hasUnsafeFPMath()) {
unsigned Flags = MI.getFlags();
auto Src32 = MIRBuilder.buildFPTrunc(S32, Src, Flags);
MIRBuilder.buildFPTrunc(Dst, Src32, Flags);
diff --git a/llvm/lib/CodeGen/MachineFunction.cpp b/llvm/lib/CodeGen/MachineFunction.cpp
index 7d504ef5a0482..d29610bfeb1b4 100644
--- a/llvm/lib/CodeGen/MachineFunction.cpp
+++ b/llvm/lib/CodeGen/MachineFunction.cpp
@@ -699,6 +699,11 @@ bool MachineFunction::needsFrameMoves() const {
!F.getParent()->debug_compile_units().empty();
}
+/// True if function attribute unsafe-fp-math is true.
+bool MachineFunction::hasUnsafeFPMath() const {
+ return F.getFnAttribute("unsafe-fp-math").getValueAsBool();
+}
+
namespace llvm {
template<>
diff --git a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
index f4caaf426de6a..2dc827a8040f5 100644
--- a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
@@ -16280,15 +16280,6 @@ ConstantFoldBITCASTofBUILD_VECTOR(SDNode *BV, EVT DstEltVT) {
return DAG.getBuildVector(VT, DL, Ops);
}
-// Returns true if floating point contraction is allowed on the FMUL-SDValue
-// `N`
-static bool isContractableFMUL(const TargetOptions &Options, SDValue N) {
- assert(N.getOpcode() == ISD::FMUL);
-
- return Options.AllowFPOpFusion == FPOpFusion::Fast || Options.UnsafeFPMath ||
- N->getFlags().hasAllowContract();
-}
-
// Returns true if `N` can assume no infinities involved in its computation.
static bool hasNoInfs(const TargetOptions &Options, SDValue N) {
return Options.NoInfsFPMath || N->getFlags().hasNoInfs();
@@ -16320,8 +16311,9 @@ SDValue DAGCombiner::visitFADDForFMACombine(SDNode *N) {
if (!HasFMAD && !HasFMA)
return SDValue();
- bool AllowFusionGlobally = (Options.AllowFPOpFusion == FPOpFusion::Fast ||
- Options.UnsafeFPMath || HasFMAD);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
+ bool AllowFusionGlobally =
+ (Options.AllowFPOpFusion == FPOpFusion::Fast || UnsafeFPMath || HasFMAD);
// If the addition is not contractable, do not combine.
if (!AllowFusionGlobally && !N->getFlags().hasAllowContract())
return SDValue();
@@ -16379,8 +16371,7 @@ SDValue DAGCombiner::visitFADDForFMACombine(SDNode *N) {
// fadd (G, (fma A, B, (fma (C, D, (fmul (E, F)))))) -->
// fma A, B, (fma C, D, fma (E, F, G)).
// This requires reassociation because it changes the order of operations.
- bool CanReassociate =
- Options.UnsafeFPMath || N->getFlags().hasAllowReassociation();
+ bool CanReassociate = UnsafeFPMath || N->getFlags().hasAllowReassociation();
if (CanReassociate) {
SDValue FMA, E;
if (isFusedOp(N0) && N0.hasOneUse()) {
@@ -16558,8 +16549,9 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
return SDValue();
const SDNodeFlags Flags = N->getFlags();
- bool AllowFusionGlobally = (Options.AllowFPOpFusion == FPOpFusion::Fast ||
- Options.UnsafeFPMath || HasFMAD);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
+ bool AllowFusionGlobally =
+ (Options.AllowFPOpFusion == FPOpFusion::Fast || UnsafeFPMath || HasFMAD);
// If the subtraction is not contractable, do not combine.
if (!AllowFusionGlobally && !N->getFlags().hasAllowContract())
@@ -16714,8 +16706,8 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
}
}
- auto isReassociable = [&Options](SDNode *N) {
- return Options.UnsafeFPMath || N->getFlags().hasAllowReassociation();
+ auto isReassociable = [UnsafeFPMath](SDNode *N) {
+ return UnsafeFPMath || N->getFlags().hasAllowReassociation();
};
auto isContractableAndReassociableFMUL = [&isContractableFMUL,
@@ -16729,7 +16721,7 @@ SDValue DAGCombiner::visitFSUBForFMACombine(SDNode *N) {
// More folding opportunities when target permits.
if (Aggressive && isReassociable(N)) {
- bool CanFuse = Options.UnsafeFPMath || N->getFlags().hasAllowContract();
+ bool CanFuse = UnsafeFPMath || N->getFlags().hasAllowContract();
// fold (fsub (fma x, y, (fmul u, v)), z)
// -> (fma x, y (fma u, v, (fneg z)))
if (CanFuse && isFusedOp(N0) &&
@@ -16878,6 +16870,16 @@ SDValue DAGCombiner::visitFMULForFMADistributiveCombine(SDNode *N) {
if (!hasNoInfs(Options, FAdd))
return SDValue();
+ // Returns true if floating point contraction is allowed on the FMUL-SDValue
+ // `N`
+ auto isContractableFMUL = [this](const TargetOptions &Options, SDValue N) {
+ assert(N.getOpcode() == ISD::FMUL);
+
+ return Options.AllowFPOpFusion == FPOpFusion::Fast ||
+ DAG.getMachineFunction().hasUnsafeFPMath() ||
+ N->getFlags().hasAllowContract();
+ };
+
// Floating-point multiply-add without intermediate rounding.
bool HasFMA =
isContractableFMUL(Options, SDValue(N, 0)) &&
@@ -16886,7 +16888,7 @@ SDValue DAGCombiner::visitFMULForFMADistributiveCombine(SDNode *N) {
// Floating-point multiply-add with intermediate rounding. This can result
// in a less precise result due to the changed rounding order.
- bool HasFMAD = Options.UnsafeFPMath &&
+ bool HasFMAD = DAG.getMachineFunction().hasUnsafeFPMath() &&
(LegalOperations && TLI.isFMADLegal(DAG, N));
// No valid opcode, do not combine.
@@ -16971,6 +16973,7 @@ SDValue DAGCombiner::visitFADD(SDNode *N) {
SDValue N1 = N->getOperand(1);
bool N0CFP = DAG.isConstantFPBuildVectorOrConstantFP(N0);
bool N1CFP = DAG.isConstantFPBuildVectorOrConstantFP(N1);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
EVT VT = N->getValueType(0);
SDLoc DL(N);
const TargetOptions &Options = DAG.getTarget().Options;
@@ -17052,7 +17055,7 @@ SDValue DAGCombiner::visitFADD(SDNode *N) {
// If 'unsafe math' or reassoc and nsz, fold lots of things.
// TODO: break out portions of the transformations below for which Unsafe is
// considered and which do not require both nsz and reassoc
- if (((Options.UnsafeFPMath && Options.NoSignedZerosFPMath) ||
+ if (((UnsafeFPMath && Options.NoSignedZerosFPMath) ||
(Flags.hasAllowReassociation() && Flags.hasNoSignedZeros())) &&
AllowNewConst) {
// fadd (fadd x, c1), c2 -> fadd x, c1 + c2
@@ -17190,6 +17193,7 @@ SDValue DAGCombiner::visitFSUB(SDNode *N) {
const TargetOptions &Options = DAG.getTarget().Options;
const SDNodeFlags Flags = N->getFlags();
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
if (SDValue R = DAG.simplifyFPBinop(N->getOpcode(), N0, N1, Flags))
return R;
@@ -17239,7 +17243,7 @@ SDValue DAGCombiner::visitFSUB(SDNode *N) {
}
}
- if (((Options.UnsafeFPMath && Options.NoSignedZerosFPMath) ||
+ if (((UnsafeFPMath && Options.NoSignedZerosFPMath) ||
(Flags.hasAllowReassociation() && Flags.hasNoSignedZeros())) &&
N1.getOpcode() == ISD::FADD) {
// X - (X + Y) -> -Y
@@ -17376,7 +17380,6 @@ SDValue DAGCombiner::visitFMUL(SDNode *N) {
ConstantFPSDNode *N1CFP = isConstOrConstSplatFP(N1, true);
EVT VT = N->getValueType(0);
SDLoc DL(N);
- const TargetOptions &Options = DAG.getTarget().Options;
const SDNodeFlags Flags = N->getFlags();
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
@@ -17400,7 +17403,8 @@ SDValue DAGCombiner::visitFMUL(SDNode *N) {
if (SDValue NewSel = foldBinOpIntoSelect(N))
return NewSel;
- if (Options.UnsafeFPMath || Flags.hasAllowReassociation()) {
+ if (DAG.getMachineFunction().hasUnsafeFPMath() ||
+ Flags.hasAllowReassociation()) {
// fmul (fmul X, C1), C2 -> fmul X, C1 * C2
if (DAG.isConstantFPBuildVectorOrConstantFP(N1) &&
N0.getOpcode() == ISD::FMUL) {
@@ -17526,10 +17530,10 @@ template <class MatchContextClass> SDValue DAGCombiner::visitFMA(SDNode *N) {
ConstantFPSDNode *N1CFP = dyn_cast<ConstantFPSDNode>(N1);
EVT VT = N->getValueType(0);
SDLoc DL(N);
- const TargetOptions &Options = DAG.getTarget().Options;
// FMA nodes have flags that propagate to the created nodes.
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
MatchContextClass matcher(DAG, TLI, N);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
// Constant fold FMA.
if (SDValue C =
@@ -17552,8 +17556,8 @@ template <class MatchContextClass> SDValue DAGCombiner::visitFMA(SDNode *N) {
return matcher.getNode(ISD::FMA, DL, VT, NegN0, NegN1, N2);
}
- // FIXME: use fast math flags instead of Options.UnsafeFPMath
- if (Options.UnsafeFPMath) {
+ // FIXME: use fast math flags instead of MachineFunction::hasUnsafeFPMath()
+ if (UnsafeFPMath) {
if (N0CFP && N0CFP->isZero())
return N2;
if (N1CFP && N1CFP->isZero())
@@ -17571,8 +17575,7 @@ template <class MatchContextClass> SDValue DAGCombiner::visitFMA(SDNode *N) {
!DAG.isConstantFPBuildVectorOrConstantFP(N1))
return matcher.getNode(ISD::FMA, DL, VT, N1, N0, N2);
- bool CanReassociate =
- Options.UnsafeFPMath || N->getFlags().hasAllowReassociation();
+ bool CanReassociate = UnsafeFPMath || N->getFlags().hasAllowReassociation();
if (CanReassociate) {
// (fma x, c1, (fmul x, c2)) -> (fmul x, c1+c2)
if (matcher.match(N2, ISD::FMUL) && N0 == N2.getOperand(0) &&
@@ -17667,7 +17670,7 @@ SDValue DAGCombiner::combineRepeatedFPDivisors(SDNode *N) {
// TODO: Limit this transform based on optsize/minsize - it always creates at
// least 1 extra instruction. But the perf win may be substantial enough
// that only minsize should restrict this.
- bool UnsafeMath = DAG.getTarget().Options.UnsafeFPMath;
+ bool UnsafeMath = DAG.getMachineFunction().hasUnsafeFPMath();
const SDNodeFlags Flags = N->getFlags();
if (LegalDAG || (!UnsafeMath && !Flags.hasAllowReciprocal()))
return SDValue();
@@ -17744,6 +17747,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
const TargetOptions &Options = DAG.getTarget().Options;
SDNodeFlags Flags = N->getFlags();
SelectionDAG::FlagInserter FlagsInserter(DAG, N);
+ bool UnsafeFPMath = DAG.getMachineFunction().hasUnsafeFPMath();
if (SDValue R = DAG.simplifyFPBinop(N->getOpcode(), N0, N1, Flags))
return R;
@@ -17774,7 +17778,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
// isn't too nasty (eg NaN, denormal, ...).
if (((st == APFloat::opOK && !Recip.isDenormal()) ||
(st == APFloat::opInexact &&
- (Options.UnsafeFPMath || Flags.hasAllowReciprocal()))) &&
+ (UnsafeFPMath || Flags.hasAllowReciprocal()))) &&
(!LegalOperations ||
// FIXME: custom lowering of ConstantFP might fail (see e.g. ARM
// backend)... we should handle this gracefully after Legalize.
@@ -17785,7 +17789,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
DAG.getConstantFP(Recip, DL, VT));
}
- if (Options.UnsafeFPMath || Flags.hasAllowReciprocal()) {
+ if (UnsafeFPMath || Flags.hasAllowReciprocal()) {
// If this FDIV is part of a reciprocal square root, it may be folded
// into a target-specific square root estimate instruction.
if (N1.getOpcode() == ISD::FSQRT) {
@@ -17860,7 +17864,7 @@ SDValue DAGCombiner::visitFDIV(SDNode *N) {
// Fold X/Sqrt(X) -> Sqrt(X)
if ((Options.NoSignedZerosFPMath || Flags.hasNoSignedZeros()) &&
- (Options.UnsafeFPMath || Flags.hasAllowReassociation()))
+ (UnsafeFPMath || Flags.hasAllowReassociation()))
if (N1.getOpcode() == ISD::FSQRT && N0 == N1.getOperand(0))
return N1;
@@ -18356,7 +18360,7 @@ SDValue DAGCombiner::visitFP_ROUND(SDNode *N) {
// single-step fp_round we want to fold to.
// In other words, double rounding isn't the same as rounding.
// Also, this is a value preserving truncation iff both fp_round's are.
- if (DAG.getTarget().Options.UnsafeFPMath || N0IsTrunc)
+ if (DAG.getMachineFunction().hasUnsafeFPMath() || N0IsTrunc)
return DAG.getNode(
ISD::FP_ROUND, DL, VT, N0.getOperand(0),
DAG.getIntPtrConstant(NIsTrunc && N0IsTrunc, DL, /*isTarget=*/true));
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
index c6475f0219903..1470410d0e1ec 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
@@ -3643,7 +3643,7 @@ bool SelectionDAGLegalize::ExpandNode(SDNode *Node) {
break;
case ISD::FP_TO_FP16:
LLVM_DEBUG(dbgs() << "Legalizing FP_TO_FP16\n");
- if (!TLI.useSoftFloat() && TM.Options.UnsafeFPMath) {
+ if (!TLI.useSoftFloat() && DAG.getMachineFunction().hasUnsafeFPMath()) {
SDValue Op = Node->getOperand(0);
MVT SVT = Op.getSimpleValueType();
if ((SVT == MVT::f64 || SVT == MVT::f80) &&
diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
index bd9994bcb669c..e65078decebfe 100644
--- a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
@@ -11327,7 +11327,7 @@ SDValue AArch64TargetLowering::LowerSELECT_CC(ISD::CondCode CC, SDValue LHS,
AArch64CC::CondCode CC1, CC2;
changeFPCCToAArch64CC(CC, CC1, CC2);
- if (DAG.getTarget().Options.UnsafeFPMath) {
+ if (DAG.getMachineFunction().hasUnsafeFPMath()) {
// Transform "a == 0.0 ? 0.0 : x" to "a == 0.0 ? a : x" and
// "a != 0.0 ? x : 0.0" to "a != 0.0 ? x : a" to avoid materializing 0.0.
ConstantFPSDNode *RHSVal = dyn_cast<ConstantFPSDNode>(RHS);
diff --git a/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp b/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
index 17dd8a073eff0..a993dbb493903 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
+++ b/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp
@@ -6351,7 +6351,7 @@ static bool isCombineInstrCandidateFP(const MachineInstr &Inst) {
TargetOptions Options = Inst.getParent()->getParent()->getTarget().Options;
// We can fuse FADD/FSUB with FMUL, if fusion is either allowed globally by
// the target options or if FADD/FSUB has the contract fast-math flag.
- return Options.UnsafeFPMath ||
+ return Inst.getParent()->getParent()->hasUnsafeFPMath() ||
Options.AllowFPOpFusion == FPOpFusion::Fast ||
Inst.getFlag(MachineInstr::FmContract);
return true;
@@ -6457,7 +6457,7 @@ bool AArch64InstrInfo::isAssociativeAndCommutative(const MachineInstr &Inst,
case AArch64::FMUL_ZZZ_H:
case AArch64::FMUL_ZZZ_S:
case AArch64::FMUL_ZZZ_D:
- return Inst.getParent()->getParent()->getTarget().Options.UnsafeFPMath ||
+ return Inst.getParent()->getParent()->hasUnsafeFPMath() ||
(Inst.getFlag(MachineInstr::MIFlag::FmReassoc) &&
Inst.getFlag(MachineInstr::MIFlag::FmNsz));
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp b/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
index cca9fa72d0ca5..9bb25bd02141a 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
@@ -2623,7 +2623,7 @@ bool AMDGPUTargetLowering::allowApproxFunc(const SelectionDAG &DAG,
if (Flags.hasApproximateFuncs())
return true;
auto &Options = DAG.getTarget().Options;
- return Options.UnsafeFPMath || Options.ApproxFuncFPMath;
+ return DAG.getMachineFunction().hasUnsafeFPMath() || Options.ApproxFuncFPMath;
}
bool AMDGPUTargetLowering::needsDenormHandlingF32(const SelectionDAG &DAG,
@@ -2746,7 +2746,7 @@ SDValue AMDGPUTargetLowering::LowerFLOGCommon(SDValue Op,
const auto &Options = getTargetMachine().Options;
if (VT == MVT::f16 || Flags.hasApproximateFuncs() ||
- Options.ApproxFuncFPMath || Options.UnsafeFPMath) {
+ Options.ApproxFuncFPMath || DAG.getMachineFunction().hasUnsafeFPMath()) {
if (VT == MVT::f16 && !Subtarget->has16BitInsts()) {
// Log and multiply in f32 is good enough for f16.
@@ -3573,7 +3573,7 @@ SDValue AMDGPUTargetLowering::LowerFP_TO_FP16(SDValue Op, SelectionDAG &DAG) con
if (N0.getValueType() == MVT::f32)
return DAG.getNode(AMDGPUISD::FP_TO_FP16, DL, Op.getValueType(), N0);
- if (getTargetMachine().Options.UnsafeFPMath) {
+ if (DAG.getMachineFunction().hasUnsafeFPMath()) {
// There is a generic expand for FP_TO_FP16 with unsafe fast math.
return SDValue();
}
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td b/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td
index 6a5065cd4a0e8..10db60a4e890e 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td
+++ b/llvm/lib/Target/AMDGPU/AMDGPUInstructions.td
@@ -94,7 +94,7 @@ def FP64Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().F
def NoFP16Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().FP64FP16Denormals == DenormalMode::getPreserveSign()">;
def NoFP32Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().FP32Denormals == DenormalMode::getPreserveSign()">;
def NoFP64Denormals : Predicate<"MF->getInfo<SIMachineFunctionInfo>()->getMode().FP64FP16Denormals == DenormalMode::getPreserveSign()">;
-def UnsafeFPMath : Predicate<"TM.Options.UnsafeFPMath">;
+def UnsafeFPMath : Predicate<"MF->hasUnsafeFPMath()">;
}
def FMA : Predicate<"Subtarget->hasFMA()">;
diff --git a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
index e9e47eaadd557..f9eea24d0b9a1 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
@@ -3270,7 +3270,7 @@ static bool allowApproxFunc(const MachineFunction &MF, unsigned Flags) {
if (Flags & MachineInstr::FmAfn)
return true;
const auto &Options = MF.getTarget().Options;
- return Options.UnsafeFPMath || Options.ApproxFuncFPMath;
+ return MF.hasUnsafeFPMath() || Options.ApproxFuncFPMath;
}
static bool needsDenormHandlingF32(const MachineFunction &MF, Register Src,
@@ -3376,7 +3376,7 @@ bool AMDGPULegalizerInfo::legalizeFlogCommon(MachineInstr &MI,
static_cast<const AMDGPUTargetMachine &>(MF.getTarg...
[truncated]
|
Address llvm#27735 incrementally. New code should use `MachineFunction::hasUnsafeFPMath`.
paperchalice
force-pushed
the
rm-unsafe-fp-math
branch
from
25a2cb2 to
d7021b3
Compare
arsenm
reviewed
Feb 18, 2025
| @@ -699,6 +699,11 @@ bool MachineFunction::needsFrameMoves() const { | |||
| !F.getParent()->debug_compile_units().empty(); | |||
| } | |||
|
|
|||
| /// True if function attribute unsafe-fp-math is true. | |||
| bool MachineFunction::hasUnsafeFPMath() const { | |||
| return F.getFnAttribute("unsafe-fp-math").getValueAsBool(); | |||
There was a problem hiding this comment.
We probably should cache this in a bit in the function instead of requerying the attribute on every use, there are a lot of these. (plus we should stop checking the attributes and move to flags, but that's a bigger ask)
There was a problem hiding this comment.
Change it to a member variable is ok. Currently resetTargetOptions has higher priority than --enable-unsafe-fp-math, this results in some unreliable codes in backends and tests, change it to Options.UnsafeFPMath || F.getFnAttribute("unsafe-fp-math").getValueAsBool() break some tests.
There was a problem hiding this comment.
Does --enable-unsafe-fp-math currently set unsafe-fp-math on all functions?
If it does not, then the change will likely affect performance, as unsafe math will only enable unsafe optimizations in explicitly marked functions only. Ignoring global Options.UnsafeFPMath may not be the best idea.
break some tests.
Can you elaborate on which tests failed and how. It's possible that it's those tests that should be fixed, instead.
There was a problem hiding this comment.
--enable-unsafe-fp-math is always ignored after resetTargetOptions, which is called before instruction selection and subtarget info initialization.
Only CodeGen/NVPTX/fast-math.ll and CodeGen/X86/change-unsafe-fp-math.ll failed.
Some backends use this option to control the behavior of legalizer. Ideally backend should consider both |
Yes
No, Options.UnsafeFPMath should be purged. unsafe-fp-math should also be removed, but that's more difficult. Users should never use --enable-unsafe-fp-math. It's only for lit tests. The cl::opt is still used in many tests, but is implemented by setting the attribute on functions that do not have have the attribute already. |
arsenm
reviewed
Feb 19, 2025
| assert(N.getOpcode() == ISD::FMUL); | ||
|
|
||
| return Options.AllowFPOpFusion == FPOpFusion::Fast || | ||
| DAG.getMachineFunction().hasUnsafeFPMath() || |
There was a problem hiding this comment.
This is one of the most egregious uses of the unsafe-fp-math. It is interpreted inconsistently and confusingly. In this case, we already have the global fusion option and the contract flag.
In a separate PR, can you drop the usage of unsafe fp math here?
This seems like the sort of thing that should be made obvious in user-facing documentation. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Address #27735 incrementally. New code should use
MachineFunction::hasUnsafeFPMath.Will migrate other options with
hasXXXmethod inMachineFunction.