[llvm] Change fp128 lowering to use f128 functions by default

[tgross35]

LLVM currently emits calls to *l (long double) libm symbols for fp128 intrinsics. This works on platforms where long double and _Float128 are the same type, but is incorrect on many platforms.

Change RuntimeLibcalls such that *f128 libcalls are used by default, which is always safe and correct but may not be available. On platforms where it is likely that sqrtf128 and similar are not available, keep the current behavior of lowering to *l symbols if long double is binary128.

The logic for whether f128 is long double is based on the platforms in Clang that set LongDoubleFormat to llvm::APFloat::IEEEquad.

Fixes: #44744
Discourse discussion: https://discourse.llvm.org/t/fp128-math-functions-strange-results/72708
Initial patchset: https://reviews.llvm.org/D157836 / http://108.170.204.19/D157836

[github-actions]

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be
notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write
permissions for the repository. In which case you can instead tag reviewers by
name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review
by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate
is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

[tgross35]

@efriedma-quic was looking at this on phabricator

[github-actions]

✅ With the latest revision this PR passed the C/C++ code formatter.

[efriedma-quic]

This is basically the approach I was expecting: we check the type of "long double" when we build the TargetLowering, and pick appropriate names based on that.

I expect that for -mlong-double-128, you just want to add a module flag that overrides the default choice.

I think I'd prefer to keep the clang type information computation independent from the backend's type information, even if it overlaps. We try to layer the clang frontend so it isn't directly tied to LLVM IR outside of CodeGen.

My first thought was that the computation of the defaults should be in the backend, not Triple.h, since nothing else needs it at the moment. But I guess it could be useful outside the backend, so maybe that's fine. (At the moment, all the relevant optimizations just check the type of the call itself, but I can imagine certain optimizations could benefit from being able to compute the type without an existing signature to consult.)

[tgross35]

I'm struggling a bit with how to handle ABI information since that affects layout (e.g. ARM aapcs), which I think explains most of the errors in https://buildkite.com/llvm-project/github-pull-requests/builds/31198#018d26e2-fd17-4e15-a1eb-08580c189056. This needs to be available at TargetLoweringBase::InitLibcalls, which calls getCLayouts.

TargetMachine is available at that time, so would it be better to move CLayouts from Triple to TargetMachine? If so subclasses could be used rather than the if block, which more closely follows the Clang side.

Also, are there currently any module flags that make it to TargetLowering? Looking for a reference on how get the -mlong-double-128 information.

[efriedma-quic]

Putting a function in TargetMachine seems reasonable.

[efriedma-quic]

For the question about querying module flags, we do that in a few different places in codegen; grep for "getModuleFlag". Not sure if there's anything specifically in TargetLowering.

[tgross35]

Finally getting around to this after more than a year. @efriedma-quic as an alternative to the current implementation of duplicating long double layout information from Clang to LLVM, would it work if LLVM lowers to *f128 calls but provides a module flag fp128_use_long_double_libcalls to prefer the *l versions? So if Clang or other frontends know that their long double is _Float128, it can select those libcalls.

The advantage is avoided code duplication and the logic is easier to follow. Also this avoids problems if linking a library built with an unexpected -mlong-double- configuration.

The disadvantage is that frontends that don't know about C's long double can't benefit from the more common *l symbols. I don't think this is too big of a problem though: it makes no difference with glibc (the f128 aliases have been around sufficiently long) or on any platforms where long double is not _Float128. And it is easy enough for frontends to set fp128_use_long_double_libcalls on a case-by-case basis if they know what math library is being used (e.g. aarch64 musl).

(I handle the f128 support for Rust and would much rather never think about *l symbols, I can alias them to *f128 if needed or set the flag)

[tgross35]

In either case, I need to have the module flags available pretty early and I'm not sure how to do that. Ideally they would be available when TargetLowering is constructed or sometime before it is used for lowering, but it only gets a TargetMachine as a paremeter. All values in TargetOptions seem to be configured once and don't pay attention to module flags or take the module as a parameter - is there a reason for that? I'm wondering if TargetMachine is intended to be unchanging across different modules.

[tgross35]

I'm not sure what the effects of this change are, but it seemed unusual that long double and f128 would be treated differently here.

[tgross35]

I don't follow this, why was the sin+cos->sincos optimization lost here?

[llvmbot]

@llvm/pr-subscribers-llvm-analysis
@llvm/pr-subscribers-backend-powerpc
@llvm/pr-subscribers-backend-arm
@llvm/pr-subscribers-backend-webassembly

@llvm/pr-subscribers-backend-systemz

Author: Trevor Gross (tgross35)

Changes

LLVM currently emits calls to *l (long double) libm symbols for fp128 intrinsics. This works on platforms where long double and _Float128 are the same type, but is incorrect on many platforms.

Change RuntimeLibcalls such that *f128 libcalls are used by default, which is always safe and correct but may not be available. On platforms where it is likely that sqrtf128 and similar are not available, keep the current behavior of lowering to *l symbols if long double is binary128.

The logic for whether f128 is long double is based on the platforms in Clang that set LongDoubleFormat to llvm::APFloat::IEEEquad.

Fixes: #44744
Discourse discussion: https://discourse.llvm.org/t/fp128-math-functions-strange-results/72708
Initial patchset: https://reviews.llvm.org/D157836 / http://108.170.204.19/D157836

---

Patch is 120.93 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/76558.diff

32 Files Affected:

• (modified) llvm/include/llvm/Analysis/TargetTransformInfoImpl.h (+19-18)
• (modified) llvm/include/llvm/IR/RuntimeLibcalls.def (+51-50)
• (modified) llvm/include/llvm/IR/RuntimeLibcalls.h (+18)
• (modified) llvm/include/llvm/TargetParser/Triple.h (+13-7)
• (modified) llvm/lib/IR/RuntimeLibcalls.cpp (+65-49)
• (modified) llvm/lib/Target/Mips/MipsCCState.cpp (+10-5)
• (modified) llvm/lib/Target/PowerPC/PPCISelLowering.cpp (-26)
• (modified) llvm/lib/Target/WebAssembly/WebAssemblyRuntimeLibcallSignatures.cpp (+48)
• (modified) llvm/lib/TargetParser/Triple.cpp (+42)
• (modified) llvm/test/CodeGen/AArch64/illegal-float-ops.ll (+38-20)
• (modified) llvm/test/CodeGen/AArch64/sincos-expansion.ll (+1-1)
• (modified) llvm/test/CodeGen/AArch64/vecreduce-fmax-legalization-nan.ll (+1-1)
• (modified) llvm/test/CodeGen/ARM/ldexp.ll (+1-1)
• (modified) llvm/test/CodeGen/ARM/llvm.sincos.ll (+1-1)
• (added) llvm/test/CodeGen/Generic/f128-math-lowering.ll (+324)
• (modified) llvm/test/CodeGen/Mips/cconv/fmaxl_call.ll (+1-1)
• (modified) llvm/test/CodeGen/Mips/cconv/roundl-call.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/llrint-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/llround-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/lrint-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/lround-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/PowerPC/f128-arith.ll (+32-24)
• (modified) llvm/test/CodeGen/SystemZ/atomicrmw-fmax-03.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/atomicrmw-fmin-03.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-libcall.ll (+10-10)
• (modified) llvm/test/CodeGen/SystemZ/fp-mul-13.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-round-01.ll (+6-6)
• (modified) llvm/test/CodeGen/SystemZ/fp-sincos-01.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-strict-mul-13.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-strict-round-01.ll (+6-6)
• (modified) llvm/test/CodeGen/X86/fp128-libcalls-strict.ll (+96-96)
• (modified) llvm/test/CodeGen/X86/fp128-libcalls.ll (+60-60)

diff --git a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
index 990252b1e5743..4cc014aaa6699 100644
--- a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
+++ b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
@@ -189,27 +189,28 @@ class TargetTransformInfoImplBase {
 
     // These will all likely lower to a single selection DAG node.
     // clang-format off
-    if (Name == "copysign" || Name == "copysignf" || Name == "copysignl" ||
-        Name == "fabs"  || Name == "fabsf"  || Name == "fabsl" ||
-        Name == "fmin"  || Name == "fminf"  || Name == "fminl" ||
-        Name == "fmax"  || Name == "fmaxf"  || Name == "fmaxl" ||
-        Name == "sin"   || Name == "sinf"   || Name == "sinl"  ||
-        Name == "cos"   || Name == "cosf"   || Name == "cosl"  ||
-        Name == "tan"   || Name == "tanf"   || Name == "tanl"  ||
-        Name == "asin"  || Name == "asinf"  || Name == "asinl" ||
-        Name == "acos"  || Name == "acosf"  || Name == "acosl" ||
-        Name == "atan"  || Name == "atanf"  || Name == "atanl" ||
-        Name == "atan2" || Name == "atan2f" || Name == "atan2l"||
-        Name == "sinh"  || Name == "sinhf"  || Name == "sinhl" ||
-        Name == "cosh"  || Name == "coshf"  || Name == "coshl" ||
-        Name == "tanh"  || Name == "tanhf"  || Name == "tanhl" ||
-        Name == "sqrt"  || Name == "sqrtf"  || Name == "sqrtl" ||
-        Name == "exp10"  || Name == "exp10l"  || Name == "exp10f")
+    if (Name == "copysign" || Name == "copysignf" || Name == "copysignl" || Name == "copysignf128" ||
+        Name == "fabs"  || Name == "fabsf"  || Name == "fabsl"  || Name  == "fabsf128" ||
+        Name == "fmin"  || Name == "fminf"  || Name == "fminl"  || Name  == "fminf128" ||
+        Name == "fmax"  || Name == "fmaxf"  || Name == "fmaxl"  || Name  == "fmaxf128" ||
+        Name == "sin"   || Name == "sinf"   || Name == "sinl"   || Name  == "sinf128"  ||
+        Name == "cos"   || Name == "cosf"   || Name == "cosl"   || Name  == "cosf128"  ||
+        Name == "tan"   || Name == "tanf"   || Name == "tanl"   || Name  == "tanf128"  ||
+        Name == "asin"  || Name == "asinf"  || Name == "asinl"  || Name  == "asinf128" ||
+        Name == "acos"  || Name == "acosf"  || Name == "acosl"  || Name  == "acosf128" ||
+        Name == "atan"  || Name == "atanf"  || Name == "atanl"  || Name  == "atanf128" ||
+        Name == "atan2" || Name == "atan2f" || Name == "atan2l" || Name  == "atan2f128"||
+        Name == "sinh"  || Name == "sinhf"  || Name == "sinhl"  || Name  == "sinhf128" ||
+        Name == "cosh"  || Name == "coshf"  || Name == "coshl"  || Name  == "coshf128" ||
+        Name == "tanh"  || Name == "tanhf"  || Name == "tanhl"  || Name  == "tanhf128" ||
+        Name == "sqrt"  || Name == "sqrtf"  || Name == "sqrtl"  || Name  == "sqrtf128" ||
+        Name == "exp10" || Name == "exp10l" || Name == "exp10f" || Name  == "exp10f128")
       return false;
     // clang-format on
     // These are all likely to be optimized into something smaller.
-    if (Name == "pow" || Name == "powf" || Name == "powl" || Name == "exp2" ||
-        Name == "exp2l" || Name == "exp2f" || Name == "floor" ||
+    if (Name == "pow" || Name == "powf" || Name == "powl" ||
+        Name == "powf128" || Name == "exp2" || Name == "exp2f" ||
+        Name == "exp2l" || Name == "exp2f128" || Name == "floor" ||
         Name == "floorf" || Name == "ceil" || Name == "round" ||
         Name == "ffs" || Name == "ffsl" || Name == "abs" || Name == "labs" ||
         Name == "llabs")
diff --git a/llvm/include/llvm/IR/RuntimeLibcalls.def b/llvm/include/llvm/IR/RuntimeLibcalls.def
index cd8e9b598044c..e5f9ee3b2f384 100644
--- a/llvm/include/llvm/IR/RuntimeLibcalls.def
+++ b/llvm/include/llvm/IR/RuntimeLibcalls.def
@@ -89,7 +89,8 @@ HANDLE_LIBCALL(CTPOP_I32, "__popcountsi2")
 HANDLE_LIBCALL(CTPOP_I64, "__popcountdi2")
 HANDLE_LIBCALL(CTPOP_I128, "__popcountti2")
 
-// Floating-point
+// Floating-point. Note that new fp128 math routines should also be added to
+// setF128LibcallFormat in RuntimeLibcalls.cpp.
 HANDLE_LIBCALL(ADD_F32, "__addsf3")
 HANDLE_LIBCALL(ADD_F64, "__adddf3")
 HANDLE_LIBCALL(ADD_F80, "__addxf3")
@@ -113,12 +114,12 @@ HANDLE_LIBCALL(DIV_PPCF128, "__gcc_qdiv")
 HANDLE_LIBCALL(REM_F32, "fmodf")
 HANDLE_LIBCALL(REM_F64, "fmod")
 HANDLE_LIBCALL(REM_F80, "fmodl")
-HANDLE_LIBCALL(REM_F128, "fmodl")
+HANDLE_LIBCALL(REM_F128, "fmodf128")
 HANDLE_LIBCALL(REM_PPCF128, "fmodl")
 HANDLE_LIBCALL(FMA_F32, "fmaf")
 HANDLE_LIBCALL(FMA_F64, "fma")
 HANDLE_LIBCALL(FMA_F80, "fmal")
-HANDLE_LIBCALL(FMA_F128, "fmal")
+HANDLE_LIBCALL(FMA_F128, "fmaf128")
 HANDLE_LIBCALL(FMA_PPCF128, "fmal")
 HANDLE_LIBCALL(POWI_F32, "__powisf2")
 HANDLE_LIBCALL(POWI_F64, "__powidf2")
@@ -128,117 +129,117 @@ HANDLE_LIBCALL(POWI_PPCF128, "__powitf2")
 HANDLE_LIBCALL(SQRT_F32, "sqrtf")
 HANDLE_LIBCALL(SQRT_F64, "sqrt")
 HANDLE_LIBCALL(SQRT_F80, "sqrtl")
-HANDLE_LIBCALL(SQRT_F128, "sqrtl")
+HANDLE_LIBCALL(SQRT_F128, "sqrtf128")
 HANDLE_LIBCALL(SQRT_PPCF128, "sqrtl")
 HANDLE_LIBCALL(CBRT_F32, "cbrtf")
 HANDLE_LIBCALL(CBRT_F64, "cbrt")
 HANDLE_LIBCALL(CBRT_F80, "cbrtl")
-HANDLE_LIBCALL(CBRT_F128, "cbrtl")
+HANDLE_LIBCALL(CBRT_F128, "cbrtf128")
 HANDLE_LIBCALL(CBRT_PPCF128, "cbrtl")
 HANDLE_LIBCALL(LOG_F32, "logf")
 HANDLE_LIBCALL(LOG_F64, "log")
 HANDLE_LIBCALL(LOG_F80, "logl")
-HANDLE_LIBCALL(LOG_F128, "logl")
+HANDLE_LIBCALL(LOG_F128, "logf128")
 HANDLE_LIBCALL(LOG_PPCF128, "logl")
 HANDLE_LIBCALL(LOG_FINITE_F32, "__logf_finite")
 HANDLE_LIBCALL(LOG_FINITE_F64, "__log_finite")
 HANDLE_LIBCALL(LOG_FINITE_F80, "__logl_finite")
-HANDLE_LIBCALL(LOG_FINITE_F128, "__logl_finite")
+HANDLE_LIBCALL(LOG_FINITE_F128, "__logf128_finite")
 HANDLE_LIBCALL(LOG_FINITE_PPCF128, "__logl_finite")
 HANDLE_LIBCALL(LOG2_F32, "log2f")
 HANDLE_LIBCALL(LOG2_F64, "log2")
 HANDLE_LIBCALL(LOG2_F80, "log2l")
-HANDLE_LIBCALL(LOG2_F128, "log2l")
+HANDLE_LIBCALL(LOG2_F128, "log2f128")
 HANDLE_LIBCALL(LOG2_PPCF128, "log2l")
 HANDLE_LIBCALL(LOG2_FINITE_F32, "__log2f_finite")
 HANDLE_LIBCALL(LOG2_FINITE_F64, "__log2_finite")
 HANDLE_LIBCALL(LOG2_FINITE_F80, "__log2l_finite")
-HANDLE_LIBCALL(LOG2_FINITE_F128, "__log2l_finite")
+HANDLE_LIBCALL(LOG2_FINITE_F128, "__log2f128_finite")
 HANDLE_LIBCALL(LOG2_FINITE_PPCF128, "__log2l_finite")
 HANDLE_LIBCALL(LOG10_F32, "log10f")
 HANDLE_LIBCALL(LOG10_F64, "log10")
 HANDLE_LIBCALL(LOG10_F80, "log10l")
-HANDLE_LIBCALL(LOG10_F128, "log10l")
+HANDLE_LIBCALL(LOG10_F128, "log10f128")
 HANDLE_LIBCALL(LOG10_PPCF128, "log10l")
 HANDLE_LIBCALL(LOG10_FINITE_F32, "__log10f_finite")
 HANDLE_LIBCALL(LOG10_FINITE_F64, "__log10_finite")
 HANDLE_LIBCALL(LOG10_FINITE_F80, "__log10l_finite")
-HANDLE_LIBCALL(LOG10_FINITE_F128, "__log10l_finite")
+HANDLE_LIBCALL(LOG10_FINITE_F128, "__log10f128_finite")
 HANDLE_LIBCALL(LOG10_FINITE_PPCF128, "__log10l_finite")
 HANDLE_LIBCALL(EXP_F32, "expf")
 HANDLE_LIBCALL(EXP_F64, "exp")
 HANDLE_LIBCALL(EXP_F80, "expl")
-HANDLE_LIBCALL(EXP_F128, "expl")
+HANDLE_LIBCALL(EXP_F128, "expf128")
 HANDLE_LIBCALL(EXP_PPCF128, "expl")
 HANDLE_LIBCALL(EXP_FINITE_F32, "__expf_finite")
 HANDLE_LIBCALL(EXP_FINITE_F64, "__exp_finite")
 HANDLE_LIBCALL(EXP_FINITE_F80, "__expl_finite")
-HANDLE_LIBCALL(EXP_FINITE_F128, "__expl_finite")
+HANDLE_LIBCALL(EXP_FINITE_F128, "__expf128_finite")
 HANDLE_LIBCALL(EXP_FINITE_PPCF128, "__expl_finite")
 HANDLE_LIBCALL(EXP2_F32, "exp2f")
 HANDLE_LIBCALL(EXP2_F64, "exp2")
 HANDLE_LIBCALL(EXP2_F80, "exp2l")
-HANDLE_LIBCALL(EXP2_F128, "exp2l")
+HANDLE_LIBCALL(EXP2_F128, "exp2f128")
 HANDLE_LIBCALL(EXP2_PPCF128, "exp2l")
 HANDLE_LIBCALL(EXP2_FINITE_F32, "__exp2f_finite")
 HANDLE_LIBCALL(EXP2_FINITE_F64, "__exp2_finite")
 HANDLE_LIBCALL(EXP2_FINITE_F80, "__exp2l_finite")
-HANDLE_LIBCALL(EXP2_FINITE_F128, "__exp2l_finite")
+HANDLE_LIBCALL(EXP2_FINITE_F128, "__exp2f128_finite")
 HANDLE_LIBCALL(EXP2_FINITE_PPCF128, "__exp2l_finite")
 HANDLE_LIBCALL(EXP10_F32, "exp10f")
 HANDLE_LIBCALL(EXP10_F64, "exp10")
 HANDLE_LIBCALL(EXP10_F80, "exp10l")
-HANDLE_LIBCALL(EXP10_F128, "exp10l")
+HANDLE_LIBCALL(EXP10_F128, "exp10f128")
 HANDLE_LIBCALL(EXP10_PPCF128, "exp10l")
 HANDLE_LIBCALL(SIN_F32, "sinf")
 HANDLE_LIBCALL(SIN_F64, "sin")
 HANDLE_LIBCALL(SIN_F80, "sinl")
-HANDLE_LIBCALL(SIN_F128, "sinl")
+HANDLE_LIBCALL(SIN_F128, "sinf128")
 HANDLE_LIBCALL(SIN_PPCF128, "sinl")
 HANDLE_LIBCALL(COS_F32, "cosf")
 HANDLE_LIBCALL(COS_F64, "cos")
 HANDLE_LIBCALL(COS_F80, "cosl")
-HANDLE_LIBCALL(COS_F128, "cosl")
+HANDLE_LIBCALL(COS_F128, "cosf128")
 HANDLE_LIBCALL(COS_PPCF128, "cosl")
 HANDLE_LIBCALL(TAN_F32, "tanf")
 HANDLE_LIBCALL(TAN_F64, "tan")
 HANDLE_LIBCALL(TAN_F80, "tanl")
-HANDLE_LIBCALL(TAN_F128,"tanl")
+HANDLE_LIBCALL(TAN_F128,"tanf128")
 HANDLE_LIBCALL(TAN_PPCF128, "tanl")
 HANDLE_LIBCALL(SINH_F32, "sinhf")
 HANDLE_LIBCALL(SINH_F64, "sinh")
 HANDLE_LIBCALL(SINH_F80, "sinhl")
-HANDLE_LIBCALL(SINH_F128, "sinhl")
+HANDLE_LIBCALL(SINH_F128, "sinhf128")
 HANDLE_LIBCALL(SINH_PPCF128, "sinhl")
 HANDLE_LIBCALL(COSH_F32, "coshf")
 HANDLE_LIBCALL(COSH_F64, "cosh")
 HANDLE_LIBCALL(COSH_F80, "coshl")
-HANDLE_LIBCALL(COSH_F128, "coshl")
+HANDLE_LIBCALL(COSH_F128, "coshf128")
 HANDLE_LIBCALL(COSH_PPCF128, "coshl")
 HANDLE_LIBCALL(TANH_F32, "tanhf")
 HANDLE_LIBCALL(TANH_F64, "tanh")
 HANDLE_LIBCALL(TANH_F80, "tanhl")
-HANDLE_LIBCALL(TANH_F128,"tanhl")
+HANDLE_LIBCALL(TANH_F128,"tanhf128")
 HANDLE_LIBCALL(TANH_PPCF128, "tanhl")
 HANDLE_LIBCALL(ASIN_F32, "asinf")
 HANDLE_LIBCALL(ASIN_F64, "asin")
 HANDLE_LIBCALL(ASIN_F80, "asinl")
-HANDLE_LIBCALL(ASIN_F128, "asinl")
+HANDLE_LIBCALL(ASIN_F128, "asinf128")
 HANDLE_LIBCALL(ASIN_PPCF128, "asinl")
 HANDLE_LIBCALL(ACOS_F32, "acosf")
 HANDLE_LIBCALL(ACOS_F64, "acos")
 HANDLE_LIBCALL(ACOS_F80, "acosl")
-HANDLE_LIBCALL(ACOS_F128, "acosl")
+HANDLE_LIBCALL(ACOS_F128, "acosf128")
 HANDLE_LIBCALL(ACOS_PPCF128, "acosl")
 HANDLE_LIBCALL(ATAN_F32, "atanf")
 HANDLE_LIBCALL(ATAN_F64, "atan")
 HANDLE_LIBCALL(ATAN_F80, "atanl")
-HANDLE_LIBCALL(ATAN_F128,"atanl")
+HANDLE_LIBCALL(ATAN_F128,"atanf128")
 HANDLE_LIBCALL(ATAN_PPCF128, "atanl")
 HANDLE_LIBCALL(ATAN2_F32, "atan2f")
 HANDLE_LIBCALL(ATAN2_F64, "atan2")
 HANDLE_LIBCALL(ATAN2_F80, "atan2l")
-HANDLE_LIBCALL(ATAN2_F128,"atan2l")
+HANDLE_LIBCALL(ATAN2_F128,"atan2f128")
 HANDLE_LIBCALL(ATAN2_PPCF128, "atan2l")
 HANDLE_LIBCALL(SINCOS_F32, nullptr)
 HANDLE_LIBCALL(SINCOS_F64, nullptr)
@@ -250,122 +251,122 @@ HANDLE_LIBCALL(SINCOS_STRET_F64, nullptr)
 HANDLE_LIBCALL(POW_F32, "powf")
 HANDLE_LIBCALL(POW_F64, "pow")
 HANDLE_LIBCALL(POW_F80, "powl")
-HANDLE_LIBCALL(POW_F128, "powl")
+HANDLE_LIBCALL(POW_F128, "powf128")
 HANDLE_LIBCALL(POW_PPCF128, "powl")
 HANDLE_LIBCALL(POW_FINITE_F32, "__powf_finite")
 HANDLE_LIBCALL(POW_FINITE_F64, "__pow_finite")
 HANDLE_LIBCALL(POW_FINITE_F80, "__powl_finite")
-HANDLE_LIBCALL(POW_FINITE_F128, "__powl_finite")
+HANDLE_LIBCALL(POW_FINITE_F128, "__powf128_finite")
 HANDLE_LIBCALL(POW_FINITE_PPCF128, "__powl_finite")
 HANDLE_LIBCALL(CEIL_F32, "ceilf")
 HANDLE_LIBCALL(CEIL_F64, "ceil")
 HANDLE_LIBCALL(CEIL_F80, "ceill")
-HANDLE_LIBCALL(CEIL_F128, "ceill")
+HANDLE_LIBCALL(CEIL_F128, "ceilf128")
 HANDLE_LIBCALL(CEIL_PPCF128, "ceill")
 HANDLE_LIBCALL(TRUNC_F32, "truncf")
 HANDLE_LIBCALL(TRUNC_F64, "trunc")
 HANDLE_LIBCALL(TRUNC_F80, "truncl")
-HANDLE_LIBCALL(TRUNC_F128, "truncl")
+HANDLE_LIBCALL(TRUNC_F128, "truncf128")
 HANDLE_LIBCALL(TRUNC_PPCF128, "truncl")
 HANDLE_LIBCALL(RINT_F32, "rintf")
 HANDLE_LIBCALL(RINT_F64, "rint")
 HANDLE_LIBCALL(RINT_F80, "rintl")
-HANDLE_LIBCALL(RINT_F128, "rintl")
+HANDLE_LIBCALL(RINT_F128, "rintf128")
 HANDLE_LIBCALL(RINT_PPCF128, "rintl")
 HANDLE_LIBCALL(NEARBYINT_F32, "nearbyintf")
 HANDLE_LIBCALL(NEARBYINT_F64, "nearbyint")
 HANDLE_LIBCALL(NEARBYINT_F80, "nearbyintl")
-HANDLE_LIBCALL(NEARBYINT_F128, "nearbyintl")
+HANDLE_LIBCALL(NEARBYINT_F128, "nearbyintf128")
 HANDLE_LIBCALL(NEARBYINT_PPCF128, "nearbyintl")
 HANDLE_LIBCALL(ROUND_F32, "roundf")
 HANDLE_LIBCALL(ROUND_F64, "round")
 HANDLE_LIBCALL(ROUND_F80, "roundl")
-HANDLE_LIBCALL(ROUND_F128, "roundl")
+HANDLE_LIBCALL(ROUND_F128, "roundf128")
 HANDLE_LIBCALL(ROUND_PPCF128, "roundl")
 HANDLE_LIBCALL(ROUNDEVEN_F32, "roundevenf")
 HANDLE_LIBCALL(ROUNDEVEN_F64, "roundeven")
 HANDLE_LIBCALL(ROUNDEVEN_F80, "roundevenl")
-HANDLE_LIBCALL(ROUNDEVEN_F128, "roundevenl")
+HANDLE_LIBCALL(ROUNDEVEN_F128, "roundevenf128")
 HANDLE_LIBCALL(ROUNDEVEN_PPCF128, "roundevenl")
 HANDLE_LIBCALL(FLOOR_F32, "floorf")
 HANDLE_LIBCALL(FLOOR_F64, "floor")
 HANDLE_LIBCALL(FLOOR_F80, "floorl")
-HANDLE_LIBCALL(FLOOR_F128, "floorl")
+HANDLE_LIBCALL(FLOOR_F128, "floorf128")
 HANDLE_LIBCALL(FLOOR_PPCF128, "floorl")
 HANDLE_LIBCALL(COPYSIGN_F32, "copysignf")
 HANDLE_LIBCALL(COPYSIGN_F64, "copysign")
 HANDLE_LIBCALL(COPYSIGN_F80, "copysignl")
-HANDLE_LIBCALL(COPYSIGN_F128, "copysignl")
+HANDLE_LIBCALL(COPYSIGN_F128, "copysignf128")
 HANDLE_LIBCALL(COPYSIGN_PPCF128, "copysignl")
 HANDLE_LIBCALL(FMIN_F32, "fminf")
 HANDLE_LIBCALL(FMIN_F64, "fmin")
 HANDLE_LIBCALL(FMIN_F80, "fminl")
-HANDLE_LIBCALL(FMIN_F128, "fminl")
+HANDLE_LIBCALL(FMIN_F128, "fminf128")
 HANDLE_LIBCALL(FMIN_PPCF128, "fminl")
 HANDLE_LIBCALL(FMAX_F32, "fmaxf")
 HANDLE_LIBCALL(FMAX_F64, "fmax")
 HANDLE_LIBCALL(FMAX_F80, "fmaxl")
-HANDLE_LIBCALL(FMAX_F128, "fmaxl")
+HANDLE_LIBCALL(FMAX_F128, "fmaxf128")
 HANDLE_LIBCALL(FMAX_PPCF128, "fmaxl")
 HANDLE_LIBCALL(FMINIMUM_F32, "fminimumf")
 HANDLE_LIBCALL(FMINIMUM_F64, "fminimum")
 HANDLE_LIBCALL(FMINIMUM_F80, "fminimuml")
-HANDLE_LIBCALL(FMINIMUM_F128, "fminimuml")
+HANDLE_LIBCALL(FMINIMUM_F128, "fminimumf128")
 HANDLE_LIBCALL(FMINIMUM_PPCF128, "fminimuml")
 HANDLE_LIBCALL(FMAXIMUM_F32, "fmaximumf")
 HANDLE_LIBCALL(FMAXIMUM_F64, "fmaximum")
 HANDLE_LIBCALL(FMAXIMUM_F80, "fmaximuml")
-HANDLE_LIBCALL(FMAXIMUM_F128, "fmaximuml")
+HANDLE_LIBCALL(FMAXIMUM_F128, "fmaximumf128")
 HANDLE_LIBCALL(FMAXIMUM_PPCF128, "fmaximum_numl")
 HANDLE_LIBCALL(FMINIMUMNUM_F32, "fminimum_numf")
 HANDLE_LIBCALL(FMINIMUMNUM_F64, "fminimum_num")
 HANDLE_LIBCALL(FMINIMUMNUM_F80, "fminimum_numl")
-HANDLE_LIBCALL(FMINIMUMNUM_F128, "fminimum_numl")
+HANDLE_LIBCALL(FMINIMUMNUM_F128, "fminimum_numf128")
 HANDLE_LIBCALL(FMINIMUMNUM_PPCF128, "fminimum_numl")
 HANDLE_LIBCALL(FMAXIMUMNUM_F32, "fmaximum_numf")
 HANDLE_LIBCALL(FMAXIMUMNUM_F64, "fmaximum_num")
 HANDLE_LIBCALL(FMAXIMUMNUM_F80, "fmaximum_numl")
-HANDLE_LIBCALL(FMAXIMUMNUM_F128, "fmaximum_numl")
+HANDLE_LIBCALL(FMAXIMUMNUM_F128, "fmaximum_numf128")
 HANDLE_LIBCALL(FMAXIMUMNUM_PPCF128, "fmaximum_numl")
 HANDLE_LIBCALL(LROUND_F32, "lroundf")
 HANDLE_LIBCALL(LROUND_F64, "lround")
 HANDLE_LIBCALL(LROUND_F80, "lroundl")
-HANDLE_LIBCALL(LROUND_F128, "lroundl")
+HANDLE_LIBCALL(LROUND_F128, "lroundf128")
 HANDLE_LIBCALL(LROUND_PPCF128, "lroundl")
 HANDLE_LIBCALL(LLROUND_F32, "llroundf")
 HANDLE_LIBCALL(LLROUND_F64, "llround")
 HANDLE_LIBCALL(LLROUND_F80, "llroundl")
-HANDLE_LIBCALL(LLROUND_F128, "llroundl")
+HANDLE_LIBCALL(LLROUND_F128, "llroundf128")
 HANDLE_LIBCALL(LLROUND_PPCF128, "llroundl")
 HANDLE_LIBCALL(LRINT_F32, "lrintf")
 HANDLE_LIBCALL(LRINT_F64, "lrint")
 HANDLE_LIBCALL(LRINT_F80, "lrintl")
-HANDLE_LIBCALL(LRINT_F128, "lrintl")
+HANDLE_LIBCALL(LRINT_F128, "lrintf128")
 HANDLE_LIBCALL(LRINT_PPCF128, "lrintl")
 HANDLE_LIBCALL(LLRINT_F32, "llrintf")
 HANDLE_LIBCALL(LLRINT_F64, "llrint")
 HANDLE_LIBCALL(LLRINT_F80, "llrintl")
-HANDLE_LIBCALL(LLRINT_F128, "llrintl")
+HANDLE_LIBCALL(LLRINT_F128, "llrintf128")
 HANDLE_LIBCALL(LLRINT_PPCF128, "llrintl")
 HANDLE_LIBCALL(LDEXP_F32, "ldexpf")
 HANDLE_LIBCALL(LDEXP_F64, "ldexp")
 HANDLE_LIBCALL(LDEXP_F80, "ldexpl")
-HANDLE_LIBCALL(LDEXP_F128, "ldexpl")
+HANDLE_LIBCALL(LDEXP_F128, "ldexpf128")
 HANDLE_LIBCALL(LDEXP_PPCF128, "ldexpl")
 HANDLE_LIBCALL(FREXP_F32, "frexpf")
 HANDLE_LIBCALL(FREXP_F64, "frexp")
 HANDLE_LIBCALL(FREXP_F80, "frexpl")
-HANDLE_LIBCALL(FREXP_F128, "frexpl")
+HANDLE_LIBCALL(FREXP_F128, "frexpf128")
 HANDLE_LIBCALL(FREXP_PPCF128, "frexpl")
 HANDLE_LIBCALL(SINCOSPI_F32, "sincospif")
 HANDLE_LIBCALL(SINCOSPI_F64, "sincospi")
 HANDLE_LIBCALL(SINCOSPI_F80, "sincospil")
-HANDLE_LIBCALL(SINCOSPI_F128, "sincospil")
+HANDLE_LIBCALL(SINCOSPI_F128, "sincospif128")
 HANDLE_LIBCALL(SINCOSPI_PPCF128, "sincospil")
 HANDLE_LIBCALL(MODF_F32, "modff")
 HANDLE_LIBCALL(MODF_F64, "modf")
 HANDLE_LIBCALL(MODF_F80, "modfl")
-HANDLE_LIBCALL(MODF_F128, "modfl")
+HANDLE_LIBCALL(MODF_F128, "modff128")
 HANDLE_LIBCALL(MODF_PPCF128, "modfl")
 
 // Floating point environment
diff --git a/llvm/include/llvm/IR/RuntimeLibcalls.h b/llvm/include/llvm/IR/RuntimeLibcalls.h
index b3648f5a31e2a..09da78720b988 100644
--- a/llvm/include/llvm/IR/RuntimeLibcalls.h
+++ b/llvm/include/llvm/IR/RuntimeLibcalls.h
@@ -20,6 +20,17 @@
 #include "llvm/TargetParser/Triple.h"
 
 namespace llvm {
+
+/// Library names to use for `fp128` libcalls.
+enum class F128LibcallFormat {
+  /// C23 `*f128` lowering, e.g. `sinf128`
+  Default = 0,
+  /// `long double` *l` lowering, e.g. `sinl`.
+  LongDouble = 1,
+  // If needed, this could be extended with an option for `q` suffixes from
+  // libquadmath.
+};
+
 namespace RTLIB {
 
 /// RTLIB::Libcall enum - This enum defines all of the runtime library calls
@@ -97,6 +108,13 @@ struct RuntimeLibcallsInfo {
   /// Set default libcall names. If a target wants to opt-out of a libcall it
   /// should be placed here.
   void initLibcalls(const Triple &TT);
+
+  /// Set a specific lowering convention for `fp128` math libcalls.
+  ///
+  /// By default, `fp128` math functions get lowered to the C23 `sinf128`-
+  /// style symbols. This allows overriding with `sinl`-style symbols on
+  /// platforms where `long double` is known to be identical to _Float128.
+  void setF128LibcallFormat(F128LibcallFormat Format);
 };
 
 } // namespace RTLIB
diff --git a/llvm/include/llvm/TargetParser/Triple.h b/llvm/include/llvm/TargetParser/Triple.h
index fb6bbc0163701..f3976704f4c7c 100644
--- a/llvm/include/llvm/TargetParser/Triple.h
+++ b/llvm/include/llvm/TargetParser/Triple.h
@@ -262,13 +262,13 @@ class Triple {
     EABIHF,
     Android,
     Musl,
-    MuslABIN32,
-    MuslABI64,
-    MuslEABI,
-    MuslEABIHF,
-    MuslF32,
-    MuslSF,
-    MuslX32,
+    MuslABIN32, // MIPS N32 ABI
+    MuslABI64,  // MIPS N64 ABI
+    MuslEABI,   // Arm32 EABI
+    MuslEABIHF, // Arm32 EABI + HF
+    MuslF32,    // LoongArch ILP32F/LP64F
+    MuslSF,     // LoongArch ILP32S/LP64S
+    MuslX32,    // Musl using 32-bit ABI on x86_64
     LLVM,
 
     MSVC,
@@ -1231,6 +1231,12 @@ class Triple {
   /// or an invalid version tuple if this triple doesn't have one.
   VersionTuple getMinimumSupportedOSVersion() const;
 
+  /// Check whether (1) f128 is the same format as `long double`, and (2)
+  /// `*f128` symbols are likely unavailable. In other words, platforms for
+  /// which this returns true may safely use sqrtl instead of sqrtf128 and
+  /// should do so because sqrtf128 would probably error at link time.
+  bool shouldLowerf128AsLongDouble() const;
+
   /// @}
   /// @name Static helpers for IDs.
   /// @{
diff --git a/llvm/lib/IR/RuntimeLibcalls.cpp b/llvm/lib/IR/RuntimeLibcalls.cpp
index 90c3bf0db0236..e00f4346984ae 100644
--- a/llvm/lib/IR/RuntimeLibcalls.cpp
+++ b/llvm/lib/IR/RuntimeLibcalls.cpp
@@ -25,54 +25,6 @@ void RuntimeLibcallsInfo::initLibcalls(const Triple &TT) {
   for (int LC = 0; LC < RTLIB::UNKNOWN_LIBCALL; ++LC)
     setLibcallCallingConv((RTLIB::Libcall)LC, CallingConv::C);
 
-  // Use the f128 variants of math functions on x86_64
-  if (TT.getArch() == Triple::ArchType::x86_64 && TT.isGNUEnvironment()) {
-    setLibcallName(RTLIB::REM_F128, "fmodf128");
-    setLibcallName(RTLIB::FMA_F128, "fmaf128");
-    setLibcallName(RTLIB::SQRT_F128, "sqrtf128");
-    setLibcallName(RTLIB::CBRT_F128, "cbrtf128");
-    setLibcallName(RTLIB::LOG_F128, "logf128");
-    setLibcallName(RTLIB::LOG_FINITE_F128, "__logf128_finite");
-    setLibcallName(RTLIB::LOG2_F128, "log2f128");
-    setLibcallName(RTLIB::LOG2_FINITE_F128, "__log2f128_finite");
-    setLibcallName(RTLIB::LOG10_F128, "log10f128");
-    setLibcallName(RTLIB::LOG10_FINITE_F128, "__log10f128_finite");
-    setLibcallName(RTLIB::EXP_F128, "expf128");
-    setLibcallName(RTLIB::EXP_FINITE_F128, "__expf128_finite");
-    setLibcallName(RTLIB::EXP2_F128, "exp2f128");
-    setLibcallName(RTLIB::EXP2_FINITE_F128, "__exp2f128_finite");
-    setLibcallName(RTLIB::EXP10_F128, "exp10f128");
-    setLibcallName(RTLIB::SIN_F128, "sinf128");
-    setLibcallName(RTLIB::COS_F1...
[truncated]

[llvmbot]

@llvm/pr-subscribers-backend-aarch64

Author: Trevor Gross (tgross35)

Changes

LLVM currently emits calls to *l (long double) libm symbols for fp128 intrinsics. This works on platforms where long double and _Float128 are the same type, but is incorrect on many platforms.

Change RuntimeLibcalls such that *f128 libcalls are used by default, which is always safe and correct but may not be available. On platforms where it is likely that sqrtf128 and similar are not available, keep the current behavior of lowering to *l symbols if long double is binary128.

The logic for whether f128 is long double is based on the platforms in Clang that set LongDoubleFormat to llvm::APFloat::IEEEquad.

Fixes: #44744
Discourse discussion: https://discourse.llvm.org/t/fp128-math-functions-strange-results/72708
Initial patchset: https://reviews.llvm.org/D157836 / http://108.170.204.19/D157836

---

Patch is 120.93 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/76558.diff

32 Files Affected:

• (modified) llvm/include/llvm/Analysis/TargetTransformInfoImpl.h (+19-18)
• (modified) llvm/include/llvm/IR/RuntimeLibcalls.def (+51-50)
• (modified) llvm/include/llvm/IR/RuntimeLibcalls.h (+18)
• (modified) llvm/include/llvm/TargetParser/Triple.h (+13-7)
• (modified) llvm/lib/IR/RuntimeLibcalls.cpp (+65-49)
• (modified) llvm/lib/Target/Mips/MipsCCState.cpp (+10-5)
• (modified) llvm/lib/Target/PowerPC/PPCISelLowering.cpp (-26)
• (modified) llvm/lib/Target/WebAssembly/WebAssemblyRuntimeLibcallSignatures.cpp (+48)
• (modified) llvm/lib/TargetParser/Triple.cpp (+42)
• (modified) llvm/test/CodeGen/AArch64/illegal-float-ops.ll (+38-20)
• (modified) llvm/test/CodeGen/AArch64/sincos-expansion.ll (+1-1)
• (modified) llvm/test/CodeGen/AArch64/vecreduce-fmax-legalization-nan.ll (+1-1)
• (modified) llvm/test/CodeGen/ARM/ldexp.ll (+1-1)
• (modified) llvm/test/CodeGen/ARM/llvm.sincos.ll (+1-1)
• (added) llvm/test/CodeGen/Generic/f128-math-lowering.ll (+324)
• (modified) llvm/test/CodeGen/Mips/cconv/fmaxl_call.ll (+1-1)
• (modified) llvm/test/CodeGen/Mips/cconv/roundl-call.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/llrint-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/llround-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/lrint-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/lround-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/PowerPC/f128-arith.ll (+32-24)
• (modified) llvm/test/CodeGen/SystemZ/atomicrmw-fmax-03.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/atomicrmw-fmin-03.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-libcall.ll (+10-10)
• (modified) llvm/test/CodeGen/SystemZ/fp-mul-13.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-round-01.ll (+6-6)
• (modified) llvm/test/CodeGen/SystemZ/fp-sincos-01.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-strict-mul-13.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-strict-round-01.ll (+6-6)
• (modified) llvm/test/CodeGen/X86/fp128-libcalls-strict.ll (+96-96)
• (modified) llvm/test/CodeGen/X86/fp128-libcalls.ll (+60-60)

diff --git a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
index 990252b1e5743..4cc014aaa6699 100644
--- a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
+++ b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
@@ -189,27 +189,28 @@ class TargetTransformInfoImplBase {
 
     // These will all likely lower to a single selection DAG node.
     // clang-format off
-    if (Name == "copysign" || Name == "copysignf" || Name == "copysignl" ||
-        Name == "fabs"  || Name == "fabsf"  || Name == "fabsl" ||
-        Name == "fmin"  || Name == "fminf"  || Name == "fminl" ||
-        Name == "fmax"  || Name == "fmaxf"  || Name == "fmaxl" ||
-        Name == "sin"   || Name == "sinf"   || Name == "sinl"  ||
-        Name == "cos"   || Name == "cosf"   || Name == "cosl"  ||
-        Name == "tan"   || Name == "tanf"   || Name == "tanl"  ||
-        Name == "asin"  || Name == "asinf"  || Name == "asinl" ||
-        Name == "acos"  || Name == "acosf"  || Name == "acosl" ||
-        Name == "atan"  || Name == "atanf"  || Name == "atanl" ||
-        Name == "atan2" || Name == "atan2f" || Name == "atan2l"||
-        Name == "sinh"  || Name == "sinhf"  || Name == "sinhl" ||
-        Name == "cosh"  || Name == "coshf"  || Name == "coshl" ||
-        Name == "tanh"  || Name == "tanhf"  || Name == "tanhl" ||
-        Name == "sqrt"  || Name == "sqrtf"  || Name == "sqrtl" ||
-        Name == "exp10"  || Name == "exp10l"  || Name == "exp10f")
+    if (Name == "copysign" || Name == "copysignf" || Name == "copysignl" || Name == "copysignf128" ||
+        Name == "fabs"  || Name == "fabsf"  || Name == "fabsl"  || Name  == "fabsf128" ||
+        Name == "fmin"  || Name == "fminf"  || Name == "fminl"  || Name  == "fminf128" ||
+        Name == "fmax"  || Name == "fmaxf"  || Name == "fmaxl"  || Name  == "fmaxf128" ||
+        Name == "sin"   || Name == "sinf"   || Name == "sinl"   || Name  == "sinf128"  ||
+        Name == "cos"   || Name == "cosf"   || Name == "cosl"   || Name  == "cosf128"  ||
+        Name == "tan"   || Name == "tanf"   || Name == "tanl"   || Name  == "tanf128"  ||
+        Name == "asin"  || Name == "asinf"  || Name == "asinl"  || Name  == "asinf128" ||
+        Name == "acos"  || Name == "acosf"  || Name == "acosl"  || Name  == "acosf128" ||
+        Name == "atan"  || Name == "atanf"  || Name == "atanl"  || Name  == "atanf128" ||
+        Name == "atan2" || Name == "atan2f" || Name == "atan2l" || Name  == "atan2f128"||
+        Name == "sinh"  || Name == "sinhf"  || Name == "sinhl"  || Name  == "sinhf128" ||
+        Name == "cosh"  || Name == "coshf"  || Name == "coshl"  || Name  == "coshf128" ||
+        Name == "tanh"  || Name == "tanhf"  || Name == "tanhl"  || Name  == "tanhf128" ||
+        Name == "sqrt"  || Name == "sqrtf"  || Name == "sqrtl"  || Name  == "sqrtf128" ||
+        Name == "exp10" || Name == "exp10l" || Name == "exp10f" || Name  == "exp10f128")
       return false;
     // clang-format on
     // These are all likely to be optimized into something smaller.
-    if (Name == "pow" || Name == "powf" || Name == "powl" || Name == "exp2" ||
-        Name == "exp2l" || Name == "exp2f" || Name == "floor" ||
+    if (Name == "pow" || Name == "powf" || Name == "powl" ||
+        Name == "powf128" || Name == "exp2" || Name == "exp2f" ||
+        Name == "exp2l" || Name == "exp2f128" || Name == "floor" ||
         Name == "floorf" || Name == "ceil" || Name == "round" ||
         Name == "ffs" || Name == "ffsl" || Name == "abs" || Name == "labs" ||
         Name == "llabs")
diff --git a/llvm/include/llvm/IR/RuntimeLibcalls.def b/llvm/include/llvm/IR/RuntimeLibcalls.def
index cd8e9b598044c..e5f9ee3b2f384 100644
--- a/llvm/include/llvm/IR/RuntimeLibcalls.def
+++ b/llvm/include/llvm/IR/RuntimeLibcalls.def
@@ -89,7 +89,8 @@ HANDLE_LIBCALL(CTPOP_I32, "__popcountsi2")
 HANDLE_LIBCALL(CTPOP_I64, "__popcountdi2")
 HANDLE_LIBCALL(CTPOP_I128, "__popcountti2")
 
-// Floating-point
+// Floating-point. Note that new fp128 math routines should also be added to
+// setF128LibcallFormat in RuntimeLibcalls.cpp.
 HANDLE_LIBCALL(ADD_F32, "__addsf3")
 HANDLE_LIBCALL(ADD_F64, "__adddf3")
 HANDLE_LIBCALL(ADD_F80, "__addxf3")
@@ -113,12 +114,12 @@ HANDLE_LIBCALL(DIV_PPCF128, "__gcc_qdiv")
 HANDLE_LIBCALL(REM_F32, "fmodf")
 HANDLE_LIBCALL(REM_F64, "fmod")
 HANDLE_LIBCALL(REM_F80, "fmodl")
-HANDLE_LIBCALL(REM_F128, "fmodl")
+HANDLE_LIBCALL(REM_F128, "fmodf128")
 HANDLE_LIBCALL(REM_PPCF128, "fmodl")
 HANDLE_LIBCALL(FMA_F32, "fmaf")
 HANDLE_LIBCALL(FMA_F64, "fma")
 HANDLE_LIBCALL(FMA_F80, "fmal")
-HANDLE_LIBCALL(FMA_F128, "fmal")
+HANDLE_LIBCALL(FMA_F128, "fmaf128")
 HANDLE_LIBCALL(FMA_PPCF128, "fmal")
 HANDLE_LIBCALL(POWI_F32, "__powisf2")
 HANDLE_LIBCALL(POWI_F64, "__powidf2")
@@ -128,117 +129,117 @@ HANDLE_LIBCALL(POWI_PPCF128, "__powitf2")
 HANDLE_LIBCALL(SQRT_F32, "sqrtf")
 HANDLE_LIBCALL(SQRT_F64, "sqrt")
 HANDLE_LIBCALL(SQRT_F80, "sqrtl")
-HANDLE_LIBCALL(SQRT_F128, "sqrtl")
+HANDLE_LIBCALL(SQRT_F128, "sqrtf128")
 HANDLE_LIBCALL(SQRT_PPCF128, "sqrtl")
 HANDLE_LIBCALL(CBRT_F32, "cbrtf")
 HANDLE_LIBCALL(CBRT_F64, "cbrt")
 HANDLE_LIBCALL(CBRT_F80, "cbrtl")
-HANDLE_LIBCALL(CBRT_F128, "cbrtl")
+HANDLE_LIBCALL(CBRT_F128, "cbrtf128")
 HANDLE_LIBCALL(CBRT_PPCF128, "cbrtl")
 HANDLE_LIBCALL(LOG_F32, "logf")
 HANDLE_LIBCALL(LOG_F64, "log")
 HANDLE_LIBCALL(LOG_F80, "logl")
-HANDLE_LIBCALL(LOG_F128, "logl")
+HANDLE_LIBCALL(LOG_F128, "logf128")
 HANDLE_LIBCALL(LOG_PPCF128, "logl")
 HANDLE_LIBCALL(LOG_FINITE_F32, "__logf_finite")
 HANDLE_LIBCALL(LOG_FINITE_F64, "__log_finite")
 HANDLE_LIBCALL(LOG_FINITE_F80, "__logl_finite")
-HANDLE_LIBCALL(LOG_FINITE_F128, "__logl_finite")
+HANDLE_LIBCALL(LOG_FINITE_F128, "__logf128_finite")
 HANDLE_LIBCALL(LOG_FINITE_PPCF128, "__logl_finite")
 HANDLE_LIBCALL(LOG2_F32, "log2f")
 HANDLE_LIBCALL(LOG2_F64, "log2")
 HANDLE_LIBCALL(LOG2_F80, "log2l")
-HANDLE_LIBCALL(LOG2_F128, "log2l")
+HANDLE_LIBCALL(LOG2_F128, "log2f128")
 HANDLE_LIBCALL(LOG2_PPCF128, "log2l")
 HANDLE_LIBCALL(LOG2_FINITE_F32, "__log2f_finite")
 HANDLE_LIBCALL(LOG2_FINITE_F64, "__log2_finite")
 HANDLE_LIBCALL(LOG2_FINITE_F80, "__log2l_finite")
-HANDLE_LIBCALL(LOG2_FINITE_F128, "__log2l_finite")
+HANDLE_LIBCALL(LOG2_FINITE_F128, "__log2f128_finite")
 HANDLE_LIBCALL(LOG2_FINITE_PPCF128, "__log2l_finite")
 HANDLE_LIBCALL(LOG10_F32, "log10f")
 HANDLE_LIBCALL(LOG10_F64, "log10")
 HANDLE_LIBCALL(LOG10_F80, "log10l")
-HANDLE_LIBCALL(LOG10_F128, "log10l")
+HANDLE_LIBCALL(LOG10_F128, "log10f128")
 HANDLE_LIBCALL(LOG10_PPCF128, "log10l")
 HANDLE_LIBCALL(LOG10_FINITE_F32, "__log10f_finite")
 HANDLE_LIBCALL(LOG10_FINITE_F64, "__log10_finite")
 HANDLE_LIBCALL(LOG10_FINITE_F80, "__log10l_finite")
-HANDLE_LIBCALL(LOG10_FINITE_F128, "__log10l_finite")
+HANDLE_LIBCALL(LOG10_FINITE_F128, "__log10f128_finite")
 HANDLE_LIBCALL(LOG10_FINITE_PPCF128, "__log10l_finite")
 HANDLE_LIBCALL(EXP_F32, "expf")
 HANDLE_LIBCALL(EXP_F64, "exp")
 HANDLE_LIBCALL(EXP_F80, "expl")
-HANDLE_LIBCALL(EXP_F128, "expl")
+HANDLE_LIBCALL(EXP_F128, "expf128")
 HANDLE_LIBCALL(EXP_PPCF128, "expl")
 HANDLE_LIBCALL(EXP_FINITE_F32, "__expf_finite")
 HANDLE_LIBCALL(EXP_FINITE_F64, "__exp_finite")
 HANDLE_LIBCALL(EXP_FINITE_F80, "__expl_finite")
-HANDLE_LIBCALL(EXP_FINITE_F128, "__expl_finite")
+HANDLE_LIBCALL(EXP_FINITE_F128, "__expf128_finite")
 HANDLE_LIBCALL(EXP_FINITE_PPCF128, "__expl_finite")
 HANDLE_LIBCALL(EXP2_F32, "exp2f")
 HANDLE_LIBCALL(EXP2_F64, "exp2")
 HANDLE_LIBCALL(EXP2_F80, "exp2l")
-HANDLE_LIBCALL(EXP2_F128, "exp2l")
+HANDLE_LIBCALL(EXP2_F128, "exp2f128")
 HANDLE_LIBCALL(EXP2_PPCF128, "exp2l")
 HANDLE_LIBCALL(EXP2_FINITE_F32, "__exp2f_finite")
 HANDLE_LIBCALL(EXP2_FINITE_F64, "__exp2_finite")
 HANDLE_LIBCALL(EXP2_FINITE_F80, "__exp2l_finite")
-HANDLE_LIBCALL(EXP2_FINITE_F128, "__exp2l_finite")
+HANDLE_LIBCALL(EXP2_FINITE_F128, "__exp2f128_finite")
 HANDLE_LIBCALL(EXP2_FINITE_PPCF128, "__exp2l_finite")
 HANDLE_LIBCALL(EXP10_F32, "exp10f")
 HANDLE_LIBCALL(EXP10_F64, "exp10")
 HANDLE_LIBCALL(EXP10_F80, "exp10l")
-HANDLE_LIBCALL(EXP10_F128, "exp10l")
+HANDLE_LIBCALL(EXP10_F128, "exp10f128")
 HANDLE_LIBCALL(EXP10_PPCF128, "exp10l")
 HANDLE_LIBCALL(SIN_F32, "sinf")
 HANDLE_LIBCALL(SIN_F64, "sin")
 HANDLE_LIBCALL(SIN_F80, "sinl")
-HANDLE_LIBCALL(SIN_F128, "sinl")
+HANDLE_LIBCALL(SIN_F128, "sinf128")
 HANDLE_LIBCALL(SIN_PPCF128, "sinl")
 HANDLE_LIBCALL(COS_F32, "cosf")
 HANDLE_LIBCALL(COS_F64, "cos")
 HANDLE_LIBCALL(COS_F80, "cosl")
-HANDLE_LIBCALL(COS_F128, "cosl")
+HANDLE_LIBCALL(COS_F128, "cosf128")
 HANDLE_LIBCALL(COS_PPCF128, "cosl")
 HANDLE_LIBCALL(TAN_F32, "tanf")
 HANDLE_LIBCALL(TAN_F64, "tan")
 HANDLE_LIBCALL(TAN_F80, "tanl")
-HANDLE_LIBCALL(TAN_F128,"tanl")
+HANDLE_LIBCALL(TAN_F128,"tanf128")
 HANDLE_LIBCALL(TAN_PPCF128, "tanl")
 HANDLE_LIBCALL(SINH_F32, "sinhf")
 HANDLE_LIBCALL(SINH_F64, "sinh")
 HANDLE_LIBCALL(SINH_F80, "sinhl")
-HANDLE_LIBCALL(SINH_F128, "sinhl")
+HANDLE_LIBCALL(SINH_F128, "sinhf128")
 HANDLE_LIBCALL(SINH_PPCF128, "sinhl")
 HANDLE_LIBCALL(COSH_F32, "coshf")
 HANDLE_LIBCALL(COSH_F64, "cosh")
 HANDLE_LIBCALL(COSH_F80, "coshl")
-HANDLE_LIBCALL(COSH_F128, "coshl")
+HANDLE_LIBCALL(COSH_F128, "coshf128")
 HANDLE_LIBCALL(COSH_PPCF128, "coshl")
 HANDLE_LIBCALL(TANH_F32, "tanhf")
 HANDLE_LIBCALL(TANH_F64, "tanh")
 HANDLE_LIBCALL(TANH_F80, "tanhl")
-HANDLE_LIBCALL(TANH_F128,"tanhl")
+HANDLE_LIBCALL(TANH_F128,"tanhf128")
 HANDLE_LIBCALL(TANH_PPCF128, "tanhl")
 HANDLE_LIBCALL(ASIN_F32, "asinf")
 HANDLE_LIBCALL(ASIN_F64, "asin")
 HANDLE_LIBCALL(ASIN_F80, "asinl")
-HANDLE_LIBCALL(ASIN_F128, "asinl")
+HANDLE_LIBCALL(ASIN_F128, "asinf128")
 HANDLE_LIBCALL(ASIN_PPCF128, "asinl")
 HANDLE_LIBCALL(ACOS_F32, "acosf")
 HANDLE_LIBCALL(ACOS_F64, "acos")
 HANDLE_LIBCALL(ACOS_F80, "acosl")
-HANDLE_LIBCALL(ACOS_F128, "acosl")
+HANDLE_LIBCALL(ACOS_F128, "acosf128")
 HANDLE_LIBCALL(ACOS_PPCF128, "acosl")
 HANDLE_LIBCALL(ATAN_F32, "atanf")
 HANDLE_LIBCALL(ATAN_F64, "atan")
 HANDLE_LIBCALL(ATAN_F80, "atanl")
-HANDLE_LIBCALL(ATAN_F128,"atanl")
+HANDLE_LIBCALL(ATAN_F128,"atanf128")
 HANDLE_LIBCALL(ATAN_PPCF128, "atanl")
 HANDLE_LIBCALL(ATAN2_F32, "atan2f")
 HANDLE_LIBCALL(ATAN2_F64, "atan2")
 HANDLE_LIBCALL(ATAN2_F80, "atan2l")
-HANDLE_LIBCALL(ATAN2_F128,"atan2l")
+HANDLE_LIBCALL(ATAN2_F128,"atan2f128")
 HANDLE_LIBCALL(ATAN2_PPCF128, "atan2l")
 HANDLE_LIBCALL(SINCOS_F32, nullptr)
 HANDLE_LIBCALL(SINCOS_F64, nullptr)
@@ -250,122 +251,122 @@ HANDLE_LIBCALL(SINCOS_STRET_F64, nullptr)
 HANDLE_LIBCALL(POW_F32, "powf")
 HANDLE_LIBCALL(POW_F64, "pow")
 HANDLE_LIBCALL(POW_F80, "powl")
-HANDLE_LIBCALL(POW_F128, "powl")
+HANDLE_LIBCALL(POW_F128, "powf128")
 HANDLE_LIBCALL(POW_PPCF128, "powl")
 HANDLE_LIBCALL(POW_FINITE_F32, "__powf_finite")
 HANDLE_LIBCALL(POW_FINITE_F64, "__pow_finite")
 HANDLE_LIBCALL(POW_FINITE_F80, "__powl_finite")
-HANDLE_LIBCALL(POW_FINITE_F128, "__powl_finite")
+HANDLE_LIBCALL(POW_FINITE_F128, "__powf128_finite")
 HANDLE_LIBCALL(POW_FINITE_PPCF128, "__powl_finite")
 HANDLE_LIBCALL(CEIL_F32, "ceilf")
 HANDLE_LIBCALL(CEIL_F64, "ceil")
 HANDLE_LIBCALL(CEIL_F80, "ceill")
-HANDLE_LIBCALL(CEIL_F128, "ceill")
+HANDLE_LIBCALL(CEIL_F128, "ceilf128")
 HANDLE_LIBCALL(CEIL_PPCF128, "ceill")
 HANDLE_LIBCALL(TRUNC_F32, "truncf")
 HANDLE_LIBCALL(TRUNC_F64, "trunc")
 HANDLE_LIBCALL(TRUNC_F80, "truncl")
-HANDLE_LIBCALL(TRUNC_F128, "truncl")
+HANDLE_LIBCALL(TRUNC_F128, "truncf128")
 HANDLE_LIBCALL(TRUNC_PPCF128, "truncl")
 HANDLE_LIBCALL(RINT_F32, "rintf")
 HANDLE_LIBCALL(RINT_F64, "rint")
 HANDLE_LIBCALL(RINT_F80, "rintl")
-HANDLE_LIBCALL(RINT_F128, "rintl")
+HANDLE_LIBCALL(RINT_F128, "rintf128")
 HANDLE_LIBCALL(RINT_PPCF128, "rintl")
 HANDLE_LIBCALL(NEARBYINT_F32, "nearbyintf")
 HANDLE_LIBCALL(NEARBYINT_F64, "nearbyint")
 HANDLE_LIBCALL(NEARBYINT_F80, "nearbyintl")
-HANDLE_LIBCALL(NEARBYINT_F128, "nearbyintl")
+HANDLE_LIBCALL(NEARBYINT_F128, "nearbyintf128")
 HANDLE_LIBCALL(NEARBYINT_PPCF128, "nearbyintl")
 HANDLE_LIBCALL(ROUND_F32, "roundf")
 HANDLE_LIBCALL(ROUND_F64, "round")
 HANDLE_LIBCALL(ROUND_F80, "roundl")
-HANDLE_LIBCALL(ROUND_F128, "roundl")
+HANDLE_LIBCALL(ROUND_F128, "roundf128")
 HANDLE_LIBCALL(ROUND_PPCF128, "roundl")
 HANDLE_LIBCALL(ROUNDEVEN_F32, "roundevenf")
 HANDLE_LIBCALL(ROUNDEVEN_F64, "roundeven")
 HANDLE_LIBCALL(ROUNDEVEN_F80, "roundevenl")
-HANDLE_LIBCALL(ROUNDEVEN_F128, "roundevenl")
+HANDLE_LIBCALL(ROUNDEVEN_F128, "roundevenf128")
 HANDLE_LIBCALL(ROUNDEVEN_PPCF128, "roundevenl")
 HANDLE_LIBCALL(FLOOR_F32, "floorf")
 HANDLE_LIBCALL(FLOOR_F64, "floor")
 HANDLE_LIBCALL(FLOOR_F80, "floorl")
-HANDLE_LIBCALL(FLOOR_F128, "floorl")
+HANDLE_LIBCALL(FLOOR_F128, "floorf128")
 HANDLE_LIBCALL(FLOOR_PPCF128, "floorl")
 HANDLE_LIBCALL(COPYSIGN_F32, "copysignf")
 HANDLE_LIBCALL(COPYSIGN_F64, "copysign")
 HANDLE_LIBCALL(COPYSIGN_F80, "copysignl")
-HANDLE_LIBCALL(COPYSIGN_F128, "copysignl")
+HANDLE_LIBCALL(COPYSIGN_F128, "copysignf128")
 HANDLE_LIBCALL(COPYSIGN_PPCF128, "copysignl")
 HANDLE_LIBCALL(FMIN_F32, "fminf")
 HANDLE_LIBCALL(FMIN_F64, "fmin")
 HANDLE_LIBCALL(FMIN_F80, "fminl")
-HANDLE_LIBCALL(FMIN_F128, "fminl")
+HANDLE_LIBCALL(FMIN_F128, "fminf128")
 HANDLE_LIBCALL(FMIN_PPCF128, "fminl")
 HANDLE_LIBCALL(FMAX_F32, "fmaxf")
 HANDLE_LIBCALL(FMAX_F64, "fmax")
 HANDLE_LIBCALL(FMAX_F80, "fmaxl")
-HANDLE_LIBCALL(FMAX_F128, "fmaxl")
+HANDLE_LIBCALL(FMAX_F128, "fmaxf128")
 HANDLE_LIBCALL(FMAX_PPCF128, "fmaxl")
 HANDLE_LIBCALL(FMINIMUM_F32, "fminimumf")
 HANDLE_LIBCALL(FMINIMUM_F64, "fminimum")
 HANDLE_LIBCALL(FMINIMUM_F80, "fminimuml")
-HANDLE_LIBCALL(FMINIMUM_F128, "fminimuml")
+HANDLE_LIBCALL(FMINIMUM_F128, "fminimumf128")
 HANDLE_LIBCALL(FMINIMUM_PPCF128, "fminimuml")
 HANDLE_LIBCALL(FMAXIMUM_F32, "fmaximumf")
 HANDLE_LIBCALL(FMAXIMUM_F64, "fmaximum")
 HANDLE_LIBCALL(FMAXIMUM_F80, "fmaximuml")
-HANDLE_LIBCALL(FMAXIMUM_F128, "fmaximuml")
+HANDLE_LIBCALL(FMAXIMUM_F128, "fmaximumf128")
 HANDLE_LIBCALL(FMAXIMUM_PPCF128, "fmaximum_numl")
 HANDLE_LIBCALL(FMINIMUMNUM_F32, "fminimum_numf")
 HANDLE_LIBCALL(FMINIMUMNUM_F64, "fminimum_num")
 HANDLE_LIBCALL(FMINIMUMNUM_F80, "fminimum_numl")
-HANDLE_LIBCALL(FMINIMUMNUM_F128, "fminimum_numl")
+HANDLE_LIBCALL(FMINIMUMNUM_F128, "fminimum_numf128")
 HANDLE_LIBCALL(FMINIMUMNUM_PPCF128, "fminimum_numl")
 HANDLE_LIBCALL(FMAXIMUMNUM_F32, "fmaximum_numf")
 HANDLE_LIBCALL(FMAXIMUMNUM_F64, "fmaximum_num")
 HANDLE_LIBCALL(FMAXIMUMNUM_F80, "fmaximum_numl")
-HANDLE_LIBCALL(FMAXIMUMNUM_F128, "fmaximum_numl")
+HANDLE_LIBCALL(FMAXIMUMNUM_F128, "fmaximum_numf128")
 HANDLE_LIBCALL(FMAXIMUMNUM_PPCF128, "fmaximum_numl")
 HANDLE_LIBCALL(LROUND_F32, "lroundf")
 HANDLE_LIBCALL(LROUND_F64, "lround")
 HANDLE_LIBCALL(LROUND_F80, "lroundl")
-HANDLE_LIBCALL(LROUND_F128, "lroundl")
+HANDLE_LIBCALL(LROUND_F128, "lroundf128")
 HANDLE_LIBCALL(LROUND_PPCF128, "lroundl")
 HANDLE_LIBCALL(LLROUND_F32, "llroundf")
 HANDLE_LIBCALL(LLROUND_F64, "llround")
 HANDLE_LIBCALL(LLROUND_F80, "llroundl")
-HANDLE_LIBCALL(LLROUND_F128, "llroundl")
+HANDLE_LIBCALL(LLROUND_F128, "llroundf128")
 HANDLE_LIBCALL(LLROUND_PPCF128, "llroundl")
 HANDLE_LIBCALL(LRINT_F32, "lrintf")
 HANDLE_LIBCALL(LRINT_F64, "lrint")
 HANDLE_LIBCALL(LRINT_F80, "lrintl")
-HANDLE_LIBCALL(LRINT_F128, "lrintl")
+HANDLE_LIBCALL(LRINT_F128, "lrintf128")
 HANDLE_LIBCALL(LRINT_PPCF128, "lrintl")
 HANDLE_LIBCALL(LLRINT_F32, "llrintf")
 HANDLE_LIBCALL(LLRINT_F64, "llrint")
 HANDLE_LIBCALL(LLRINT_F80, "llrintl")
-HANDLE_LIBCALL(LLRINT_F128, "llrintl")
+HANDLE_LIBCALL(LLRINT_F128, "llrintf128")
 HANDLE_LIBCALL(LLRINT_PPCF128, "llrintl")
 HANDLE_LIBCALL(LDEXP_F32, "ldexpf")
 HANDLE_LIBCALL(LDEXP_F64, "ldexp")
 HANDLE_LIBCALL(LDEXP_F80, "ldexpl")
-HANDLE_LIBCALL(LDEXP_F128, "ldexpl")
+HANDLE_LIBCALL(LDEXP_F128, "ldexpf128")
 HANDLE_LIBCALL(LDEXP_PPCF128, "ldexpl")
 HANDLE_LIBCALL(FREXP_F32, "frexpf")
 HANDLE_LIBCALL(FREXP_F64, "frexp")
 HANDLE_LIBCALL(FREXP_F80, "frexpl")
-HANDLE_LIBCALL(FREXP_F128, "frexpl")
+HANDLE_LIBCALL(FREXP_F128, "frexpf128")
 HANDLE_LIBCALL(FREXP_PPCF128, "frexpl")
 HANDLE_LIBCALL(SINCOSPI_F32, "sincospif")
 HANDLE_LIBCALL(SINCOSPI_F64, "sincospi")
 HANDLE_LIBCALL(SINCOSPI_F80, "sincospil")
-HANDLE_LIBCALL(SINCOSPI_F128, "sincospil")
+HANDLE_LIBCALL(SINCOSPI_F128, "sincospif128")
 HANDLE_LIBCALL(SINCOSPI_PPCF128, "sincospil")
 HANDLE_LIBCALL(MODF_F32, "modff")
 HANDLE_LIBCALL(MODF_F64, "modf")
 HANDLE_LIBCALL(MODF_F80, "modfl")
-HANDLE_LIBCALL(MODF_F128, "modfl")
+HANDLE_LIBCALL(MODF_F128, "modff128")
 HANDLE_LIBCALL(MODF_PPCF128, "modfl")
 
 // Floating point environment
diff --git a/llvm/include/llvm/IR/RuntimeLibcalls.h b/llvm/include/llvm/IR/RuntimeLibcalls.h
index b3648f5a31e2a..09da78720b988 100644
--- a/llvm/include/llvm/IR/RuntimeLibcalls.h
+++ b/llvm/include/llvm/IR/RuntimeLibcalls.h
@@ -20,6 +20,17 @@
 #include "llvm/TargetParser/Triple.h"
 
 namespace llvm {
+
+/// Library names to use for `fp128` libcalls.
+enum class F128LibcallFormat {
+  /// C23 `*f128` lowering, e.g. `sinf128`
+  Default = 0,
+  /// `long double` *l` lowering, e.g. `sinl`.
+  LongDouble = 1,
+  // If needed, this could be extended with an option for `q` suffixes from
+  // libquadmath.
+};
+
 namespace RTLIB {
 
 /// RTLIB::Libcall enum - This enum defines all of the runtime library calls
@@ -97,6 +108,13 @@ struct RuntimeLibcallsInfo {
   /// Set default libcall names. If a target wants to opt-out of a libcall it
   /// should be placed here.
   void initLibcalls(const Triple &TT);
+
+  /// Set a specific lowering convention for `fp128` math libcalls.
+  ///
+  /// By default, `fp128` math functions get lowered to the C23 `sinf128`-
+  /// style symbols. This allows overriding with `sinl`-style symbols on
+  /// platforms where `long double` is known to be identical to _Float128.
+  void setF128LibcallFormat(F128LibcallFormat Format);
 };
 
 } // namespace RTLIB
diff --git a/llvm/include/llvm/TargetParser/Triple.h b/llvm/include/llvm/TargetParser/Triple.h
index fb6bbc0163701..f3976704f4c7c 100644
--- a/llvm/include/llvm/TargetParser/Triple.h
+++ b/llvm/include/llvm/TargetParser/Triple.h
@@ -262,13 +262,13 @@ class Triple {
     EABIHF,
     Android,
     Musl,
-    MuslABIN32,
-    MuslABI64,
-    MuslEABI,
-    MuslEABIHF,
-    MuslF32,
-    MuslSF,
-    MuslX32,
+    MuslABIN32, // MIPS N32 ABI
+    MuslABI64,  // MIPS N64 ABI
+    MuslEABI,   // Arm32 EABI
+    MuslEABIHF, // Arm32 EABI + HF
+    MuslF32,    // LoongArch ILP32F/LP64F
+    MuslSF,     // LoongArch ILP32S/LP64S
+    MuslX32,    // Musl using 32-bit ABI on x86_64
     LLVM,
 
     MSVC,
@@ -1231,6 +1231,12 @@ class Triple {
   /// or an invalid version tuple if this triple doesn't have one.
   VersionTuple getMinimumSupportedOSVersion() const;
 
+  /// Check whether (1) f128 is the same format as `long double`, and (2)
+  /// `*f128` symbols are likely unavailable. In other words, platforms for
+  /// which this returns true may safely use sqrtl instead of sqrtf128 and
+  /// should do so because sqrtf128 would probably error at link time.
+  bool shouldLowerf128AsLongDouble() const;
+
   /// @}
   /// @name Static helpers for IDs.
   /// @{
diff --git a/llvm/lib/IR/RuntimeLibcalls.cpp b/llvm/lib/IR/RuntimeLibcalls.cpp
index 90c3bf0db0236..e00f4346984ae 100644
--- a/llvm/lib/IR/RuntimeLibcalls.cpp
+++ b/llvm/lib/IR/RuntimeLibcalls.cpp
@@ -25,54 +25,6 @@ void RuntimeLibcallsInfo::initLibcalls(const Triple &TT) {
   for (int LC = 0; LC < RTLIB::UNKNOWN_LIBCALL; ++LC)
     setLibcallCallingConv((RTLIB::Libcall)LC, CallingConv::C);
 
-  // Use the f128 variants of math functions on x86_64
-  if (TT.getArch() == Triple::ArchType::x86_64 && TT.isGNUEnvironment()) {
-    setLibcallName(RTLIB::REM_F128, "fmodf128");
-    setLibcallName(RTLIB::FMA_F128, "fmaf128");
-    setLibcallName(RTLIB::SQRT_F128, "sqrtf128");
-    setLibcallName(RTLIB::CBRT_F128, "cbrtf128");
-    setLibcallName(RTLIB::LOG_F128, "logf128");
-    setLibcallName(RTLIB::LOG_FINITE_F128, "__logf128_finite");
-    setLibcallName(RTLIB::LOG2_F128, "log2f128");
-    setLibcallName(RTLIB::LOG2_FINITE_F128, "__log2f128_finite");
-    setLibcallName(RTLIB::LOG10_F128, "log10f128");
-    setLibcallName(RTLIB::LOG10_FINITE_F128, "__log10f128_finite");
-    setLibcallName(RTLIB::EXP_F128, "expf128");
-    setLibcallName(RTLIB::EXP_FINITE_F128, "__expf128_finite");
-    setLibcallName(RTLIB::EXP2_F128, "exp2f128");
-    setLibcallName(RTLIB::EXP2_FINITE_F128, "__exp2f128_finite");
-    setLibcallName(RTLIB::EXP10_F128, "exp10f128");
-    setLibcallName(RTLIB::SIN_F128, "sinf128");
-    setLibcallName(RTLIB::COS_F1...
[truncated]

[llvmbot]

@llvm/pr-subscribers-backend-x86

Author: Trevor Gross (tgross35)

Changes

LLVM currently emits calls to *l (long double) libm symbols for fp128 intrinsics. This works on platforms where long double and _Float128 are the same type, but is incorrect on many platforms.

Change RuntimeLibcalls such that *f128 libcalls are used by default, which is always safe and correct but may not be available. On platforms where it is likely that sqrtf128 and similar are not available, keep the current behavior of lowering to *l symbols if long double is binary128.

The logic for whether f128 is long double is based on the platforms in Clang that set LongDoubleFormat to llvm::APFloat::IEEEquad.

Fixes: #44744
Discourse discussion: https://discourse.llvm.org/t/fp128-math-functions-strange-results/72708
Initial patchset: https://reviews.llvm.org/D157836 / http://108.170.204.19/D157836

---

Patch is 120.93 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/76558.diff

32 Files Affected:

• (modified) llvm/include/llvm/Analysis/TargetTransformInfoImpl.h (+19-18)
• (modified) llvm/include/llvm/IR/RuntimeLibcalls.def (+51-50)
• (modified) llvm/include/llvm/IR/RuntimeLibcalls.h (+18)
• (modified) llvm/include/llvm/TargetParser/Triple.h (+13-7)
• (modified) llvm/lib/IR/RuntimeLibcalls.cpp (+65-49)
• (modified) llvm/lib/Target/Mips/MipsCCState.cpp (+10-5)
• (modified) llvm/lib/Target/PowerPC/PPCISelLowering.cpp (-26)
• (modified) llvm/lib/Target/WebAssembly/WebAssemblyRuntimeLibcallSignatures.cpp (+48)
• (modified) llvm/lib/TargetParser/Triple.cpp (+42)
• (modified) llvm/test/CodeGen/AArch64/illegal-float-ops.ll (+38-20)
• (modified) llvm/test/CodeGen/AArch64/sincos-expansion.ll (+1-1)
• (modified) llvm/test/CodeGen/AArch64/vecreduce-fmax-legalization-nan.ll (+1-1)
• (modified) llvm/test/CodeGen/ARM/ldexp.ll (+1-1)
• (modified) llvm/test/CodeGen/ARM/llvm.sincos.ll (+1-1)
• (added) llvm/test/CodeGen/Generic/f128-math-lowering.ll (+324)
• (modified) llvm/test/CodeGen/Mips/cconv/fmaxl_call.ll (+1-1)
• (modified) llvm/test/CodeGen/Mips/cconv/roundl-call.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/llrint-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/llround-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/lrint-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/Mips/lround-conv.ll (+2-2)
• (modified) llvm/test/CodeGen/PowerPC/f128-arith.ll (+32-24)
• (modified) llvm/test/CodeGen/SystemZ/atomicrmw-fmax-03.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/atomicrmw-fmin-03.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-libcall.ll (+10-10)
• (modified) llvm/test/CodeGen/SystemZ/fp-mul-13.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-round-01.ll (+6-6)
• (modified) llvm/test/CodeGen/SystemZ/fp-sincos-01.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-strict-mul-13.ll (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/fp-strict-round-01.ll (+6-6)
• (modified) llvm/test/CodeGen/X86/fp128-libcalls-strict.ll (+96-96)
• (modified) llvm/test/CodeGen/X86/fp128-libcalls.ll (+60-60)

diff --git a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
index 990252b1e5743..4cc014aaa6699 100644
--- a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
+++ b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
@@ -189,27 +189,28 @@ class TargetTransformInfoImplBase {
 
     // These will all likely lower to a single selection DAG node.
     // clang-format off
-    if (Name == "copysign" || Name == "copysignf" || Name == "copysignl" ||
-        Name == "fabs"  || Name == "fabsf"  || Name == "fabsl" ||
-        Name == "fmin"  || Name == "fminf"  || Name == "fminl" ||
-        Name == "fmax"  || Name == "fmaxf"  || Name == "fmaxl" ||
-        Name == "sin"   || Name == "sinf"   || Name == "sinl"  ||
-        Name == "cos"   || Name == "cosf"   || Name == "cosl"  ||
-        Name == "tan"   || Name == "tanf"   || Name == "tanl"  ||
-        Name == "asin"  || Name == "asinf"  || Name == "asinl" ||
-        Name == "acos"  || Name == "acosf"  || Name == "acosl" ||
-        Name == "atan"  || Name == "atanf"  || Name == "atanl" ||
-        Name == "atan2" || Name == "atan2f" || Name == "atan2l"||
-        Name == "sinh"  || Name == "sinhf"  || Name == "sinhl" ||
-        Name == "cosh"  || Name == "coshf"  || Name == "coshl" ||
-        Name == "tanh"  || Name == "tanhf"  || Name == "tanhl" ||
-        Name == "sqrt"  || Name == "sqrtf"  || Name == "sqrtl" ||
-        Name == "exp10"  || Name == "exp10l"  || Name == "exp10f")
+    if (Name == "copysign" || Name == "copysignf" || Name == "copysignl" || Name == "copysignf128" ||
+        Name == "fabs"  || Name == "fabsf"  || Name == "fabsl"  || Name  == "fabsf128" ||
+        Name == "fmin"  || Name == "fminf"  || Name == "fminl"  || Name  == "fminf128" ||
+        Name == "fmax"  || Name == "fmaxf"  || Name == "fmaxl"  || Name  == "fmaxf128" ||
+        Name == "sin"   || Name == "sinf"   || Name == "sinl"   || Name  == "sinf128"  ||
+        Name == "cos"   || Name == "cosf"   || Name == "cosl"   || Name  == "cosf128"  ||
+        Name == "tan"   || Name == "tanf"   || Name == "tanl"   || Name  == "tanf128"  ||
+        Name == "asin"  || Name == "asinf"  || Name == "asinl"  || Name  == "asinf128" ||
+        Name == "acos"  || Name == "acosf"  || Name == "acosl"  || Name  == "acosf128" ||
+        Name == "atan"  || Name == "atanf"  || Name == "atanl"  || Name  == "atanf128" ||
+        Name == "atan2" || Name == "atan2f" || Name == "atan2l" || Name  == "atan2f128"||
+        Name == "sinh"  || Name == "sinhf"  || Name == "sinhl"  || Name  == "sinhf128" ||
+        Name == "cosh"  || Name == "coshf"  || Name == "coshl"  || Name  == "coshf128" ||
+        Name == "tanh"  || Name == "tanhf"  || Name == "tanhl"  || Name  == "tanhf128" ||
+        Name == "sqrt"  || Name == "sqrtf"  || Name == "sqrtl"  || Name  == "sqrtf128" ||
+        Name == "exp10" || Name == "exp10l" || Name == "exp10f" || Name  == "exp10f128")
       return false;
     // clang-format on
     // These are all likely to be optimized into something smaller.
-    if (Name == "pow" || Name == "powf" || Name == "powl" || Name == "exp2" ||
-        Name == "exp2l" || Name == "exp2f" || Name == "floor" ||
+    if (Name == "pow" || Name == "powf" || Name == "powl" ||
+        Name == "powf128" || Name == "exp2" || Name == "exp2f" ||
+        Name == "exp2l" || Name == "exp2f128" || Name == "floor" ||
         Name == "floorf" || Name == "ceil" || Name == "round" ||
         Name == "ffs" || Name == "ffsl" || Name == "abs" || Name == "labs" ||
         Name == "llabs")
diff --git a/llvm/include/llvm/IR/RuntimeLibcalls.def b/llvm/include/llvm/IR/RuntimeLibcalls.def
index cd8e9b598044c..e5f9ee3b2f384 100644
--- a/llvm/include/llvm/IR/RuntimeLibcalls.def
+++ b/llvm/include/llvm/IR/RuntimeLibcalls.def
@@ -89,7 +89,8 @@ HANDLE_LIBCALL(CTPOP_I32, "__popcountsi2")
 HANDLE_LIBCALL(CTPOP_I64, "__popcountdi2")
 HANDLE_LIBCALL(CTPOP_I128, "__popcountti2")
 
-// Floating-point
+// Floating-point. Note that new fp128 math routines should also be added to
+// setF128LibcallFormat in RuntimeLibcalls.cpp.
 HANDLE_LIBCALL(ADD_F32, "__addsf3")
 HANDLE_LIBCALL(ADD_F64, "__adddf3")
 HANDLE_LIBCALL(ADD_F80, "__addxf3")
@@ -113,12 +114,12 @@ HANDLE_LIBCALL(DIV_PPCF128, "__gcc_qdiv")
 HANDLE_LIBCALL(REM_F32, "fmodf")
 HANDLE_LIBCALL(REM_F64, "fmod")
 HANDLE_LIBCALL(REM_F80, "fmodl")
-HANDLE_LIBCALL(REM_F128, "fmodl")
+HANDLE_LIBCALL(REM_F128, "fmodf128")
 HANDLE_LIBCALL(REM_PPCF128, "fmodl")
 HANDLE_LIBCALL(FMA_F32, "fmaf")
 HANDLE_LIBCALL(FMA_F64, "fma")
 HANDLE_LIBCALL(FMA_F80, "fmal")
-HANDLE_LIBCALL(FMA_F128, "fmal")
+HANDLE_LIBCALL(FMA_F128, "fmaf128")
 HANDLE_LIBCALL(FMA_PPCF128, "fmal")
 HANDLE_LIBCALL(POWI_F32, "__powisf2")
 HANDLE_LIBCALL(POWI_F64, "__powidf2")
@@ -128,117 +129,117 @@ HANDLE_LIBCALL(POWI_PPCF128, "__powitf2")
 HANDLE_LIBCALL(SQRT_F32, "sqrtf")
 HANDLE_LIBCALL(SQRT_F64, "sqrt")
 HANDLE_LIBCALL(SQRT_F80, "sqrtl")
-HANDLE_LIBCALL(SQRT_F128, "sqrtl")
+HANDLE_LIBCALL(SQRT_F128, "sqrtf128")
 HANDLE_LIBCALL(SQRT_PPCF128, "sqrtl")
 HANDLE_LIBCALL(CBRT_F32, "cbrtf")
 HANDLE_LIBCALL(CBRT_F64, "cbrt")
 HANDLE_LIBCALL(CBRT_F80, "cbrtl")
-HANDLE_LIBCALL(CBRT_F128, "cbrtl")
+HANDLE_LIBCALL(CBRT_F128, "cbrtf128")
 HANDLE_LIBCALL(CBRT_PPCF128, "cbrtl")
 HANDLE_LIBCALL(LOG_F32, "logf")
 HANDLE_LIBCALL(LOG_F64, "log")
 HANDLE_LIBCALL(LOG_F80, "logl")
-HANDLE_LIBCALL(LOG_F128, "logl")
+HANDLE_LIBCALL(LOG_F128, "logf128")
 HANDLE_LIBCALL(LOG_PPCF128, "logl")
 HANDLE_LIBCALL(LOG_FINITE_F32, "__logf_finite")
 HANDLE_LIBCALL(LOG_FINITE_F64, "__log_finite")
 HANDLE_LIBCALL(LOG_FINITE_F80, "__logl_finite")
-HANDLE_LIBCALL(LOG_FINITE_F128, "__logl_finite")
+HANDLE_LIBCALL(LOG_FINITE_F128, "__logf128_finite")
 HANDLE_LIBCALL(LOG_FINITE_PPCF128, "__logl_finite")
 HANDLE_LIBCALL(LOG2_F32, "log2f")
 HANDLE_LIBCALL(LOG2_F64, "log2")
 HANDLE_LIBCALL(LOG2_F80, "log2l")
-HANDLE_LIBCALL(LOG2_F128, "log2l")
+HANDLE_LIBCALL(LOG2_F128, "log2f128")
 HANDLE_LIBCALL(LOG2_PPCF128, "log2l")
 HANDLE_LIBCALL(LOG2_FINITE_F32, "__log2f_finite")
 HANDLE_LIBCALL(LOG2_FINITE_F64, "__log2_finite")
 HANDLE_LIBCALL(LOG2_FINITE_F80, "__log2l_finite")
-HANDLE_LIBCALL(LOG2_FINITE_F128, "__log2l_finite")
+HANDLE_LIBCALL(LOG2_FINITE_F128, "__log2f128_finite")
 HANDLE_LIBCALL(LOG2_FINITE_PPCF128, "__log2l_finite")
 HANDLE_LIBCALL(LOG10_F32, "log10f")
 HANDLE_LIBCALL(LOG10_F64, "log10")
 HANDLE_LIBCALL(LOG10_F80, "log10l")
-HANDLE_LIBCALL(LOG10_F128, "log10l")
+HANDLE_LIBCALL(LOG10_F128, "log10f128")
 HANDLE_LIBCALL(LOG10_PPCF128, "log10l")
 HANDLE_LIBCALL(LOG10_FINITE_F32, "__log10f_finite")
 HANDLE_LIBCALL(LOG10_FINITE_F64, "__log10_finite")
 HANDLE_LIBCALL(LOG10_FINITE_F80, "__log10l_finite")
-HANDLE_LIBCALL(LOG10_FINITE_F128, "__log10l_finite")
+HANDLE_LIBCALL(LOG10_FINITE_F128, "__log10f128_finite")
 HANDLE_LIBCALL(LOG10_FINITE_PPCF128, "__log10l_finite")
 HANDLE_LIBCALL(EXP_F32, "expf")
 HANDLE_LIBCALL(EXP_F64, "exp")
 HANDLE_LIBCALL(EXP_F80, "expl")
-HANDLE_LIBCALL(EXP_F128, "expl")
+HANDLE_LIBCALL(EXP_F128, "expf128")
 HANDLE_LIBCALL(EXP_PPCF128, "expl")
 HANDLE_LIBCALL(EXP_FINITE_F32, "__expf_finite")
 HANDLE_LIBCALL(EXP_FINITE_F64, "__exp_finite")
 HANDLE_LIBCALL(EXP_FINITE_F80, "__expl_finite")
-HANDLE_LIBCALL(EXP_FINITE_F128, "__expl_finite")
+HANDLE_LIBCALL(EXP_FINITE_F128, "__expf128_finite")
 HANDLE_LIBCALL(EXP_FINITE_PPCF128, "__expl_finite")
 HANDLE_LIBCALL(EXP2_F32, "exp2f")
 HANDLE_LIBCALL(EXP2_F64, "exp2")
 HANDLE_LIBCALL(EXP2_F80, "exp2l")
-HANDLE_LIBCALL(EXP2_F128, "exp2l")
+HANDLE_LIBCALL(EXP2_F128, "exp2f128")
 HANDLE_LIBCALL(EXP2_PPCF128, "exp2l")
 HANDLE_LIBCALL(EXP2_FINITE_F32, "__exp2f_finite")
 HANDLE_LIBCALL(EXP2_FINITE_F64, "__exp2_finite")
 HANDLE_LIBCALL(EXP2_FINITE_F80, "__exp2l_finite")
-HANDLE_LIBCALL(EXP2_FINITE_F128, "__exp2l_finite")
+HANDLE_LIBCALL(EXP2_FINITE_F128, "__exp2f128_finite")
 HANDLE_LIBCALL(EXP2_FINITE_PPCF128, "__exp2l_finite")
 HANDLE_LIBCALL(EXP10_F32, "exp10f")
 HANDLE_LIBCALL(EXP10_F64, "exp10")
 HANDLE_LIBCALL(EXP10_F80, "exp10l")
-HANDLE_LIBCALL(EXP10_F128, "exp10l")
+HANDLE_LIBCALL(EXP10_F128, "exp10f128")
 HANDLE_LIBCALL(EXP10_PPCF128, "exp10l")
 HANDLE_LIBCALL(SIN_F32, "sinf")
 HANDLE_LIBCALL(SIN_F64, "sin")
 HANDLE_LIBCALL(SIN_F80, "sinl")
-HANDLE_LIBCALL(SIN_F128, "sinl")
+HANDLE_LIBCALL(SIN_F128, "sinf128")
 HANDLE_LIBCALL(SIN_PPCF128, "sinl")
 HANDLE_LIBCALL(COS_F32, "cosf")
 HANDLE_LIBCALL(COS_F64, "cos")
 HANDLE_LIBCALL(COS_F80, "cosl")
-HANDLE_LIBCALL(COS_F128, "cosl")
+HANDLE_LIBCALL(COS_F128, "cosf128")
 HANDLE_LIBCALL(COS_PPCF128, "cosl")
 HANDLE_LIBCALL(TAN_F32, "tanf")
 HANDLE_LIBCALL(TAN_F64, "tan")
 HANDLE_LIBCALL(TAN_F80, "tanl")
-HANDLE_LIBCALL(TAN_F128,"tanl")
+HANDLE_LIBCALL(TAN_F128,"tanf128")
 HANDLE_LIBCALL(TAN_PPCF128, "tanl")
 HANDLE_LIBCALL(SINH_F32, "sinhf")
 HANDLE_LIBCALL(SINH_F64, "sinh")
 HANDLE_LIBCALL(SINH_F80, "sinhl")
-HANDLE_LIBCALL(SINH_F128, "sinhl")
+HANDLE_LIBCALL(SINH_F128, "sinhf128")
 HANDLE_LIBCALL(SINH_PPCF128, "sinhl")
 HANDLE_LIBCALL(COSH_F32, "coshf")
 HANDLE_LIBCALL(COSH_F64, "cosh")
 HANDLE_LIBCALL(COSH_F80, "coshl")
-HANDLE_LIBCALL(COSH_F128, "coshl")
+HANDLE_LIBCALL(COSH_F128, "coshf128")
 HANDLE_LIBCALL(COSH_PPCF128, "coshl")
 HANDLE_LIBCALL(TANH_F32, "tanhf")
 HANDLE_LIBCALL(TANH_F64, "tanh")
 HANDLE_LIBCALL(TANH_F80, "tanhl")
-HANDLE_LIBCALL(TANH_F128,"tanhl")
+HANDLE_LIBCALL(TANH_F128,"tanhf128")
 HANDLE_LIBCALL(TANH_PPCF128, "tanhl")
 HANDLE_LIBCALL(ASIN_F32, "asinf")
 HANDLE_LIBCALL(ASIN_F64, "asin")
 HANDLE_LIBCALL(ASIN_F80, "asinl")
-HANDLE_LIBCALL(ASIN_F128, "asinl")
+HANDLE_LIBCALL(ASIN_F128, "asinf128")
 HANDLE_LIBCALL(ASIN_PPCF128, "asinl")
 HANDLE_LIBCALL(ACOS_F32, "acosf")
 HANDLE_LIBCALL(ACOS_F64, "acos")
 HANDLE_LIBCALL(ACOS_F80, "acosl")
-HANDLE_LIBCALL(ACOS_F128, "acosl")
+HANDLE_LIBCALL(ACOS_F128, "acosf128")
 HANDLE_LIBCALL(ACOS_PPCF128, "acosl")
 HANDLE_LIBCALL(ATAN_F32, "atanf")
 HANDLE_LIBCALL(ATAN_F64, "atan")
 HANDLE_LIBCALL(ATAN_F80, "atanl")
-HANDLE_LIBCALL(ATAN_F128,"atanl")
+HANDLE_LIBCALL(ATAN_F128,"atanf128")
 HANDLE_LIBCALL(ATAN_PPCF128, "atanl")
 HANDLE_LIBCALL(ATAN2_F32, "atan2f")
 HANDLE_LIBCALL(ATAN2_F64, "atan2")
 HANDLE_LIBCALL(ATAN2_F80, "atan2l")
-HANDLE_LIBCALL(ATAN2_F128,"atan2l")
+HANDLE_LIBCALL(ATAN2_F128,"atan2f128")
 HANDLE_LIBCALL(ATAN2_PPCF128, "atan2l")
 HANDLE_LIBCALL(SINCOS_F32, nullptr)
 HANDLE_LIBCALL(SINCOS_F64, nullptr)
@@ -250,122 +251,122 @@ HANDLE_LIBCALL(SINCOS_STRET_F64, nullptr)
 HANDLE_LIBCALL(POW_F32, "powf")
 HANDLE_LIBCALL(POW_F64, "pow")
 HANDLE_LIBCALL(POW_F80, "powl")
-HANDLE_LIBCALL(POW_F128, "powl")
+HANDLE_LIBCALL(POW_F128, "powf128")
 HANDLE_LIBCALL(POW_PPCF128, "powl")
 HANDLE_LIBCALL(POW_FINITE_F32, "__powf_finite")
 HANDLE_LIBCALL(POW_FINITE_F64, "__pow_finite")
 HANDLE_LIBCALL(POW_FINITE_F80, "__powl_finite")
-HANDLE_LIBCALL(POW_FINITE_F128, "__powl_finite")
+HANDLE_LIBCALL(POW_FINITE_F128, "__powf128_finite")
 HANDLE_LIBCALL(POW_FINITE_PPCF128, "__powl_finite")
 HANDLE_LIBCALL(CEIL_F32, "ceilf")
 HANDLE_LIBCALL(CEIL_F64, "ceil")
 HANDLE_LIBCALL(CEIL_F80, "ceill")
-HANDLE_LIBCALL(CEIL_F128, "ceill")
+HANDLE_LIBCALL(CEIL_F128, "ceilf128")
 HANDLE_LIBCALL(CEIL_PPCF128, "ceill")
 HANDLE_LIBCALL(TRUNC_F32, "truncf")
 HANDLE_LIBCALL(TRUNC_F64, "trunc")
 HANDLE_LIBCALL(TRUNC_F80, "truncl")
-HANDLE_LIBCALL(TRUNC_F128, "truncl")
+HANDLE_LIBCALL(TRUNC_F128, "truncf128")
 HANDLE_LIBCALL(TRUNC_PPCF128, "truncl")
 HANDLE_LIBCALL(RINT_F32, "rintf")
 HANDLE_LIBCALL(RINT_F64, "rint")
 HANDLE_LIBCALL(RINT_F80, "rintl")
-HANDLE_LIBCALL(RINT_F128, "rintl")
+HANDLE_LIBCALL(RINT_F128, "rintf128")
 HANDLE_LIBCALL(RINT_PPCF128, "rintl")
 HANDLE_LIBCALL(NEARBYINT_F32, "nearbyintf")
 HANDLE_LIBCALL(NEARBYINT_F64, "nearbyint")
 HANDLE_LIBCALL(NEARBYINT_F80, "nearbyintl")
-HANDLE_LIBCALL(NEARBYINT_F128, "nearbyintl")
+HANDLE_LIBCALL(NEARBYINT_F128, "nearbyintf128")
 HANDLE_LIBCALL(NEARBYINT_PPCF128, "nearbyintl")
 HANDLE_LIBCALL(ROUND_F32, "roundf")
 HANDLE_LIBCALL(ROUND_F64, "round")
 HANDLE_LIBCALL(ROUND_F80, "roundl")
-HANDLE_LIBCALL(ROUND_F128, "roundl")
+HANDLE_LIBCALL(ROUND_F128, "roundf128")
 HANDLE_LIBCALL(ROUND_PPCF128, "roundl")
 HANDLE_LIBCALL(ROUNDEVEN_F32, "roundevenf")
 HANDLE_LIBCALL(ROUNDEVEN_F64, "roundeven")
 HANDLE_LIBCALL(ROUNDEVEN_F80, "roundevenl")
-HANDLE_LIBCALL(ROUNDEVEN_F128, "roundevenl")
+HANDLE_LIBCALL(ROUNDEVEN_F128, "roundevenf128")
 HANDLE_LIBCALL(ROUNDEVEN_PPCF128, "roundevenl")
 HANDLE_LIBCALL(FLOOR_F32, "floorf")
 HANDLE_LIBCALL(FLOOR_F64, "floor")
 HANDLE_LIBCALL(FLOOR_F80, "floorl")
-HANDLE_LIBCALL(FLOOR_F128, "floorl")
+HANDLE_LIBCALL(FLOOR_F128, "floorf128")
 HANDLE_LIBCALL(FLOOR_PPCF128, "floorl")
 HANDLE_LIBCALL(COPYSIGN_F32, "copysignf")
 HANDLE_LIBCALL(COPYSIGN_F64, "copysign")
 HANDLE_LIBCALL(COPYSIGN_F80, "copysignl")
-HANDLE_LIBCALL(COPYSIGN_F128, "copysignl")
+HANDLE_LIBCALL(COPYSIGN_F128, "copysignf128")
 HANDLE_LIBCALL(COPYSIGN_PPCF128, "copysignl")
 HANDLE_LIBCALL(FMIN_F32, "fminf")
 HANDLE_LIBCALL(FMIN_F64, "fmin")
 HANDLE_LIBCALL(FMIN_F80, "fminl")
-HANDLE_LIBCALL(FMIN_F128, "fminl")
+HANDLE_LIBCALL(FMIN_F128, "fminf128")
 HANDLE_LIBCALL(FMIN_PPCF128, "fminl")
 HANDLE_LIBCALL(FMAX_F32, "fmaxf")
 HANDLE_LIBCALL(FMAX_F64, "fmax")
 HANDLE_LIBCALL(FMAX_F80, "fmaxl")
-HANDLE_LIBCALL(FMAX_F128, "fmaxl")
+HANDLE_LIBCALL(FMAX_F128, "fmaxf128")
 HANDLE_LIBCALL(FMAX_PPCF128, "fmaxl")
 HANDLE_LIBCALL(FMINIMUM_F32, "fminimumf")
 HANDLE_LIBCALL(FMINIMUM_F64, "fminimum")
 HANDLE_LIBCALL(FMINIMUM_F80, "fminimuml")
-HANDLE_LIBCALL(FMINIMUM_F128, "fminimuml")
+HANDLE_LIBCALL(FMINIMUM_F128, "fminimumf128")
 HANDLE_LIBCALL(FMINIMUM_PPCF128, "fminimuml")
 HANDLE_LIBCALL(FMAXIMUM_F32, "fmaximumf")
 HANDLE_LIBCALL(FMAXIMUM_F64, "fmaximum")
 HANDLE_LIBCALL(FMAXIMUM_F80, "fmaximuml")
-HANDLE_LIBCALL(FMAXIMUM_F128, "fmaximuml")
+HANDLE_LIBCALL(FMAXIMUM_F128, "fmaximumf128")
 HANDLE_LIBCALL(FMAXIMUM_PPCF128, "fmaximum_numl")
 HANDLE_LIBCALL(FMINIMUMNUM_F32, "fminimum_numf")
 HANDLE_LIBCALL(FMINIMUMNUM_F64, "fminimum_num")
 HANDLE_LIBCALL(FMINIMUMNUM_F80, "fminimum_numl")
-HANDLE_LIBCALL(FMINIMUMNUM_F128, "fminimum_numl")
+HANDLE_LIBCALL(FMINIMUMNUM_F128, "fminimum_numf128")
 HANDLE_LIBCALL(FMINIMUMNUM_PPCF128, "fminimum_numl")
 HANDLE_LIBCALL(FMAXIMUMNUM_F32, "fmaximum_numf")
 HANDLE_LIBCALL(FMAXIMUMNUM_F64, "fmaximum_num")
 HANDLE_LIBCALL(FMAXIMUMNUM_F80, "fmaximum_numl")
-HANDLE_LIBCALL(FMAXIMUMNUM_F128, "fmaximum_numl")
+HANDLE_LIBCALL(FMAXIMUMNUM_F128, "fmaximum_numf128")
 HANDLE_LIBCALL(FMAXIMUMNUM_PPCF128, "fmaximum_numl")
 HANDLE_LIBCALL(LROUND_F32, "lroundf")
 HANDLE_LIBCALL(LROUND_F64, "lround")
 HANDLE_LIBCALL(LROUND_F80, "lroundl")
-HANDLE_LIBCALL(LROUND_F128, "lroundl")
+HANDLE_LIBCALL(LROUND_F128, "lroundf128")
 HANDLE_LIBCALL(LROUND_PPCF128, "lroundl")
 HANDLE_LIBCALL(LLROUND_F32, "llroundf")
 HANDLE_LIBCALL(LLROUND_F64, "llround")
 HANDLE_LIBCALL(LLROUND_F80, "llroundl")
-HANDLE_LIBCALL(LLROUND_F128, "llroundl")
+HANDLE_LIBCALL(LLROUND_F128, "llroundf128")
 HANDLE_LIBCALL(LLROUND_PPCF128, "llroundl")
 HANDLE_LIBCALL(LRINT_F32, "lrintf")
 HANDLE_LIBCALL(LRINT_F64, "lrint")
 HANDLE_LIBCALL(LRINT_F80, "lrintl")
-HANDLE_LIBCALL(LRINT_F128, "lrintl")
+HANDLE_LIBCALL(LRINT_F128, "lrintf128")
 HANDLE_LIBCALL(LRINT_PPCF128, "lrintl")
 HANDLE_LIBCALL(LLRINT_F32, "llrintf")
 HANDLE_LIBCALL(LLRINT_F64, "llrint")
 HANDLE_LIBCALL(LLRINT_F80, "llrintl")
-HANDLE_LIBCALL(LLRINT_F128, "llrintl")
+HANDLE_LIBCALL(LLRINT_F128, "llrintf128")
 HANDLE_LIBCALL(LLRINT_PPCF128, "llrintl")
 HANDLE_LIBCALL(LDEXP_F32, "ldexpf")
 HANDLE_LIBCALL(LDEXP_F64, "ldexp")
 HANDLE_LIBCALL(LDEXP_F80, "ldexpl")
-HANDLE_LIBCALL(LDEXP_F128, "ldexpl")
+HANDLE_LIBCALL(LDEXP_F128, "ldexpf128")
 HANDLE_LIBCALL(LDEXP_PPCF128, "ldexpl")
 HANDLE_LIBCALL(FREXP_F32, "frexpf")
 HANDLE_LIBCALL(FREXP_F64, "frexp")
 HANDLE_LIBCALL(FREXP_F80, "frexpl")
-HANDLE_LIBCALL(FREXP_F128, "frexpl")
+HANDLE_LIBCALL(FREXP_F128, "frexpf128")
 HANDLE_LIBCALL(FREXP_PPCF128, "frexpl")
 HANDLE_LIBCALL(SINCOSPI_F32, "sincospif")
 HANDLE_LIBCALL(SINCOSPI_F64, "sincospi")
 HANDLE_LIBCALL(SINCOSPI_F80, "sincospil")
-HANDLE_LIBCALL(SINCOSPI_F128, "sincospil")
+HANDLE_LIBCALL(SINCOSPI_F128, "sincospif128")
 HANDLE_LIBCALL(SINCOSPI_PPCF128, "sincospil")
 HANDLE_LIBCALL(MODF_F32, "modff")
 HANDLE_LIBCALL(MODF_F64, "modf")
 HANDLE_LIBCALL(MODF_F80, "modfl")
-HANDLE_LIBCALL(MODF_F128, "modfl")
+HANDLE_LIBCALL(MODF_F128, "modff128")
 HANDLE_LIBCALL(MODF_PPCF128, "modfl")
 
 // Floating point environment
diff --git a/llvm/include/llvm/IR/RuntimeLibcalls.h b/llvm/include/llvm/IR/RuntimeLibcalls.h
index b3648f5a31e2a..09da78720b988 100644
--- a/llvm/include/llvm/IR/RuntimeLibcalls.h
+++ b/llvm/include/llvm/IR/RuntimeLibcalls.h
@@ -20,6 +20,17 @@
 #include "llvm/TargetParser/Triple.h"
 
 namespace llvm {
+
+/// Library names to use for `fp128` libcalls.
+enum class F128LibcallFormat {
+  /// C23 `*f128` lowering, e.g. `sinf128`
+  Default = 0,
+  /// `long double` *l` lowering, e.g. `sinl`.
+  LongDouble = 1,
+  // If needed, this could be extended with an option for `q` suffixes from
+  // libquadmath.
+};
+
 namespace RTLIB {
 
 /// RTLIB::Libcall enum - This enum defines all of the runtime library calls
@@ -97,6 +108,13 @@ struct RuntimeLibcallsInfo {
   /// Set default libcall names. If a target wants to opt-out of a libcall it
   /// should be placed here.
   void initLibcalls(const Triple &TT);
+
+  /// Set a specific lowering convention for `fp128` math libcalls.
+  ///
+  /// By default, `fp128` math functions get lowered to the C23 `sinf128`-
+  /// style symbols. This allows overriding with `sinl`-style symbols on
+  /// platforms where `long double` is known to be identical to _Float128.
+  void setF128LibcallFormat(F128LibcallFormat Format);
 };
 
 } // namespace RTLIB
diff --git a/llvm/include/llvm/TargetParser/Triple.h b/llvm/include/llvm/TargetParser/Triple.h
index fb6bbc0163701..f3976704f4c7c 100644
--- a/llvm/include/llvm/TargetParser/Triple.h
+++ b/llvm/include/llvm/TargetParser/Triple.h
@@ -262,13 +262,13 @@ class Triple {
     EABIHF,
     Android,
     Musl,
-    MuslABIN32,
-    MuslABI64,
-    MuslEABI,
-    MuslEABIHF,
-    MuslF32,
-    MuslSF,
-    MuslX32,
+    MuslABIN32, // MIPS N32 ABI
+    MuslABI64,  // MIPS N64 ABI
+    MuslEABI,   // Arm32 EABI
+    MuslEABIHF, // Arm32 EABI + HF
+    MuslF32,    // LoongArch ILP32F/LP64F
+    MuslSF,     // LoongArch ILP32S/LP64S
+    MuslX32,    // Musl using 32-bit ABI on x86_64
     LLVM,
 
     MSVC,
@@ -1231,6 +1231,12 @@ class Triple {
   /// or an invalid version tuple if this triple doesn't have one.
   VersionTuple getMinimumSupportedOSVersion() const;
 
+  /// Check whether (1) f128 is the same format as `long double`, and (2)
+  /// `*f128` symbols are likely unavailable. In other words, platforms for
+  /// which this returns true may safely use sqrtl instead of sqrtf128 and
+  /// should do so because sqrtf128 would probably error at link time.
+  bool shouldLowerf128AsLongDouble() const;
+
   /// @}
   /// @name Static helpers for IDs.
   /// @{
diff --git a/llvm/lib/IR/RuntimeLibcalls.cpp b/llvm/lib/IR/RuntimeLibcalls.cpp
index 90c3bf0db0236..e00f4346984ae 100644
--- a/llvm/lib/IR/RuntimeLibcalls.cpp
+++ b/llvm/lib/IR/RuntimeLibcalls.cpp
@@ -25,54 +25,6 @@ void RuntimeLibcallsInfo::initLibcalls(const Triple &TT) {
   for (int LC = 0; LC < RTLIB::UNKNOWN_LIBCALL; ++LC)
     setLibcallCallingConv((RTLIB::Libcall)LC, CallingConv::C);
 
-  // Use the f128 variants of math functions on x86_64
-  if (TT.getArch() == Triple::ArchType::x86_64 && TT.isGNUEnvironment()) {
-    setLibcallName(RTLIB::REM_F128, "fmodf128");
-    setLibcallName(RTLIB::FMA_F128, "fmaf128");
-    setLibcallName(RTLIB::SQRT_F128, "sqrtf128");
-    setLibcallName(RTLIB::CBRT_F128, "cbrtf128");
-    setLibcallName(RTLIB::LOG_F128, "logf128");
-    setLibcallName(RTLIB::LOG_FINITE_F128, "__logf128_finite");
-    setLibcallName(RTLIB::LOG2_F128, "log2f128");
-    setLibcallName(RTLIB::LOG2_FINITE_F128, "__log2f128_finite");
-    setLibcallName(RTLIB::LOG10_F128, "log10f128");
-    setLibcallName(RTLIB::LOG10_FINITE_F128, "__log10f128_finite");
-    setLibcallName(RTLIB::EXP_F128, "expf128");
-    setLibcallName(RTLIB::EXP_FINITE_F128, "__expf128_finite");
-    setLibcallName(RTLIB::EXP2_F128, "exp2f128");
-    setLibcallName(RTLIB::EXP2_FINITE_F128, "__exp2f128_finite");
-    setLibcallName(RTLIB::EXP10_F128, "exp10f128");
-    setLibcallName(RTLIB::SIN_F128, "sinf128");
-    setLibcallName(RTLIB::COS_F1...
[truncated]

[tgross35]

@arsenm would you mind taking a look at this? It should fix most of the problems with unsoundly lowering f128 to long double calls. No control via target string, I'll do that separately.

Does

llvm-project/llvm/lib/CodeGen/IntrinsicLowering.cpp

Lines 386 to 445 in 50aacb9

	case Intrinsic::sqrt: {
	ReplaceFPIntrinsicWithCall(CI, "sqrtf", "sqrt", "sqrtl");
	break;
	}
	case Intrinsic::log: {
	ReplaceFPIntrinsicWithCall(CI, "logf", "log", "logl");
	break;
	}
	case Intrinsic::log2: {
	ReplaceFPIntrinsicWithCall(CI, "log2f", "log2", "log2l");
	break;
	}
	case Intrinsic::log10: {
	ReplaceFPIntrinsicWithCall(CI, "log10f", "log10", "log10l");
	break;
	}
	case Intrinsic::exp: {
	ReplaceFPIntrinsicWithCall(CI, "expf", "exp", "expl");
	break;
	}
	case Intrinsic::exp2: {
	ReplaceFPIntrinsicWithCall(CI, "exp2f", "exp2", "exp2l");
	break;
	}
	case Intrinsic::pow: {
	ReplaceFPIntrinsicWithCall(CI, "powf", "pow", "powl");
	break;
	}
	case Intrinsic::sin: {
	ReplaceFPIntrinsicWithCall(CI, "sinf", "sin", "sinl");
	break;
	}
	case Intrinsic::cos: {
	ReplaceFPIntrinsicWithCall(CI, "cosf", "cos", "cosl");
	break;
	}
	case Intrinsic::floor: {
	ReplaceFPIntrinsicWithCall(CI, "floorf", "floor", "floorl");
	break;
	}
	case Intrinsic::ceil: {
	ReplaceFPIntrinsicWithCall(CI, "ceilf", "ceil", "ceill");
	break;
	}
	case Intrinsic::trunc: {
	ReplaceFPIntrinsicWithCall(CI, "truncf", "trunc", "truncl");
	break;
	}
	case Intrinsic::round: {
	ReplaceFPIntrinsicWithCall(CI, "roundf", "round", "roundl");
	break;
	}
	case Intrinsic::roundeven: {
	ReplaceFPIntrinsicWithCall(CI, "roundevenf", "roundeven", "roundevenl");
	break;
	}
	case Intrinsic::copysign: {
	ReplaceFPIntrinsicWithCall(CI, "copysignf", "copysign", "copysignl");
	break;
	}

need to be updated somehow?

[uweigand]

Why are we seeing all these changes in the SystemZ tests? On SystemZ we should continue to use the *l names (I thought this is what the shouldLowerf128AsLongDouble logic was supposed to achieve?)

[tgross35]

The logic I have with that function is that if available, *f128 should be preferred over *l if because that is unconditionally correct for fp128. *l would be incorrect in the unusual case that the math library was compiled with a different -mlong-double flag. On s390x (and others where l-d is f128) the *f128 versions are only used on glibc targets, which this test is showing, because glibc aliases the *f128 and *l functions when it knows long double is binary128.

The downside of course is that compiling for glibc but bringing your own libm will need to add aliases as well. This seems acceptable to me because it should be the library's responsibility to indicate that it expects long double to be the same as _Float128 rather than anything else. (Also per @arsenm on Discord LLVM assumes that the libm from your libc is used, or at least has the same interfaces).

If you're fine with the change then I can update this tests showing that a musl target is unaffected. Or if you would prefer I can drop the isGNUEnvironment check and always use *l on s390x, since it's less likely to be a problem here.

[uweigand]

I'm a bit concerned about making this change by default, I don't see that this has any actual advantages on s390x (*l is always correct here anyway), and it might cause some surprises. For example, while current glibc version do indeed provide the *f128 aliases, that has not always been the case - e.g. RHEL 7 doesn't have them.

[beetrees]

This also covers MinGW *-pc-windows-gnu targets, which don't use glibc. The logic is still correct (as Windows needs to use the *f128 symbols), but swapping the order of this if statement and the if (isOSWindows() || isOSDarwin()) below will make the comment correct too.