Skip to content

[AArch64] Implement NEON vamin/vamax intrinsics #99041

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Sep 9, 2024
Merged

[AArch64] Implement NEON vamin/vamax intrinsics #99041

merged 4 commits into from
Sep 9, 2024

Conversation

momchil-velikov
Copy link
Collaborator

This patch implements the intrinsics of the form

floatNxM_t vamin[q]_fN(floatNxM_t vn, floatNxM_t vm);
floatNxM_t vamax[q]_fN(floatNxM_t vn, floatNxM_t vm);

as defined in ARM-software/acle#324

@llvmbot llvmbot added clang Clang issues not falling into any other category backend:AArch64 clang:frontend Language frontend issues, e.g. anything involving "Sema" clang:codegen IR generation bugs: mangling, exceptions, etc. llvm:ir labels Jul 16, 2024
@llvmbot
Copy link
Member

llvmbot commented Jul 16, 2024 •
edited
Loading

@llvm/pr-subscribers-backend-aarch64

@llvm/pr-subscribers-clang

Author: Momchil Velikov (momchil-velikov)

Changes

This patch implements the intrinsics of the form

floatNxM_t vamin[q]_fN(floatNxM_t vn, floatNxM_t vm);
floatNxM_t vamax[q]_fN(floatNxM_t vn, floatNxM_t vm);

as defined in ARM-software/acle#324

Full diff: https://github.com/llvm/llvm-project/pull/99041.diff

7 Files Affected:

  • (modified) clang/include/clang/Basic/arm_neon.td (+5)
  • (modified) clang/lib/CodeGen/CGBuiltin.cpp (+17)
  • (added) clang/test/CodeGen/aarch64-neon-faminmax-intrinsics.c (+112)
  • (modified) llvm/include/llvm/IR/IntrinsicsAArch64.td (+3)
  • (modified) llvm/lib/Target/AArch64/AArch64InstrFormats.td (+20)
  • (modified) llvm/lib/Target/AArch64/AArch64InstrInfo.td (+2-2)
  • (added) llvm/test/CodeGen/AArch64/neon-famin-famax.ll (+96)
diff --git a/clang/include/clang/Basic/arm_neon.td b/clang/include/clang/Basic/arm_neon.td
index 6390ba3f9fe5e..3746b3667ad99 100644
--- a/clang/include/clang/Basic/arm_neon.td
+++ b/clang/include/clang/Basic/arm_neon.td
@@ -2096,3 +2096,8 @@ let ArchGuard = "defined(__aarch64__) || defined(__arm64ec__)", TargetGuard = "r
   def VLDAP1_LANE : WInst<"vldap1_lane", ".(c*!).I", "QUlQlUlldQdPlQPl">;
   def VSTL1_LANE  : WInst<"vstl1_lane", "v*(.!)I", "QUlQlUlldQdPlQPl">;
 }
+
+let ArchGuard = "defined(__aarch64__)", TargetGuard = "faminmax" in {
+  def FAMIN : WInst<"vamin", "...", "fhQdQfQh">;
+  def FAMAX : WInst<"vamax", "...", "fhQdQfQh">;
+}
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index a54fa7bf87aad..bb6094aa31805 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -13398,6 +13398,23 @@ Value *CodeGenFunction::EmitAArch64BuiltinExpr(unsigned BuiltinID,
     Int = Intrinsic::aarch64_neon_suqadd;
     return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "vuqadd");
   }
+
+  case NEON::BI__builtin_neon_vamin_f16:
+  case NEON::BI__builtin_neon_vaminq_f16:
+  case NEON::BI__builtin_neon_vamin_f32:
+  case NEON::BI__builtin_neon_vaminq_f32:
+  case NEON::BI__builtin_neon_vaminq_f64: {
+    Int = Intrinsic::aarch64_neon_famin;
+    return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "famin");
+  }
+  case NEON::BI__builtin_neon_vamax_f16:
+  case NEON::BI__builtin_neon_vamaxq_f16:
+  case NEON::BI__builtin_neon_vamax_f32:
+  case NEON::BI__builtin_neon_vamaxq_f32:
+  case NEON::BI__builtin_neon_vamaxq_f64: {
+    Int = Intrinsic::aarch64_neon_famax;
+    return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "famax");
+  }
   }
 }
 
diff --git a/clang/test/CodeGen/aarch64-neon-faminmax-intrinsics.c b/clang/test/CodeGen/aarch64-neon-faminmax-intrinsics.c
new file mode 100644
index 0000000000000..631e9738b85c5
--- /dev/null
+++ b/clang/test/CodeGen/aarch64-neon-faminmax-intrinsics.c
@@ -0,0 +1,112 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py UTC_ARGS: --version 5
+#include <arm_neon.h>
+
+// RUN: %clang_cc1 -triple aarch64-none-linux-gnu -target-feature +neon -target-feature +faminmax -O3 -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -triple aarch64-none-linux-gnu -target-feature +neon -target-feature +faminmax -S -O3 -Werror -Wall -o /dev/null %s
+
+// CHECK-LABEL: define dso_local <4 x half> @test_vamin_f16(
+// CHECK-SAME: <4 x half> noundef [[VN:%.*]], <4 x half> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMIN2_I:%.*]] = tail call <4 x half> @llvm.aarch64.neon.famin.v4f16(<4 x half> [[VN]], <4 x half> [[VM]])
+// CHECK-NEXT:    ret <4 x half> [[FAMIN2_I]]
+//
+float16x4_t test_vamin_f16(float16x4_t vn, float16x4_t vm) {
+  return vamin_f16(vn, vm);
+}
+
+// CHECK-LABEL: define dso_local <8 x half> @test_vaminq_f16(
+// CHECK-SAME: <8 x half> noundef [[VN:%.*]], <8 x half> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMIN2_I:%.*]] = tail call <8 x half> @llvm.aarch64.neon.famin.v8f16(<8 x half> [[VN]], <8 x half> [[VM]])
+// CHECK-NEXT:    ret <8 x half> [[FAMIN2_I]]
+//
+float16x8_t test_vaminq_f16(float16x8_t vn, float16x8_t vm) {
+  return vaminq_f16(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <2 x float> @test_vamin_f32(
+// CHECK-SAME: <2 x float> noundef [[VN:%.*]], <2 x float> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMIN2_I:%.*]] = tail call <2 x float> @llvm.aarch64.neon.famin.v2f32(<2 x float> [[VN]], <2 x float> [[VM]])
+// CHECK-NEXT:    ret <2 x float> [[FAMIN2_I]]
+//
+float32x2_t test_vamin_f32(float32x2_t vn, float32x2_t vm) {
+  return vamin_f32(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <4 x float> @test_vaminq_f32(
+// CHECK-SAME: <4 x float> noundef [[VN:%.*]], <4 x float> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMIN2_I:%.*]] = tail call <4 x float> @llvm.aarch64.neon.famin.v4f32(<4 x float> [[VN]], <4 x float> [[VM]])
+// CHECK-NEXT:    ret <4 x float> [[FAMIN2_I]]
+//
+float32x4_t test_vaminq_f32(float32x4_t vn, float32x4_t vm) {
+  return vaminq_f32(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <2 x double> @test_vaminq_f64(
+// CHECK-SAME: <2 x double> noundef [[VN:%.*]], <2 x double> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMIN2_I:%.*]] = tail call <2 x double> @llvm.aarch64.neon.famin.v2f64(<2 x double> [[VN]], <2 x double> [[VM]])
+// CHECK-NEXT:    ret <2 x double> [[FAMIN2_I]]
+//
+float64x2_t test_vaminq_f64(float64x2_t vn, float64x2_t vm) {
+  return vaminq_f64(vn, vm);
+}
+
+
+// CHECK-LABEL: define dso_local <4 x half> @test_vamax_f16(
+// CHECK-SAME: <4 x half> noundef [[VN:%.*]], <4 x half> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMAX2_I:%.*]] = tail call <4 x half> @llvm.aarch64.neon.famax.v4f16(<4 x half> [[VN]], <4 x half> [[VM]])
+// CHECK-NEXT:    ret <4 x half> [[FAMAX2_I]]
+//
+float16x4_t test_vamax_f16(float16x4_t vn, float16x4_t vm) {
+  return vamax_f16(vn, vm);
+}
+
+// CHECK-LABEL: define dso_local <8 x half> @test_vamaxq_f16(
+// CHECK-SAME: <8 x half> noundef [[VN:%.*]], <8 x half> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMAX2_I:%.*]] = tail call <8 x half> @llvm.aarch64.neon.famax.v8f16(<8 x half> [[VN]], <8 x half> [[VM]])
+// CHECK-NEXT:    ret <8 x half> [[FAMAX2_I]]
+//
+float16x8_t test_vamaxq_f16(float16x8_t vn, float16x8_t vm) {
+  return vamaxq_f16(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <2 x float> @test_vamax_f32(
+// CHECK-SAME: <2 x float> noundef [[VN:%.*]], <2 x float> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMAX2_I:%.*]] = tail call <2 x float> @llvm.aarch64.neon.famax.v2f32(<2 x float> [[VN]], <2 x float> [[VM]])
+// CHECK-NEXT:    ret <2 x float> [[FAMAX2_I]]
+//
+float32x2_t test_vamax_f32(float32x2_t vn, float32x2_t vm) {
+  return vamax_f32(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <4 x float> @test_vamaxq_f32(
+// CHECK-SAME: <4 x float> noundef [[VN:%.*]], <4 x float> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMAX2_I:%.*]] = tail call <4 x float> @llvm.aarch64.neon.famax.v4f32(<4 x float> [[VN]], <4 x float> [[VM]])
+// CHECK-NEXT:    ret <4 x float> [[FAMAX2_I]]
+//
+float32x4_t test_vamaxq_f32(float32x4_t vn, float32x4_t vm) {
+  return vamaxq_f32(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <2 x double> @test_vamaxq_f64(
+// CHECK-SAME: <2 x double> noundef [[VN:%.*]], <2 x double> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMAX2_I:%.*]] = tail call <2 x double> @llvm.aarch64.neon.famax.v2f64(<2 x double> [[VN]], <2 x double> [[VM]])
+// CHECK-NEXT:    ret <2 x double> [[FAMAX2_I]]
+//
+float64x2_t test_vamaxq_f64(float64x2_t vn, float64x2_t vm) {
+  return vamaxq_f64(vn, vm);
+}
diff --git a/llvm/include/llvm/IR/IntrinsicsAArch64.td b/llvm/include/llvm/IR/IntrinsicsAArch64.td
index 3735bf5222fce..f778973880703 100644
--- a/llvm/include/llvm/IR/IntrinsicsAArch64.td
+++ b/llvm/include/llvm/IR/IntrinsicsAArch64.td
@@ -3730,3 +3730,6 @@ def int_aarch64_sve_pmov_to_vector_lane_zeroing : SVE2_Pred_1VectorArg_Intrinsic
 def int_aarch64_sme_mopa_nonwide : SME_OuterProduct_Intrinsic;
 def int_aarch64_sme_mops_nonwide : SME_OuterProduct_Intrinsic;
 
+// Neon absolute maximum and minimum
+def int_aarch64_neon_famax :  AdvSIMD_2VectorArg_Intrinsic;
+def int_aarch64_neon_famin :  AdvSIMD_2VectorArg_Intrinsic;
diff --git a/llvm/lib/Target/AArch64/AArch64InstrFormats.td b/llvm/lib/Target/AArch64/AArch64InstrFormats.td
index e1ecc5a57dd26..3cbc41d943eba 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrFormats.td
+++ b/llvm/lib/Target/AArch64/AArch64InstrFormats.td
@@ -5985,6 +5985,26 @@ multiclass SIMDThreeSameVectorFP<bit U, bit S, bits<3> opc,
         [(set (v2f64 V128:$Rd), (OpNode (v2f64 V128:$Rn), (v2f64 V128:$Rm)))]>;
 }
 
+let mayRaiseFPException = 1, Uses = [FPCR] in
+multiclass SIMDThreeVectorFP<bit U, bit S, bits<3> opc,
+                             string asm, SDPatternOperator OpNode> {
+  def v4f16 : BaseSIMDThreeSameVector<0, U, {S,0b10}, {0b00,opc}, V64,
+                                      asm, ".4h",
+        [(set (v4f16 V64:$Rd), (OpNode (v4f16 V64:$Rn), (v4i16 V64:$Rm)))]>;
+  def v8f16 : BaseSIMDThreeSameVector<1, U, {S,0b10}, {0b00,opc}, V128,
+                                      asm, ".8h",
+        [(set (v8f16 V128:$Rd), (OpNode (v8f16 V128:$Rn), (v8i16 V128:$Rm)))]>;
+  def v2f32 : BaseSIMDThreeSameVector<0, U, {S,0b01}, {0b11,opc}, V64,
+                                      asm, ".2s",
+        [(set (v2f32 V64:$Rd), (OpNode (v2f32 V64:$Rn), (v2i32 V64:$Rm)))]>;
+  def v4f32 : BaseSIMDThreeSameVector<1, U, {S,0b01}, {0b11,opc}, V128,
+                                      asm, ".4s",
+        [(set (v4f32 V128:$Rd), (OpNode (v4f32 V128:$Rn), (v4i32 V128:$Rm)))]>;
+  def v2f64 : BaseSIMDThreeSameVector<1, U, {S,0b11}, {0b11,opc}, V128,
+                                      asm, ".2d",
+        [(set (v2f64 V128:$Rd), (OpNode (v2f64 V128:$Rn), (v2i64 V128:$Rm)))]>;
+}
+
 let mayRaiseFPException = 1, Uses = [FPCR] in
 multiclass SIMDThreeSameVectorFPCmp<bit U, bit S, bits<3> opc,
                                     string asm,
diff --git a/llvm/lib/Target/AArch64/AArch64InstrInfo.td b/llvm/lib/Target/AArch64/AArch64InstrInfo.td
index dd11f74882115..6a9b7256e31bd 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrInfo.td
+++ b/llvm/lib/Target/AArch64/AArch64InstrInfo.td
@@ -10016,8 +10016,8 @@ let Predicates = [HasFP8] in {
 } // End let Predicates = [HasFP8]
 
 let Predicates = [HasFAMINMAX] in {
- defm FAMAX : SIMDThreeSameVectorFP<0b0, 0b1, 0b011, "famax", null_frag>;
- defm FAMIN : SIMDThreeSameVectorFP<0b1, 0b1, 0b011, "famin", null_frag>;
+  defm FAMAX : SIMDThreeSameVectorFP<0b0, 0b1, 0b011, "famax", int_aarch64_neon_famax>;
+  defm FAMIN : SIMDThreeSameVectorFP<0b1, 0b1, 0b011, "famin", int_aarch64_neon_famin>;
 } // End let Predicates = [HasFAMAXMIN]
 
 let Predicates = [HasFP8FMA] in {
diff --git a/llvm/test/CodeGen/AArch64/neon-famin-famax.ll b/llvm/test/CodeGen/AArch64/neon-famin-famax.ll
new file mode 100644
index 0000000000000..97fb0a0891452
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/neon-famin-famax.ll
@@ -0,0 +1,96 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s | FileCheck %s
+
+target triple = "aarch64-linux"
+
+define <4 x half> @test_famin_f16(<4 x half> %vn, <4 x half> %vm) #0 {
+; CHECK-LABEL: test_famin_f16:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famin v0.4h, v0.4h, v1.4h
+; CHECK-NEXT:    ret
+  %res = call <4 x half> @llvm.aarch64.neon.famin.v4f16(<4 x half> %vn, <4 x half> %vm)
+  ret <4 x half> %res
+}
+
+define <8 x half> @test_famin2_f16(<8 x half> %vn, <8 x half> %vm) #0 {
+; CHECK-LABEL: test_famin2_f16:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famin v0.8h, v0.8h, v1.8h
+; CHECK-NEXT:    ret
+  %res = call <8 x half> @llvm.aarch64.neon.famin.v8f16(<8 x half> %vn, <8 x half> %vm)
+  ret <8 x half> %res
+}
+
+define <2 x float> @test_famin_f32(<2 x float> %vn, <2 x float> %vm) #0 {
+; CHECK-LABEL: test_famin_f32:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famin v0.2s, v0.2s, v1.2s
+; CHECK-NEXT:    ret
+  %res = call <2 x float> @llvm.aarch64.neon.famin.v2f32(<2 x float> %vn, <2 x float> %vm)
+  ret <2 x float> %res
+}
+
+define <4 x float> @test_famin2_f32(<4 x float> %vn, <4 x float> %vm) #0 {
+; CHECK-LABEL: test_famin2_f32:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famin v0.4s, v0.4s, v1.4s
+; CHECK-NEXT:    ret
+  %res = call <4 x float> @llvm.aarch64.neon.famin.v4f32(<4 x float> %vn, <4 x float> %vm)
+  ret <4 x float> %res
+}
+
+define <2 x double> @test_famin_f64(<2 x double> %vn, <2 x double> %vm) #0 {
+; CHECK-LABEL: test_famin_f64:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famin v0.2d, v0.2d, v1.2d
+; CHECK-NEXT:    ret
+  %res = call <2 x double> @llvm.aarch64.neon.famin.v2f64(<2 x double> %vn, <2 x double> %vm)
+  ret <2 x double> %res
+}
+
+define <4 x half> @test_famax_f16(<4 x half> %vn, <4 x half> %vm) #0 {
+; CHECK-LABEL: test_famax_f16:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famax v0.4h, v0.4h, v1.4h
+; CHECK-NEXT:    ret
+  %res = call <4 x half> @llvm.aarch64.neon.famax.v4f16(<4 x half> %vn, <4 x half> %vm)
+  ret <4 x half> %res
+}
+
+define <8 x half> @test_famax2_f16(<8 x half> %vn, <8 x half> %vm) #0 {
+; CHECK-LABEL: test_famax2_f16:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famax v0.8h, v0.8h, v1.8h
+; CHECK-NEXT:    ret
+  %res = call <8 x half> @llvm.aarch64.neon.famax.v8f16(<8 x half> %vn, <8 x half> %vm)
+  ret <8 x half> %res
+}
+
+define <2 x float> @test_famax_f32(<2 x float> %vn, <2 x float> %vm) #0 {
+; CHECK-LABEL: test_famax_f32:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famax v0.2s, v0.2s, v1.2s
+; CHECK-NEXT:    ret
+  %res = call <2 x float> @llvm.aarch64.neon.famax.v2f32(<2 x float> %vn, <2 x float> %vm)
+  ret <2 x float> %res
+}
+
+define <4 x float> @test_famax2_f32(<4 x float> %vn, <4 x float> %vm) #0 {
+; CHECK-LABEL: test_famax2_f32:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famax v0.4s, v0.4s, v1.4s
+; CHECK-NEXT:    ret
+  %res = call <4 x float> @llvm.aarch64.neon.famax.v4f32(<4 x float> %vn, <4 x float> %vm)
+  ret <4 x float> %res
+}
+
+define <2 x double> @test_famax_f64(<2 x double> %vn, <2 x double> %vm) #0 {
+; CHECK-LABEL: test_famax_f64:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famax v0.2d, v0.2d, v1.2d
+; CHECK-NEXT:    ret
+  %res = call <2 x double> @llvm.aarch64.neon.famax.v2f64(<2 x double> %vn, <2 x double> %vm)
+  ret <2 x double> %res
+}
+
+attributes #0 = { "target-features"="+neon,+faminmax" }

@llvmbot
Copy link
Member

llvmbot commented Jul 16, 2024

@llvm/pr-subscribers-clang-codegen

Author: Momchil Velikov (momchil-velikov)

Changes

This patch implements the intrinsics of the form

floatNxM_t vamin[q]_fN(floatNxM_t vn, floatNxM_t vm);
floatNxM_t vamax[q]_fN(floatNxM_t vn, floatNxM_t vm);

as defined in ARM-software/acle#324

Full diff: https://github.com/llvm/llvm-project/pull/99041.diff

7 Files Affected:

  • (modified) clang/include/clang/Basic/arm_neon.td (+5)
  • (modified) clang/lib/CodeGen/CGBuiltin.cpp (+17)
  • (added) clang/test/CodeGen/aarch64-neon-faminmax-intrinsics.c (+112)
  • (modified) llvm/include/llvm/IR/IntrinsicsAArch64.td (+3)
  • (modified) llvm/lib/Target/AArch64/AArch64InstrFormats.td (+20)
  • (modified) llvm/lib/Target/AArch64/AArch64InstrInfo.td (+2-2)
  • (added) llvm/test/CodeGen/AArch64/neon-famin-famax.ll (+96)
diff --git a/clang/include/clang/Basic/arm_neon.td b/clang/include/clang/Basic/arm_neon.td
index 6390ba3f9fe5e..3746b3667ad99 100644
--- a/clang/include/clang/Basic/arm_neon.td
+++ b/clang/include/clang/Basic/arm_neon.td
@@ -2096,3 +2096,8 @@ let ArchGuard = "defined(__aarch64__) || defined(__arm64ec__)", TargetGuard = "r
   def VLDAP1_LANE : WInst<"vldap1_lane", ".(c*!).I", "QUlQlUlldQdPlQPl">;
   def VSTL1_LANE  : WInst<"vstl1_lane", "v*(.!)I", "QUlQlUlldQdPlQPl">;
 }
+
+let ArchGuard = "defined(__aarch64__)", TargetGuard = "faminmax" in {
+  def FAMIN : WInst<"vamin", "...", "fhQdQfQh">;
+  def FAMAX : WInst<"vamax", "...", "fhQdQfQh">;
+}
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index a54fa7bf87aad..bb6094aa31805 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -13398,6 +13398,23 @@ Value *CodeGenFunction::EmitAArch64BuiltinExpr(unsigned BuiltinID,
     Int = Intrinsic::aarch64_neon_suqadd;
     return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "vuqadd");
   }
+
+  case NEON::BI__builtin_neon_vamin_f16:
+  case NEON::BI__builtin_neon_vaminq_f16:
+  case NEON::BI__builtin_neon_vamin_f32:
+  case NEON::BI__builtin_neon_vaminq_f32:
+  case NEON::BI__builtin_neon_vaminq_f64: {
+    Int = Intrinsic::aarch64_neon_famin;
+    return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "famin");
+  }
+  case NEON::BI__builtin_neon_vamax_f16:
+  case NEON::BI__builtin_neon_vamaxq_f16:
+  case NEON::BI__builtin_neon_vamax_f32:
+  case NEON::BI__builtin_neon_vamaxq_f32:
+  case NEON::BI__builtin_neon_vamaxq_f64: {
+    Int = Intrinsic::aarch64_neon_famax;
+    return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "famax");
+  }
   }
 }
 
diff --git a/clang/test/CodeGen/aarch64-neon-faminmax-intrinsics.c b/clang/test/CodeGen/aarch64-neon-faminmax-intrinsics.c
new file mode 100644
index 0000000000000..631e9738b85c5
--- /dev/null
+++ b/clang/test/CodeGen/aarch64-neon-faminmax-intrinsics.c
@@ -0,0 +1,112 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py UTC_ARGS: --version 5
+#include <arm_neon.h>
+
+// RUN: %clang_cc1 -triple aarch64-none-linux-gnu -target-feature +neon -target-feature +faminmax -O3 -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -triple aarch64-none-linux-gnu -target-feature +neon -target-feature +faminmax -S -O3 -Werror -Wall -o /dev/null %s
+
+// CHECK-LABEL: define dso_local <4 x half> @test_vamin_f16(
+// CHECK-SAME: <4 x half> noundef [[VN:%.*]], <4 x half> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMIN2_I:%.*]] = tail call <4 x half> @llvm.aarch64.neon.famin.v4f16(<4 x half> [[VN]], <4 x half> [[VM]])
+// CHECK-NEXT:    ret <4 x half> [[FAMIN2_I]]
+//
+float16x4_t test_vamin_f16(float16x4_t vn, float16x4_t vm) {
+  return vamin_f16(vn, vm);
+}
+
+// CHECK-LABEL: define dso_local <8 x half> @test_vaminq_f16(
+// CHECK-SAME: <8 x half> noundef [[VN:%.*]], <8 x half> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMIN2_I:%.*]] = tail call <8 x half> @llvm.aarch64.neon.famin.v8f16(<8 x half> [[VN]], <8 x half> [[VM]])
+// CHECK-NEXT:    ret <8 x half> [[FAMIN2_I]]
+//
+float16x8_t test_vaminq_f16(float16x8_t vn, float16x8_t vm) {
+  return vaminq_f16(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <2 x float> @test_vamin_f32(
+// CHECK-SAME: <2 x float> noundef [[VN:%.*]], <2 x float> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMIN2_I:%.*]] = tail call <2 x float> @llvm.aarch64.neon.famin.v2f32(<2 x float> [[VN]], <2 x float> [[VM]])
+// CHECK-NEXT:    ret <2 x float> [[FAMIN2_I]]
+//
+float32x2_t test_vamin_f32(float32x2_t vn, float32x2_t vm) {
+  return vamin_f32(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <4 x float> @test_vaminq_f32(
+// CHECK-SAME: <4 x float> noundef [[VN:%.*]], <4 x float> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMIN2_I:%.*]] = tail call <4 x float> @llvm.aarch64.neon.famin.v4f32(<4 x float> [[VN]], <4 x float> [[VM]])
+// CHECK-NEXT:    ret <4 x float> [[FAMIN2_I]]
+//
+float32x4_t test_vaminq_f32(float32x4_t vn, float32x4_t vm) {
+  return vaminq_f32(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <2 x double> @test_vaminq_f64(
+// CHECK-SAME: <2 x double> noundef [[VN:%.*]], <2 x double> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMIN2_I:%.*]] = tail call <2 x double> @llvm.aarch64.neon.famin.v2f64(<2 x double> [[VN]], <2 x double> [[VM]])
+// CHECK-NEXT:    ret <2 x double> [[FAMIN2_I]]
+//
+float64x2_t test_vaminq_f64(float64x2_t vn, float64x2_t vm) {
+  return vaminq_f64(vn, vm);
+}
+
+
+// CHECK-LABEL: define dso_local <4 x half> @test_vamax_f16(
+// CHECK-SAME: <4 x half> noundef [[VN:%.*]], <4 x half> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMAX2_I:%.*]] = tail call <4 x half> @llvm.aarch64.neon.famax.v4f16(<4 x half> [[VN]], <4 x half> [[VM]])
+// CHECK-NEXT:    ret <4 x half> [[FAMAX2_I]]
+//
+float16x4_t test_vamax_f16(float16x4_t vn, float16x4_t vm) {
+  return vamax_f16(vn, vm);
+}
+
+// CHECK-LABEL: define dso_local <8 x half> @test_vamaxq_f16(
+// CHECK-SAME: <8 x half> noundef [[VN:%.*]], <8 x half> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMAX2_I:%.*]] = tail call <8 x half> @llvm.aarch64.neon.famax.v8f16(<8 x half> [[VN]], <8 x half> [[VM]])
+// CHECK-NEXT:    ret <8 x half> [[FAMAX2_I]]
+//
+float16x8_t test_vamaxq_f16(float16x8_t vn, float16x8_t vm) {
+  return vamaxq_f16(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <2 x float> @test_vamax_f32(
+// CHECK-SAME: <2 x float> noundef [[VN:%.*]], <2 x float> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMAX2_I:%.*]] = tail call <2 x float> @llvm.aarch64.neon.famax.v2f32(<2 x float> [[VN]], <2 x float> [[VM]])
+// CHECK-NEXT:    ret <2 x float> [[FAMAX2_I]]
+//
+float32x2_t test_vamax_f32(float32x2_t vn, float32x2_t vm) {
+  return vamax_f32(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <4 x float> @test_vamaxq_f32(
+// CHECK-SAME: <4 x float> noundef [[VN:%.*]], <4 x float> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMAX2_I:%.*]] = tail call <4 x float> @llvm.aarch64.neon.famax.v4f32(<4 x float> [[VN]], <4 x float> [[VM]])
+// CHECK-NEXT:    ret <4 x float> [[FAMAX2_I]]
+//
+float32x4_t test_vamaxq_f32(float32x4_t vn, float32x4_t vm) {
+  return vamaxq_f32(vn, vm);
+
+}
+
+// CHECK-LABEL: define dso_local <2 x double> @test_vamaxq_f64(
+// CHECK-SAME: <2 x double> noundef [[VN:%.*]], <2 x double> noundef [[VM:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK-NEXT:  [[ENTRY:.*:]]
+// CHECK-NEXT:    [[FAMAX2_I:%.*]] = tail call <2 x double> @llvm.aarch64.neon.famax.v2f64(<2 x double> [[VN]], <2 x double> [[VM]])
+// CHECK-NEXT:    ret <2 x double> [[FAMAX2_I]]
+//
+float64x2_t test_vamaxq_f64(float64x2_t vn, float64x2_t vm) {
+  return vamaxq_f64(vn, vm);
+}
diff --git a/llvm/include/llvm/IR/IntrinsicsAArch64.td b/llvm/include/llvm/IR/IntrinsicsAArch64.td
index 3735bf5222fce..f778973880703 100644
--- a/llvm/include/llvm/IR/IntrinsicsAArch64.td
+++ b/llvm/include/llvm/IR/IntrinsicsAArch64.td
@@ -3730,3 +3730,6 @@ def int_aarch64_sve_pmov_to_vector_lane_zeroing : SVE2_Pred_1VectorArg_Intrinsic
 def int_aarch64_sme_mopa_nonwide : SME_OuterProduct_Intrinsic;
 def int_aarch64_sme_mops_nonwide : SME_OuterProduct_Intrinsic;
 
+// Neon absolute maximum and minimum
+def int_aarch64_neon_famax :  AdvSIMD_2VectorArg_Intrinsic;
+def int_aarch64_neon_famin :  AdvSIMD_2VectorArg_Intrinsic;
diff --git a/llvm/lib/Target/AArch64/AArch64InstrFormats.td b/llvm/lib/Target/AArch64/AArch64InstrFormats.td
index e1ecc5a57dd26..3cbc41d943eba 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrFormats.td
+++ b/llvm/lib/Target/AArch64/AArch64InstrFormats.td
@@ -5985,6 +5985,26 @@ multiclass SIMDThreeSameVectorFP<bit U, bit S, bits<3> opc,
         [(set (v2f64 V128:$Rd), (OpNode (v2f64 V128:$Rn), (v2f64 V128:$Rm)))]>;
 }
 
+let mayRaiseFPException = 1, Uses = [FPCR] in
+multiclass SIMDThreeVectorFP<bit U, bit S, bits<3> opc,
+                             string asm, SDPatternOperator OpNode> {
+  def v4f16 : BaseSIMDThreeSameVector<0, U, {S,0b10}, {0b00,opc}, V64,
+                                      asm, ".4h",
+        [(set (v4f16 V64:$Rd), (OpNode (v4f16 V64:$Rn), (v4i16 V64:$Rm)))]>;
+  def v8f16 : BaseSIMDThreeSameVector<1, U, {S,0b10}, {0b00,opc}, V128,
+                                      asm, ".8h",
+        [(set (v8f16 V128:$Rd), (OpNode (v8f16 V128:$Rn), (v8i16 V128:$Rm)))]>;
+  def v2f32 : BaseSIMDThreeSameVector<0, U, {S,0b01}, {0b11,opc}, V64,
+                                      asm, ".2s",
+        [(set (v2f32 V64:$Rd), (OpNode (v2f32 V64:$Rn), (v2i32 V64:$Rm)))]>;
+  def v4f32 : BaseSIMDThreeSameVector<1, U, {S,0b01}, {0b11,opc}, V128,
+                                      asm, ".4s",
+        [(set (v4f32 V128:$Rd), (OpNode (v4f32 V128:$Rn), (v4i32 V128:$Rm)))]>;
+  def v2f64 : BaseSIMDThreeSameVector<1, U, {S,0b11}, {0b11,opc}, V128,
+                                      asm, ".2d",
+        [(set (v2f64 V128:$Rd), (OpNode (v2f64 V128:$Rn), (v2i64 V128:$Rm)))]>;
+}
+
 let mayRaiseFPException = 1, Uses = [FPCR] in
 multiclass SIMDThreeSameVectorFPCmp<bit U, bit S, bits<3> opc,
                                     string asm,
diff --git a/llvm/lib/Target/AArch64/AArch64InstrInfo.td b/llvm/lib/Target/AArch64/AArch64InstrInfo.td
index dd11f74882115..6a9b7256e31bd 100644
--- a/llvm/lib/Target/AArch64/AArch64InstrInfo.td
+++ b/llvm/lib/Target/AArch64/AArch64InstrInfo.td
@@ -10016,8 +10016,8 @@ let Predicates = [HasFP8] in {
 } // End let Predicates = [HasFP8]
 
 let Predicates = [HasFAMINMAX] in {
- defm FAMAX : SIMDThreeSameVectorFP<0b0, 0b1, 0b011, "famax", null_frag>;
- defm FAMIN : SIMDThreeSameVectorFP<0b1, 0b1, 0b011, "famin", null_frag>;
+  defm FAMAX : SIMDThreeSameVectorFP<0b0, 0b1, 0b011, "famax", int_aarch64_neon_famax>;
+  defm FAMIN : SIMDThreeSameVectorFP<0b1, 0b1, 0b011, "famin", int_aarch64_neon_famin>;
 } // End let Predicates = [HasFAMAXMIN]
 
 let Predicates = [HasFP8FMA] in {
diff --git a/llvm/test/CodeGen/AArch64/neon-famin-famax.ll b/llvm/test/CodeGen/AArch64/neon-famin-famax.ll
new file mode 100644
index 0000000000000..97fb0a0891452
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/neon-famin-famax.ll
@@ -0,0 +1,96 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc < %s | FileCheck %s
+
+target triple = "aarch64-linux"
+
+define <4 x half> @test_famin_f16(<4 x half> %vn, <4 x half> %vm) #0 {
+; CHECK-LABEL: test_famin_f16:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famin v0.4h, v0.4h, v1.4h
+; CHECK-NEXT:    ret
+  %res = call <4 x half> @llvm.aarch64.neon.famin.v4f16(<4 x half> %vn, <4 x half> %vm)
+  ret <4 x half> %res
+}
+
+define <8 x half> @test_famin2_f16(<8 x half> %vn, <8 x half> %vm) #0 {
+; CHECK-LABEL: test_famin2_f16:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famin v0.8h, v0.8h, v1.8h
+; CHECK-NEXT:    ret
+  %res = call <8 x half> @llvm.aarch64.neon.famin.v8f16(<8 x half> %vn, <8 x half> %vm)
+  ret <8 x half> %res
+}
+
+define <2 x float> @test_famin_f32(<2 x float> %vn, <2 x float> %vm) #0 {
+; CHECK-LABEL: test_famin_f32:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famin v0.2s, v0.2s, v1.2s
+; CHECK-NEXT:    ret
+  %res = call <2 x float> @llvm.aarch64.neon.famin.v2f32(<2 x float> %vn, <2 x float> %vm)
+  ret <2 x float> %res
+}
+
+define <4 x float> @test_famin2_f32(<4 x float> %vn, <4 x float> %vm) #0 {
+; CHECK-LABEL: test_famin2_f32:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famin v0.4s, v0.4s, v1.4s
+; CHECK-NEXT:    ret
+  %res = call <4 x float> @llvm.aarch64.neon.famin.v4f32(<4 x float> %vn, <4 x float> %vm)
+  ret <4 x float> %res
+}
+
+define <2 x double> @test_famin_f64(<2 x double> %vn, <2 x double> %vm) #0 {
+; CHECK-LABEL: test_famin_f64:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famin v0.2d, v0.2d, v1.2d
+; CHECK-NEXT:    ret
+  %res = call <2 x double> @llvm.aarch64.neon.famin.v2f64(<2 x double> %vn, <2 x double> %vm)
+  ret <2 x double> %res
+}
+
+define <4 x half> @test_famax_f16(<4 x half> %vn, <4 x half> %vm) #0 {
+; CHECK-LABEL: test_famax_f16:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famax v0.4h, v0.4h, v1.4h
+; CHECK-NEXT:    ret
+  %res = call <4 x half> @llvm.aarch64.neon.famax.v4f16(<4 x half> %vn, <4 x half> %vm)
+  ret <4 x half> %res
+}
+
+define <8 x half> @test_famax2_f16(<8 x half> %vn, <8 x half> %vm) #0 {
+; CHECK-LABEL: test_famax2_f16:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famax v0.8h, v0.8h, v1.8h
+; CHECK-NEXT:    ret
+  %res = call <8 x half> @llvm.aarch64.neon.famax.v8f16(<8 x half> %vn, <8 x half> %vm)
+  ret <8 x half> %res
+}
+
+define <2 x float> @test_famax_f32(<2 x float> %vn, <2 x float> %vm) #0 {
+; CHECK-LABEL: test_famax_f32:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famax v0.2s, v0.2s, v1.2s
+; CHECK-NEXT:    ret
+  %res = call <2 x float> @llvm.aarch64.neon.famax.v2f32(<2 x float> %vn, <2 x float> %vm)
+  ret <2 x float> %res
+}
+
+define <4 x float> @test_famax2_f32(<4 x float> %vn, <4 x float> %vm) #0 {
+; CHECK-LABEL: test_famax2_f32:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famax v0.4s, v0.4s, v1.4s
+; CHECK-NEXT:    ret
+  %res = call <4 x float> @llvm.aarch64.neon.famax.v4f32(<4 x float> %vn, <4 x float> %vm)
+  ret <4 x float> %res
+}
+
+define <2 x double> @test_famax_f64(<2 x double> %vn, <2 x double> %vm) #0 {
+; CHECK-LABEL: test_famax_f64:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    famax v0.2d, v0.2d, v1.2d
+; CHECK-NEXT:    ret
+  %res = call <2 x double> @llvm.aarch64.neon.famax.v2f64(<2 x double> %vn, <2 x double> %vm)
+  ret <2 x double> %res
+}
+
+attributes #0 = { "target-features"="+neon,+faminmax" }

davemgreen
davemgreen reviewed Jul 16, 2024
View reviewed changes
Copy link
Collaborator

@davemgreen davemgreen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you consider emitting llvm.fmin(llvm.fabs(x), llvm.fabs(y))?

@momchil-velikov
Copy link
Collaborator Author

Did you consider emitting llvm.fmin(llvm.fabs(x), llvm.fabs(y))?

Nope. I'll have a look.

CarolineConcatto
CarolineConcatto reviewed Jul 25, 2024
View reviewed changes
@momchil-velikov
Copy link
Collaborator Author

Ping?

SpencerAbson
SpencerAbson reviewed Sep 5, 2024
View reviewed changes
momchil-velikov and others added 4 commits September 9, 2024 10:12
@momchil-velikov @hassnaaHamdi
[AArch64] Implement NEON vamin/vamax intrinsics
0cca71a 
This patch implements the intrinsics of the form

    floatNxM_t vamin[q]_fN(floatNxM_t vn, floatNxM_t vm);
    floatNxM_t vamax[q]_fN(floatNxM_t vn, floatNxM_t vm);

as defined in ARM-software/acle#324

Co-authored-by: Hassnaa Hamdi <[email protected]>
@momchil-velikov
[fixup] Remove unused tablegen class
67fe989
@momchil-velikov
[fixup] Add a REQUIRES clause to a test and rebase
634302b
@momchil-velikov
[fixup] Add neon feature requirement
65101a0
@momchil-velikov momchil-velikov merged commit cf8fb43 into llvm:main Sep 9, 2024
8 checks passed
@momchil-velikov momchil-velikov deleted the faminmax-neon branch November 13, 2024 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davemgreen davemgreen davemgreen left review comments

@SpencerAbson SpencerAbson SpencerAbson left review comments

@CarolineConcatto CarolineConcatto CarolineConcatto approved these changes

@Lukacma Lukacma Awaiting requested review from Lukacma

Assignees
No one assigned
Labels
backend:AArch64 clang:codegen IR generation bugs: mangling, exceptions, etc. clang:frontend Language frontend issues, e.g. anything involving "Sema" clang Clang issues not falling into any other category llvm:ir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@momchil-velikov @llvmbot @davemgreen @CarolineConcatto @SpencerAbson