[LowerBufferFatPointers] Fix support for GEP T, p7, <N x T> idxs #126126
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The lowering for GEP didn't properly support the case where the pointer argument was being implicitly broadcast by a vector of indices. Fix that.
|
@llvm/pr-subscribers-backend-amdgpu Author: Krzysztof Drewniak (krzysz00) ChangesThe lowering for GEP didn't properly support the case where the pointer argument was being implicitly broadcast by a vector of indices. Fix that. Full diff: https://github.com/llvm/llvm-project/pull/126126.diff 2 Files Affected:
diff --git a/llvm/lib/Target/AMDGPU/AMDGPULowerBufferFatPointers.cpp b/llvm/lib/Target/AMDGPU/AMDGPULowerBufferFatPointers.cpp
index ccb874e6a934e7..8ef491f657c156 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPULowerBufferFatPointers.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPULowerBufferFatPointers.cpp
@@ -1804,14 +1804,26 @@ PtrParts SplitPtrStructs::visitGetElementPtrInst(GetElementPtrInst &GEP) {
bool IsNUW = GEP.hasNoUnsignedWrap();
bool IsNUSW = GEP.hasNoUnsignedSignedWrap();
+ Type *ResTy = GEP.getType();
+ std::optional<ElementCount> ResEC;
+ if (auto *ResVT = dyn_cast<VectorType>(ResTy->getStructElementType(0)))
+ ResEC = ResVT->getElementCount();
+ bool HasPtrVecIn = isa<VectorType>(Off->getType());
+ bool BroadcastsPtr = ResEC.has_value() && !HasPtrVecIn;
+
// In order to call emitGEPOffset() and thus not have to reimplement it,
// we need the GEP result to have ptr addrspace(7) type.
Type *FatPtrTy = IRB.getPtrTy(AMDGPUAS::BUFFER_FAT_POINTER);
- if (auto *VT = dyn_cast<VectorType>(Off->getType()))
- FatPtrTy = VectorType::get(FatPtrTy, VT->getElementCount());
+ if (ResEC.has_value())
+ FatPtrTy = VectorType::get(FatPtrTy, *ResEC);
GEP.mutateType(FatPtrTy);
Value *OffAccum = emitGEPOffset(&IRB, DL, &GEP);
- GEP.mutateType(Ptr->getType());
+ GEP.mutateType(ResTy);
+
+ if (BroadcastsPtr) {
+ Rsrc = IRB.CreateVectorSplat(*ResEC, Rsrc, Rsrc->getName());
+ Off = IRB.CreateVectorSplat(*ResEC, Off, Off->getName());
+ }
if (match(OffAccum, m_Zero())) { // Constant-zero offset
SplitUsers.insert(&GEP);
return {Rsrc, Off};
diff --git a/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-pointer-ops.ll b/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-pointer-ops.ll
index e7589690cd6702..99fcbc595ff7f3 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-pointer-ops.ll
+++ b/llvm/test/CodeGen/AMDGPU/lower-buffer-fat-pointers-pointer-ops.ll
@@ -59,6 +59,24 @@ define <2 x ptr addrspace(7)> @gep_vector_scalar(<2 x ptr addrspace(7)> %in, i64
ret <2 x ptr addrspace(7)> %ret
}
+define <2 x ptr addrspace(7)> @gep_scalar_vector(ptr addrspace(7) %in, <2 x i32> %idxs) {
+; CHECK-LABEL: define { <2 x ptr addrspace(8)>, <2 x i32> } @gep_scalar_vector
+; CHECK-SAME: ({ ptr addrspace(8), i32 } [[IN:%.*]], <2 x i32> [[IDXS:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT: [[IN_RSRC:%.*]] = extractvalue { ptr addrspace(8), i32 } [[IN]], 0
+; CHECK-NEXT: [[IN_OFF:%.*]] = extractvalue { ptr addrspace(8), i32 } [[IN]], 1
+; CHECK-NEXT: [[IN_RSRC_SPLATINSERT:%.*]] = insertelement <2 x ptr addrspace(8)> poison, ptr addrspace(8) [[IN_RSRC]], i64 0
+; CHECK-NEXT: [[IN_RSRC_SPLAT:%.*]] = shufflevector <2 x ptr addrspace(8)> [[IN_RSRC_SPLATINSERT]], <2 x ptr addrspace(8)> poison, <2 x i32> zeroinitializer
+; CHECK-NEXT: [[IN_OFF_SPLATINSERT:%.*]] = insertelement <2 x i32> poison, i32 [[IN_OFF]], i64 0
+; CHECK-NEXT: [[IN_OFF_SPLAT:%.*]] = shufflevector <2 x i32> [[IN_OFF_SPLATINSERT]], <2 x i32> poison, <2 x i32> zeroinitializer
+; CHECK-NEXT: [[RET:%.*]] = add <2 x i32> [[IN_OFF_SPLAT]], [[IDXS]]
+; CHECK-NEXT: [[TMP1:%.*]] = insertvalue { <2 x ptr addrspace(8)>, <2 x i32> } poison, <2 x ptr addrspace(8)> [[IN_RSRC_SPLAT]], 0
+; CHECK-NEXT: [[TMP2:%.*]] = insertvalue { <2 x ptr addrspace(8)>, <2 x i32> } [[TMP1]], <2 x i32> [[RET]], 1
+; CHECK-NEXT: ret { <2 x ptr addrspace(8)>, <2 x i32> } [[TMP2]]
+;
+ %ret = getelementptr inbounds i8, ptr addrspace(7) %in, <2 x i32> %idxs
+ ret <2 x ptr addrspace(7)> %ret
+}
+
define ptr addrspace(7) @simple_gep(ptr addrspace(7) %ptr, i32 %off) {
; CHECK-LABEL: define { ptr addrspace(8), i32 } @simple_gep
; CHECK-SAME: ({ ptr addrspace(8), i32 } [[PTR:%.*]], i32 [[OFF:%.*]]) #[[ATTR0]] {
|
arsenm
reviewed
Feb 7, 2025
| @@ -1804,14 +1804,26 @@ PtrParts SplitPtrStructs::visitGetElementPtrInst(GetElementPtrInst &GEP) { | |||
| bool IsNUW = GEP.hasNoUnsignedWrap(); | |||
| bool IsNUSW = GEP.hasNoUnsignedSignedWrap(); | |||
|
|
|||
| Type *ResTy = GEP.getType(); | |||
| std::optional<ElementCount> ResEC; | |||
There was a problem hiding this comment.
I don't see the point of ResEC, can just use the dyn_cast<VectorType> result later?
arsenm
approved these changes
Feb 11, 2025
Co-authored-by: Matt Arsenault <[email protected]>
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/30/builds/15617 Here is the relevant piece of the build log for the reference |
|
Anyone know where this PGO issue could've come from? My commit seems entirely unrelated |
flovent
pushed a commit
to flovent/llvm-project
that referenced
this pull request
Feb 13, 2025
…m#126126) The lowering for GEP didn't properly support the case where the pointer argument was being implicitly broadcast by a vector of indices. Fix that. --------- Co-authored-by: Matt Arsenault <[email protected]>
joaosaffran
pushed a commit
to joaosaffran/llvm-project
that referenced
this pull request
Feb 14, 2025
…m#126126) The lowering for GEP didn't properly support the case where the pointer argument was being implicitly broadcast by a vector of indices. Fix that. --------- Co-authored-by: Matt Arsenault <[email protected]>
sivan-shani
pushed a commit
to sivan-shani/llvm-project
that referenced
this pull request
Feb 24, 2025
…m#126126) The lowering for GEP didn't properly support the case where the pointer argument was being implicitly broadcast by a vector of indices. Fix that. --------- Co-authored-by: Matt Arsenault <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The lowering for GEP didn't properly support the case where the pointer argument was being implicitly broadcast by a vector of indices. Fix that.