Skip to content

[AMDGPU][TTI] Add Target Hook for Instruction Uniformity (getInstructionUniformity) #137639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[AMDGPU][TTI] Add Target Hook for Instruction Uniformity (getInstructionUniformity) #137639

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a771611
adds target hook to support more inst uniformity
PankajDwivedi-25 Apr 28, 2025
8cca1b2
[WIP] currently users of a divergent value are marked as divergent, i…
PankajDwivedi-25 May 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions llvm/include/llvm/Analysis/TargetTransformInfo.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@

#include "llvm/ADT/APInt.h"
#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/Uniformity.h"
#include "llvm/Analysis/IVDescriptors.h"
#include "llvm/IR/FMF.h"
#include "llvm/IR/InstrTypes.h"
Expand Down Expand Up @@ -1916,6 +1917,13 @@ class TargetTransformInfo {
const Function &F,
SmallVectorImpl<std::pair<StringRef, int64_t>> &LB) const;

/// Target can implement more complex patterns for getting Uniformity of an
/// instruction.Currently Uniformity analysis catagorises instructions with a
/// fixed set of InstructionUniformity values: Default, AlwaysUniform and
/// NeverUniform.
std::optional<InstructionUniformity>
getInstructionUniformity(const Instruction &I) const;

private:
std::unique_ptr<const TargetTransformInfoImplBase> TTIImpl;
};
Expand Down
5 changes: 5 additions & 0 deletions llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1147,6 +1147,11 @@ class TargetTransformInfoImplBase {
const Function &F,
SmallVectorImpl<std::pair<StringRef, int64_t>> &LB) const {}

virtual std::optional<InstructionUniformity>
getInstructionUniformity(const Instruction &I) const {
return std::nullopt;
}

protected:
// Obtain the minimum required size to hold the value (without the sign)
// In case of a vector it returns the min required size for one element.
Expand Down
5 changes: 5 additions & 0 deletions llvm/lib/Analysis/TargetTransformInfo.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1476,6 +1476,11 @@ void TargetTransformInfo::collectKernelLaunchBounds(
return TTIImpl->collectKernelLaunchBounds(F, LB);
}

std::optional<InstructionUniformity>
TargetTransformInfo::getInstructionUniformity(const Instruction &I) const {
return TTIImpl->getInstructionUniformity(I);
}

TargetTransformInfoImplBase::~TargetTransformInfoImplBase() = default;

TargetIRAnalysis::TargetIRAnalysis() : TTICallback(&getDefaultTTI) {}
Expand Down
13 changes: 13 additions & 0 deletions llvm/lib/Analysis/UniformityAnalysis.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,20 @@ template <> void llvm::GenericUniformityAnalysisImpl<SSAContext>::initialize() {
markDivergent(I);
else if (TTI->isAlwaysUniform(&I))
addUniformOverride(I);
else if (auto Uniformity = TTI->getInstructionUniformity(I)) {
switch (*Uniformity) {
case InstructionUniformity::AlwaysUniform:
addUniformOverride(I);
break;
case InstructionUniformity::NeverUniform:
markDivergent(I);
break;
case InstructionUniformity::Default:
break;
}
}
}

for (auto &Arg : F.args()) {
if (TTI->isSourceOfDivergence(&Arg)) {
markDivergent(&Arg);
Expand Down
21 changes: 21 additions & 0 deletions llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1422,3 +1422,24 @@ void GCNTTIImpl::collectKernelLaunchBounds(
LB.push_back({"amdgpu-waves-per-eu[0]", WavesPerEU.first});
LB.push_back({"amdgpu-waves-per-eu[1]", WavesPerEU.second});
}

std::optional<InstructionUniformity>
GCNTTIImpl::getInstructionUniformity(const Instruction &I) const {
if (const auto *II = dyn_cast<IntrinsicInst>(&I)) {
// We can define the custom rules for the intrinsics uniformity, depending
// on argument.
switch (II->getIntrinsicID()) {
case Intrinsic::amdgcn_permlane64:
// If either operand is uniform, the result is uniform.
for (unsigned Arg_i = 0, NumArg = II->arg_size(); Arg_i < NumArg;
Arg_i++) {
if (!isSourceOfDivergence(II->getArgOperand(Arg_i)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. Here we need to be able to query "is this operand uniform according to the current state of the uniformity analysis", either by calling back into the analysis class, or having it pass in a vector saying which operands are already known to be uniform, or something like that. Just checking for sources of divergence is not enough.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it was the first thing that arose in my mind while writing this. At this point, we don't have a way to call back to UI about isUniform(operand_i) or isDivergentUse(opeand_i).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we don't need to actually query uniformity. We could extend the enum to have something like InstructionUniformity::SameAsOperand0. Not exactly that, but it's information that the calling UA can then use to decide whether a returned value is uniform based on the operands.

return InstructionUniformity::AlwaysUniform;
}
return InstructionUniformity::Default;
default:
break;
}
}
return std::nullopt;
}
2 changes: 2 additions & 0 deletions llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,8 @@ class GCNTTIImpl final : public BasicTTIImplBase<GCNTTIImpl> {
void collectKernelLaunchBounds(
const Function &F,
SmallVectorImpl<std::pair<StringRef, int64_t>> &LB) const override;
std::optional<InstructionUniformity>
getInstructionUniformity(const Instruction &I) const override;
};

} // end namespace llvm
Expand Down
25 changes: 25 additions & 0 deletions llvm/test/Analysis/UniformityAnalysis/AMDGPU/uniform_intrinsic.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
; RUN: opt -mtriple amdgcn-unknown-amdhsa -passes='print<uniformity>' -disable-output %s 2>&1 | FileCheck %s

; CHECK: ALL VALUES UNIFORM
define amdgpu_kernel void @permlane64_constant(ptr addrspace(1) %out) {
%v = call i32 @llvm.amdgcn.permlane64(i32 7)
store i32 %v, ptr addrspace(1) %out
ret void
}

; CHECK: ALL VALUES UNIFORM
define amdgpu_kernel void @permlane64_uniform(ptr addrspace(1) %out, i32 %src) {
%v = call i32 @llvm.amdgcn.permlane64(i32 %src)
store i32 %v, ptr addrspace(1) %out
ret void
}

; CHECK: DIVERGENT: %tid = call i32 @llvm.amdgcn.workitem.id.x()
; CHECK: DIVERGENT: %v = call i32 @llvm.amdgcn.permlane64.i32(i32 %tid)
define amdgpu_kernel void @permlane64_nonuniform(i32 addrspace(1)* %out) {
%tid = call i32 @llvm.amdgcn.workitem.id.x()
%v = call i32 @llvm.amdgcn.permlane64(i32 %tid)
%out_ptr = getelementptr i32, i32 addrspace(1)* %out, i32 %tid
store i32 %v, i32 addrspace(1)* %out_ptr
ret void
}
Loading