[NVPTX] Improve folding to mad with immediate 1 #93628
Conversation
| @@ -5614,17 +5614,98 @@ static SDValue TryMULWIDECombine(SDNode *N, | |||
| return DCI.DAG.getNode(Opc, DL, MulType, TruncLHS, TruncRHS); | |||
| } | |||
|
|
|||
| static SDValue matchMADConstOnePattern(SDValue X, SDValue Add) { | |||
| return SDValue(); | ||
|
|
||
| SDValue Y = Add->getOperand(0); | ||
| ConstantSDNode *Const = dyn_cast<ConstantSDNode>(Add->getOperand(1)); |
There was a problem hiding this comment.
Are we guaranteed to have const operand to be last? I think we normalize them, but I'm not 100% sure it's always the case.
There was a problem hiding this comment.
Good point, I've added the other case as well just in case.
|
|
||
| SDValue Y = Add->getOperand(0); | ||
| ConstantSDNode *Const = dyn_cast<ConstantSDNode>(Add->getOperand(1)); | ||
| if (!Const || Const->getZExtValue() != 1) |
There was a problem hiding this comment.
Nit. Phrasing the condition in positive terms would be more readable, IMO.
if (Const && Const->getZExtValue() == 1) return Y;
| @@ -0,0 +1,101 @@ | |||
| ; RUN: llc < %s -march=nvptx -mcpu=sm_20 -O1 | FileCheck %s | |||
There was a problem hiding this comment.
Another test which could use autogenerated CHECK patterns.
| ret i32 %mul | ||
| } | ||
|
|
||
| ; Transpose (mul (select)) if it can then be folded to mad |
There was a problem hiding this comment.
Does it buy us anything?
mul(m,select(1,n)) will probably have the same performance as select(mul(m,n), m) as the critical path will always have mul and select, just in different order.
There was a problem hiding this comment.
By itself this transform doesn't help much, I agree. However, if m or n are add(x,1) then it enables the other transformation. In the code we're checking for this case and only running the transformation when it would enable further folding. A rare case to be sure, but better to support it than not.
There was a problem hiding this comment.
This kind of optimization is not target-specific and should probably be done somewhere in instcombine. Perhaps move the optimization of mul(m,select(1,n)) there as a separate patch?
There was a problem hiding this comment.
instcombine already canonicalizes in the opposite direction, select(mul(m,n), m) -> mul(m,select(1,n)). I think this is target specific because it is only worth doing to improve mad folding.
| ret i32 %mul | ||
| } | ||
|
|
||
| ; Transpose (mul (select)) if it can then be folded to mad |
|
|
||
| unsigned ConstOpNo = 1; | ||
| auto *Const = dyn_cast<ConstantSDNode>(Select->getOperand(ConstOpNo)); | ||
| if (!Const || Const->getZExtValue() != 1) { |
There was a problem hiding this comment.
It looks like we could extract the common pattern into a helper function:
bool isConstOne(Operand) {
const auto *Const = dyn_cast<ConstantSDNode>(Operand);
return Const && Const->getZExtValue() == 1;
}
and then use it in handful of instances of this pattern throughout the code.
| ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5 | ||
| ; RUN: llc < %s -mtriple=nvptx -mcpu=sm_20 -O1 | FileCheck %s | ||
| ; RUN: llc < %s -mtriple=nvptx64 -mcpu=sm_20 -O1 | FileCheck %s | ||
| ; RUN: %if ptxas %{ llc < %s -mtriple=nvptx -mcpu=sm_20 -O1 | %ptxas-verify %} |
There was a problem hiding this comment.
This needs to be disabled with newer ptxas.
RUN: %if ptxas && !ptxas-12.0 %{ llc < %s -march=nvptx -mcpu=sm_20 | %ptxas-verify %}
Extend NVPTX DAG combining logic to distribute a mul instruction across an add of 1 into a mad where possible. In addition, add support for transposing a mul through a select with an option of 1, if that would allow further mul folding.