548.exchange2_r of SPEC CPU 2017 has 30% performance regression between LLVM 18/19 and LLVM 20 on amd64

[jiegec]

TL; DR: -fwrapv resolves the regression.

548.exchange2_r built using flang-new-18:

548.exchange2_r: time (ms) = 181666
548.exchange2_r: clock freq (MHz) = 5701.6834
548.exchange2_r: instructions = 4.05239e+12
548.exchange2_r: branch instructions = 5.80567e+11
548.exchange2_r: ipc = 3.9123
548.exchange2_r: misprediction rate (%) = 0.8200
548.exchange2_r: mpki = 1.1747

548.exchange2_r built using flang-new-19:

548.exchange2_r: time (ms) = 182113
548.exchange2_r: clock freq (MHz) = 5701.3884
548.exchange2_r: instructions = 4.03508e+12
548.exchange2_r: branch instructions = 5.50304e+11
548.exchange2_r: ipc = 3.8863
548.exchange2_r: misprediction rate (%) = 0.7708
548.exchange2_r: mpki = 1.0513

548.exchange2_r built using flang-new-20:

548.exchange2_r: time (ms) = 240537
548.exchange2_r: clock freq (MHz) = 5700.9669
548.exchange2_r: instructions = 5.15502e+12
548.exchange2_r: branch instructions = 5.42299e+11
548.exchange2_r: ipc = 3.7592
548.exchange2_r: misprediction rate (%) = 0.7719
548.exchange2_r: mpki = 0.8121

A performance regression of ~30% is observed.

Hardware: Intel Core i9-14900K 5.7 GHz

Full results and compilation flags are available at https://jia.je/benchmark/ (in chinese).

[AaronBallman]

Thank you for the report! Can you provide source code that we can use to reproduce the issue?

[jiegec]

Thank you for the report! Can you provide source code that we can use to reproduce the issue?

It is part of the SPEC CPU 2017 benchmark suite, I can email you a copy if necessary.

[AaronBallman]

No worries, that should be sufficient for us to find it (also, I just noticed this was a flang issue and not a clang one)

[jiegec]

... I just noticed this was a flang issue and not a clang one

Correct. The compiler is flang-new-{18,19,20}.

[jiegec]

To reproduce:

1. extract the source from SPEC CPU 2017 and cd into it
2. run ./build.sh
3. run ./run.sh

Result:

$ ./run.sh
+ echo LLVM 19:
LLVM 19:
+ time numactl -C 0 ./exchange2_llvm19 6
Fortran STOP
181.98user 0.00system 3:02.07elapsed 99%CPU (0avgtext+0avgdata 1628maxresident)k
0inputs+19outputs (0major+206minor)pagefaults 0swaps
+ echo LLVM 20:
LLVM 20:
+ time numactl -C 0 ./exchange2_llvm20 6
Fortran STOP
240.16user 0.00system 4:00.30elapsed 99%CPU (0avgtext+0avgdata 1628maxresident)k
1952inputs+19outputs (119major+186minor)pagefaults 0swaps

[vzakhari]

This is probably the same as #117318 (comment)

Please try it with -fwrapv option.

[mcolg]

Hi jiegec,

Note that CPU2017 is a licensed product and only source contained in the "redistributable_source" directory may be shared publicly. The Exchange2 source is licensed to SPEC directly by the author, so not a source that can be shared publicly. Posting here is a violation of your license, so please remove the link to the source.

You may share directly between other SPEC CPU2017 license holders or other SPEC members and associates, just not publicly. Given Aaron is with Intel, a SPEC Member, he should be able to contact his primary representative to obtain a copy, so no need to post here.

FYI, I'm with NVIDIA's NVHPC Compiler team, but also NVIDIA's SPEC primary rep. If you have questions on what you can and can't share publicly, please feel free to reach out.

Thanks,
Mat