[LV] wrf in SPEC2017 emits "Verification Error" on Grace when using llvmorg-20.1.0-rc1

[yus3710-fj]

• CPU2017.*.log

runcpu v6612 started at 2025-02-12 10:12:40 on "***"
runcpu is: /path/to/cpu2017-1.1.0/bin/harness/runcpu
runcpu: runcpu -I --action run -T base -c /path/to/config.cfg -i ref -n 1 --threads=1 --output_root=/path/to/output -o asc 521
toolset: linux-aarch64
suite: CPU2017 1.1.9
 :
Benchmarks selected: 521.wrf_r
Compiling Binaries
------------------------------------------------------------------------
 :
O: OPTIMIZE="-O2 -mcpu=neoverse-v2"
 :
flang version 20.1.0-rc1 (https://github.com/llvm/llvm-project.git af7f483a9d801252247b6c72e3763c1f55c5a506)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /path/to/bin
clang version 20.1.0-rc1 (https://github.com/llvm/llvm-project.git af7f483a9d801252247b6c72e3763c1f55c5a506)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /path/to/bin
 :
Setting Up Run Directories
  Setting up 521.wrf_r refrate (ref) base mytest-64 (1 copy): run_base_refrate_mytest-64.0000
-----------------------------------
Running Benchmarks
-----------------------------------
 :
Error with '/path/to/cpu2017-1.1.0/bin/specinvoke -d /path/to/output/benchspec/CPU/521.wrf_r/run/run_base_refrate_mytest-64.0000 -f compare.cmd -E -e compare.err -o compare.stdout'; no non-empty output files exist
  Command returned exit code 1
*** Miscompare of diffwrf_output_01.txt; for details see
    /path/to/output/benchspec/CPU/521.wrf_r/run/run_base_refrate_mytest-64.0000/diffwrf_output_01.txt.mis
0015:           W    PASSED
                W    121182    1   0.7117559909E-01   0.6272285812E-01   0   0.3873E-01   0.1085E+01
                  ^
0105:       RAINC    PASSED
            RAINC      1132    2   0.5266002302E+01   0.7384289461E+01   0   0.2787E+01   0.4166E+00
                  ^
0107:      RAINNC    PASSED
           RAINNC      4114    1   0.4236885721E+01   0.3284801810E+01   0   0.1728E+01   0.9835E+00
                  ^
0244:       RAINC    PASSED
            RAINC      1359    2   0.9852032964E+01   0.1287943984E+02   0   0.4379E+01   0.4672E+00
                  ^
0246:      RAINNC    PASSED
           RAINNC      4118    1   0.1017513524E+02   0.8983057159E+01   0   0.2827E+01   0.6375E+00
                  ^
0383:       RAINC    PASSED
            RAINC      1510    2   0.1283821850E+02   0.1616687672E+02   0   0.4926E+01   0.4845E+00
                  ^
Benchmark Times:
  Run Start:    2025-02-12 10:12:46 (1739322766)
  Rate Start:   2025-02-12 10:12:46 (1739322766.13335)
  Rate End:     2025-02-12 10:15:34 (1739322934.19951)
  Run Stop:     2025-02-12 10:15:34 (1739322934)
  Run Elapsed:  00:02:48 (168)
  Run Reported: 00:02:48 (168 66154003 168.066154)
 Error 521.wrf_r base refrate ratio=13.33, runtime=168.066154, copies=1, threads=1, power=0.00W, temp=0.00 degC, humidity=0.00%, errorcode=VE
Error: 1x521.wrf_r
 :

The verification is passed when the option -mcpu=neoverse-v2 is removed.
(The option -mcpu=neoverse-v2+nosve doesn't work for this.)

[yus3710-fj]

Roughly, the regression occured between f5a35a3 and 69d3ba3.

[llvmbot]

@llvm/issue-subscribers-backend-aarch64

Author: Yusuke MINATO (yus3710-fj)

* CPU2017.*.log ``` runcpu v6612 started at 2025-02-12 10:12:40 on "***" runcpu is: /path/to/cpu2017-1.1.0/bin/harness/runcpu runcpu: runcpu -I --action run -T base -c /path/to/config.cfg -i ref -n 1 --threads=1 --output_root=/path/to/output -o asc 521 toolset: linux-aarch64 suite: CPU2017 1.1.9 : Benchmarks selected: 521.wrf_r Compiling Binaries ------------------------------------------------------------------------ : O: OPTIMIZE="-O2 -mcpu=neoverse-v2" : flang version 20.1.0-rc1 (https://github.com/llvm/llvm-project.git af7f483) Target: aarch64-unknown-linux-gnu Thread model: posix InstalledDir: /path/to/bin clang version 20.1.0-rc1 (https://github.com/llvm/llvm-project.git af7f483) Target: aarch64-unknown-linux-gnu Thread model: posix InstalledDir: /path/to/bin : Setting Up Run Directories Setting up 521.wrf_r refrate (ref) base mytest-64 (1 copy): run_base_refrate_mytest-64.0000 ----------------------------------- Running Benchmarks ----------------------------------- : Error with '/path/to/cpu2017-1.1.0/bin/specinvoke -d /path/to/output/benchspec/CPU/521.wrf_r/run/run_base_refrate_mytest-64.0000 -f compare.cmd -E -e compare.err -o compare.stdout'; no non-empty output files exist Command returned exit code 1 *** Miscompare of diffwrf_output_01.txt; for details see /path/to/output/benchspec/CPU/521.wrf_r/run/run_base_refrate_mytest-64.0000/diffwrf_output_01.txt.mis 0015: W PASSED W 121182 1 0.7117559909E-01 0.6272285812E-01 0 0.3873E-01 0.1085E+01 ^ 0105: RAINC PASSED RAINC 1132 2 0.5266002302E+01 0.7384289461E+01 0 0.2787E+01 0.4166E+00 ^ 0107: RAINNC PASSED RAINNC 4114 1 0.4236885721E+01 0.3284801810E+01 0 0.1728E+01 0.9835E+00 ^ 0244: RAINC PASSED RAINC 1359 2 0.9852032964E+01 0.1287943984E+02 0 0.4379E+01 0.4672E+00 ^ 0246: RAINNC PASSED RAINNC 4118 1 0.1017513524E+02 0.8983057159E+01 0 0.2827E+01 0.6375E+00 ^ 0383: RAINC PASSED RAINC 1510 2 0.1283821850E+02 0.1616687672E+02 0 0.4926E+01 0.4845E+00 ^ Benchmark Times: Run Start: 2025-02-12 10:12:46 (1739322766) Rate Start: 2025-02-12 10:12:46 (1739322766.13335) Rate End: 2025-02-12 10:15:34 (1739322934.19951) Run Stop: 2025-02-12 10:15:34 (1739322934) Run Elapsed: 00:02:48 (168) Run Reported: 00:02:48 (168 66154003 168.066154) Error 521.wrf_r base refrate ratio=13.33, runtime=168.066154, copies=1, threads=1, power=0.00W, temp=0.00 degC, humidity=0.00%, errorcode=VE Error: 1x521.wrf_r : ``` The verification is passed when the option `-mcpu=neoverse-v2` is removed.

[yus3710-fj]

The verification is also passed when the option -mllvm -vectorize-loops=false is specified.
FYI: Flang doesn't support the option -fno-vectorize.

[fhahn]

@yus3710-fj would you be able to isolate the loop/function that's causing the issue and share the input IR?

[kiranchandramohan]

The verification is also passed when the option -mllvm -vectorize-loops=false is specified. FYI: Flang doesn't support the option -fno-vectorize.

There is a patch up for review for -fno-vectorize #119718.

Does it work correctly for -O3 or -Ofast?