LLVM ERROR: Cannot select: intrinsic %llvm.x86.aesni.aesenc

[smsxgli]

Related issue in LLVM

Compiling firefox with rustc and clang, failed during PGO while linking libxul.so with thin LTO

32:33.70 LLVM ERROR: Cannot select: intrinsic %llvm.x86.aesni.aesenc

I expected to see this happen: compile succeed

Instead, this happened: compile failed

Meta

build with stable rust, rustc version 1.58.1

And same error in freebsd can be found here

OS: arch linux.
llvm, clang and lld version: 13.0.1
CPU: Intel(R) Core(TM) i7-4790 CPU, haswell
CFLAGS: -march=native -mtune=native -O2 -pipe -fno-plt -fexceptions -fasynchronous-unwind-tables -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-protector-strong -fstack-clash-protection -fcf-protection
CXXFLAGS: $CFLAGS -Wp,-D_GLIBCXX_ASSERTIONS
LDFLAGS: -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now,-z,noexecstack
RUSTFLAGS: -C opt-level=2 -C target-cpu=native

Building with same configuration succeed with Intel(R) Core(TM) i7-11800H CPU tigerlake

The LLVM developer's opinion can be found here, please take a look.

[workingjubilee]

I believe this is happening due to the get_mut function on hashbrown depending on a hasher which is inlined, specifically it appears to be using
https://searchfox.org/mozilla-central/source/third_party/rust/ahash/src/aes_hash.rs
But the code appears to cfg on a feature test correctly, so I am not sure what is going on.
https://searchfox.org/mozilla-central/source/third_party/rust/ahash/src/operations.rs
We can see the IR for this instruction is in fact linked in stdarch:
https://github.com/rust-lang/stdarch/blob/6495bb0e33578443c21764655a4dd8b55399c008/crates/core_arch/src/x86/aes.rs#L21-L22

[smsxgli]

Well, in fact I am not very familiar with rust. So this may not be a bug of rustc ? Just a bug of a crate?

[workingjubilee]

Possibly. Can't say for sure at the moment, it might be an error in how we steer inlining and feature annotations in general. Also, what's weird is that an i7-4790 CPU has AESNI support, so there's no clear reason why building it natively shouldn't work.

[smsxgli]

Got it. If more information needed, please let me know.

[eddyb]

Does this happen during the first build or the post-profiling one that does PGO?
-march=native -mtune=native/-C target-cpu=native seem suspicious with PGO, is every step ran locally on the same CPU? It seems like it would be very easy for those to get out of sync.

If PGO is not relevant, that's more worrying - but also, I'm surprised the LLVM devs didn't bring up PGO.

[smsxgli]

This occurs in second build, I mean the post-profiling one that does PGO, with thin LTO

[smsxgli]

And yes, every steps ran on the same CPU. I build firefox locally on my own PC.

[workingjubilee]

Oh. I don't believe the Rust toolchain has anything to do with PGO or ThinLTO except sometimes turning it on via flags, am I wrong?

[smsxgli]

I think thin LTO is handled by lld, and PGO is handled by LLVM. And firefox is build with PGO, and first build without LTO, post-profiling one with thin LTO across clang and rustc. As I said, I am using arch linux, so if you need, I can post my build script PKGBUILD, which contains the build steps.

[smsxgli]

cross-language LTO in Firefox for all platforms
reddit

[eddyb]

And yes, every steps ran on the same CPU.

What happens if you replace all occurrences of "native" with the appropriate value? (haswell, I assume?)
So -C target-cpu=haswell in RUSTFLAGS and -mcpu=haswell in CFLAGS.

Right now I'm expecting either native detection to silently fail (is any containerization done as part of the build, that could hide some CPU details somehow?) or something to be mismatched between Clang and rustc.

[smsxgli]

Ok, I will have a try.
During buiding, I used "clean buid" called in arch linux, using systemd-nspawn, I am afraid it's not containerization.
FreeBSD have the same problem according to this