building the std for aarch64_be-unknown-linux-gnu_ilp32 twice in a row, the binary libcompiler_builtins.rlib is inconsistent

[lxy19980601]

We have experimented with recent versions of rust(from 2023-10-30 to 2023-12-27).
use RUSTC_BOOTSTRAP=1 cargo build --target aarch64_be-unknown-linux-gnu_ilp32 -Z build-std
It has been found that the 2023-11-05 version has binary inconsistencies in libcompiler_builtins.rmeta, and the 11-06 version is completely consistent. So th problem is fixed on 11-05. And I also found that 11-05 uses compiler_builtins v0.1.101, but 11-06 uses compiler_builtins v0.1.103.
I replaced the compiler_builtins version in rust-2023-11-05 from v0.1.101 to v0.1.103, and then found libcompiler_builtins.rmeta is consistent.

[lxy19980601]

Maybe some changes of crate compiler_builtin from version v0.1.101 to version v0.1.103 caused this problem.

[bjorn3]

If I understand you correctly this issue has been fixed already since compiler-builtins 0.1.103. rust-lang/compiler-builtins#549 was probably the fix. Given that this has been fixed already, can this issue be closed?

[aerions]

I guess I need to add more context to clarify more about our issue. The beginning of the event is that, when building rust std for target aarch64_be-unknown-linux-gnu_ilp32, most rlib exhibit binary inconsistencies, including the libstd so itself. After some investigation, it comes accross that the main issue was hash difference for compiler_builtins between two build.
rust-lang/compiler-builtins#549 did fix the problem. However, at the end of the day, I assume rust should ensure the compile result to be consistent between different builds when using the same enviroment+source code right?

Maybe there could be some verification setup for binary comparison? For example, ci running every week or monthly check on those issue

[bjorn3]

Compiler builtins itself was giving a nondeterministic output of it's build script, which caused libcompiler_builtins.rlib to change and as a result everything that depends on it to change too. The build script of compiler builtins has been fixed in 0.1.103.

There is not much we can do about non-deterministic build scripts and proc macros. They are not sandboxed and may intentionally depend on the environment. For example they could be reading C header files from the system to produce bindings (eg because the target doesn't have a stable ABI) or read from a database. Optional sandboxing of proc macros by compiling them to wasm has been concidered, but for build scripts sandboxing is guaranteed to break things as one of the main reasons they exist is to invoke an external C compiler. And even with sandboxing there can be sources of non-determinism through for example multithreading inside build scripts. (or even deliberate random number generation)

[davidtwco]

I'm going to close this as there's nothing for us to do - the binary inconsistency described in this issue has been resolved. rust-lang/compiler-builtins#549 fixed the issue by changing from a HashMap to a BTreeMap in compiler-builtins' build.rs (HashMap doesn't guarantee an iteration order, which meant that a generated source file would change each build for no reason).