Large const allocations no longer lead to graceful errors on aarch64

[saethlin]

I just spun up an aarch64 (c6g.metal) instance to look at codegen, and I noticed that two of our UI tests do not pass:

    [ui] tests/ui/consts/large_const_alloc.rs
    [ui] tests/ui/consts/promoted_running_out_of_memory_issue-130687.rs

I can reduce one of the failing tests to this:

#![crate_type = "lib"]
pub const FOO: &[u8] = &[0_u8; (1 << 47) - 1];

If I am on an x86_64 host targeting aarch64-unknown-linux-gnu, using stable or nightly, this will immediately fail to compile with:

error[E0080]: evaluation of constant value failed
 --> demo.rs:2:25
  |
2 | pub const FOO: &[u8] = &[0_u8; (1 << 47) - 1];
  |                         ^^^^^^^^^^^^^^^^^^^^^ tried to allocate more memory than available to compiler

error: aborting due to 1 previous error

If my host is aarch64-unknown-linux-gnu using stable, I also get that diagnostic. But nightly toolchains on an aarch64 host allocate more and more memory, eventually dying with SIGKILL.

Bisection points to #135262 as the cause, which suggests that this is a miscompile.

searched nightlies: from nightly-2024-01-01 to nightly-2025-01-23
regressed nightly: nightly-2025-01-13
searched commit range: eb54a50...48a426e
regressed commit: c0f6a1c

bisected with cargo-bisect-rustc v0.6.9

Host triple: aarch64-unknown-linux-gnu
Reproduce with:

cargo bisect-rustc --script script.sh --start 2024-01-01 

This is worth further investigation. Of course it is plausible that a change to the CI configuration would alter the build artifacts from CI, but bisecting a test failure from a stage1 and stage2 toolchain to a CI change is odd.

[saethlin]

The difference in behavior here is of course that the PR in question also changed the compiler to use jemalloc, which is now passing MAP_NORESERVE to the big mmap call.

I thought we were already using jemalloc on x86_64; my x86_64 desktop has the same logical cores and memory as my c6g.metal instance, as well as the same overcommit settings. So it's quite odd that there is a difference in behavior here.

I wonder if this is somehow related to the variable page size on aarch64?

[saethlin]

but bisecting a test failure from a stage1 and stage2 toolchain to a CI change is odd.

The difference-maker here is definitely enabling jemalloc = true in config.toml, which is one of the things that linked PR does in our dist aarch64 builds.

[workingjubilee]

cc @mrkajetanp

[mrkajetanp]

two of our UI tests do not pass

We were never able to reproduce this outside of Docker, when running either natively on AArch64 Linux or in qemu AArch64 Linux those tests pass just fine, that's why we filed this under a Docker issue and skipped them in the CI. More investigation needed indeed, thanks for pointing it out!

[workingjubilee]

That is something that seems to be a valid reason to reject a PR.

[mrkajetanp]

To clarify, whatever the original problem is it was not introduced with the CI changes in question. We were skipping large_const_alloc inside our internal CI runs ~6 months ago already, and those runs weren't even using jemalloc.
The test suite was just not run on AArch64 on here prior to the CI changes moving the dist build.

I'm a little surprised that the CI changes modify the behaviour there. It'd be nice to confirm that it's definitely switching on jemalloc that's making it behave differently and not something else. What I don't get is, how come the CI changes make the diagnostic not pop-up? Does that specific diagnostic behave differently depending on whether jemalloc is switched on or not?

[WaffleLapkin]

The diagnostic originates here (this error is later turned into the diagnostic):

rust/compiler/rustc_middle/src/mir/interpret/allocation.rs

Line 304 in 99768c8

let bytes = Bytes::zeroed(size, align).ok_or_else(fail)?;

Since Bytes::zeroed allocates memory, it can be affected by the allocator.