Miscompilation of Bevy with MSVC

[PROMETHIA-27]

Compiling on windows with the latest CI artifact build.

Full error message:

   Compiling bevy_reflect v0.7.0
error: could not compile `bevy_reflect`

Caused by:
  process didn't exit successfully: `rustc --crate-name bevy_reflect --edition=2021 C:\Users\elect\.cargo\registry\src\github.com-1ecc6299db9ec823\bevy_reflect-0.7.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C debuginfo=2 -C debug-assertions=on -C metadata=93cc29b344f5fd9d -C extra-filename=-93cc29b344f5fd9d --out-dir C:\Assets\ProgrammingProjects\Rust\testing\target\debug\deps -L dependency=C:\Assets\ProgrammingProjects\Rust\testing\target\debug\deps --extern bevy_reflect_derive=C:\Assets\ProgrammingProjects\Rust\testing\target\debug\deps\bevy_reflect_derive-7a7ec19481b36f7b.dll --extern bevy_utils=C:\Assets\ProgrammingProjects\Rust\testing\target\debug\deps\libbevy_utils-638c5a2ec5f001df.rmeta --extern downcast_rs=C:\Assets\ProgrammingProjects\Rust\testing\target\debug\deps\libdowncast_rs-f56d5e827a193966.rmeta --extern erased_serde=C:\Assets\ProgrammingProjects\Rust\testing\target\debug\deps\liberased_serde-9c11269d65f6973f.rmeta --extern parking_lot=C:\Assets\ProgrammingProjects\Rust\testing\target\debug\deps\libparking_lot-63f537677751ed65.rmeta --extern serde=C:\Assets\ProgrammingProjects\Rust\testing\target\debug\deps\libserde-b6c12e91e4361d2b.rmeta --extern thiserror=C:\Assets\ProgrammingProjects\Rust\testing\target\debug\deps\libthiserror-546b84b988865560.rmeta --cap-lints allow -Cpanic=abort -Zpanic-abort-tests -Zcodegen-backend=C:\users\elect\desktop\cg_clif\build\bin\rustc_codegen_cranelift.dll --sysroot C:\users\elect\desktop\cg_clif\build -L native=C:\Users\elect\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib` (exit code: 0xc0000005, STATUS_ACCESS_VIOLATION)

Not sure what's causing this. Happens consistently, so it's not random, but I have no idea what's in bevy_reflect that causes it. It's a pretty simple crate, doesn't do anything particularly weird.

Unfortunately this crate is depended on by the most important bevy crates, so bevy as a whole can't be compiled on cranelift.
Not certain if this is limited to windows/just my PC, unable to test it anywhere else. Going to try and play with it to see if I can learn anything about it.

EDIT: This appears to not apply to the main branch of bevy. Instead, the main branch has the same error on bevy_hierarchy.
EDIT 2: This appears to also not apply to compiling the main branch directly, rather than as a git dependency. That works fine.
EDIT 3: After some testing, these appear to be the conditions of the crash:

1. There must be a tuple struct type T present in a dependency with the following conditions: (named structs and any enum types do not cause this issue)
2. The type must contain a field which is an instantiation of a generic type, in which the instantiation passes a slice or array type of (seemingly) any type or length directly. Passing a &[T] does not trigger it; only a [T]. As such this does not occur if nesting types, unless the nesting is to produce a multidimensional array. This also does not occur if the field itself is an array or slice. Not all dynamically sized types cause this issue; str is fine. Not all types taking a const generic parameter cause this issue; a Custom<const T: u32> is fine.
3. The type must #[derive(Component)], referring to bevy_ecs's Component trait. Manual impls do not cause this issue. The manual impl is as follows:

impl Component for T {
    type Storage = TableStorage;
}

No other derive I tested causes this issue.

This issue occurs for the Children type in bevy_hierarchy which has the following definition:

/// Contains references to the child entities of this entity
#[derive(Component, Debug, Reflect)]
#[reflect(Component, MapEntities)]
pub struct Children(pub(crate) SmallVec<[Entity; 8]>);

Unable to identify the problem in bevy_reflect. There are no instances of #[derive(Component)], but many other macros which are significantly more complex than Component.

[bjorn3]

Are you able to attach a debugger and get a backtrace when it crashes?

[PROMETHIA-27]

Yes, but I'm not sure what to look for. It might be hitting a UD2 instruction but I don't really think so, that's about all that stands out to me. The stack is just a bunch of offsets in the assembly. I tried building from source with --debug to see if that would put debug info back in, but that doesn't seem to have done anything.

Edit: Also just noticed that jit and lazy-jit have the same issue with a hello world project.

[bjorn3]

I'm probably not going to get around debugging this myself soon as I don't have a windows dev environment for rust set up. I unfortunately also don't have a clue where the bug could be, so I can't do a blind attempt at fixing it either. Linux has the best support as that is what I use myself. *Bsd also works really well as it is close enough to Linux. macOS should work too most of the time, but support is a tiny bit worse as it uses a different object file format. Windows support is not the best as you noticed. Only the MinGW toolchain is supported at all.

[PROMETHIA-27]

That's unfortunate. I don't have the expertise to debug any further than I have, but hopefully someone will be able to pick this up at a later date.

[bjorn3]

https://github.com/bjorn3/rustc_codegen_cranelift/pull/1255 shows an ABI incompatibility with MSVC. Debugging that one should probably be easier than this.

[afonso360]

Hey @PROMETHIA-27, would you be able to test this branch to see if it fixes the issues you are experiencing?

It contains a fix for an ABI issue that sounds suspiciously like what you are hitting.

[PROMETHIA-27]

I was unable to build the project (kept getting error: the 'cargo.exe' binary, normally provided by the 'cargo' component, is not applicable to the 'nightly-2022-07-27-x86_64-pc-windows-gnu' toolchain), but I grabbed an artifact off of CI and that worked! Looks like bevy can now run on cargo_cranelift on windows. Unfortunately jit and lazy-jit seem to still just STATUS_ACCESS_VIOLATION rustc while compiling. Maybe it's not properly removing the execution protection on the allocations it uses to emit code?

[afonso360]

but I grabbed an artifact off of CI and that worked!

That's great! 🎉

Unfortunately jit and lazy-jit seem to still just STATUS_ACCESS_VIOLATION rustc while compiling. Maybe it's not properly removing the execution protection on the allocations it uses to emit code?

I haven't really tested anything with the jit, I think I disabled it early on for windows when porting the testsuite into rust.

Hopefully it's something simple 😄

[bjorn3]

but I grabbed an artifact off of CI and that worked!

Nice!

Unfortunately jit and lazy-jit seem to still just STATUS_ACCESS_VIOLATION rustc while compiling. Maybe it's not properly removing the execution protection on the allocations it uses to emit code?

Cranelift-jit should set the mappings to RX before an attempt at executing the code is made.

[afonso360]

We do run the cranelift test suite on the JIT, and it runs fine on windows. But I don't know much beyond that.

[bjorn3]

@PROMETHIA-27 Did https://github.com/bjorn3/rustc_codegen_cranelift/pull/1284 fix this?