LLVM miscompilation/assert in `ui/issues/issue-23036.rs` on wasm32-unknown-unknown target

[the8472]

Context: #93697 - the test had to be disabled there to get a fix in

With patched library/std/src/path.rs running the UI tests targeting wasm32 causes this failure with LLVM asserts enabled:

error: test compilation failed although it shouldn't!
status: signal: 6 (core dumped)
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/issues/issue-23036.rs" "-Zthreads=1" "--target=wasm32-unknown-unknown" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/issues/issue-23036/a.wasm" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/wasm32-unknown-unknown/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/issues/issue-23036/auxiliary"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
rustc: /checkout/src/llvm-project/llvm/lib/MC/WasmObjectWriter.cpp:149: void {anonymous}::writePatchableLEB(llvm::raw_pwrite_stream&, uint64_t, uint64_t) [with int W = 5; uint64_t = long unsigned int]: Assertion `SizeLen == W' failed.
------------------------------------------

With LLVM asserts disabled it produces invalid wasm bytecode:

/home/the8472/workspace/rust/src/etc/wasm32-shim.js:17
let m = new WebAssembly.Module(buffer);
        ^

CompileError: WebAssembly.Module(): Compiling function #3:"_ZN11issue_230364main17hf6fa4aae1344ccadE" failed: Invalid opcode 0xff @+874
    at Object.<anonymous> (/home/the8472/workspace/rust/src/etc/wasm32-shim.js:17:9)
    at Module._compile (node:internal/modules/cjs/loader:1097:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1149:10)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47

Manually executing the rustc command printed by the UI test and adding --emit=llvm-ir generates the following IR repro.ir.txt. Running that through llc (built by bootstrap) also produces invalid bytecode (checked with wasm-dis).

Tested on base commit 686663a + PR #93697 (currently 45082b0)

@rustbot ping llvm

[rustbot]

Hey LLVM ICE-breakers! This bug has been identified as a good
"LLVM ICE-breaking candidate". In case it's useful, here are some
instructions for tackling these sorts of bugs. Maybe take a look?
Thanks! <3

cc @ayazhafiz @camelid @comex @cuviper @hdhoang @henryboisdequin @heyrutvik @higuoxing @JOE1994 @jryans @mmilenko @nagisa @nikic @Noah-Kennedy @SiavoshZarrasvand @spastorino @vertexclique

[comex]

Quick observations:

The failing assert:

template <int W>
void writePatchableLEB(raw_pwrite_stream &Stream, uint64_t X, uint64_t Offset) {
  uint8_t Buffer[W];
  unsigned SizeLen = encodeULEB128(X, Buffer, W);
  assert(SizeLen == W);
  Stream.pwrite((char *)Buffer, SizeLen, Offset);
}

Looking at the implementation of encodeULEB128, it seems like the only way this assert could fail is if the value is too large to fit in the specified number of bytes.

There are two calls to writePatchableLEB<5>, and one of them has a check for out-of-range values already, so it's probably the other one in WasmObjectWriter::applyRelocations.

[the8472]

No longer blocks the PR, it now disables the test on wasm32 instead.

[nikic]

Upstream issue: llvm/llvm-project#54012