[MIR] Stage1 `libcore` crashes with SIGILL on ARM

[MagaTailor]

Since a few days ago, I'm unable to bootstrap rustc on ARM. Stage0 passes ok, but then during libcore compilation I'm getting a consistent crash:

(gdb) run
Starting program: ../arm-unknown-linux-gnueabihf/stage1/bin/rustc --cfg stage1 -C codegen-units=1 -C target-cpu=cortex-a5 -C target-feature=-neon,-slowfpvmlx -Z orbit -C link-args=-s -O --cfg rtopt -C rpath -C prefer-dynamic --target=arm-unknown-linux-gnueabihf -C target-feature=+v7,+vfp3 -L arm-unknown-linux-gnueabihf/rt -L native=../arm-unknown-linux-gnueabihf/llvm/Release/lib --out-dir arm-unknown-linux-gnueabihf/stage1/lib/rustlib/arm-unknown-linux-gnueabihf/lib -C extra-filename=-fe3cdf61 -C metadata=fe3cdf61 src/libcore/lib.rs
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
[New Thread 0xb23c8280 (LWP 10171)]

Program received signal SIGILL, Illegal instruction.
[Switching to Thread 0xb23c8280 (LWP 10171)]
0xb5ccd4c8 in rustc_trans::mir::constant::MirConstContext::new::hfbf8e74461aa3c9a ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
(gdb) bt
#0  0xb5ccd4c8 in rustc_trans::mir::constant::MirConstContext::new::hfbf8e74461aa3c9a ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#1  0xb5cca2bc in rustc_trans::mir::constant::_$LT$impl$u20$mir..MirContext$LT$$u27$bcx$C$$u20$$u27$tcx$GT$$GT$::trans_constant::h0fc06663a5c5fd5c () from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#2  0xb5cc97e8 in rustc_trans::mir::operand::_$LT$impl$u20$mir..MirContext$LT$$u27$bcx$C$$u20$$u27$tcx$GT$$GT$::trans_operand::hf76be65ffae28ad3 () from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#3  0xb5cda61c in rustc_trans::mir::rvalue::_$LT$impl$u20$mir..MirContext$LT$$u27$bcx$C$$u20$$u27$tcx$GT$$GT$::trans_rvalue_operand::hd939e0a331e9c597 () from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#4  0xb5cbff28 in rustc_trans::mir::block::_$LT$impl$u20$mir..MirContext$LT$$u27$bcx$C$$u20$$u27$tcx$GT$$GT$::trans_block::hb1d87007c9e87f30 () from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#5  0xb5ba9aa8 in rustc_trans::mir::trans_mir::hf51ccc32c7a8ddd3 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#6  0xb5ba1ff0 in rustc_trans::base::trans_closure::hcb1c30ddf81aba40 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#7  0xb5bab570 in rustc_trans::base::trans_fn::h1508ebeec7351f87 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#8  0xb5bb8950 in rustc_trans::base::trans_item::hd6a260625a685bf2 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#9  0xb5bd3e10 in _$LT$base..TransItemsWithinModVisitor$LT$$u27$a$C$$u20$$u27$tcx$GT$$u20$as$u20$rustc..hir..intravisit..Visitor$LT$$u27$v$GT$$GT$::visit_item::heb661116fa8a1e87 () from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#10 0xb5bd2524 in rustc::hir::intravisit::walk_item::hf7ed747a4fa46491 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#11 0xb5bcbbe4 in _$LT$base..TransModVisitor$LT$$u27$a$C$$u20$$u27$tcx$GT$$u20$as$u20$rustc..hir..intravisit..Visitor$LT$$u27$v$GT$$GT$::visit_item::h65d871e71b3668c9 () from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#12 0xb5bbf8ec in rustc_trans::base::trans_crate::hca67d2711539a441 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_trans-fe3cdf61.so
#13 0xb6e06194 in rustc_driver::driver::phase_4_translate_to_llvm::h5f028163cc963293 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_driver-fe3cdf61.so
#14 0xb6e04640 in rustc_driver::driver::compile_input::_$u7b$$u7b$closure$u7d$$u7d$::h540343a1efd859c9 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_driver-fe3cdf61.so
#15 0xb6e10f78 in rustc_driver::driver::phase_3_run_analysis_passes::_$u7b$$u7b$closure$u7d$$u7d$::h0c96940e550f2879 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_driver-fe3cdf61.so
#16 0xb6e16264 in rustc::ty::context::TyCtxt::create_and_enter::h167e75cbc44e6d50 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_driver-fe3cdf61.so
#17 0xb6e0a56c in rustc_driver::driver::phase_3_run_analysis_passes::h75db68819aee0b65 ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_driver-fe3cdf61.so
#18 0xb6dd47b8 in rustc_driver::driver::compile_input::h5e5e4501615d7d4b ()
   from ../arm-unknown-linux-gnueabihf/stage1/lib/librustc_driver-fe3cdf61.so
Backtrace stopped: Cannot access memory at address 0x1d1cfff8

(gdb) disass 0xb5ccd4c8,+1
Dump of assembler code from 0xb5ccd4c8 to 0xb5ccd4c9:
=> 0xb5ccd4c8 <_ZN11rustc_trans3mir8constant15MirConstContext3new17hfbf8e74461aa3c9aE+404>:     str     r0, [r0, r8]!

Before getting that trace I had no idea it was related to MIR in any way. I'll have to give old trans a shot.

EDIT:
Yeah, old trans chugs along without a hitch.

[MagaTailor]

@eddyb Looks like an LLVM codegen issue that's triggered by MIR - any debugging suggestions?

@alexcrichton Provided it's not just my llvm, do ARM issues have any bearing on switching to MIR by default?

[eddyb]

@petevine I don't understand, is the instruction being executed not supported on the architecture you're running it on? Because then it's a coincidence that the librustc_trans/mir/constant.rs is compiled with that instruction and you could probably reproduce with code using types of the same size and shape.

[MagaTailor]

The instruction is just a standard register store (word size) with preindexed offset, so it's probably about wrong offset or alignment.

[MagaTailor]

About three days ago it was still crashing (a full rebuild of llvm didn't help either) but today, it's back to normal.

[MagaTailor]

No, I was a little too hasty. It was just my little experiment (done stage0 with old trans) which allowed stage1 to proceed even with -Z orbit but the same scenario has just happened during stage2 libcore.

So this bug could be summarised like this:
rustc built with-Zorbitcrashes when trying to use-Zorbitbut not old trans codegen

[MagaTailor]

@eddyb I've got a debug trace:

Program received signal SIGILL, Illegal instruction.
[Switching to Thread 0xb2a282b0 (LWP 25309)]
collections::vec::{{impl}}::extend_desugared<core::option::Option<rustc_trans::mir::constant::Const>,core::iter::Map<core::ops::Range<usize>, closure>> (self=<optimized out>, iterator=...)
    at src/libcollections/vec.rs:1424
1424                    ptr::write(self.get_unchecked_mut(len), element);
(gdb) bt
#0  collections::vec::{{impl}}::extend_desugared<core::option::Option<rustc_trans::mir::constant::Const>,core::iter::Map<core::ops::Range<usize>, closure>> (self=<optimized out>, iterator=...) at src/libcollections/vec.rs:1424
#1  collections::vec::{{impl}}::from_iter<core::option::Option<rustc_trans::mir::constant::Const>,core::iter::Map<core::ops::Range<usize>, closure>> (iter=...) at src/libcollections/vec.rs:1325
#2  rustc_data_structures::indexed_vec::{{impl}}::from_iter<rustc::mir::repr::Local,core::option::Option<rustc_trans::mir::constant::Const>,core::iter::Map<core::ops::Range<usize>, closure>> (iter=...) at src/librustc_data_structures/indexed_vec.rs:180
#3  core::iter::iterator::Iterator::collect<core::iter::Map<core::ops::Range<usize>, closure>,rustc_data_structures::indexed_vec::IndexVec<rustc::mir::repr::Local, core::option::Option<rustc_trans::mir::constant::Const>>> (self=...) at src/libcore/iter/iterator.rs:1207
#4  rustc_trans::mir::constant::{{impl}}::new (ccx=<optimized out>, mir=<optimized out>, substs=<optimized out>, args=...)
    at src/librustc_trans/mir/constant.rs:221
#5  0xb6554cd4 in rustc_trans::mir::constant::{{impl}}::trans_constant (self=0xb2a15170, bcx=<optimized out>, constant=<optimized out>)
    at src/librustc_trans/mir/constant.rs:911
#6  0xb65541e0 in rustc_trans::mir::operand::{{impl}}::trans_operand (self=0x0, bcx=0xb2a13b60, operand=<optimized out>)
    at src/librustc_trans/mir/operand.rs:234
#7  0xb65644cc in rustc_trans::mir::rvalue::{{impl}}::trans_rvalue_operand (self=<optimized out>, bcx=..., rvalue=<optimized out>, 
    debug_loc=...) at src/librustc_trans/mir/rvalue.rs:476
#8  0xb654aa28 in rustc_trans::mir::statement::{{impl}}::trans_statement (statement=<optimized out>, self=<optimized out>, bcx=...)
    at src/librustc_trans/mir/statement.rs:36
#9  rustc_trans::mir::block::{{impl}}::trans_block (self=<optimized out>, bb=...) at src/librustc_trans/mir/block.rs:103
#10 0xb64354e8 in rustc_trans::mir::trans_mir (fcx=<optimized out>) at src/librustc_trans/mir/mod.rs:249
#11 0xb642df4c in rustc_trans::base::trans_closure (ccx=<optimized out>, decl=<optimized out>, body=<optimized out>, 
    llfndecl=<optimized out>, instance=..., inlined_id=271, sig=<optimized out>, abi=<optimized out>, closure_env=...)
    at src/librustc_trans/base.rs:1854
#12 0xb6456f78 in rustc_trans::base::trans_instance (ccx=<optimized out>, instance=...) at src/librustc_trans/base.rs:1953
#13 rustc_trans::trans_item::{{impl}}::define (self=<optimized out>, ccx=<optimized out>) at src/librustc_trans/trans_item.rs:89
#14 0xb6446660 in rustc_trans::base::trans_crate (tcx=..., mir_map=<optimized out>, analysis=...) at src/librustc_trans/base.rs:2638
#15 0xb6e0bc74 in rustc_driver::driver::phase_4_translate_to_llvm::{{closure}} () at src/librustc_driver/driver.rs:1055
#16 rustc::util::common::time<rustc_trans::CrateTranslation,closure> (what=..., f=..., do_it=<optimized out>)
    at src/librustc/util/common.rs:38
#17 rustc_driver::driver::phase_4_translate_to_llvm (tcx=..., mir_map=..., analysis=...) at src/librustc_driver/driver.rs:1053
#18 0xb6e0a1a4 in rustc_driver::driver::compile_input::{{closure}} (tcx=..., mir_map=..., analysis=..., result=...)
    at src/librustc_driver/driver.rs:204
#19 0xb6e167f4 in rustc_driver::driver::phase_3_run_analysis_passes::{{closure}}<closure,core::result::Result<(rustc::session::config::OutputFilenames, rustc_trans::CrateTranslation), usize>> (tcx=...) at src/librustc_driver/driver.rs:1016
#20 0xb6e1b8a4 in std::thread::local::{{impl}}::with<core::cell::Cell<core::option::Option<(*const rustc::ty::context::tls::ThreadLocalGlobalCtxt, *const rustc::ty::context::tls::ThreadLocalInterners)>>,closure,core::result::Result<core::result::Result<(rustc::session::config::OutputFilenames, rustc_trans::CrateTranslation), usize>, usize>> (f=..., self=<optimized out>) at src/libstd/thread/local.rs:245
#21 rustc::ty::context::tls::enter<closure,core::result::Result<core::result::Result<(rustc::session::config::OutputFilenames, rustc_trans::CrateTranslation), usize>, usize>> (gcx=<optimized out>, interners=<optimized out>, f=...) at src/librustc/ty/context.rs:903
#22 std::thread::local::{{impl}}::with<core::cell::Cell<fn(syntax_pos::Span, &mut core::fmt::Formatter) -> core::result::Result<(), core::fmt::Error>>,closure,core::result::Result<core::result::Result<(rustc::session::config::OutputFilenames, rustc_trans::CrateTranslation), usize>, usize>> (f=..., self=<optimized out>) at src/libstd/thread/local.rs:245
#23 rustc::ty::context::tls::enter_global<closure,core::result::Result<core::result::Result<(rustc::session::config::OutputFilenames, rustc_trans::CrateTranslation), usize>, usize>> (gcx=..., f=...) at src/librustc/ty/context.rs:887
#24 rustc::ty::context::{{impl}}::create_and_enter<closure,core::result::Result<core::result::Result<(rustc::session::config::OutputFilenames, rustc_trans::CrateTranslation), usize>, usize>> (s=<optimized out>, arenas=<optimized out>, def_map=..., named_region_map=..., 
    map=..., freevars=..., maybe_unused_trait_imports=..., region_maps=..., lang_items=..., stability=..., crate_name=..., f=...)
    at src/librustc/ty/context.rs:685
#25 0xb6e0ff50 in rustc_driver::driver::phase_3_run_analysis_passes<closure,core::result::Result<(rustc::session::config::OutputFilenames, rustc_trans::CrateTranslation), usize>> (sess=<optimized out>, hir_map=..., analysis=..., resolutions=..., arenas=<optimized out>, 
    name=..., f=...) at src/librustc_driver/driver.rs:898
#26 0xb6dd55e0 in rustc_driver::driver::compile_input (sess=<optimized out>, cstore=<optimized out>, cfg=..., input=0xb2a25ff8, 
    outdir=0xb2a26028, output=0xb2a26018, addl_plugins=..., control=0xb2a257e8) at src/librustc_driver/driver.rs:170
#27 0xb6dbf10c in rustc_driver::run_compiler_with_file_loader<syntax::codemap::RealFileLoader> (args=..., callbacks=..., 
    loader=<optimized out>) at src/librustc_driver/lib.rs:220
#28 rustc_driver::run_compiler (args=..., callbacks=...) at src/librustc_driver/lib.rs:157
#29 0xb6dcce68 in rustc_driver::run::{{closure}} () at src/librustc_driver/lib.rs:135
#30 rustc_driver::monitor::{{closure}}<closure> () at src/librustc_driver/lib.rs:1043
#31 std::panic::{{impl}}::call_once<(),closure> (self=...) at src/libstd/panic.rs:251
#32 std::panicking::try::call<closure> (f=0xb2a27c54) at src/libstd/panicking.rs:272
#33 0xb6b32238 in panic_unwind::__rust_maybe_catch_panic (f=0x40000000, data=0xaa482e18 "", data_ptr=0xb2a27c50, vtable_ptr=0xb2a27c4c)
    at src/libpanic_unwind/lib.rs:91
#34 0xb6dcd8a8 in std::thread::local::{{impl}}::with<core::cell::Cell<usize>,closure,core::result::Result<(), Box<Any>>> (
    self=<optimized out>, f=...) at src/libstd/thread/local.rs:245
#33 0xb6b32238 in panic_unwind::__rust_maybe_catch_panic (f=0x40000000, data=0xaa482e18 "", data_ptr=0xb2a27c50, vtable_ptr=0xb2a27c4c)
    at src/libpanic_unwind/lib.rs:91
#34 0xb6dcd8a8 in std::thread::local::{{impl}}::with<core::cell::Cell<usize>,closure,core::result::Result<(), Box<Any>>> (
    self=<optimized out>, f=...) at src/libstd/thread/local.rs:245
#35 std::panicking::try<(),std::panic::AssertUnwindSafe<closure>> (f=...) at src/libstd/panicking.rs:235
#36 std::panic::catch_unwind<std::panic::AssertUnwindSafe<closure>,()> (f=...) at src/libstd/panic.rs:307
#37 alloc::boxed::{{impl}}::call_box<(),closure> (self=0x7f56eb88, args=<optimized out>) at src/liballoc/boxed.rs:544
#38 0xb6b218e8 in alloc::boxed::{{impl}}::call_once<(),()> (self=...) at src/liballoc/boxed.rs:554
#39 std::sys_common::thread::start_thread (main=<optimized out>) at src/libstd/sys/common/thread.rs:23
#40 std::sys::thread::{{impl}}::new::thread_start (main=0x7f56ec48) at src/libstd/sys/unix/thread.rs:74
#41 0xb2cd5fa0 in start_thread (arg=0xb2a282b0) at pthread_create.c:314
#42 0xb69a8d9c in ?? () at ../ports/sysdeps/unix/sysv/linux/arm/nptl/../clone.S:92 from /lib/arm-linux-gnueabihf/libc.so.6

Remember the get<usize> jemalloc segfault? There's no jemalloc here and MIR bootstrap used to work fine about 3 weeks ago.

[eddyb]

Looks like this is indeed a coincidence, you should be able to reproduce using types with the same layout.

[eddyb]

This snippet should generate the same instructions and trigger SIGILL:

#[inline(never)]
fn foo(n: usize) -> Vec<Option<(*mut (), &'static ())>> {
    (0..n).map(|_| None).collect()
}

fn main() {
    let _ = (foo(10), foo(32));
}

[MagaTailor]

Yes it does! I initially forgot to use optimizations and the code ran fine. As expected, it also runs fine without -Zorbit or at opt-level=1.

[MagaTailor]

Here's the difference in the generated assembly:

21c21
<     25b8:       ba000070        blt     2780 <_ZN6sigill3foo17hfaa1209845a9b530E+0x214>
---
>     25b8:       ba00006f        blt     277c <_ZN6sigill3foo17hfaa1209845a9b530E+0x210>
26c26
<     25cc:       eb003fc5        bl      124e8 <__rust_allocate>
---
>     25cc:       eb003fbf        bl      124d0 <__rust_allocate>
29c29
<     25d8:       0a00006d        beq     2794 <_ZN6sigill3foo17hfaa1209845a9b530E+0x228>
---
>     25d8:       0a00006b        beq     278c <_ZN6sigill3foo17hfaa1209845a9b530E+0x220>
49c49
< 2628:       23000000        movwcs  r0, #0
---
>     2628:       23000000        movwcs  r0, #0
67c67
<     2670:       eb003fae        bl      12530 <__rust_reallocate>
---
>     2670:       eb003fa8        bl      12518 <__rust_reallocate>
71c71
<     2680:       eb003f98        bl      124e8 <__rust_allocate>
---
>     2680:       eb003f92        bl      124d0 <__rust_allocate>
79c79
<     26a0:       e7a00008        str     r0, [r0, r8]!
---
>     26a0:       e7a01008        str     r1, [r0, r8]!

dump-mir.txt
dump-nomir.txt

[eddyb]

If anything, this looks like a LLVM bug. cc @alexcrichton

[MagaTailor]

I was about to ask if opening a bug upstream with both llvm-ir and assembly versions side by side was a good course of action?

[eddyb]

@petevine Our LLVM has been getting put of date because upgrading it was blocked by moving to only cmake.
But now that's done and we're waiting for the upgrade to happen, AFAIK.
I've already seen a few bugs that already had fixes upstream, some of which got cherry-picked.
I pinged @alexcrichton to see what he knows about the upgrade and about testing if it helps with this issue, since finding completely new bugs in LLVM can be hit and miss.