segfault on ppc64le when compiling master using nightly-2016-10-19

[amboar]

Hi,

I see that nightlies and betas are now up for ppc{,64{,le}}! I've had a crack at building master on a ppc64le host using nightly-2016-10-19 as stage0, but no luck:

rustc: powerpc64le-unknown-linux-gnu/stage0/lib/rustlib/powerpc64le-unknown-linux-gnu/lib/libcore
compile: powerpc64le-unknown-linux-gnu/rustllvm/RustWrapper.o
compile: powerpc64le-unknown-linux-gnu/rustllvm/PassWrapper.o
compile: powerpc64le-unknown-linux-gnu/rustllvm/ArchiveWrapper.o
Segmentation fault
/home/arj/rust/mk/target.mk:191: recipe for target 'powerpc64le-unknown-linux-gnu/stage0/lib/rustlib/powerpc64le-unknown-linux-gnu/lib/stamp.core' failed
make: *** [powerpc64le-unknown-linux-gnu/stage0/lib/rustlib/powerpc64le-unknown-linux-gnu/lib/stamp.core] Error 139
make: *** Waiting for unfinished jobs....

My naive gdb bt gave:

$ CFG_LLVM_LINKAGE_FILE=/home/arj/rust/powerpc64le-unknown-linux-gnu/rt/llvmdeps.rs LD_LIBRARY_PATH=/home/arj/rust/powerpc64le-unknown-linux-gnu/st[30/1939]
/home/arj/rust/powerpc64le-unknown-linux-gnu/llvm/lib:$LD_LIBRARY_PATH  gdb powerpc64le-unknown-linux-gnu/stage0/bin/rustc
Reading symbols from powerpc64le-unknown-linux-gnu/stage0/bin/rustc...(no debugging symbols found)...done.
(gdb) run --cfg stage0  -O --cfg rtopt -C rpath -C prefer-dynamic -C no-stack-check --target=powerpc64le-unknown-linux-gnu   -L "powerpc64le-unknown-linux-gnu/rt" -L native="/h
ome/arj/rust/powerpc64le-unknown-linux-gnu/llvm/lib"     --out-dir powerpc64le-unknown-linux-gnu/stage0/lib/rustlib/powerpc64le-unknown-linux-gnu/lib -C extra-filename=-3d98e1d
d -C metadata=3d98e1dd src/libcore/lib.rs
Starting program: /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/rustc --cfg stage0  -O --cfg rtopt -C rpath -C prefer-dynamic -C no-stack-check --target=powerpc64le-u
nknown-linux-gnu   -L "powerpc64le-unknown-linux-gnu/rt" -L native="/home/arj/rust/powerpc64le-unknown-linux-gnu/llvm/lib"     --out-dir powerpc64le-unknown-linux-gnu/stage0/li
b/rustlib/powerpc64le-unknown-linux-gnu/lib -C extra-filename=-3d98e1dd -C metadata=3d98e1dd src/libcore/lib.rs
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
[New Thread 0x3fffb2dfee80 (LWP 17275)]

Thread 2 "rustc" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x3fffb2dfee80 (LWP 17275)]
0x00003fffb7e25e48 in drop::hf6b6fd210b1b8022 () from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/librustc_driver-1ed44d7390b9470e.so
(gdb) bt
#0  0x00003fffb7e25e48 in drop::hf6b6fd210b1b8022 () from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/librustc_driver-1ed44d7390b9470e.so
#1  0x00003fffb7e23fe8 in drop::he6b5c1de4328d7c4 () from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/librustc_driver-1ed44d7390b9470e.so
#2  0x00003fffb7e1b144 in drop::ha6e76cd3e2aaf918 () from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/librustc_driver-1ed44d7390b9470e.so
#3  0x00003fffb7e25ea4 in drop::hf6b6fd210b1b8022 () from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/librustc_driver-1ed44d7390b9470e.so
#4  0x00003fffb7e145a8 in drop::h6c5bef721832e06f () from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/librustc_driver-1ed44d7390b9470e.so
#5  0x00003fffb7eb4f4c in rustc_driver::driver::compile_input::h67df2cb32ed2d9fe ()
   from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/librustc_driver-1ed44d7390b9470e.so
#6  0x00003fffb7ed9574 in rustc_driver::run_compiler::hf62e42444e71fbda ()
   from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/librustc_driver-1ed44d7390b9470e.so
#7  0x00003fffb7e005bc in std::panicking::try::do_call::h0565df63f4851a92 ()
   from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/librustc_driver-1ed44d7390b9470e.so
#8  0x00003fffb7d088c8 in __rust_maybe_catch_panic () from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/libstd-17342542cc541012.so
#9  0x00003fffb7e26724 in _$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::hebe081d66cfa068a ()
   from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/librustc_driver-1ed44d7390b9470e.so
#10 0x00003fffb7cfb3f8 in std::sys::thread::Thread::new::thread_start::hbaf2dd6b96963058 ()
   from /home/arj/rust/powerpc64le-unknown-linux-gnu/stage0/bin/../lib/libstd-17342542cc541012.so
#11 0x00003fffb34a84a0 in start_thread (arg=0x3fffb2dfee80) at pthread_create.c:335
#12 0x00003fffb7ba7e74 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Happy to dig in further if the debug symbols are available somewhere.

[alexcrichton]

Note that for the most part we're just producing cross compilers, not actually running them. In that sense it may be possible that they just segfault on startup (as this appears to be doing) and it may be a legit bug in trans or something like that (unsure)

[nagisa]

T-compiler. Might also want a powerpc tag of some sort.

[amboar]

The cross-build I created a while back didn't have this issue. I plan on cross-building again when I have a moment to see if I can reproduce. Off the top of my head two differences are they were built from different commits on master, and my initial cross-build was done with the standard build system, not rustbuild. I'll try to iron out whether either of those has had an effect.

[cuviper]

I believe PPC has multiple possible page sizes -- perhaps this is a case like #36994? If your page size is larger than what jemalloc was compiled with, then madvise(MADV_DONTNEED) behaves badly, and you may be able to get around this with MALLOC_CONF=lg_dirty_mult:-1. If your actual page size is smaller than jemalloc's expectations, it's still possible for other errors to crop up. You can check your own with getconf PAGESIZE, but I'm not sure there's an easy way to check how some binary was compiled.

[amboar]

@cuviper thanks, exporting MALLOC_CONF=lg_dirty_mult:-1 appears to work around the crash. Page size on the ppc64le host was 64K.

[alexcrichton]

Whoa thanks for the diagnosis @cuviper! I've opened a PR to disable jemalloc on powerpc: #37392