rustdoc crashing on linux-sparc64 due to unaligned access

[glaubitz]

Since version 1.31.0, rustdoc crashes when running the test src/test/run-make-fulldeps/rustdoc-error-lines:

(sid_sparc64-dchroot)glaubitz@kyoto:~/rust$ cd src/test/run-make-fulldeps/rustdoc-error-lines
(sid_sparc64-dchroot)glaubitz@kyoto:~/rust/src/test/run-make-fulldeps/rustdoc-error-lines$ LD_LIBRARY_PATH="/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/test/run-make-fulldeps/rustdoc-error-lines/rustdoc-error-lines:/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/lib:/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage0-bootstrap-tools/sparc64-unknown-linux-gnu/release/deps:/usr/lib:" '/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/bin/rustdoc' -L /home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/lib/rustlib/sparc64-unknown-linux-gnu/lib --test input.rs
Bus error
(sid_sparc64-dchroot)glaubitz@kyoto:~/rust/src/test/run-make-fulldeps/rustdoc-error-lines$
(sid_sparc64-dchroot)glaubitz@kyoto:~/rust/src/test/run-make-fulldeps/rustdoc-error-lines$ LD_LIBRARY_PATH="/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/test/run-make-fulldeps/rustdoc-error-lines/rustdoc-error-lines:/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/lib:/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage0-bootstrap-tools/sparc64-unknown-linux-gnu/release/deps:/usr/lib:" '/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/bin/rustdoc' -L /home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/lib/rustlib/sparc64-unknown-linux-gnu/lib --test input.rs
Bus error
(sid_sparc64-dchroot)glaubitz@kyoto:~/rust/src/test/run-make-fulldeps/rustdoc-error-lines$ gdb /home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/bin/rustdoc       GNU gdb (Debian 8.2-1) 8.2                                                                                                                                                  Copyright (C) 2018 Free Software Foundation, Inc.                                                                                                                           License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "sparc64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/bin/rustdoc...(no debugging symbols found)...done.
(gdb) set environment LD_LIBRARY_PATH "/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/test/run-make-fulldeps/rustdoc-error-lines/rustdoc-error-lines:/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/lib:/home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage0-bootstrap-tools/sparc64-unknown-linux-gnu/release/deps:/usr/lib:"
(gdb) r -L /home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/lib/rustlib/sparc64-unknown-linux-gnu/lib --test input.rs
Starting program: /home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/bin/rustdoc -L /home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/lib/rustlib/sparc64-unknown-linux-gnu/lib --test input.rs
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/sparc64-linux-gnu/libthread_db.so.1".
[New Thread 0xffff80010793e100 (LWP 16987)]

Thread 2 "rustdoc" received signal SIGBUS, Bus error.
[Switching to Thread 0xffff80010793e100 (LWP 16987)]
0x000001000038a48c in pulldown_cmark::parse::RawParser::new_with_links_and_callback ()
(gdb) bt
#0  0x000001000038a48c in pulldown_cmark::parse::RawParser::new_with_links_and_callback ()
#1  0x00000100003855c0 in pulldown_cmark::passes::Parser::new_with_broken_link_callback ()
#2  0x000001000038546c in pulldown_cmark::passes::Parser::new ()
#3  0x000001000014a8a4 in rustdoc::html::markdown::find_testable_code ()
#4  0x00000100003357a4 in <rustdoc::test::HirCollector<'a, 'hir> as rustc::hir::intravisit::Visitor<'hir>>::visit_item ()
#5  0x00000100002e2178 in <scoped_tls::ScopedKey<T>>::set ()
#6  0x00000100003304d4 in rustdoc::test::run ()
#7  0x000001000009b3fc in rustdoc::main_args ()
#8  0x00000100002e0b68 in <scoped_tls::ScopedKey<T>>::set ()
#9  0x0000010000082ae4 in syntax::with_globals ()
#10 0x000001000015b600 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#11 0x000001000015bf40 in std::panicking::try::do_call ()
#12 0xffff8001044fdf58 in __rust_maybe_catch_panic () from /home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/lib/libstd-3b39188da0a8cce7.so
#13 0x0000010000108be4 in <F as alloc::boxed::FnBox<A>>::call_box ()
#14 0xffff8001044eddc8 in std::sys_common::thread::start_thread () from /home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/lib/libstd-3b39188da0a8cce7.so
#15 0xffff8001044b6300 in std::sys::unix::thread::Thread::new::thread_start ()
   from /home/glaubitz/rust/build/sparc64-unknown-linux-gnu/stage2/lib/libstd-3b39188da0a8cce7.so
#16 0xffff80010478fe68 in start_thread () from /lib/sparc64-linux-gnu/libpthread.so.0
#17 0xffff800104aa8274 in ?? () from /lib/sparc64-linux-gnu/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)

I have not worked out yet whether this is a regression in the Rust compiler itself such that it generates code with unaligned access (we had this before) or whether rustdoc itself contains broken code.

CC @jrtc27 @psumbera @karcherm

[glaubitz]

Looking at the build log (https://buildd.debian.org/status/fetch.php?pkg=rustc&arch=sparc64&ver=1.31.0%2Bdfsg1-2&stamp=1545047706&raw=0), it looks like it's affecting rustdoc only. And it looks like an issue in the crate https://github.com/raphlinus/pulldown-cmark.

[nagisa]

Would it be possible to disassemble the function in question? Does the failure occur with the official builds? If so, bisecting with cargo-bisect-rustc might help to quickly narrow down the cause.

I would do it myself, but I’ve been unsuccessful in setting up sparc VMs (installed debian fails to boot, and I haven’t gotten around to dealing with it yet).

---

pulldown-cmark itself has no unsafe code so the bug is definitely in either:

1. The compiler;
2. Libraries;
3. Dependencies (but there aren’t many of them…)

[glaubitz]

I am currently bisecting this, I'm almost there :).

As for SPARC machines, we have set up a SPARC T5 VM in the gcc compile farm which every developer can use who applies for a free account at the gcc compile farm, see: https://gcc.gnu.org/wiki/CompileFarm There are both SPARC machines running Linux as well as Solaris available.

[glaubitz]

Found the commit which introduced the bug:

06b8c3ee5b52dc849259655578afca70cb32dda3 is the first bad commit
commit 06b8c3ee5b52dc849259655578afca70cb32dda3
Author: Colin Pronovost <colin.pron@live.com>
Date:   Mon Sep 24 23:09:44 2018 -0400

    Rely only on base alignment and offset for computing field alignment
    
    Fix #54028

:040000 040000 b5af5c83972da8e9ff9f10e923608400e82f2964 439ea89453a897b6c984eb58b7cd43836b299e3e M      src

[nagisa]

This smellls similar to #56267, although the fixes for that particular problem have been backported and are available in stable and the lint pass does not complain about pulldown_cmark in any way...

[pnkfelix]

discussed at T-compiler meeting. P-high, since this may be symptom of broader UB beyond the case outlined here.

[nagisa]

cc @eddyb do you have any idea in what way could the bisected PR have broken things related to alignment?

The relevant PR here is #54547

[infinity0]

Since you guys don't run tests on sparc64, is there a way to write a test for this that crashes on x86-64? I originally told @glaubitz to ignore the sparc64 failures but it looks like it was good that he chased it down!