1.14.0 mipsel test failures: f32

[infinity0]

Build machine: https://db.debian.org/machines.cgi?host=mipsel-aql-02
Build log: https://buildd.debian.org/status/fetch.php?pkg=rustc&arch=mipsel&ver=1.14.0%2Bdfsg1-3&stamp=1484088148
Raw build log: https://buildd.debian.org/status/fetch.php?pkg=rustc&arch=mipsel&ver=1.14.0%2Bdfsg1-3&stamp=1484088148&raw=1

This is after applying #38650 and #38675. The LLVM diff between Debian vs Rust is here.

Note that, because of these failures, the whole-compiler tests (e.g. run-pass etc) have not been run, but they probably also have failures.

Relevant links:
https://github.com/rust-lang/rust/blob/1.14.0/src/librustc_back/target/mipsel_unknown_linux_gnu.rs
https://github.com/rust-lang/rust/blob/1.14.0/mk/cfg/mipsel-unknown-linux-gnu.mk

test f32::tests::test_asinh ... ok
test f32::tests::test_ceil ... ok
thread '<unnamed>' panicked at 'NaN is not approximately equal to 0.54930615', src/libstd/f32.rs:1831
stack backtrace:
test f32::tests::test_classify ... ok
test f32::tests::test_exp ... ok
test f32::tests::test_exp2 ... ok
test f32::tests::test_floor ... ok
test f32::tests::test_fract ... ok
thread '<unnamed>' panicked at 'assertion failed: `(left == right)` (left: `(0, -122)`, right: `(0.5, -122)`)', src/libstd/f32.rs:1759
   1: 0x56672dbb - std::sys::imp::backtrace::tracing::imp::write::h33de6ffd1ea231bf
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:42
   2: 0x566c0307 - std::panicking::default_hook::{{closure}}::hcabac60ea7b48baf
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:247
   3: 0x566a0357 - std::panicking::rust_panic_with_hook::h059b28291193443c
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:263
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:451
   4: 0x5669fd4f - std::panicking::begin_panic::h929be95e04ca164b
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:413
   5: 0x5669fb3f - std::panicking::begin_panic_fmt::ha7d9cbddb8702778
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:397
   6: 0x565069e7 - std::f32::tests::test_atanh::h8bd5519b0026e3d0
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/f32.rs:1831
   7: 0x566d31e3 - <F as test::FnBox<T>>::call_box::he8581d59e8028413
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libtest/lib.rs:1265
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libtest/lib.rs:141
   8: 0x566c4497 - std::panicking::try::do_call::h83b58d81c874206d
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libtest/lib.rs:1211
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panic.rs:295
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:356
   9: 0x5671999f - __rust_maybe_catch_panic
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libpanic_unwind/lib.rs:97
  10: 0x566c3913 - std::panicking::try::do_call::h636b19c00d03e824
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:332
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panic.rs:351
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libtest/lib.rs:1210
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panic.rs:295
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:356
  11: 0x5671999f - __rust_maybe_catch_panic
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libpanic_unwind/lib.rs:97
  12: 0x566cc717 - <F as alloc::boxed::FnBox<A>>::call_box::h15256864a2eaf226
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:332
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panic.rs:351
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/thread/mod.rs:287
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/liballoc/boxed.rs:595
  13: 0x5670f07b - std::sys::imp::thread::Thread::new::thread_start::ha102a6120fc52763
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/liballoc/boxed.rs:605
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/sys_common/thread.rs:21
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/sys/unix/thread.rs:84
  14: 0x77ad68bf - <unknown>
stack backtrace:
   1: 0x56672dbb - std::sys::imp::backtrace::tracing::imp::write::h33de6ffd1ea231bf
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:42
   2: 0x566c0307 - std::panicking::default_hook::{{closure}}::hcabac60ea7b48baf
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:247
   3: 0x566a0357 - std::panicking::rust_panic_with_hook::h059b28291193443c
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:263
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:451
   4: 0x5669fd4f - std::panicking::begin_panic::h929be95e04ca164b
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:413
   5: 0x5669fb3f - std::panicking::begin_panic_fmt::ha7d9cbddb8702778
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:397
   6: 0x56505e13 - std::f32::tests::test_frexp::h3a1b32397c1dbb90
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/f32.rs:1759
   7: 0x566d31e3 - <F as test::FnBox<T>>::call_box::he8581d59e8028413
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libtest/lib.rs:1265
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libtest/lib.rs:141
   8: 0x566c4497 - std::panicking::try::do_call::h83b58d81c874206d
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libtest/lib.rs:1211
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panic.rs:295
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:356
   9: 0x5671999f - __rust_maybe_catch_panic
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libpanic_unwind/lib.rs:97
  10: 0x566c3913 - std::panicking::try::do_call::h636b19c00d03e824
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:332
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panic.rs:351
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libtest/lib.rs:1210
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panic.rs:295
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:356
  11: 0x5671999f - __rust_maybe_catch_panic
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libpanic_unwind/lib.rs:97
  12: 0x566cc717 - <F as alloc::boxed::FnBox<A>>::call_box::h15256864a2eaf226
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panicking.rs:332
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/panic.rs:351
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/thread/mod.rs:287
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/liballoc/boxed.rs:595
  13: 0x5670f07b - std::sys::imp::thread::Thread::new::thread_start::ha102a6120fc52763
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/liballoc/boxed.rs:605
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/sys_common/thread.rs:21
                at /«BUILDDIR»/rustc-1.14.0+dfsg1/src/libstd/sys/unix/thread.rs:84
  14: 0x77ad68bf - <unknown>
test f32::tests::test_frexp ... FAILED
test f32::tests::test_frexp_nowin ... ok
test f32::tests::test_atanh ... FAILED
[..]
failures:
    f32::tests::test_atanh
    f32::tests::test_frexp
    f32::tests::test_log

test result: FAILED. 772 passed; 3 failed; 0 ignored; 0 measured

/«BUILDDIR»/rustc-1.14.0+dfsg1/mk/tests.mk:423: recipe for target 'tmp/check-stage2-T-mipsel-unknown-linux-gnu-H-mipsel-unknown-linux-gnu-std.ok' failed
make[2]: *** [tmp/check-stage2-T-mipsel-unknown-linux-gnu-H-mipsel-unknown-linux-gnu-std.ok] Error 101

[jcowgill]

This could potentially be the the Loongson FPU bug which we see every so often. I'm currently building it on a non-Loongson machine to see if it passes there.

[jcowgill]

It looks like this was the Loongson FPU bug. After building it on a different machine those tests pass. The tests in #39014 also pass (so those must be big-endian specific).

It now gets to the concurrency docs tests where test _10 and test _11 hang. These look like the 2 channels examples from https://github.com/rust-lang/rust/blob/master/src/doc/book/concurrency.md#channels

touch tmp/check-stage2-T-mipsel-unknown-linux-gnu-H-mipsel-unknown-linux-gnu-doc-book-concurrency.ok.start_time
LD_LIBRARY_PATH=/<<BUILDDIR>>/rustc-1.14.0+dfsg1/mipsel-unknown-linux-gnu/stage2/lib:/usr/lib/llvm-3.9/lib:$LD_LIBRARY_PATH mipsel-unknown-linux-gnu/stage2/bin/rustdoc --cfg dox --test /<<BUILDDIR>>/rustc-1.14.0
+dfsg1/src/doc/book/concurrency.md --test-args "" && touch -r tmp/check-stage2-T-mipsel-unknown-linux-gnu-H-mipsel-unknown-linux-gnu-doc-book-concurrency.ok.start_time tmp/check-stage2-T-mipsel-unknown-linux-gnu
-H-mipsel-unknown-linux-gnu-doc-book-concurrency.ok && rm tmp/check-stage2-T-mipsel-unknown-linux-gnu-H-mipsel-unknown-linux-gnu-doc-book-concurrency.ok.start_time

running 13 tests
test _0 ... ok
test _1 ... ok
test _2 ... ignored
test _3 ... ok
test _4 ... ignored
test _5 ... ignored
test _6 ... ignored
test _12 ... ok
test _8 ... ignored
test _7 ... ok
test _9 ... ok
test _10 has been running for over 60 seconds
test _11 has been running for over 60 seconds
E: Build killed with signal TERM after 150 minutes of inactivity
--------------------------------------------------------------------------------
Build finished at 2017-01-17T02:01:53Z

[jcowgill]

The hang occurs in this loop in _ZN4core4sync6atomic23atomic_compare_exchange17hd108e2f5e9a6b935

=> 0x5556de48 <+720>:   lw      at,148(sp)
   0x5556de4c <+724>:   ll      v0,0(at)
   0x5556de50 <+728>:   lw      v1,152(sp)
   0x5556de54 <+732>:   sw      v0,92(sp)
   0x5556de58 <+736>:   bne     v0,v1,0x5556de74 <_ZN4core4sync6atomic23atomic_compare_exchange17hd108e2f5e9a6b935E+764>
   0x5556de5c <+740>:   nop
   0x5556de60 <+744>:   lw      at,156(sp)
   0x5556de64 <+748>:   lw      v0,148(sp)
   0x5556de68 <+752>:   sc      at,0(v0)
   0x5556de6c <+756>:   beqz    at,0x5556de48 <_ZN4core4sync6atomic23atomic_compare_exchange17hd108e2f5e9a6b935E+720>

This code is highly dodgy. It executes 3 loads and 1 store within an LL/SC block which is probably why the SC instruction always fails and the loop goes around forever. I guess this is a bug in LLVM's atomics implementation.

[jcowgill]

FYI the LLVM atomics bug was fixed here: https://bugs.llvm.org/show_bug.cgi?id=32020

I haven't tested rust with a fixed LLVM yet though.

[infinity0]

Just noting that at the time of writing, the LLVM bug was reopened, the previous fix was not good.