`--test -Z lto` optimizes the benchmarks out

Without LTO:

running 4 tests
test xxhash::rust::test ... ok
test xxhash::native::test ... ok
test xxhash::native::bench ... bench:     75535 ns/iter (+/- 564) = 3470 MB/s
test xxhash::rust::bench   ... bench:     68471 ns/iter (+/- 953) = 3828 MB/s

test result: ok. 2 passed; 0 failed; 0 ignored; 2 measured

With LTO:

running 4 tests
test xxhash::rust::test ... ok
test xxhash::native::test ... ok
test xxhash::native::bench ... bench:     75598 ns/iter (+/- 109) = 3467 MB/s
test xxhash::rust::bench   ... bench:         0 ns/iter (+/- 0) = 262144000 MB/s

test result: ok. 2 passed; 0 failed; 0 ignored; 2 measured

[huonw]

This is a fundamental problem of micro-benchmarks with an optimising compiler: it can and will change code in unexpected ways (including removing code that it sees as dead, i.e. results that are not used where the computation is has no external effects).

One way around this is to trick the compiler into thinking that a result is used. E.g. for a sufficiently compilcated computation (so the optimiser cannot constant-fold it):

bh.iter(|| {
    let result = /* calculation */;
    if result == 0 { println!("no dead code elimination"); }
})

(works best if result is never zero, or choosing some other value to compare against.)

Even the stdlib has many instances of benchmarks that have their contents removed with optimisations on.

I would think that the change that LTO introduces is (a) just giving LLVM more passes with which to reduce and eliminate code, and (b) allowing the benchmarking harness (specifically the .iter function) itself to be inlined. (Compiling with --opt-level=3 may show a similar elimination of dead code.)

In any case, I believe this is a dupe of #8261.