compiletest should be built with the stage2 compiler not the stage0 compiler

[infinity0]

See #57709. Unfortunately the "fix" to that bug breaks the build when building using 1.31.0 and you have to unapply the patch to make it work against 1.31.0.

e.g. from https://buildd.debian.org/status/fetch.php?pkg=rustc&arch=amd64&ver=1.32.0%2Bdfsg1-1&stamp=1548672224&raw=0 :

rustc command: "LD_LIBRARY_PATH"="/usr/lib:/<<BUILDDIR>>/rustc-1.32.0+dfsg1/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/release/deps:/usr/lib" "/usr/bin/rustc" "--crate-name" "compiletest" "

The problem here is that if stage0 is 1.31.0 then the patch MUST be unapplied, if stage0 is 1.32.0 then the patch MUST be applied. Obviously we can't do both at once.

However in both Debian and Fedora we occasionally use either the current or the previous version of rustc to build a given version of rustc, and both of these versions should work.

cc @cuviper

[infinity0]

Doing things this way (using stage2) would also have ensured that #57709 was detected early, and never made it into a release.

[Mark-Simulacrum]

We would prefer to build tools such as compiletest with the bootstrap compiler because stage1, stage2 compilers take a long time to build.

I think largely we just can't at this point support building with multiple compilers -- i.e., the bootstrap compiler should be the same one we use in tree. We just don't have the CI budget and resources to test and support multiple versions of the compiler for crates using unstable features.

As such, I think the only resolution here will be to close this...

cc @rust-lang/infra and @alexcrichton though since this is somewhat of a policy type decision (and we don't have a firm policy AFAIK).

[cuviper]

As a matter of policy, we do need such rebuild-bootstrapping to continue to be supported to some extent, even if it's only a "tier-2" use case. That is, fixing these sort of issues reactively is still better than nothing.

[infinity0]

Is conditional compilation based on the compiler version possible in Rust today? That could be an alternative solution, instead of always using stage2. The language here is a bit technical and I couldn't figure it out.

[infinity0]

I guess for now, I could hack that sort of logic into the Debian build script manually in an ad-hoc way - applying the patch vs not, based on the stage0 compiler version.

[alexcrichton]

I think if we wanted to solve this issue the best way to do it would be to:

• Require compiletest compiles on stable, except for the dependency on the test crate
• Have compiletest depend on std/test in-tree binaries, so we always build them (with the stage0 compiler)

That way compiletest works with rebuild-bootstraping and normal use cases because it's not using anything unstable except libtest, which is built from source to link against and already solves this problem otherwise.

[Mark-Simulacrum]

Have compiletest depend on std/test in-tree binaries, so we always build them (with the stage0 compiler)

I think in general a policy of unstable tools/dependencies having the requirement of being in-tree deps instead of beta is reasonable, yeah.

I can try to make a patch that makes compiletest (and if there are any other such tools) do this sometime this week.

[infinity0]

Same issue with 1.40.0 again, when using itself as stage0:

   Compiling compiletest v0.0.0 (/<<PKGBUILDDIR>>/src/tools/compiletest)
     Running `CARGO=/usr/bin/cargo CARGO_MANIFEST_DIR=/<<PKGBUILDDIR>>/src/tools/compiletest CARGO_PKG_VERSION=0.0.0 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_MINOR=0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_NAME=compiletest CARGO_PKG_DESCRIPTION= CARGO_PKG_REPOSITORY= CARGO_PKG_AUTHORS='The Rust Project Developers' CARGO_PKG_HOMEPAGE= CARGO_PKG_VERSION_PRE= LD_LIBRARY_PATH='/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/release/deps:/usr/lib' /<<PKGBUILDDIR>>/build/bootstrap/debug/rustc --edition=2018 --crate-name compiletest src/tools/compiletest/src/main.rs --error-format json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=2 -C debuginfo=0 -C metadata=3bff92eb77ed034a -C extra-filename=-3bff92eb77ed034a --out-dir /<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps --target powerpc64le-unknown-linux-gnu -C linker=powerpc64le-linux-gnu-gcc -L dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps -L dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/release/deps --extern diff=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libdiff-f7c70ead82d3bb7b.rlib --extern env_logger=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libenv_logger-42bd3e818814c443.rlib --extern getopts=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libgetopts-9ad175f6b0afc4b5.rlib --extern lazy_static=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblazy_static-0d6c5f722411bcab.rlib --extern libc=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblibc-0f34c56984de760b.rlib --extern log=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblog-1d69c45aedc3973c.rlib --extern regex=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libregex-877563e5fb3e2b70.rlib --extern rustfix=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/librustfix-52c76b39e6b4bcbd.rlib --extern serde=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde-6b72e6c8366be6dd.rlib --extern serde_json=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde_json-0db31fa1a064a5dd.rlib --extern walkdir=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libwalkdir-243dfb4a0f7f76f8.rlib -C link-args=-Wl,-z,relro --cap-lints warn --remap-path-prefix=/<<PKGBUILDDIR>>=/usr/src/rustc-1.40.0 -Zexternal-macro-backtrace -Zbinary-dep-depinfo -L native=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/build/backtrace-sys-6a5e41a89dbb2581/out`
rustc command: "LD_LIBRARY_PATH"="/usr/lib:/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/release/deps:/usr/lib" "/usr/bin/rustc" "--edition=2018" "--crate-name" "compiletest" "src/tools/compiletest/src/main.rs" "--error-format" "json" "--json=diagnostic-rendered-ansi" "--crate-type" "bin" "--emit=dep-info,link" "-C" "opt-level=2" "-C" "debuginfo=0" "-C" "metadata=3bff92eb77ed034a" "-C" "extra-filename=-3bff92eb77ed034a" "--out-dir" "/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps" "--target" "powerpc64le-unknown-linux-gnu" "-C" "linker=powerpc64le-linux-gnu-gcc" "-L" "dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps" "-L" "dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/release/deps" "--extern" "diff=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libdiff-f7c70ead82d3bb7b.rlib" "--extern" "env_logger=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libenv_logger-42bd3e818814c443.rlib" "--extern" "getopts=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libgetopts-9ad175f6b0afc4b5.rlib" "--extern" "lazy_static=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblazy_static-0d6c5f722411bcab.rlib" "--extern" "libc=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblibc-0f34c56984de760b.rlib" "--extern" "log=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblog-1d69c45aedc3973c.rlib" "--extern" "regex=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libregex-877563e5fb3e2b70.rlib" "--extern" "rustfix=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/librustfix-52c76b39e6b4bcbd.rlib" "--extern" "serde=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde-6b72e6c8366be6dd.rlib" "--extern" "serde_json=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde_json-0db31fa1a064a5dd.rlib" "--extern" "walkdir=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libwalkdir-243dfb4a0f7f76f8.rlib" "-C" "link-args=-Wl,-z,relro" "--cap-lints" "warn" "--remap-path-prefix=/<<PKGBUILDDIR>>=/usr/src/rustc-1.40.0" "-Zexternal-macro-backtrace" "-Zbinary-dep-depinfo" "-L" "native=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/build/backtrace-sys-6a5e41a89dbb2581/out" "--sysroot" "/usr" "-C" "debug-assertions=n"
sysroot: "/usr"
libdir: "/usr/lib"
error[E0063]: missing field `time_options` in initializer of `test::TestOpts`
   --> src/tools/compiletest/src/main.rs:545:5
    |
545 |     test::TestOpts {
    |     ^^^^^^^^^^^^^^ missing `time_options`

error[E0063]: missing field `test_type` in initializer of `test::TestDesc`
   --> src/tools/compiletest/src/main.rs:701:23
    |
701 |                 desc: test::TestDesc {
    |                       ^^^^^^^^^^^^^^ missing `test_type`

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0063`.

Did not run successfully: Ok(ExitStatus(ExitStatus(256)))
"/usr/bin/rustc" "--edition=2018" "--crate-name" "compiletest" "src/tools/compiletest/src/main.rs" "--error-format" "json" "--json=diagnostic-rendered-ansi" "--crate-type" "bin" "--emit=dep-info,link" "-C" "opt-level=2" "-C" "debuginfo=0" "-C" "metadata=3bff92eb77ed034a" "-C" "extra-filename=-3bff92eb77ed034a" "--out-dir" "/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps" "--target" "powerpc64le-unknown-linux-gnu" "-C" "linker=powerpc64le-linux-gnu-gcc" "-L" "dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps" "-L" "dependency=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/release/deps" "--extern" "diff=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libdiff-f7c70ead82d3bb7b.rlib" "--extern" "env_logger=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libenv_logger-42bd3e818814c443.rlib" "--extern" "getopts=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libgetopts-9ad175f6b0afc4b5.rlib" "--extern" "lazy_static=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblazy_static-0d6c5f722411bcab.rlib" "--extern" "libc=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblibc-0f34c56984de760b.rlib" "--extern" "log=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/liblog-1d69c45aedc3973c.rlib" "--extern" "regex=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libregex-877563e5fb3e2b70.rlib" "--extern" "rustfix=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/librustfix-52c76b39e6b4bcbd.rlib" "--extern" "serde=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde-6b72e6c8366be6dd.rlib" "--extern" "serde_json=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libserde_json-0db31fa1a064a5dd.rlib" "--extern" "walkdir=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/deps/libwalkdir-243dfb4a0f7f76f8.rlib" "-C" "link-args=-Wl,-z,relro" "--cap-lints" "warn" "--remap-path-prefix=/<<PKGBUILDDIR>>=/usr/src/rustc-1.40.0" "-Zexternal-macro-backtrace" "-Zbinary-dep-depinfo" "-L" "native=/<<PKGBUILDDIR>>/build/powerpc64le-unknown-linux-gnu/stage0-bootstrap-tools/powerpc64le-unknown-linux-gnu/release/build/backtrace-sys-6a5e41a89dbb2581/out" "--sysroot" "/usr" "-C" "debug-assertions=n"

CC @cuviper AFAICT you do not have a patch that fixes this on Fedora either.

[infinity0]

For 1.40.0 the relevant commit to conditionally-apply or not is f6832ad.

For the upcoming 1.41.0 (or is it 1.42.0) stable release it seems we will have to do the same thing for 241d2e7.