Problem linking libunwind when compiling clippy

[jfgoog]

We are building the Rust compiler from nightly, from which we define a Bazel toolchain.

In addition to rustc, the toolchain includes other tools, such as clippy and rustfmt, which we build as well.

We require the binaries to be statically linked, so they are self-contained and do not depend on shared libraries.

In particular, we need to statically link the version of libunwind that gets built when compiling llvm.

So in config.toml we have

[rust]
llvm-libunwind = "in-tree"

[llvm]
static-libstdcpp = true
cflags = "-D_GLIBCXX_USE_CXX11_ABI=0 -stdlib=libc++ --rtlib=compiler-rt -unwindlib=libunwind -static-libstdc++ -static-libgcc -DUCHAR_TYPE=uint16_t -g0"
ldflags = "-L<staging_dir>/llvm/lib -lunwind -Bstatic -static-libstdc++ -static-libgcc  -g0"
ninja = true

Then, after compiling llvm, we hackishly force static linking by removing the .so: rm llvm/lib/libunwind.so*. This works around #29527

This works for rustc and rustfmt, but not clippy. It seems that clippy is not being told to use the in-tree version of libunwind, and so we get a linker error:

(using <BD> for brevity to indicate the common build directory path)

Building stage1 tool clippy-driver (x86_64-unknown-linux-gnu)
   Compiling ucd-trie v0.1.3
   Compiling pulldown-cmark v0.8.0
   Compiling rustc-semver v1.1.0
   Compiling if_chain v1.0.0
   Compiling rustc_tools_util v0.2.0 (/tmp/build-rustc-staging.g27EHWwDms/rustc-src/src/tools/clippy/rustc_tools_util)
   Compiling quine-mc_cluskey v0.2.4
   Compiling unicase v2.6.0
   Compiling itertools v0.9.0
   Compiling pest v2.1.3
   Compiling clippy v0.1.54 (/tmp/build-rustc-staging.g27EHWwDms/rustc-src/src/tools/clippy)
   Compiling clippy_utils v0.1.54 (/tmp/build-rustc-staging.g27EHWwDms/rustc-src/src/tools/clippy/clippy_utils)
   Compiling semver-parser v0.10.2
   Compiling semver v0.11.0
   Compiling cargo_metadata v0.12.0
   Compiling clippy_lints v0.1.54 (/tmp/build-rustc-staging.g27EHWwDms/rustc-src/src/tools/clippy/clippy_lints)
error: linking with `/usr/bin/clang++` failed: exit status: 1
  |
  = note: "/usr/bin/clang++" "-m64" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-Wl,--as-needed" "-L" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.0.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.1.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.10.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.11.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.12.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.13.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.14.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.15.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.2.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.3.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.4.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.5.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.6.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.7.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.8.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.9.rcgu.o" "-o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps" "-L" "<BD>/stage1-tools/release/deps" "-L" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libclippy_lints-2c94a74107bddb69.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libtoml-dd7456c55c44c90a.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libregex_syntax-4e074f16cdab16e6.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libcargo_metadata-572da3e0be85a7d1.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libserde_json-4b2f39e93e914c91.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libryu-fe03cf9014177fe1.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libitoa-254a369192b0e41b.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libpulldown_cmark-a3cb3d622d5299ec.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libmemchr-5fc53eaed5b93baf.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libunicase-08b754bdb3e79b40.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libbitflags-fa4146283c7dcf7f.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/liburl-e334d4999ae40e26.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libidna-4ccd292e8ff413b7.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libunicode_normalization-977261e1ba455667.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libtinyvec-5931f1f70217b006.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libunicode_bidi-306ecb356465bbde.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libform_urlencoded-c5adc592c24847fe.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libpercent_encoding-c42753e7ff933803.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libmatches-1ba42f8e6736589f.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libitertools-91c2c298f57ac430.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libeither-285099727ca0d3a5.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libquine_mc_cluskey-6ff02b0b5df613fb.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libsemver-cc320f53f6085e6e.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libsemver_parser-71420aeaef3a7d9b.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libpest-41b2d2072c136e80.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libucd_trie-848f9c70f929d69f.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libserde-bba5b8307b4c2e1f.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libclippy_utils-8af687cb6957b8f8.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/librustc_semver-f95ec72f9354327c.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libif_chain-14b305b09fe6df2e.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/librustc_tools_util-ddfcfb1da96998e3.rlib" "-L" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bdynamic" "-lrustc_driver-d0d9704efb4ebe56" "-Wl,--start-group" "-L" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lstd-bc7a0745f1fbbd95" "-Wl,--end-group" "-Wl,-Bstatic" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-d845064f5e2f3da0.rlib" "-Wl,-Bdynamic" "-lrt" "-ldl" "-lpthread" "-lm" "-lunwind" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,-rpath,$ORIGIN/../lib"
  = note: /usr/bin/ld: cannot find -lunwind
          clang: error: linker command failed with exit code 1 (use -v to see invocation)


error: aborting due to previous error

error: could not compile `clippy`

To learn more, run the command again with --verbose.
command did not execute successfully: "/tmp/build-rustc-staging.g27EHWwDms/rustc-beta-toolchain/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "12" "--release" "--frozen" "--manifest-path" "/tmp/build-rustc-staging.g27EHWwDms/rustc-src/src/tools/clippy/Cargo.toml" "--message-format" "json-render-diagnostics"
expected success, got: exit code: 101
Building stage1 tool cargo-clippy (x86_64-unknown-linux-gnu)
   Compiling clippy v0.1.54 (/tmp/build-rustc-staging.g27EHWwDms/rustc-src/src/tools/clippy)
error: linking with `/usr/bin/clang++` failed: exit status: 1
  |
  = note: "/usr/bin/clang++" "-m64" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-Wl,--as-needed" "-L" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.0.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.1.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.10.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.11.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.12.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.13.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.14.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.15.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.2.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.3.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.4.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.5.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.6.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.7.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.8.rcgu.o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.9.rcgu.o" "-o" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps" "-L" "<BD>/stage1-tools/release/deps" "-L" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libclippy_lints-2c94a74107bddb69.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libtoml-dd7456c55c44c90a.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libregex_syntax-4e074f16cdab16e6.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libcargo_metadata-572da3e0be85a7d1.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libserde_json-4b2f39e93e914c91.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libryu-fe03cf9014177fe1.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libitoa-254a369192b0e41b.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libpulldown_cmark-a3cb3d622d5299ec.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libmemchr-5fc53eaed5b93baf.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libunicase-08b754bdb3e79b40.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libbitflags-fa4146283c7dcf7f.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/liburl-e334d4999ae40e26.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libidna-4ccd292e8ff413b7.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libunicode_normalization-977261e1ba455667.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libtinyvec-5931f1f70217b006.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libunicode_bidi-306ecb356465bbde.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libform_urlencoded-c5adc592c24847fe.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libpercent_encoding-c42753e7ff933803.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libmatches-1ba42f8e6736589f.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libitertools-91c2c298f57ac430.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libeither-285099727ca0d3a5.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libquine_mc_cluskey-6ff02b0b5df613fb.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libsemver-cc320f53f6085e6e.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libsemver_parser-71420aeaef3a7d9b.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libpest-41b2d2072c136e80.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libucd_trie-848f9c70f929d69f.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libserde-bba5b8307b4c2e1f.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libclippy_utils-8af687cb6957b8f8.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/librustc_semver-f95ec72f9354327c.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libif_chain-14b305b09fe6df2e.rlib" "<BD>/stage1-tools/x86_64-unknown-linux-gnu/release/deps/librustc_tools_util-ddfcfb1da96998e3.rlib" "-L" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bdynamic" "-lrustc_driver-d0d9704efb4ebe56" "-Wl,--start-group" "-L" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lstd-bc7a0745f1fbbd95" "-Wl,--end-group" "-Wl,-Bstatic" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-d845064f5e2f3da0.rlib" "-Wl,-Bdynamic" "-lrt" "-ldl" "-lpthread" "-lm" "-lunwind" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,-rpath,$ORIGIN/../lib"
  = note: /usr/bin/ld: cannot find -lunwind
          clang: error: linker command failed with exit code 1 (use -v to see invocation)


error: aborting due to previous error

error: could not compile `clippy`

To learn more, run the command again with --verbose.
command did not execute successfully: "/tmp/build-rustc-staging.g27EHWwDms/rustc-beta-toolchain/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "12" "--release" "--frozen" "--manifest-path" "/tmp/build-rustc-staging.g27EHWwDms/rustc-src/src/tools/clippy/Cargo.toml" "--message-format" "json-render-diagnostics"
expected success, got: exit code: 101

Additional information:

• We are compiling in a debian:buster docker image.
• Possibly very relevant work in progress: build llvm libunwind.a in rustbuild #85600
• We are currently using the 2021-05-18 nightly (git version 3e99439), and building it with the 2021-04-07 beta compiler (git version 2157381).

[petrhosek]

Can you try using the following flags: -Clink-arg=-static-libgcc -Clink-arg=-unwindlib=libunwind? That will tell Clang to link libunwind statically, that is Clang will pass -l:libunwind.a to the linker.

[petrochenkov]

cc @12101111

[12101111]

The linking command line is assembled in linker_with_args

This is the end part of linking command, and the are added to command line by link_local_crate_native_libs_and_dependent_crate_libs:

-Wl,-Bstatic <Many rlibs>
"-Wl,-Bdynamic" "-lrustc_driver-d0d9704efb4ebe56" 
"-Wl,--start-group" "-L" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lstd-bc7a0745f1fbbd95" "-Wl,--end-group"
"-Wl,-Bstatic" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-d845064f5e2f3da0.rlib" 
"-Wl,-Bdynamic" "-lrt" "-ldl" "-lpthread" "-lm" "-lunwind" 
"-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc"

This part, among all rlib files before this are handled by add_upstream_rust_crates.

-Wl,-Bstatic <Many rlibs>
"-Wl,-Bdynamic" "-lrustc_driver-d0d9704efb4ebe56" 
"-Wl,--start-group" "-L" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lstd-bc7a0745f1fbbd95" "-Wl,--end-group"
"-Wl,-Bstatic" "<BD>/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-d845064f5e2f3da0.rlib" 

And compiler_builtins is alaways the last object file added here:

rust/compiler/rustc_codegen_ssa/src/back/link.rs

Lines 1952 to 1959 in df3d86b

	// compiler-builtins are always placed last to ensure that they're
	// linked correctly.
	// We must always link the `compiler_builtins` crate statically. Even if it
	// was already "included" in a dylib (e.g., `libstd` when `-C prefer-dynamic`
	// is used)
	if let Some(cnum) = compiler_builtins {
	add_static_crate::<B>(cmd, sess, codegen_results, tmpdir, crate_type, cnum);
	}

This part is handled by add_upstream_native_libraries

"-Wl,-Bdynamic" "-lrt" "-ldl" "-lpthread" "-lm" "-lunwind" 
"-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc"

"-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" clearly come from libc:

https://github.com/rust-lang/libc/blob/9c4944a6e5d71e11009461941b3999b1c7d7e51f/src/unix/mod.rs#L324

But which crate link to "-lrt" "-ldl" "-lpthread" "-lm" "-lunwind"?

Can you apply this patch:

--- a/compiler/rustc_codegen_ssa/src/back/link.rs
+++ b/compiler/rustc_codegen_ssa/src/back/link.rs
@@ -2157,6 +2157,7 @@ fn add_upstream_native_libraries(
             last = if (lib.kind, lib.name) == last { continue } else { (lib.kind, lib.name) };
 
             let verbatim = lib.verbatim.unwrap_or(false);
+            info!("link native lib: {:?} from crate #{}", lib, codegen_results.crate_info.crate_name[&cnum]);
             match lib.kind {
                 NativeLibKind::Dylib { as_needed } => {
                     cmd.link_dylib(name, verbatim, as_needed.unwrap_or(true))

and rebuild rustc and clippy with this:

export RUSTC_LOG=rustc_codegen_ssa::back::link=info

It should output like this:

2:rustcINFO rustc_codegen_ssa::back::link preparing Executable to "/home/han/rust-master/build/x86_64-unknown-linux-musl/stage2-tools/x86_64-unknown-linux-musl/release/deps/clippy_driver-46f8584c225f05ae"
2:rustcINFO rustc_codegen_ssa::back::link link native lib: NativeLib { kind: Static { bundle: None, whole_archive: None }, name: Some("llvm-wrapper"), cfg: None, verbatim: None } from crate #rustc_llvm
2:rustcINFO rustc_codegen_ssa::back::link link native lib: NativeLib { kind: Dylib { as_needed: None }, name: Some("LLVM-12libcxx"), cfg: None, verbatim: None } from crate #rustc_llvm
2:rustcINFO rustc_codegen_ssa::back::link link native lib: NativeLib { kind: Unspecified, name: Some("c++"), cfg: None, verbatim: None } from crate #rustc_llvm
2:rustcINFO rustc_codegen_ssa::back::link link native lib: NativeLib { kind: Static { bundle: None, whole_archive: None }, name: Some("psm_s"), cfg: None, verbatim: None } from crate #psm
2:rustcINFO rustc_codegen_ssa::back::link link native lib: NativeLib { kind: Static { bundle: None, whole_archive: None }, name: Some("unwind"), cfg: None, verbatim: None } from crate #unwind
2:rustcINFO rustc_codegen_ssa::back::link link native lib: NativeLib { kind: Unspecified, name: Some("c"), cfg: Some(MetaItem { path: Path { span: /home/han/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.93/src/unix/mod.rs:336:41: 336:44 (#0), segments: [PathSegment { ident: not#0, id: NodeId(4294967040), args: None }], tokens: None }, kind: List([MetaItem(MetaItem { path: Path { span: /home/han/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.93/src/unix/mod.rs:336:45: 336:59 (#0), segments: [PathSegment { ident: target_feature#0, id: NodeId(4294967040), args: None }], tokens: None }, kind: NameValue(Lit { token: Lit { kind: Str, symbol: "crt-static", suffix: None }, kind: Str("crt-static", Cooked), span: /home/han/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.93/src/unix/mod.rs:336:62: 336:74 (#0) }), span: /home/han/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.93/src/unix/mod.rs:336:45: 336:74 (#0) })]), span: /home/han/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.93/src/unix/mod.rs:336:41: 336:75 (#0) }), verbatim: None } from crate #libc
2:rustcINFO rustc_codegen_ssa::back::link link native lib: NativeLib { kind: Static { bundle: None, whole_archive: None }, name: Some("compiler-rt"), cfg: None, verbatim: None } from crate #compiler_builtins

[jfgoog]

(Note: wrote this reply before seeing 12101111's response, which I am still digesting. This is in response to petrhosek's comment)

Thanks! I'm working on following your suggestion. I thought I could do this by modifying rustflags in src/tools/clippy/.cargo/config:

[build]
rustflags = ["-Zunstable-options", "-Clink-arg=--static-libgcc", "-Clink-arg=--unwindlib=libunwind"]

But that cargo config doesn't seem to be picked up when we build clippy, because the link command, and the associated error, seems to be the same as what I posted above.

The overall build command we are using in the docker image is: python3 x.py build library/std compiler/rustc src/tools/rustfmt src/tools/cargo src/tools/clippy, which seems to use the following "cargo build" command to build clippy:

/tmp/build-rustc-staging.J6gyafxhMk/rustc-beta-toolchain/bin/cargo build --target x86_64-unknown-linux-gnu -Zbinary-dep-depinfo -j 12 --release --frozen --manifest-path /tmp/build-rustc-staging.J6gyafxhMk/rustc-src/src/tools/clippy/Cargo.toml --message-format json-render-diagnostics

[jfgoog]

Working on following 12101111's instructions. This will be a little complicated because, if I understand correctly, I need to patch and rebuild the beta rustc, because that's what's used to build the nightly. So I need to compile the compiler to compile my compiler. :)

[12101111]

You don't need build beta compiler, stage1 will contain this change and that's enough

https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html

[jfgoog]

OK. I am surprised by that, because the "cargo build" command for clippy uses the version of cargo in the beta toolchain (see my earlier comment)

[12101111]

You should build rust toolchain using ./x.py dist

[jfgoog]

output from "Building stage1 tool clippy-driver" to the linker error:

clippy-build.txt

This was built as described above. I am inquiring as to why we are not doing ./x.py dist

[12101111]

There should be lines that contain the log from clippy crate:
INFO rustc_codegen_ssa::back::link preparing Executable to ".....clippy_driver-.....
INFO rustc_codegen_ssa::back::link "clang++ ... -o clippy_driver-......
But I don't find them in your log.

[jfgoog]

I checked all the output I have from the build command. There are no instances of "preparing Executable to.*clippy".

Are you sure I don't need to apply your patch to the beta compiler? As I said, clippy seems to be getting built using "cargo build" from the beta compiler. It looks like we may be setting this explicitly in our config.toml:

[build]
vendor = true
docs = false
sanitizers = true
rustc = "<path to beta toolchain>/bin/rustc"
cargo = "<path to beta toolchain>/bin/cargo"
rustfmt = "<path to beta toolchain>/bin/rustfmt"
target = ["x86_64-unknown-linux-gnu", "wasm32-unknown-unknown"]

I'm going to try patching and rebuilding the beta toolchain.

[jfgoog]

Here's the output with the patch applied to the beta toolchain, but not the nightly (I hope I don't need it applied to both, but if so I can do that next)

clippy-build.txt

This does contain the following lines which may be of interest to you based on your earlier comment:

INFO rustc_codegen_ssa::back::link preparing Executable to "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5"
INFO rustc_codegen_ssa::back::link "/usr/bin/clang++" "-m64" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-Wl,--as-needed" "-L" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.0.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.1.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.10.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.11.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.12.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.13.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.14.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.15.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.2.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.3.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.4.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.5.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.6.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.7.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.8.rcgu.o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5.clippy_driver.b4xue1do-cgu.9.rcgu.o" "-o" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-2fc6d04ae4266cd5" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps" "-L" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps" "-L" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libclippy_lints-2c94a74107bddb69.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libtoml-dd7456c55c44c90a.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libregex_syntax-4e074f16cdab16e6.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libcargo_metadata-572da3e0be85a7d1.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libserde_json-4b2f39e93e914c91.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libryu-fe03cf9014177fe1.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libitoa-254a369192b0e41b.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libpulldown_cmark-a3cb3d622d5299ec.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libmemchr-5fc53eaed5b93baf.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libunicase-08b754bdb3e79b40.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libbitflags-fa4146283c7dcf7f.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/liburl-e334d4999ae40e26.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libidna-4ccd292e8ff413b7.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libunicode_normalization-977261e1ba455667.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libtinyvec-5931f1f70217b006.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libunicode_bidi-306ecb356465bbde.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libform_urlencoded-c5adc592c24847fe.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libpercent_encoding-c42753e7ff933803.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libmatches-1ba42f8e6736589f.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libitertools-91c2c298f57ac430.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libeither-285099727ca0d3a5.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libquine_mc_cluskey-6ff02b0b5df613fb.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libsemver-cc320f53f6085e6e.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libsemver_parser-71420aeaef3a7d9b.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libpest-41b2d2072c136e80.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libucd_trie-848f9c70f929d69f.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libserde-bba5b8307b4c2e1f.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libclippy_utils-8af687cb6957b8f8.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/librustc_semver-f95ec72f9354327c.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/libif_chain-14b305b09fe6df2e.rlib" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/librustc_tools_util-ddfcfb1da96998e3.rlib" "-L" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bdynamic" "-lrustc_driver-d0d9704efb4ebe56" "-Wl,--start-group" "-L" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lstd-bc7a0745f1fbbd95" "-Wl,--end-group" "-Wl,-Bstatic" "/tmp/build-rustc-staging.3nWtqsywEA/rustc-src/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-d845064f5e2f3da0.rlib" "-Wl,-Bdynamic" "-lrt" "-ldl" "-lpthread" "-lm" "-lunwind" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,-rpath,$ORIGIN/../lib"

But I'm not sure this adds much new information, because the linker command and error were in my original report.

[jfgoog]

Full output from building rustc, clippy, etc. after building llvm (a superset of the file above)

full-rustc-build.txt

[jfgoog]

This looks suspicious:

clang: warning: argument unused during compilation: '-u nwindlib=libunwind' [-Wunused-command-line-argument]

Our llvm.cflags include -unwindlib=libunwind. Maybe should be --unwindlib=libunwind with a double dash?