-Clinker-plugin-lto doesn't work without extra manual work

[glandium]

$ cargo new testcase
$ cd testcase
$ RUSTFLAGS="-Clinker-plugin-lto" cargo run --release

Yields the following on mac:

 
   Compiling testcase v0.1.0 (/Users/glandium/testcase)
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-m64" "-Wl,-plugin-opt=O3" "-Wl,-plugin-opt=mcpu=core2" "-L" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/Users/glandium/testcase/target/release/deps/testcase-d97266bc083c0e9e.testcase.4ank1dht-cgu.0.rcgu.o" "/Users/glandium/testcase/target/release/deps/testcase-d97266bc083c0e9e.testcase.4ank1dht-cgu.1.rcgu.o" "-o" "/Users/glandium/testcase/target/release/deps/testcase-d97266bc083c0e9e" "/Users/glandium/testcase/target/release/deps/testcase-d97266bc083c0e9e.3pvtkuuqiy7p6wge.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/Users/glandium/testcase/target/release/deps" "-L" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-64d1544b9dc8a8d7.rlib" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-47702365139f147e.rlib" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace_sys-0aefa3a2bfa44649.rlib" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libunwind-c2b22c88cacffeb6.rlib" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-fca4484aa9be2d09.rlib" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liblibc-4728c64ee20d89f8.rlib" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc-1722fbf72ce989c9.rlib" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-0836ff3f3d6a6ee6.rlib" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcore-91c9fbc323ad09b7.rlib" "/Users/glandium/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-43b96ba2cdcc7cb3.rlib" "-lSystem" "-lresolv" "-lc" "-lm"
  = note: ld: unknown option: -plugin-opt=O3
          clang: error: linker command failed with exit code 1 (use -v to see invocation)
          

error: aborting due to previous error

error: Could not compile `testcase`.

To learn more, run the command again with --verbose.

On mac, the default linker is ld64. rustc really invokes cc, which invokes ld64 with the flags passed with -Wl and some others depending on the other command line arguments. ld64 does support the LLVM plugin... but doesn't support the -plugin-opt option to pass arguments to it. I know some things can be passed with -Wl,-mllvm,... but I don't know if that includes things that rust is trying to pass here.

On Linux, it's funnier:

   Compiling testcase v0.1.0 (/tmp/testcase)
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-Wl,-plugin-opt=O3" "-Wl,-plugin-opt=mcpu=x86-64" "-L" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/tmp/testcase/target/release/deps/testcase-576934ab8201a4ea.testcase.dl00diw9-cgu.0.rcgu.o" "/tmp/testcase/target/release/deps/testcase-576934ab8201a4ea.testcase.dl00diw9-cgu.1.rcgu.o" "-o" "/tmp/testcase/target/release/deps/testcase-576934ab8201a4ea" "/tmp/testcase/target/release/deps/testcase-576934ab8201a4ea.y29oqvxl224jt9k.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/tmp/testcase/target/release/deps" "-L" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-bbd8cb236ab3b537.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-334e405e4bdf1791.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-1e14a089a9f63178.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-f1aae4818bd13556.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-a32c94e7da1105b4.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-e214e2acd110aec9.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-fbf429991e30afee.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-1734308ff05fb551.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-b349c8b817f959a5.rlib" "-Wl,--end-group" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-c4b4b16c70e666d9.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
  = note: /tmp/testcase/target/release/deps/testcase-576934ab8201a4ea.testcase.dl00diw9-cgu.0.rcgu.o: file not recognized: file format not recognized
          collect2: error: ld returned 1 exit status
          

error: aborting due to previous error

error: Could not compile `testcase`.

To learn more, run the command again with --verbose.

Because cc is gcc, this just plain doesn't work, for no obvious reason. At the very least, it seems rust should try to use clang instead of cc in that case.

But that also fails:

RUSTFLAGS="-Clinker=clang-8 -Clinker-plugin-lto" cargo run --release
   Compiling testcase v0.1.0 (/tmp/testcase)
error: linking with `clang-8` failed: exit code: 1
  |
  = note: "clang-8" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-Wl,-plugin-opt=O3" "-Wl,-plugin-opt=mcpu=x86-64" "-L" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/tmp/testcase/target/release/deps/testcase-e3dc53d119c25549.testcase.ci8upasg-cgu.0.rcgu.o" "/tmp/testcase/target/release/deps/testcase-e3dc53d119c25549.testcase.ci8upasg-cgu.1.rcgu.o" "-o" "/tmp/testcase/target/release/deps/testcase-e3dc53d119c25549" "/tmp/testcase/target/release/deps/testcase-e3dc53d119c25549.61wo3nlknhbro8d.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/tmp/testcase/target/release/deps" "-L" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-bbd8cb236ab3b537.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-334e405e4bdf1791.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-1e14a089a9f63178.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-f1aae4818bd13556.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-a32c94e7da1105b4.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-e214e2acd110aec9.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-fbf429991e30afee.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-1734308ff05fb551.rlib" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-b349c8b817f959a5.rlib" "-Wl,--end-group" "/home/glandium/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-c4b4b16c70e666d9.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
  = note: /usr/bin/ld: bad -plugin-opt option
          clang: error: linker command failed with exit code 1 (use -v to see invocation)
          

error: aborting due to previous error

error: Could not compile `testcase`.

To learn more, run the command again with --verbose.

And here, the reason is essentially the same: the underlying linker doesn't support the -plugin-opt flag... except it does, but not when it's not passed -plugin, which happens when the compiler passes it, which happens when -flto was on its command line:

$ RUSTFLAGS="-Clinker=clang-8 -Clinker-plugin-lto -Clink-arg=-flto" cargo run --release   Compiling testcase v0.1.0 (/tmp/testcase)
    Finished release [optimized] target(s) in 0.28s
     Running `target/release/testcase`
Hello, world!

Using lld works too, because it doesn't need an explicit -plugin:

$ RUSTFLAGS="-Clinker=clang-8 -Clinker-plugin-lto -Clink-arg=-fuse-ld=lld" cargo run --release
   Compiling testcase v0.1.0 (/tmp/testcase)
    Finished release [optimized] target(s) in 0.36s
     Running `target/release/testcase`
Hello, world!

[froydnj]

FWIW, you can pass an argument to -Clinker-plugin-lto to give it the name of the plugin to load, which would fix the clang case. (You could make an argument that said requirement is exactly the problem in the clang case.) The OS X and plain cc cases seem like out-and-out bugs, though.

[glandium]

Indeed RUSTFLAGS="-Clinker=clang-8 -Clinker-plugin-lto=/usr/lib/llvm-8/lib/LLVMgold.so" cargo run --release works. It actually even works without -Clinker=clang-8.

[nikomatsakis]

Nominating for (hopefully brief) discussion in the @rust-lang/compiler meeting -- this is holding up Firefox's efforts to use ThinLTO to eliminate glue code. It would be good to get it fixed or, at minimum, to specify what the correct behavior is and figure out who to ping about it (I'm not that familiar with these options).

[glandium]

FWIW, we have a workaround for Firefox, which is that rust is not involved in linking at all. This is more about the general case than Firefox itself.

[nagisa]

It seems that this issue would be mostly resolved by documenting which cases and linker/flag combinations we know to work. We could then see if there are any changes to the CLI that could be made to improve the experience here.

[nikomatsakis]

Discussion from Zulip

[dwightguth]

I recently ran into this issue and wanted to mention that if you just want to have lto'd libraries that you are using to link an executable with clang, but can't build the rust code you need because crates that build binaries fail to link due to this error on mac OS, you can sidestep the issue by using the following script on the .rlib files generated by cargo build: https://github.com/kframework/llvm-backend/blob/6ec0694fe63f5c552ea3b64022ff8ba197d1252d/bin/llvm-kompile-rust-lto

It will not work if you want to link those libraries with rustc though (although it could be made to work with a small additional effort)