Shipping clang as a rustup component

[michaelwoerister]

Sometimes it is beneficial (or even necessary) for mixed-language projects to build C/C++ with a Clang version that matches Rust's LLVM version. Cross-language LTO is one of these cases.

Since Rust's LLVM hardly ever matches a specific release and is often ahead of the current stable release, it would be great if we provided an easy way to get ahold of the right Clang binaries. It looks like we are already building Clang for Rust-enabled LLDB anyway. Could we make also make it available via rustup? Either as a standalone component, or as part of lldb or llvm-tools?

cc @rust-lang/infra @rust-lang/release

[alexcrichton]

I personally feel like this is going perhaps a step too far in our distribution strategy in that I think we want to try to become a one-stop-shop for all your toolchain needs. Shipping a C compiler seems fraught with complications like "will we start shipping header files and runtime libraries next?"

I agree that cross-language-LTO is a good use case, but I think I'd probably prefer to document and tests that any particular version of a Rust compiler is compatible with some set of clang releases (probably just one). That way users can figure out the best way to install clang on their end (which seems like it's unlikely to be from our sources?) and we'd be testing it worked.

(I'm not 100% certain how the debugger/llvm tools stories will play out in the long run, it already feels to me like we're overstepping our bounds slightly)

[Centril]

Maybe we can try this out as a "preview" and see how it develops and if there's a big need for it?

[dwightguth]

I can say that shipping clang wouldn't really be enough for me because we also have to use the llvm API to generate code on our own. So it would definitely be better for our purposes if rust stable was compatible with some specific release of llvm.

[BatmanAoD]

It would probably be helpful for rustc to provide a programmatic way to query the recommended Clang version for LTO compatibility, then. This would provide a straightforward way for any interested parties to write a script that can install/update Rust and then install or update Clang to match.

…

On Fri, Nov 30, 2018, 10:33 AM dwightguth ***@***.***> wrote: I can say that shipping clang wouldn't really be enough for me because we also have to use the llvm API to generate code on our own. So it would definitely be better for our purposes if rust stable was compatible with some specific release of llvm. — You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub <#56371 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACNOT4Gf2J4ZuSJhYh9JNB4skpx3nHJZks5u0WvjgaJpZM4Y7XJ8> .

[bstrie]

rustc has a policy of supporting at least one stable release of LLVM in order to satisfy distro packaging, correct? Maybe rather than shipping Clang, we could provide a rustup option where you receive a version of the compiler built with a stable LLVM, which one can surely then find a suitable Clang for.

[cuviper]

Or such users can just switch to a distro rustc with its external LLVM and matching clang... 😇

[bstrie]

Sure, but if that's the case then this entire bug is pointless. :) No such luck for Windows users, sadly!

[nikic]

rustc has a policy of supporting at least one stable release of LLVM in order to satisfy distro packaging, correct? Maybe rather than shipping Clang, we could provide a rustup option where you receive a version of the compiler built with a stable LLVM, which one can surely then find a suitable Clang for.

The main issue with that is probably the additional CI time this would take. At the very least, it requires building an additional codegen backend (not so expensive), but probably also running the full testsuite with it at least on tier 1 platforms. (On the other hand, that would probably alleviate distributor concerns that Rust is not sufficiently tested with system LLVM.)

It also seems kind of odd to go back to an older LLVM version (and possibly lose performance or features) to make use of cross-lang LTO.

[gnzlbg]

Now that cross-lang LTO is being stabilized (#58057) we should definitely ship rust-clang with rustup. We are pretty much always using a non-release LLVM version pinned at some particular commit, which would mean that for people to use cross-lang LTO in nightly, they would need to get the LLVM sources used by the current nightly (potentially with Rust patches), find the respective clang sources, and compile LLVM and clang (maybe also compiler-builtins and friends), just to get started with the really FUN part of the process [0].

[0] FUN: setting up a C/C++ toolchain that differs from the platform toolchains when compiling C/C++ for Rust projects, such linking against other C/C++ code compiled by other different toolchains like the platforms one still works.

[bmmoore]

@bstrie Where is that policy stated? It would really be quite useful to have rustc promise to be LTO-compatible with a release version of LLVM, or at least ship such a rustc.