Inside Rust: Add "Exploring PGO for the Rust compiler" blog post. #718
Conversation
|
|
||
| [![Performance improvements gained from apply PGO to LLVM][rustc-perf-pgo-llvm-thumb]][rustc-perf-pgo-llvm] | ||
| [rustc-perf-pgo-llvm-thumb]: /images/inside-rust/2020-10-30-exploring-pgo-for-the-rust-compiler/rustc-perf-pgo-llvm-thumb.png | ||
| [rustc-perf-pgo-llvm]: /images/inside-rust/2020-10-30-exploring-pgo-for-the-rust-compiler/rustc-perf-pgo-llvm.png |
There was a problem hiding this comment.
FWIW I'd be willing to work with you if you have the time to try and get these results on perf.rust-lang.org (even if they're not collected by the official collector), it should be possible to import them. Let me know.
There was a problem hiding this comment.
I could send you the results.db. Would that help?
There was a problem hiding this comment.
Yeah, it shouldn't be too hard to pull results from there and into the postgres db used by the official backend, which would mean that we can provide URLs here rather than pulling images (which should be all around nicer I imagine).
I expect to name the artifacts e.g. "pgo-llvm" as you've done here fwiw.
posts/inside-rust/2020-10-30-exploring-pgo-for-the-rust-compiler.md
Outdated
Show resolved
Hide resolved
posts/inside-rust/2020-10-30-exploring-pgo-for-the-rust-compiler.md
Outdated
Show resolved
Hide resolved
posts/inside-rust/2020-10-30-exploring-pgo-for-the-rust-compiler.md
Outdated
Show resolved
Hide resolved
| Rust's CI build times have always been too long and we already forgo some optimizations because of them | ||
| (e.g. macOS still does not get the 10% performance boost from using a ThinLTOed LLVM because the build machines on that platform are especially slow). |
There was a problem hiding this comment.
Do we currently use LTO on the compiler itself? I couldn't find anything in config.toml or any of the Cargo.toml files. That might be an easy start to this, running LTO on only the stable and beta channels without having to mess with the new PGO build system yet.
There was a problem hiding this comment.
LLVM (used to be) ThinLTOd on x86_64-unknown-linux-gnu, see rust-lang/rust#78148 and linked issues/PRs for details. It wasn't a big loss to disable it (3-4% IIRC).
There was a problem hiding this comment.
Right - did we ever run LTO on the rust code though (as separate from LLVM)?
There was a problem hiding this comment.
Rust code I think is always ThinLTOd, though I might be misremembering. I don't recall if we've tried full LTO though. It should be relatively easy to try.
There was a problem hiding this comment.
LLVM (used to be) ThinLTOd on x86_64-unknown-linux-gnu, see rust-lang/rust#78148 and linked issues/PRs for details. It wasn't a big loss to disable it (3-4% IIRC).
Interesting. I saw a bigger impact recently (~10% instruction count reduction, I think). I'll take another look.
adding newlines as suggested by @jyn514 to resolve image display issue. Co-authored-by: Joshua Nelson <[email protected]>
|
Looks great to me. I dont think I have r+ privileges on this repo, though! :) |
|
I am inclined to merge this but want to hear back from @michaelwoerister on the upstreaming of the results (since that would give a nicer UI here). It's scheduled for the 30th as well anyway. |
|
I hope to get to the uploading this week, it shouldn't be too much work and I just need to sit down and do it :) |
…ks point to perf.rlo instead of images.
|
OK, the post is done now, with links pointing to perf.rlo instead of static images (thanks to @Mark-Simulacrum). The only thing that might need fixing is the publishing date (which is still Oct 30). What date should I put there? Nov 12 maybe? |
|
November 12th sounds fine, or we can do today. Just give me a ping on Zulip when the date is updated (whichever you choose). |
|
Would rebuilding just https://packages.debian.org/sid/lld-11 (why not libc/libc++ too and static link for the heck of it) with PGO be enough on the LLVM side for most of the performance gain? Also, https://github.com/llvm/llvm-project has a cmake configure that takes an entire minute on my box. Might save some time by caching parts of it you know that won't change. Cmake itself needs a re-write to allow parallelism in the language - if not coding the non-performant parts in C++/rust instead of interpreting them as scripts. |
|
https://packages.debian.org/sid/lld-11 is just for the lld linker, not for |
This is a blog post about the effects that profile-guided optimization has on rustc's performance. It might be of interest to members of @rust-lang/infra, @rust-lang/release, @rust-lang/compiler, and @rust-lang/compiler-contributors.
r? @pnkfelix