Parallel compiler jobserver management strategy

[alexcrichton]

I tested out a full crate build of the Cargo crate graph in release mode with a parallel rustc compiler today but it unfortunately regressed the build time relative to nightly by about 20%! The cause of the regression appears to be an enormous amount of time spent in the kernel, particularly related to jobserver management (although there's some other thread-related stuff in the profile too like waiting on cvars).

After a brief discussion on Zulip we decided to open an issue here about the jobserver strategy in general. To be clear, the current downsides that I've seen are:

• Tokens are acquired/reacquired excessively, causing most compilation time get sunk into the kernel.
• Tokens aren't managed quite right because the maximum parallelism went above the jobserver limit. My -j28 build, for example, at one point had 45 rustc instances running.

I'm not personally too familiar with the current jobserver code in rustc, but this is something that likely warrants improvement before shipping!

cc @Mark-Simulacrum @Zoxc

[cuviper]

• Tokens aren't managed quite right because the maximum parallelism went above the jobserver limit. My -j28 build, for example, at one point had 45 rustc instances running.

One of rustc-rayon's forked additions is the acquire/release thread handler, which is being used for jobserver tokens. So when a rayon thread starts working, it grabs a token, and it releases when going idle. The independent rustc thread also releases its own token while waiting for rayon tasks. But since this isn't a direct handoff, I suspect you're seeing a race where a different process intercepts the token, and the losing process ends up waiting with no tokens at all for a bit. Thus 45 processes, but probably still only 28 tokens-worth of threads are actually running anywhere, the rest blocked.

[alexcrichton]

Yes I believe that would do it! I saw similar behavior when implementing jobserver stuff recently in cc since I used the same strategy there. While I think it was ok for cc, though, I think we need to improve the management for rustc. The purpose of jobservers I believe are twofold: one is to limit the CPU parallelism but another is to limit the number of active processes and pressure on memory. The compiler can often use quite a bit of memory so an idle compiler, despite it not using CPU, I think would still be a pretty heavyweight thing to have around due to its memory impact.

[the8472]

But since this isn't a direct handoff, I suspect you're seeing a race where a different process intercepts the token, and the losing process ends up waiting with no tokens at all for a bit. Thus 45 processes, but probably still only 28 tokens-worth of threads are actually running anywhere, the rest blocked.

Note that #84407 bumped the jobserver dependency to include rust-lang/jobserver-rs#31 which should fix the thundering herd problem if you're on a sufficiently recent linux kernel.

That won't address the second point in OP of course, so it's only a partial fix.

[nnethercote]

The parallel compiler is now shipping in nightly and gives decent speedups. The jobserver doesn't seem to be a problem. I think this can be closed.