rust-analyzer tries to check all crates in vscode workspace in parallel

[jbg]

After installing this extension for the first time, and shortly after opening a VSCode workspace with around 40-50 crates in it, my laptop becomes completely unresponsive.

Investigation shows around 200 rustc processes running. rust-analyzer seems to be running cargo check on every one of the crates in my workspace at the same time, regardless of how many crates there are and how many CPUs I have.

After killall rustc I am able to use my computer again, but when working in the workspace the problem periodically recurs.

Is there a way to limit how many cargo check processes will be run at a time? Could this default to something based on the number of CPUs the system has?

[flodiebold]

Related to #8622, we should maybe run the initial cargo check lazily (maybe except for a top-level workspace).

Assuming this is just a folder with a bunch of crates in it, and not a Cargo workspace, I would probably recommend opening the specific crate folders instead of the top-level folders, at least for now.

[jonas-schievink]

This sounds like we might want to provide a shared jobserver to all Cargo processes we spawn

[jbg]

It's a VSCode "workspace" containing around 40-50 individual Cargo crates, but not a Cargo workspace. I've turned off cargo check for now as I can't find a way to open them separately without having multiple windows open, which makes simultaneously working on multiple crates unpleasant.

[flodiebold]

@jonas-schievink that's a good idea, but I don't think it would really solve the usability problem with such a large workspace. I.e. presumably you want to work on one or two crates out of the 50, but you still have to wait for all 50 to be built for everything to work.

[jbg]

In my specific case that'd be fine, but in general I agree it would make sense for projects with opened files to be built first.

[flodiebold]

Although actually the initial check shouldn't take that long, since we don't actually compile the crates. But flycheck (check on save) should really be done lazily only for crates that are actually opened, if we're not doing that already.

[flodiebold]

Still, I would tend to say that probably support for such huge workspaces isn't really a goal right now (but maybe @matklad and @jonas-schievink disagree).

[matklad]

I don't think we can make wins here with laziness. As soon as the user invokes "workspace symbols", we'd have to process the world anyways. It feels like another case where our automated discovery of the project structure falls down, and where an explicit configuration in terms of linkedProjects is warranted.

For the time being, I suggest not fixing this issue.

Long term, I think the right fix would be to, if our auto-discovery found > 5 crates, show a message to the user about the project load.

[flodiebold]

As soon as the user invokes "workspace symbols", we'd have to process the world anyways

... unless we don't do that for crates that aren't loaded currently (that was what I was imagining). Of course that would make it very non-obvious what will be covered by workspace symbols / find references etc., which I guess is kind of what you refer to in #8622. It seems to me what we really want would be a nice way for the user to dynamically control which crates are loaded and which aren't; you're probably right that doing it completely lazily/implicitly is also not a good user experience. Maybe this is best left to the client.

[matklad]

Yeah, IntelliJ has unloaded modules thing for that: https://www.jetbrains.com/help/idea/unloading-modules.html#7a53c16f.

IIRC, it was added because IJ struggled with JetBrains own monrepo :D

[jbg]

FWIW, every other feature of the extension that I've tried works snappily in my large workspace, including find references, workspace symbols, etc. It's only the spawning of 200 rustc processes that have caused me a problem.

(Maybe find references etc only have a partial view of the "world" though since I disabled cargo check after attempting to open the workspace a few times?)

[matklad]

Ahh, wait, I think the problem is not the initial cargo check (we do that sequentially) but the checkOnSave cargo check. That one is fully parallel, and probably should be made sequential as well.

[jbg]

That aligns with my experience. My laptop didn't start to melt as soon as I opened the workspace, but a while later — probably shortly after I saved a file.