"LoadLibraryExW failed" when loading proc macros with too many -L search paths

[goffrie]

On Windows, when the total length of -L search paths exceeds about 32k characters, rustc fails to load proc macro DLLs with the message:

error: LoadLibraryExW failed (retried 5 times)
 --> user.rs:2:1
  |
2 | extern crate procmacro;
  | ^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

WinDbg suggests that the error returned is

LastErrorValue: (Win32) 0x8 (8) - Not enough memory resources are available to process this command.
LastStatusValue: (NTSTATUS) 0xc0000017 - {Not Enough Quota}  Not enough virtual memory or paging file quota is available to complete the specified operation.

I noticed that the search paths get added to the PATH environment variable (not sure where), but environment variables have a limit of 32,767 characters. So my guess is that an overly full environment causes LoadLibraryExW to fail.

Repro gist: https://gist.github.com/goffrie/4fd3553a5417fda17d78e0f8a7252335
Run the commands in repro.ps1. Basically we just need an argfile to run rustc with a very long list of search paths, and then importing proc-macros fails.

This scenario is not likely when using Cargo, but rules_rust for Bazel generates one -L per transitive crate dependency which makes this not super difficult to hit.

Meta

rustc --version --verbose:

rustc 1.69.0 (84c898d65 2023-04-16)
binary: rustc
commit-hash: 84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc
commit-date: 2023-04-16
host: x86_64-pc-windows-msvc
release: 1.69.0
LLVM version: 15.0.7

[goffrie]

OK, I see that the PATH extension happens intentionally here:

rust/compiler/rustc_interface/src/passes.rs

Lines 195 to 225 in 8b8110e

	// Windows dlls do not have rpaths, so they don't know how to find their
	// dependencies. It's up to us to tell the system where to find all the
	// dependent dlls. Note that this uses cfg!(windows) as opposed to
	// targ_cfg because syntax extensions are always loaded for the host
	// compiler, not for the target.
	//
	// This is somewhat of an inherently racy operation, however, as
	// multiple threads calling this function could possibly continue
	// extending PATH far beyond what it should. To solve this for now we
	// just don't add any new elements to PATH which are already there
	// within PATH. This is basically a targeted fix at #17360 for rustdoc
	// which runs rustc in parallel but has been seen (#33844) to cause
	// problems with PATH becoming too long.
	let mut old_path = OsString::new();
	if cfg!(windows) {
	old_path = env::var_os("PATH").unwrap_or(old_path);
	let mut new_path = sess.host_filesearch(PathKind::All).search_path_dirs();
	for path in env::split_paths(&old_path) {
	if !new_path.contains(&path) {
	new_path.push(path);
	}
	}
	env::set_var(
	"PATH",
	&env::join_paths(
	new_path.iter().filter(|p| env::join_paths(iter::once(p)).is_ok()),
	)
	.unwrap(),
	);
	}

It's not clear to me when exactly this is necessary or if there's something smarter that could be done to reduce which paths actually get added to %PATH%, as most dependency search paths in practice are not relevant (e.g. are only for plain rlibs).