rustdoc: accept a "test runner" argument to wrap around doctest executables

[QuietMisdreavus]

Unit tests are currently built and run in separate steps: First you call rustc --test to build up a test runner executable, then you run the executable it outputs to actually execute the tests. This provides a great advantage: The unit tests can be built for a different platform than the host. Cargo takes advantage of this by allowing you to hand it a "test runner" command that it will run the tests in.

Rustdoc, on the other hand, collects, compiles, and runs the tests in one fell swoop. Specifically, it compiles the doctest as part of "running" the test, so compile errors in doctests are reported as test errors rather than as compile errors prior to test execution. Moreover, it runs the resulting executable directly:

rust/src/librustdoc/test.rs

Lines 338 to 354 in fddb46e

	// Run the code!
	//
	// We're careful to prepend the *target* dylib search path to the child's
	// environment to ensure that the target loads the right libraries at
	// runtime. It would be a sad day if the *host* libraries were loaded as a
	// mistake.
	let mut cmd = Command::new(&outdir.lock().unwrap().path().join("rust_out"));
	let var = DynamicLibrary::envvar();
	let newpath = {
	let path = env::var_os(var).unwrap_or(OsString::new());
	let mut path = env::split_paths(&path).collect::<Vec<_>>();
	path.insert(0, libdir.clone());
	env::join_paths(path).unwrap()
	};
	cmd.env(var, &newpath);
	match cmd.output() {

This can be a problem when you're trying to run doctests for a platform that's fairly far removed from your host platform - e.g. for a different processor architecture. It would be ideal if rustdoc could accept some wrapper command so that people could run their doctests in e.g. qemu rather than on the host environment.

[QuietMisdreavus]

There are some relevant prior issues that relate to this; i'll add them later today.

[QuietMisdreavus]

• rustdoc --test ignores --target #44404 is related, but not directly. Having the option to run doctests in a separate environment won't help if rustdoc still builds them for the host!
• Having something like this could provide a way to sidestep rustdoc: add the ability to build a doctest executable without running it #43030, by allowing one to create some environment for doctests (or, worst-case, copy them out entirely).

[fhaefemeier]

I am really interested of this feature and happy to find the enhancement request. But it is three years old. Any chance to see a release date?

[ehuss]

I'm going to close this as I believe this is resolved by #64245 and the --test-runtool and --test-runtool-arg CLI flags.