Description
Proposal
Problem statement
In Windows, stdio handles are (semantically speaking) Option<Handle> where Handle is a non-zero value and None is zero. When spawning a process with Stdio::Inherit, Rust currently turns zero values into -1 values. This has the unfortunate effect of breaking console subprocesses (which typically need stdio) that are spawned from gui applications (that lack stdio by default) because the console process won't be assigned handles from the newly created console (as they usually would in that situation). Worse, -1 is actually a valid handle which means "the current process". So if a console process, for example, waits on stdin and it has a -1 value then the process will end up waiting on itself instead of input.
Motivation, use-cases
Process a.exe:
#![windows_subsystem = "windows"]
fn main() {
// Run `b.exe`
// This process has stdio values of `0` because it's using the "windows" subsystem.
// However, the child process is spawned with `-1` values.
std::process::Command::new("b.exe").spawn().unwrap();
}Process b.exe:
fn main() {
// When starting this process from a "windows" subsystem parent process,
// there will be no console so one will be created.
// The usual Windows behaviour is to then replace any unset stdio handles to the new console.
// (the same is done when manually allocating a console).
// However, because the parent process set the stdio to a non-zero value, this is suppressed.
// Which means that this won't print anything.
println!("hello world!");
}Solution sketches
These problems can be fixed by properly propagating the zero values instead of converting them to -1. This does mean that there is behaviour change here but I feel they're justified.
Links and related work
What happens now?
This issue is part of the libs-api team API change proposal process. Once this issue is filed the libs-api team will review open proposals in its weekly meeting. You should receive feedback within a week or two.