Does I/O safety forbid `dup`ing arbitrary file descriptors?

[RalfJung]

I/O safety is summarized as

a guarantee that if one part of a program holds a raw handle privately, other parts cannot access it

I would argue then that if a function is written foo(fd: OwnedFd), then it should be guaranteed that it is the only one with read/write access to the underlying resource managed by this FD. (Of course if it is a file, others might be able to open the file again, but not all FDs are for files that have a name in the file system.) And this guarantee almost holds, except for OwnedFd::try_clone and BorrowedFd::try_clone_to_owned. These will take a mere reference and return a fully owned OwnedFd (via dup), so our foo function has been foo'led and it is not actually exclusively reading/writing the underlying resource.

I am somewhat surprised to discover this now; these functions were not mentioned in the original RFC. They fundamentally change what it means to hold an OwnedFd, and I think not for the better. They are incompatible with the mental model I used when helping design and defend I/O safety.

So, what shall we do about this?

• Document the actually remaining guarantees for OwnedFd carefully. They are very weak (basically just "nobody will close this FD while you have it").
• Attempt to change the past and deprecate the safety of these methods.

Also, we should clarify what this means for I/O safety overall. Would it be sound to do dup(rand()) and return the result as an OwnedFd? I think the answer should be "absolutely not"; if I create an FD and don't share it with anyone I should be guaranteed that no duplicates exist. This would at least make it possible for a user crate to define NonDupOwnedFd/NonDupBorrowedFd types that guarantee an absence of duplicates.

Cc @sunfishcode

[RalfJung]

This is @rust-lang/libs-api I guess (certainly the part about potentially deprecating safe try_clone), but note that some of this is an ecosystem-wide decision, so I'll also add the T-lang label.

[sunfishcode]

Even without try_clone in Rust's own API, this kind of exclusivity is undermined by the possibility that someone could call OwnedFd::from_raw_fd on a raw fd that has already been duped.

This might suggest adding a rule to from_raw_fd requiring that there exist no dups, however that's sometimes difficult to know. For example, if a process receives a fd from its parent when it is spawned, the parent could still be holding onto what is effectively a dup of the fd, and there's no way the child can tell.

Further, this kind of exclusivity is undermined by the fact that explicit dup isn't the only API in play. mmap performs what is effectively a dup, because it's guaranteed that you can mmap a fd and immediately close the fd, and the mapped memory can continue to access the data referenced by the fd. fork performs what is effectively a dup on all non-O_CLOEXEC fds into the child. sendmsg sending a fd to another process over a Unix-domain socket performs what is effectively a dup into the destination process.

And, on some platforms, including macOS, it's not even possible to set O_CLOEXEC atomically on all fds, meaning an ill-timed fork on another thread could implicitly dup fds before O_CLOEXEC can be set, and there's nothing that can be done to prevent this.

This is all in addition to the fact that files can be opened multiple times, potentially even by different names, from the same program or different programs.

There are so many ways to undermine this kind of exclusivity, that it would be tricky and limiting to maintain, even without try_clone in Rust's own API.

From another angle, it also happens that this kind of exclusivity is much less useful for fds than it is for memory. In Rust we're all accustomed to thinking about exclusive ownership. However, when working with fds, this kind of exclusivity is almost never what matters for avoiding UB. Among other things, OS's implicitly perform thread synchronization under the covers, so there's never a data race on a fd.

There are a few situations where this kind of exclusivity would matter. For example, if one does memfd_create and then mmap and then slice::from_raw_parts, one may be relying in keeping the fd encapsulated and not allowing anyone to write to it. However, in those cases, one usually shouldn't implement AsFd at all, since BorrowedFd as a type doesn't say that you can't write to it. And if a fd is held in a type that doesn't implement AsFd (or AsRawFd), I/O safety says that, with care, it can be fully encapsulated, so that no one can dup it without the type's knowledge.

Consequently, this kind of exclusivity guarantee at the OwnedFd level isn't that useful in practice.

Also, we should clarify what this means for I/O safety overall. Would it be sound to do dup(rand()) and return the result as an OwnedFd? I think the answer should be "absolutely not"; if I create an FD and don't share it with anyone I should be guaranteed that no duplicates exist. This would at least make it possible for a user crate to define NonDupOwnedFd/NonDupBorrowedFd types that guarantee an absence of duplicates.

I agree. dup under I/O safety conceptually has an argument with type BorrowedFd. If you're calling it in libc, where it's just a c_int, then when you type the unsafe { } needed to call it, your safety obligation is to provide a fd that could be passed to BorrowedFd::borrow_raw at that point. Consequently, dup(rand()) would violate I/O safety, because a random integer value isn't guaranteed to be the value of an open fd.

And yes, you can create NonDupOwnedFd/NonDupBorrowedFd if you encapsulate your fds and take appropriate care. To prevent them from being dupd, those types should not implement AsFd or AsRawFd.

[bjorn3]

I think exclusivity over the FD itself is important so nobody else can close it and dup2 something else over it, but exclusivity over the underlying file description pointed to by the FD (which stays the same in case of dup) is not as important.

[sunfishcode]

dup2's second argument is special and we have documentation about that very issue.

[RalfJung]

Even without try_clone in Rust's own API, this kind of exclusivity is undermined by the possibility that someone could call OwnedFd::from_raw_fd on a raw fd that has already been duped.

Yes I would have expected that to be unsound.

Further, this kind of exclusivity is undermined by the fact that explicit dup isn't the only API in play. mmap performs what is effectively a dup, because it's guaranteed that you can mmap a fd and immediately close the fd, and the mapped memory can continue to access the data referenced by the fd. fork performs what is effectively a dup on all non-O_CLOEXEC fds into the child. sendmsg sending a fd to another process over a Unix-domain socket performs what is effectively a dup into the destination process.

Fork is terribly unsafe already anyway.

But the others are valid point... I guess you would want to still get "close-on-drop" behavior for them.

And yes, you can create NonDupOwnedFd/NonDupBorrowedFd if you encapsulate your fds and take appropriate care. To prevent them from being dupd, those types should not implement AsFd or AsRawFd.

Okay, so "exclusively owning a non-duped FD" is a thing we accept one can express, but we don't provide a type for that. I think I can live with that. It needs documentation though.

[joshtriplett]

Confirming that @sunfishcode's description is what I'd always expected from I/O safety as well, since it was first proposed. The safety there is about not using closed FDs or random numbers as FDs, and about preventing double-close errors, not about preventing you from calling dup.