os: support creating FIFOs on Windows

[RayyanAnsari]

os.mkfifo() creates a named pipe that is accessible on the file system. It is currently only supported on Unix.
Windows also supports creating named pipes with CreateNamedPipeW() which returns a handle to the server end of the pipe. However, they can only be created on a special filesystem in the \\.\pipe\ directory.

The Windows docs say that:

An instance of a named pipe is always deleted when the last handle to the instance of the named pipe is closed.

And os.mkfifo doesn't return the pipe handles, it just creates the pipe:

Note that mkfifo() doesn’t open the FIFO — it just creates the rendezvous point.

This would mean that implementing Windows support into os.mkfifo would mean that the pipe is deleted as soon as it is created (if we close the handle returned by CreateNamedPipeW()), or we have a pipe that we can't delete (if we leave the pipe handle open).
Unless there is some way to pass that back to the calling code - like os.pipe() that's supported on Windows.

I'm not sure how I would go about implementing this. Returning the handle only on Windows? Making a new cross-platform function that always returns the handle (perhaps called os.fifo)?

Linked PRs

• gh-103510: Support os.mkfifo on Windows #129420

[eryksun]

The semantics are significantly different for a FIFO on POSIX compared to a named pipe on Windows.

On POSIX, a FIFO is half-duplex, and it opens as a single file in the kernel that has a write end and a read end. Either end of the FIFO can be opened by name multiple times, depending on whether open() is called with O_RDONLY, O_WRONLY, or O_RDWR.

On Windows, a named pipe is opened as a pipe instance in the named-pipe filesystem ('\\.\PIPE\"), which connects two kernel file objects -- one for each end of the pipe, the server side and the client side. The number of allowed instances of a pipe can be limited up to 254, or unlimited. The client end can be remote (over SMB) if the pipe mode isn't restricted to local clients. A named pipe can be inbound (server reads; client writes), outbound (server writes; client reads), or duplex. For the latter, the server end reads what the client end writes, and the client end reads what the server end writes. Unlike a POSIX FIFO, a named pipe can be opened by name only once on each end of an instance of the pipe -- once by the server end via CreateNamePipeW() and once on the client end via CreateFileW(). That said, multiple handles for the kernel file object on either end of a pipe instance may be duplicated to or inherited by other processes.

[SuibianP]

What about this -

On Windows, pipe servers are created by os.mkfifo, and pipe clients are created by normal open. os.mkfifo takes a new keyword argument open_mode=None, which causes the function to return an open fd corresponding to the mode if it is not None. The pipe will always be created in full duplex mode for now, but it should be trivial to extend another kwarg for pipe_mode if the need arises.

FIFO_PATH = "/tmp/myfifo"
os.mkfifo(FIFO_PATH)
with open(FIFO_PATH, "w") as mypipe:
    mypipe.write("Message\n")

becomes

FIFO_PATH = "/tmp/myfifo"
pipe_fd = os.mkfifo(FIFO_PATH, open_flags=os.WRONLY)
with os.fdopen(pipe_fd, "w") as mypipe:
    mypipe.write("Message\n")

The deletion semantics will be different between Unix and Windows, although I don't think it matters as Windows pipes cannot be persistent. Moreover, the API will of course only present identical semantics for the case of one single read & write end. Opening a named pipe multiple times will by nature exhibit different semantics between Unix and Windows.

Comments? @eryksun

[2trvl]

@SuibianP From Eryk Sun's comment and my thoughts, it can be seen that Unix FIFO and Windows Named Pipes are very different. Mimicking Unix FIFO is difficult.

So I propose creating a new function os.namedpipe(). See updated description.

The code will look like this:

# name just string, default mode "x+", raise an OSError if pipe exists, FILE_FLAG_FIRST_PIPE_INSTANCE
mypipe_fd = os.namedpipe(name, mode)
# since 3.12 available as SetNamedPipeHandleState(PIPE_NOWAIT)
os.set_blocking(mypipe_fd, False)
# create a new mypipe instance
mypipe2_fd = os.namedpipe(name, "w+")

Duplex Modes:

• x+, create PIPE_ACCESS_DUPLEX
• r+, open existing
• w+, open new instance

Server Write, Client Read Modes:

• x, create PIPE_ACCESS_OUTBOUND
• r, open existing with read only
• w, open new instance

New Instance Exclusive Modes:

• a+, open new PIPE_ACCESS_DUPLEX instance (equivalent to w+)
• a, open new PIPE_ACCESS_OUTBOUND instance (equivalent to w)

[SuibianP]

(Continuing the discussion from #129420)

I dislike option 2, since existing code using mode would suddenly leaks a file descriptor whereas the code works fine with Python 3.13. I suggest to continue the discussion in the issue instead.

You may add a new function instead.

@vstinner open_mode was to be a new keyword argument, distinct from the existing octal mode argument. As such, there was to be no change of behaviour for existing code.

---

@2trvl

(Sorry for not replying in one batch; still grokking your proposals)

The simplest case will be os.mkfifo, which allows only 2 processes, but this is not very cool.

This is exactly what I meant by "simple IPC case", which is the scope of common semantics I intended to support. IMHO this would cover the majority of named pipe use cases, and is self-contained without opening a can of worms as you noted. I considered platform-specific features (message-based FIFOs, MPMC semantics) out of scope for my original proposal.

Frankly, I am not sure if anything beyond that would be very useful with a semantic difference this great -- a quick search did not find real-world Python code using os.mkfifo or pywin32 win32pipe.CreateNamedPipe in multi-producer or multi-consumer ways. Would it be better if we first implement the building blocks by adding new keyword arguments into mkfifo, and add the higher level abstraction only when there is a more established usage pattern in the wild?

[2trvl]

@SuibianP It is preferable to use existing Python file calls rather than creating new ones.

os.namedpipe(npfs: str, mode: str = "w+", wait: bool = True, open: bool = False) -> int

Creates a new Windows named pipe instance. The function's behavior is something between os.mkfifo and os.pipe. It creates the rendezvous point and opens a server end.

Args:

• npfs - Named pipe file system path in win32 device namespace \\.\pipe\name.
• mode - Create mode. Must be the same for all instances. Defaults to "w+".
• wait - Wait for client end connection. Ensures clear synchronization. When combined with os.set_blocking(fd, wait) it can make a named pipe nonblocking. Defaults to True.
• open - Open named pipe instance for reading and writing by Everyone. This can be used to communicate with processes of other users or remote clients over SMB. Defaults to False.

Returns:

• int - Named pipe server end. The new file descriptor is non-inheritable.

Raises:

• OSError - Unknown or invalid create mode. Access denied.

Available create modes:

Mode	Access	Internal
w+	read & write	PIPE_ACCESS_DUPLEX
w	write	PIPE_ACCESS_OUTBOUND
r	read	PIPE_ACCESS_INBOUND

Local nonblocking write:

mypipe_fd = os.namedpipe(r"\\.\pipe\mypipe", wait=False)
os.set_blocking(mypipe_fd, False)
mypipe = os.fdopen(mypipe_fd, "w")
mypipe.write("Hello from mypipe server!")

mypipe = open(r"\\.\pipe\mypipe")
print(mypipe.read())
# Hello from mypipe server!

Remote:

mypipe_fd = os.namedpipe(r"\\.\pipe\mypipe", open=True)
mypipe = os.fdopen(mypipe_fd, "w+")

mypipe = open(r"\\ComputerName\pipe\mypipe", "w+")
mypipe.write("Message over SMB")

Microsoft Named Pipe Operations