File: low-level API for creating pipe #122806
Description
Background and motivation
As of today, when System.Diagnostics.Process is started with redirected standard input, output, or error, we internally create anonymous pipes to communicate with the child process. This can't be done without resorting to P/Invoke or taking dependency on System.IO.Pipes and using AnonymousPipeServerStream:
using AnonymousPipeServerStream pipeServer = new(PipeDirection.Out, HandleInheritability.Inheritable);
SafePipeHandle outHandle = pipeServer.SafePipeHandle;
SafePipeHandle inHandle = pipeServer.ClientSafePipeHandle;In addition to that, we need to call DisposeLocalCopyOfClientHandle in most of the cases.
This could be simplified by providing a method to create anonymous pipes directly on System.IO.File. And it would allow for the users of the upcoming low-level Process APIs to implement scenarios like piping.
We also need to be able to open the pipes for async IO. It allows for:
- cancellation support
- draining remaining data without risk of getting blocked
The high-level APIs are usually going to open sync write handle and async read handle. Some users, like PowerShell, need to have both async when running certain tools.
And when given a SafeFileHandle, check whether it's a pipe or not (the copy of the write end opened by the parent process needs to be closed after the process is spawned in order to allow receiving EOF when child process exits).
API Proposal
namespace System.IO;
public static class File
{
public static void CreateAnonymousPipe(out SafeFileHandle read, out SafeFileHandle write, bool asyncRead = false, bool asyncWrite = false);
}
public enum FileType
{
Unknown = 0,
RegularFile,
Pipe,
Socket,
CharacterDevice,
Directory,
SymbolicLink,
[UnsupportedOSPlatform("windows")]
BlockDevice
}namespace Microsoft.Win32.SafeHandles;
public sealed partial class SafeFileHandle
{
public FileType GetFileType();
}FileType implementation: #124561
API Usage
File.CreateAnonymousPipe(out SafeFileHandle readPipe, out SafeFileHandle writePipe);
using (readPipe)
using (writePipe)
{
ProcessStartOptions producer = new("sh")
{
Arguments = { "-c", "printf 'hello world\\ntest line\\nanother test\\n'" }
};
ProcessStartOptions consumer= new("grep")
{
Arguments = { "test" }
};
using var producerHandle = SafeProcessHandle.Start(producer, input: null, output: writePipe, error: null);
using (SafeFileHandle outputHandle = File.OpenHandle("output.txt", FileMode.Create, FileAccess.Write, FileShare.ReadWrite))
{
using var consumerHandle = SafeProcessHandleStart(consumer, readPipe, outputHandle, error: null);
await producerHandle.WaitForExitAsync();
await consumerHandle.WaitForExitAsync();
}
}Alternative Designs
As pointed by @davidfowl in an offline API review, the method could return a PipePair struct instead of using out parameters. With a deconstruct method to allow tuple like unpacking:
public readonly struct PipePair(SafeFileHandle read, SafeFileHandle write) : IDisposable
{
public SafeFileHandle Read { get; } = read;
public SafeFileHandle Write { get; } = write;
public void Deconstruct(out SafeFileHandle read, out SafeFileHandle write)
{
read = Read;
write = Write;
}
public void Dispose()
{
Read.Dispose();
Write.Dispose();
}
}Then the users could use it like this:
var (read, write) = File.CreatePipe();But it does not work well with using statements, as following code would not compile:
using var (read, write) = File.CreatePipe();And according to this SO answer, we would still need two lines of code:
using var pipePair = File.CreatePipe();
var (read, write) = pipePair;Which is not much different from the out parameter approach, but adds one more type to the API surface.
Risks
Exposing an API that returns a pipe handle created with O_NONBLOCK should be paired with exposing an ability to use it. We should consider exposing RandomAccess (this would require new API, as there is no way to represent EWOULDBLOCK with current Read APIs: 0 means EOF and throwing an exception would not be acceptable) or FileStream with such capabilities. FileStream could just attempt to read, in case of EWOULDBLOCK use Poll and wait for more data.
Not in the API design, but the implementations need to enforce CLOEXEC (Unix) / bInheritHandle: false (Windows) semantics on the opened handle to avoid leaking it to child processes unintentionally. Which is very important for pipes, as EOF is signaled only when the parent process closes the copy of the child handles. If multiple processes derive the same handle, the EOF won't be signaled until all of them close it!