Refactor Process.StartCore to accept SafeFileHandle parameters for stdin/stdout/stderr#125766
Refactor Process.StartCore to accept SafeFileHandle parameters for stdin/stdout/stderr#125766
Conversation
…ation and stream setup to Process.cs Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…nterop.Pipe.cs to csproj Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
|
Tagging subscribers to this area: @dotnet/area-system-diagnostics-process |
adamsitnik
left a comment
There was a problem hiding this comment.
@copilot address my feedback
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Windows.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Windows.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System.Diagnostics.Process.csproj
Outdated
Show resolved
Hide resolved
…eAnonymousPipe directly, simplify Windows StartWithCreateProcess Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
All feedback addressed in commit 07d9369:
|
|
@copilot please update the PR description based on my commits and then invoke the code-review skill and post your analysis/results as a comment on this PR |
PR description updated. Code review follows below. 🤖 Copilot Code ReviewHolistic AssessmentMotivation: This refactoring lifts pipe creation out of platform-specific code into the shared Approach: The approach is sound — pipes are created uniformly via Summary: ✅ LGTM. The refactoring is correct, handles resource management properly, and the locking scheme is well-designed. Two models (Gemini 2.5 Pro and Claude Opus 4.5) reviewed this independently; their findings converge. No blocking issues found. Detailed Findings✅ Resource management — Console handles are safe to disposeThe
Disposing a non-owning ✅ Handle ownership transfer in Unix OpenStream — No double-disposeThe Unix SafePipeHandle safePipeHandle = new(handle.DangerousGetHandle(), ownsHandle: true);
handle.SetHandleAsInvalid();
handle.Dispose();The parent pipe handles ( ✅ Platform-conditional handle creation — Assertions are correctThe condition
✅ Native dup2 now unconditional — SafeThe native code now always calls ✅ Locking scheme — Correctly prevents handle inheritance leaks
The lock is shared across platforms via 💡 Minor — Comment could be more preciseThe comment at line 1372 ("We MUST close the parent copies of the child handles") is slightly misleading for the non-redirect case where handles came from Models contributing to this review: Claude Sonnet 4 (primary), Gemini 2.5 Pro, Claude Opus 4.5 |
- STARTF_USESTDHANDLES is always provided now - we may use more flags in the future - so ProcessWindowStyle tests should check only if STARTF_USESHOWWINDOW (1) was applied
There was a problem hiding this comment.
Pull request overview
This PR refactors System.Diagnostics.Process startup so that platform StartCore implementations accept SafeFileHandle instances for stdin/stdout/stderr, with pipe creation moved into the shared Process.Start() path. It also updates the native/interop layers to accept provided fds instead of creating pipes in native code, enabling future scenarios where callers supply pre-existing standard stream handles.
Changes:
- Managed: Move stdio pipe creation + stream setup into
Process.Start(), and update Unix/WindowsStartCoreto acceptSafeFileHandleparameters. - Native/Interop: Change
SystemNative_ForkAndExecProcessto takestdinFd/stdoutFd/stderrFdas inputs and add a native probe forpipe2support (SystemNative_IsAtomicNonInheritablePipeCreationSupported). - Tests/Project: Update Windows window-style test expectations and add new Unix interop sources +
NullableBoolinclusion in the Process project.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/native/libs/System.Native/pal_process_wasi.c | Updates WASI ForkAndExecProcess signature to accept stdin/out/err fds as inputs. |
| src/native/libs/System.Native/pal_process.h | Adjusts native API contract to take stdio fds by value (no redirect booleans / out fds). |
| src/native/libs/System.Native/pal_process.c | Removes native pipe creation/locking for stdio and unconditionally dup2s provided fds. |
| src/native/libs/System.Native/pal_io.h | Declares new native capability probe for atomic non-inheritable pipe creation. |
| src/native/libs/System.Native/pal_io.c | Implements the pipe2 capability probe. |
| src/native/libs/System.Native/entrypoints.c | Exposes the new native probe via System.Native entrypoints. |
| src/libraries/System.Diagnostics.Process/tests/ProcessStartInfoTests.Windows.cs | Fixes window-style assertions to mask dwFlags rather than compare full value. |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/ProcessUtils.cs | Centralizes the cross-platform process-start ReaderWriterLockSlim. |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs | Creates stdio handles/pipes in shared Start() and initializes redirected streams via platform OpenStream. |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Windows.cs | Refactors CreateProcess path to take provided handles and duplicates them as inheritable. |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Win32.cs | Updates StartCore signature, adds atomic pipe creation support flag for Windows, and moves redirect validation to shared code. |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs | Updates StartCore + ForkAndExec interop usage to use provided handles; adjusts terminal detection to use IsATty(handle). |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.ConfigureTerminalForChildProcesses.Unix.cs | Switches to ProcessUtils.s_processStartLock. |
| src/libraries/System.Diagnostics.Process/src/System.Diagnostics.Process.csproj | Adds new Unix interop sources and moves NullableBool to the broader non-Windows group. |
| src/libraries/Common/src/Interop/Unix/System.Native/Interop.IsAtomicNonInheritablePipeCreationSupported.cs | Adds cached managed probe for pipe2 support. |
| src/libraries/Common/src/Interop/Unix/System.Native/Interop.IsATty.cs | Adds SafeFileHandle overload for IsATty. |
| src/libraries/Common/src/Interop/Unix/System.Native/Interop.ForkAndExecProcess.cs | Updates P/Invoke to pass stdio handles (no redirect booleans / out fds). |
You can also share your feedback on Copilot code review. Take the survey.
| { | ||
| childInputPipeHandle = Console.OpenStandardInputHandle(); | ||
| } | ||
|
|
||
| if (startInfo.RedirectStandardOutput) | ||
| { | ||
| SafeFileHandle.CreateAnonymousPipe(out parentOutputPipeHandle, out childOutputPipeHandle, asyncRead: OperatingSystem.IsWindows()); | ||
| } | ||
| else | ||
| { | ||
| childOutputPipeHandle = Console.OpenStandardOutputHandle(); | ||
| } | ||
|
|
||
| if (startInfo.RedirectStandardError) | ||
| { | ||
| SafeFileHandle.CreateAnonymousPipe(out parentErrorPipeHandle, out childErrorPipeHandle, asyncRead: OperatingSystem.IsWindows()); | ||
| } | ||
| else | ||
| { | ||
| childErrorPipeHandle = Console.OpenStandardErrorHandle(); | ||
| } |
There was a problem hiding this comment.
@jkotas The alternative would be to make Console.OpenStandard*Handle not throw and just return 0/1/2. Or use File.OpenNullHandle for Android (and iOS?). What do you think?
There was a problem hiding this comment.
I do not know what works on mobile wrt. Process.Start today. I agree with copilot that we should avoid regressing it.
There was a problem hiding this comment.
@lateralusX Anything to validate here specifically on mobile platforms?
There was a problem hiding this comment.
Couple of things to have in mind:
On both Android/iOS/TvOS fd 0,1,2 just points to /dev/null (with some exception running under debugger), in order to log to "console" you need to use platform specific API's, logcat on Android and NSLogStream on iOS/TvOS, that is what we do in the console PAL when we open OpenStandardOutput and OpenStandardError. There is no stdin on these platforms, that's why they throw PNS, but fd 0 should still represent stdin and wired to /dev/null, so won't work in practice.
Fork/exec is not allowed in most sandbox scenarios, so on iOS/TvOS, you can't run child process at all, StartCore will even throw PNS.
On Android there are security restrictions around fork/exec, but it should be possible to run system binaries and binaries deployed through the same APK without root permissions. Having that said, Android OS have restricted some of the capabilities over time.
In default case, child process will get fd 0,1,2 pointing to /dev/null. Since most logging in apps on Android goes through logcat, it will never hit stdout or stderr, but if you run shell commands or other processes using regular stderr/stdout, then it's possible to redirect and capture output in parent process.
Looking through this change, it doesn't seem to change current state around fork/exec on any of the mobile platforms, it just adding capability to get hold of stdin/stderr/stdout handles on Android, all pointing to /dev/null (if not redirected). What seems to change is that we now always dup stdin/stdout/stderr in the child process, even when not redirected, so that change might need to be verified on Android.
Doesn't look like we have any explicit test coverage on Android for Process.Start today, some tests are explicitly excluded, while others are guarded with RemoteExecutor.IsSupported. We should probably add something simple to verify the change on Android.
It could be argued if we should change the public API to return handles instead of PNS on Android. Another option could be to do Android specific handles inside the Process class, removing the need to expose it through the public API's.
There was a problem hiding this comment.
There is no stdin on these platforms, that's why they throw PNS, but fd 0 should still represent stdin and wired to /dev/null, so won't work in practice.
It could be argued if we should change the public API to return handles instead of PNS on Android. Another option could be to do Android specific handles inside the Process class, removing the need to expose it through the public API's.
The Console.OpenStandard*Handle could return 0/1/2 and it would represent /dev/null if it was opened for given process. We could also just return File.OpenNullHandle (an API that was added very recently).
But my understanding is that as of today, we should just not provide any handles if none were specified.
Fork/exec is not allowed in most sandbox scenarios, so on iOS/TvOS, you can't run child process at all, StartCore will even throw PNS.
How about posix_spawn? I will need to use it on macOS to implement some other features like #13943. Based on my research it's allowed on iOS if the user has the right permissions. And based on my performance benchmarks, it gives HUGE perf boost for starting processes on macOS itself.
We should probably add something simple to verify the change on Android.
Could you please tell me a bit more of how this could be tested without RemoteExecutor? Are all our tests run via emulation and for example I could just run some simple sh script because it's most likely run on some Unix machine?
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Windows.cs
Outdated
Show resolved
Hide resolved
| short expectedWindowFlag = short.Parse(argSplit[1]); | ||
|
|
||
| Assert.Equal(expectedDwFlag, si.dwFlags); | ||
| Assert.Equal(expectUsesShowWindow, (si.dwFlags & 0x1) != 0); // STARTF_USESHOWWINDOW is 0x1 |
There was a problem hiding this comment.
To other reviewers: This test was checking exact flag value. Now we just check if the flag is applied, allowing other flags like STARTF_USESTDHANDLES
| c_static_assert(PAL_PRIO_USER == (int)PRIO_USER); | ||
|
|
||
| #if !HAVE_PIPE2 | ||
| static pthread_mutex_t ProcessCreateLock = PTHREAD_MUTEX_INITIALIZER; |
There was a problem hiding this comment.
To other reviewers: now this lock is taken in managed code, only for the time of creating the pipes.
| bool usesTerminal = !(startInfo.RedirectStandardInput && | ||
| startInfo.RedirectStandardOutput && | ||
| startInfo.RedirectStandardError); | ||
| bool usesTerminal = Interop.Sys.IsATty(stdinHandle) || Interop.Sys.IsATty(stdoutHandle) || Interop.Sys.IsATty(stderrHandle); |
There was a problem hiding this comment.
To other reviewers: each of these handles could point to a terminal. So we perform a detailed check.
| } | ||
| else | ||
| { | ||
| childOutputPipeHandle = Console.OpenStandardOutputHandle(); |
There was a problem hiding this comment.
This and the other logic in this PR assume that stdin/stdout/stderr are distinct handles. The earlier versions had code to deal with the situation where these handle overlap. Do we need to worry about that?
There was a problem hiding this comment.
This and the other logic in this PR assume that stdin/stdout/stderr are distinct handles. The earlier versions had code to deal with the situation where these handle overlap. Do we need to worry about that?
That version was using an allow list which prohibits duplicates. It will be needed for #13943
In case of this PR, in the worst case (for example a process started with all handles redirected to NUL), we are going to perform 2 extra handle duplicates (it will be neglible when compared to the cost of starting a new process)
There was a problem hiding this comment.
It is the problem that Tom commented on. It is not clear what exact API shapes is this refactoring expected to support.
| { | ||
| childInputPipeHandle = Console.OpenStandardInputHandle(); | ||
| } | ||
|
|
||
| if (startInfo.RedirectStandardOutput) | ||
| { | ||
| SafeFileHandle.CreateAnonymousPipe(out parentOutputPipeHandle, out childOutputPipeHandle, asyncRead: OperatingSystem.IsWindows()); | ||
| } | ||
| else | ||
| { | ||
| childOutputPipeHandle = Console.OpenStandardOutputHandle(); | ||
| } | ||
|
|
||
| if (startInfo.RedirectStandardError) | ||
| { | ||
| SafeFileHandle.CreateAnonymousPipe(out parentErrorPipeHandle, out childErrorPipeHandle, asyncRead: OperatingSystem.IsWindows()); | ||
| } | ||
| else | ||
| { | ||
| childErrorPipeHandle = Console.OpenStandardErrorHandle(); | ||
| } |
There was a problem hiding this comment.
I do not know what works on mobile wrt. Process.Start today. I agree with copilot that we should avoid regressing it.
I think Deciding to no longer do
The API this is refactoring for is not worked out. It depends on what the API is whether it is useful to refactor this. |
…n Android (it's not supported)
The conclusion was that having more than one option bag would cause too much confusion (which one to use? is the old one deprecated? etc). So even if we come up with a better
We have at least two options:
If everything goes right, I am going to propose this API next Tuesday. With this refactor, exposing it publicly will be trivial. Another simple refactor is going to make it possible to move the Moreover, this refactor is going to make #13943 easier (and this API was already approved). @tmds please let me know if you need any more details. Thanks! |
| { | ||
| SafeFileHandle.CreateAnonymousPipe(out childInputPipeHandle, out parentInputPipeHandle); | ||
| } | ||
| else if (OperatingSystem.IsAndroid()) |
There was a problem hiding this comment.
This looks odd. Should we make OpenStandardInputHandle work on Android instead of this?
There was a problem hiding this comment.
@copilot address this feedback, make Console.OpenStandard*Handle APIs work on android by returning 0/1/2 values. Update the implementation, ref and tests. And then remove the Android checks from this PR
There was a problem hiding this comment.
Done in bba0144. Changed ConsolePal.Android.cs to return new SafeFileHandle(0/1/2, ownsHandle: false) (matching Unix/Wasi pattern), removed [UnsupportedOSPlatform("android")] from all three methods in both Console.cs and the ref assembly, updated tests, and removed the Android workarounds from Process.cs. All builds pass (ref, src including Android target, and Process).
I am not sure this API shape is going to fly well - it is inconsistent with what's there today. I think the SafeFileHandles should be members on ProcessStartInfo and be mutually exclusive with the other ways to redirect input/output. |
… 0/1/2; remove Android checks from Process.cs Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
| public static Stream OpenStandardError() => new LogcatStream(OutputEncoding); | ||
|
|
||
| public static SafeFileHandle OpenStandardInputHandle() => throw new PlatformNotSupportedException(); | ||
| public static SafeFileHandle OpenStandardInputHandle() => new SafeFileHandle(0, ownsHandle: false); |
There was a problem hiding this comment.
Hmm, this is inconsistent with what OpenStandardInput/OpenStandardOutput/OpenStandardError does now.
Can we just leave the standard handles alone when there is nothing to redirect?
|
We should start from the use-cases/issues, propose the API and design (which may involve some prototyping/derisking) and then the actual implementation follows. You may have a picture about the API and design. I don't think it is written down so others can understand what way we're heading and how it solves the user issues. I can technically review if this refactoring isn't breaking anything, but currently I can't say why we need it, which is the more important thing. Personally, I prefer we capture such information in an issue. PRs are already implementation centric and the coding agent workflows make it even more focused on code and code changes. |
…rning fd 0/1/2; remove Android checks from Process.cs" This reverts commit bba0144.

Description
Refactors
StartCoreon both Windows and Unix to acceptSafeFileHandleinstances for child process standard streams, enabling future scenarios where callers provide pre-existing handles. Pipe creation and stream setup are lifted into the sharedProcess.cs:Start()method, with platform-specificOpenStreamand encoding helper methods defined directly in each platform file (no partial methods).Native layer (
pal_process.c,pal_process.h,pal_process_wasi.c)stdinFd/stdoutFd/stderrFdchanged fromint32_t*(out) toint32_t(in) — caller now owns pipe creationSystemNative_Pipecalls) and associatedProcessCreateLockdup2of the provided fds ontoSTDIN_FILENO/STDOUT_FILENO/STDERR_FILENOis now unconditional (CLOEXEC flags are stripped bydup2itself since it doesn't copy descriptor flags)redirectStdin/redirectStdout/redirectStderrparameters from native signatureNew native entry point (
pal_io.c,pal_io.h)SystemNative_IsAtomicNonInheritablePipeCreationSupported— returns 1 whenHAVE_PIPE2is defined, 0 otherwise; used by managed code to decide whether pipe creation requires a write lockInterop layer (
Interop.ForkAndExecProcess.cs)out int stdinFd/stdoutFd/stderrFdtoSafeFileHandleparameters (marshalled asint32_tvalues)redirectStdin/redirectStdout/redirectStderrboolean parametersNew interop declarations
Interop.IsATty.cs: AddedSafeFileHandleoverload ofIsATtyInterop.IsAtomicNonInheritablePipeCreationSupported.cs: New P/Invoke withNullableBoolcaching forSystemNative_IsAtomicNonInheritablePipeCreationSupportedManaged layer
Process.cs:Start()now creates pipes viaSafeFileHandle.CreateAnonymousPipedirectly (withConsole.OpenStandard*Handle()fallbacks for non-redirected streams). Pipe creation is guarded by!startInfo.UseShellExecute || !OperatingSystem.IsWindows(). On platforms withoutpipe2, aWriteLockwraps pipe creation to prevent accidental handle inheritance. AfterStartCore, sets up_standardInput/_standardOutput/_standardErrorstreams via platform-specificOpenStream. TheCantRedirectStreamsvalidation forUseShellExecuteis moved here from platform files.Process.Unix.cs:StartCoretakesSafeFileHandle?parameters (asserted non-null). Terminal detection (usesTerminal) changed from checking redirect booleans toIsATty()calls on the actual handles.OpenStreamtransfers ownership fromSafeFileHandletoSafePipeHandleand wraps inAnonymousPipeClientStream.GetStandardInputEncoding/GetStandardOutputEncodingreturnEncoding.Default.s_processStartLockmoved to sharedProcessUtils.Process.Windows.cs:StartWithCreateProcesstakes non-nullableSafeFileHandleparameters. Handles are duplicated as inheritable viaDuplicateAsInheritable. The oldCreatePipehelper ands_createProcessLockare removed. The lock is replaced byProcessUtils.s_processStartLock.EnterWriteLock().OpenStreamwraps handle inFileStream.GetStandardInputEncoding/GetStandardOutputEncodinguse console code pages.Process.Win32.cs:StartCoresignature updated; asserts thatUseShellExecuteimplies all-null handles.CantRedirectStreamscheck removed (moved to sharedProcess.cs).ProcessUtils.cs: Now hostss_processStartLock(shared across platforms).Process.ConfigureTerminalForChildProcesses.Unix.cs: ReferencesProcessUtils.s_processStartLockinstead of the removed instance field.Console changes (Android support)
ConsolePal.Android.cs:OpenStandardInputHandle/OpenStandardOutputHandle/OpenStandardErrorHandlenow returnSafeFileHandle(0/1/2, ownsHandle: false)instead of throwingPlatformNotSupportedException, matching the existing Unix/Wasi behavior.Console.cs: Removed[UnsupportedOSPlatform("android")]from all threeOpenStandard*Handlemethods.System.Console.cs): Removed[UnsupportedOSPlatformAttribute("android")]from all three methods.ConsoleHandles.cs): Updated platform filters to include Android in valid-handle tests and exclude it from throws-on-unsupported tests.Build verification
TestGetProcesses)🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.