Hash-based pipe naming for fast node discovery on Unix#13337
Open
JakeRadMSFT wants to merge 4 commits intodotnet:mainfrom
Open
Hash-based pipe naming for fast node discovery on Unix#13337JakeRadMSFT wants to merge 4 commits intodotnet:mainfrom
JakeRadMSFT wants to merge 4 commits intodotnet:mainfrom
Conversation
This was referenced Mar 9, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Implements hash-based Unix named-pipe naming and discovery to speed up MSBuild reusable node lookup by avoiding O(N) probing of all dotnet processes, using pipe globbing under /tmp keyed by a stable handshake hash.
Changes:
- Move/expand handshake hashing so both sides can compute a stable hash for pipe naming.
- Add hash-based pipe naming (
MSBuild-{hash}-{pid}) and Unix discovery via/tmpdirectory listing. - Switch Unix node reuse connection attempts to use the hash-based pipe name and a non-zero reuse timeout.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Shared/NodeEndpointOutOfProcBase.cs | Reduces client connect timeout to limit impact of failed reuse probes. |
| src/Shared/NamedPipeUtil.cs | Adds hash-based pipe naming and Unix PID discovery from /tmp pipe filenames. |
| src/Shared/CommunicationsUtilities.cs | Adds Handshake.ComputeHash, adjusts Unix session ID behavior, tweaks default timeouts, and normalizes * architecture for hashing. |
| src/MSBuild/NodeEndpointOutOfProcTaskHost.cs | Uses hash-based pipe names on Unix for task host endpoints. |
| src/Build/BackEnd/Components/Communications/NodeProviderOutOfProcBase.cs | Switches Unix node discovery to /tmp listing and uses hash-based pipe names when connecting. |
| src/Build/BackEnd/Components/Communications/NodeEndpointOutOfProc.cs | Uses hash-based pipe names on Unix for worker node endpoints. |
| src/Build.UnitTests/BackEnd/UnixNodeReuseFixes_Tests.cs | Adds unit tests validating Unix session ID behavior in handshake keys. |
| src/Build.UnitTests/BackEnd/HashBasedPipeNaming_Tests.cs | Adds unit tests for hashing, pipe naming, and Unix pipe discovery behavior. |
Move ComputeHash() from ServerNodeHandshake to base Handshake class
so both client and server handshakes can compute their hash for pipe
naming.
Add GetHashBasedPipeName() and FindNodesByHandshakeHash() to
NamedPipeUtil for O(1) discovery of compatible nodes on Unix by
listing /tmp/MSBuild-{hash}-* instead of probing all dotnet processes.
Update NodeEndpointOutOfProc to create hash-based pipe names on Unix.
Update NodeProviderOutOfProcBase to use hash-based discovery on Unix
and hash-based pipe names when connecting.
Includes 12 unit tests covering ComputeHash, GetHashBasedPipeName,
and FindNodesByHandshakeHash.
The parent MSBuild uses hash-based pipe names on Unix via
TryConnectToProcess, but the task host child was still creating
pipes with the legacy MSBuild{pid} naming. This caused MSB4216
'Could not create or connect to a task host' errors on
Linux and macOS.
Apply the same hash-based pipe naming pattern used by
NodeEndpointOutOfProc to the task host endpoint.
When TaskHostParameters specify architecture '*' (any), resolve it to the actual current architecture so parent and child compute identical HandshakeOptions and hash-based pipe names on Unix. This fixes TransientAndSidecarNodeCanCoexist and TaskHostLifecycle tests that were failing because the parent had no architecture bits in the handshake while the child (using TaskHostParameters.Empty) resolved to the current arch.
JakeRadMSFT
force-pushed
the
feature/hash-based-pipe-naming
branch
from
845361a to
6414a83
Compare
- Replace runtime platform gates with [UnixOnlyFact]/[WindowsOnlyFact] - Use Directory.EnumerateFiles instead of GetFiles for streaming - Clarify fallback comment in FindNodesByHandshakeHash
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hash-based pipe naming for fast node discovery on Unix
Problem
On Unix/macOS, MSBuild discovers reusable worker nodes by enumerating all running
dotnetprocesses and probing each one with a handshake. With many processes running, this is slow and creates O(N) connection attempts.Solution
Encode both handshake hash and PID in the pipe name:
MSBuild-{hash}-{pid}. This allows the scheduler to find compatible nodes by listing/tmp/MSBuild-{hash}-*— an O(1) directory glob instead of trial-and-error probing.Changes
ComputeHash()fromServerNodeHandshaketo baseHandshakeclass so both client and server can compute their hashGetHashBasedPipeName()andFindNodesByHandshakeHash()for hash-based pipe naming and discoveryTests
12 new unit tests covering
ComputeHash,GetHashBasedPipeName, andFindNodesByHandshakeHash.This is now a standalone PR (previously stacked on #13336, which has been split into #13354–#13357).