Skip to content

Enlighten RAR task#13319

Open
AR-May wants to merge 27 commits intodotnet:mainfrom
AR-May:enlighten-RAR-2
Open

Enlighten RAR task#13319
AR-May wants to merge 27 commits intodotnet:mainfrom
AR-May:enlighten-RAR-2

Conversation

@AR-May
Copy link
Copy Markdown
Member

@AR-May AR-May commented Mar 3, 2026

Fixes #12483

Context

RAR task enlightening. De-facto the only inputs that I saw unenlightened in the logs were StateFile, AppConfigFile and CandidateAssemblyFiles. However multiple other inputs, like TargetFrameworkDirectories or HintPath of assemblies are not guaranteed to be the full path. I added additional logic that ensures that the full path is used for them too.

Changes Made

  • RAR out-of-proc node now uses the multithreaded TaskEnvironment for proper path resolution (instead of just updating StateFile and AppConfigFile)
  • Moved Drivers to the Framework project - was required for above. It is in a separate PR Move task environment drivers to Framework. #13380.
  • Resolvers now take TaskEnvironement object and use them for path absolutisation of the inputs that are not absolutized prior.
  • Some of the inputs now are backed up with AbsolutePath structs instead of strings.

Breaking changes under a ChangeWave:

  • Empty AppConfigFile resulted before in failure while now it is skipped (as if it was null before)
  • StateFile.ItemSpec is used for the path instead of StateFile.ToString() which could be otherwise defined in a custom ITaskItem

Commits overview:

  1. Make RAR as a service use TaskEnvironment: Updates out-of-process RAR client and supporting classes to be thread-safe and work through TaskEnvironment.
  2. Absolutize RAR task inputs: Makes RAR inputs absolute paths and updates ReferenceTable to use TaskEnvironment instead of process-level environment.
  3. Make resolvers use task environment: Updates all assembly resolvers (AssemblyFoldersResolver, GacResolver, HintPathResolver, etc.) to use TaskEnvironment instead of process environment.
  4. Enlighten GetReferenceAssemblyPaths: Updates GetReferenceAssemblyPaths task and GlobalAssemblyCache to work with TaskEnvironment. This is required because GlobalAssemblyCache refactoring is needed for RAR task
  5. Fix tests: Updates test files to work with the new TaskEnvironment-based RAR implementation, adding necessary setup and configuration.

Testing (in progress)

Checklist:

  • Manual tests (building OC in mt mode)
  • Unit tests
  • Exp insertion
  • Test RAR node mode (building OC with RAR out-of-proc node) - found issue, does not work!
  • VMR
  • Analyzer

@AR-May AR-May self-assigned this Mar 4, 2026
Comment thread src/Framework/FrameworkCommunicationsUtilities.cs Outdated
@AR-May AR-May marked this pull request as ready for review March 4, 2026 13:48
Copilot AI review requested due to automatic review settings March 4, 2026 13:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR advances “RAR task enlightening” by routing path resolution and environment access through TaskEnvironment (including in the out-of-proc RAR node), and by upgrading several RAR inputs from strings to AbsolutePath to ensure stable, project-relative-to-absolute resolution in multithreaded builds.

Changes:

  • RAR and related resolvers now use TaskEnvironment for absolute path conversion and environment variable access.
  • Out-of-proc RAR node now hydrates a multithreaded TaskEnvironment using the client project directory.
  • Introduces a new ChangeWave (18.6) and updates task/unit test code to provide TaskEnvironment.

Reviewed changes

Copilot reviewed 51 out of 51 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/Tasks/SystemState.cs Uses TaskEnvironment for state file path handling during precomputed cache load.
src/Tasks/StateFileBase.cs Clarifies expectation that state file path is absolute.
src/Tasks/RedistList.cs Moves redist list paths to AbsolutePath and enables TaskEnvironment-based absolutization.
src/Tasks/InstalledSDKResolver.cs Caches absolute SDK root paths for resolver probing.
src/Tasks/GetReferenceAssemblyPaths.cs Marks task as multi-threadable and passes TaskEnvironment into GAC resolution.
src/Tasks/AssemblyDependency/Resolver.cs Adds TaskEnvironment to resolver base for consistent path/env usage.
src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs Converts several inputs to AbsolutePath, adds MT task interface, and updates RAR execution to pass TaskEnvironment through the pipeline.
src/Tasks/AssemblyDependency/ReferenceTable.cs Propagates TaskEnvironment into resolution pipeline and normalizes resolved paths.
src/Tasks/AssemblyDependency/Reference.cs Adds IsFrameworkFile overload for AbsolutePath[] framework paths.
src/Tasks/AssemblyDependency/RawFilenameResolver.cs Resolves raw filename candidate via TaskEnvironment.
src/Tasks/AssemblyDependency/Node/RarNodeExecuteRequest.cs Captures project directory for out-of-proc node TaskEnvironment creation.
src/Tasks/AssemblyDependency/Node/OutOfProcRarNodeEndpoint.cs Creates a multithreaded TaskEnvironment per request using the client project directory.
src/Tasks/AssemblyDependency/HintPathResolver.cs Resolves hint path using TaskEnvironment and canonicalizes.
src/Tasks/AssemblyDependency/GlobalAssemblyCache.cs Switches env-var access to TaskEnvironment and threads it through GetLocation APIs.
src/Tasks/AssemblyDependency/GacResolver.cs Threads TaskEnvironment into GAC resolver construction.
src/Tasks/AssemblyDependency/FrameworkPathResolver.cs Threads TaskEnvironment into framework path resolver construction.
src/Tasks/AssemblyDependency/DirectoryResolver.cs Caches absolute search path via TaskEnvironment.
src/Tasks/AssemblyDependency/CandidateAssemblyFilesResolver.cs Threads TaskEnvironment into candidate assembly files resolver construction.
src/Tasks/AssemblyDependency/AssemblyResolution.cs Threads TaskEnvironment through resolver compilation APIs.
src/Tasks/AssemblyDependency/AssemblyFoldersResolver.cs Threads TaskEnvironment into AssemblyFolders resolver.
src/Tasks/AssemblyDependency/AssemblyFoldersFromConfig/AssemblyFoldersFromConfigResolver.cs Uses TaskEnvironment for cache escape-hatch env var access and passes it into cache.
src/Tasks/AssemblyDependency/AssemblyFoldersFromConfig/AssemblyFoldersFromConfigCache.cs Uses TaskEnvironment for cache escape-hatch env var access.
src/Tasks/AssemblyDependency/AssemblyFoldersExResolver.cs Uses TaskEnvironment for cache escape-hatch env var access and passes it into cache.
src/Tasks.UnitTests/RARPrecomputedCache_Tests.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/HintPathResolver_Tests.cs Updates resolver test to provide TaskEnvironment.
src/Tasks.UnitTests/GetReferencePaths_Tests.cs Updates task test to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/WinMDTests.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/VerifyTargetFrameworkHigherThanRedist.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/VerifyTargetFrameworkAttribute.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/VerifyIgnoreVersionForFrameworkReference.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/SuggestedRedirects.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/StronglyNamedDependencyAutoUnify.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/StronglyNamedDependencyAppConfig.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/StronglyNamedDependency.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/SpecificVersionPrimary.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/ResolveAssemblyReferenceTestFixture.cs Updates helper to pass TaskEnvironment into GAC resolution.
src/Tasks.UnitTests/AssemblyDependency/Perf.cs Updates perf tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/NonSpecificVersionStrictPrimary.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/Node/RarNodeExecuteRequest_Tests.cs Updates node request tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/Node/OutOfProcRarNode_Tests.cs Updates node tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/Miscellaneous.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/InstalledSDKResolverFixture.cs Updates resolver fixture to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/GlobalAssemblyCacheTests.cs Updates cache tests to pass TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/FilePrimary.cs Updates RAR tests to provide TaskEnvironment.
src/Tasks.UnitTests/AssemblyDependency/AssemblyFoldersFromConfig_Tests.cs Updates tests to provide TaskEnvironment.
src/Framework/MultiThreadedTaskEnvironmentDriver.cs Moves driver into Framework layer and updates env-var comparer dependency.
src/Framework/MultiProcessTaskEnvironmentDriver.cs Moves driver into Framework layer and routes env-var ops through Framework utilities.
src/Framework/FrameworkCommunicationsUtilities.cs Adds Framework-local env-var utilities needed by moved drivers.
src/Framework/ChangeWaves.cs Adds new wave 18.6 and includes it in AllWaves.
src/Build/Microsoft.Build.csproj Removes compilation of drivers from Build project (now in Framework).
src/Build/BackEnd/Components/RequestBuilder/RequestBuilder.cs Updates comment to reflect new Framework env-var utilities usage.
Comments suppressed due to low confidence (1)

src/Tasks/AssemblyDependency/ReferenceTable.cs:496

  • When assemblyFileName is relative, reference.FullPath is made absolute via TaskEnvironment, but the subsequent _fileExists, _directoryExists, _getAssemblyName, and ResolvedSearchPath logic still uses the original (potentially relative) assemblyFileName. In multithreaded/out-of-proc scenarios this can probe the wrong directory and fail to resolve assemblies. Use the resolved absolute path (e.g., reference.FullPath) consistently for file system checks and assembly name probing.

Comment thread src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs
Comment thread src/Tasks/SystemState.cs Outdated
Comment thread src/Tasks/AssemblyDependency/RawFilenameResolver.cs Outdated
Comment thread src/Tasks/InstalledSDKResolver.cs Outdated
Comment thread src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs
Comment thread src/Tasks/AssemblyDependency/ReferenceTable.cs Outdated
Comment thread src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs Outdated
Comment thread src/Build/Microsoft.Build.csproj Outdated
Comment thread src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs Outdated
Comment thread src/Tasks/AssemblyDependency/ResolveAssemblyReference.cs
Comment thread src/Tasks/StateFileBase.cs
Comment thread src/Tasks/AssemblyDependency/AssemblyFoldersResolver.cs Outdated
Comment thread src/Tasks/AssemblyDependency/ReferenceTable.cs Outdated
@AR-May
Copy link
Copy Markdown
Member Author

AR-May commented Mar 13, 2026

I separated moving drivers to the Framework project to another PR: #13380.
Merge this changes after #13380 is merged. There would be build errors meanwhile.

Comment thread src/Tasks.UnitTests/AssemblyDependency/GlobalAssemblyCacheTests.cs
Comment thread src/Tasks/AssemblyDependency/Node/OutOfProcRarNodeEndpoint.cs
@AR-May
Copy link
Copy Markdown
Member Author

AR-May commented Apr 16, 2026

/review

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 16, 2026

Expert Code Review (command) failed. Please review the logs for details.

Review complete — the expert-reviewer sub-agent posted 6 inline comments and 1 summary COMMENT review on PR #13319. No additional action needed from the orchestrator.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expert Review — PR #13319: RAR Path Handling Enlightenment (Part 2)

Summary

This PR continues the enlightenment of RAR (ResolveAssemblyReference) for the TaskEnvironment API. Key changes include: defensive ?? [] null-coalescing for array property setters, removal of redundant _taskEnvironment field in AssemblyFoldersExResolver, removal of .GetCanonicalForm() from individual resolvers (centralizing canonicalization in ReferenceTable), a new AbsolutePath overload for StateFileBase.SerializeCache, path absolutization in AssemblyFoldersFromConfig, ChangeWave 18_6 tests for empty AppConfigFile, and an InstalledSDKResolver comparer fix.

Severity Legend

  • 🔴 BLOCKING — Must be fixed before merge
  • 🟠 MAJOR — Should be fixed; significant quality/correctness concern
  • 🟡 MINOR — Nice to fix; low-risk style or completeness issue
  • 🔵 ADVISORY — Informational, no action required

Issues Found

# Severity Dimension File Finding
1 🟠 Null Safety AssemblyFoldersFromConfigCache.cs:49-51 GetAbsolutePath called before null/empty filter — could throw for edge-case empty DirectoryPath after env-var expansion
2 🟠 Null Safety AssemblyFoldersFromConfigResolver.cs:185 Same issue — no guard before GetAbsolutePath on config directory paths
3 🟡 Code Duplication StateFileBase.cs:78-108 AbsolutePath overload is near-copy of string overload (acceptable per migration plan)
4 🟡 C# Modernization Miscellaneous.cs:1189,1215 new ITaskItem[] { ... }[...] collection expressions
5 🟡 Test Resilience Miscellaneous.cs:1223 ResetStateForTests() skipped if assertion throws
6 🔵 Good Fix InstalledSDKResolver.cs:36 Comparer propagation fix — prevents key-mismatch bugs

Verification Notes

Canonicalization removal is correct: Confirmed that ReferenceTable.cs:1368 canonicalizes all resolved paths centrally before storing them in _externallyResolvedImmutableFiles. Removing .GetCanonicalForm() from individual resolvers avoids redundant Path.GetFullPath calls.

ChangeWave 18_6 is the correct wave version: Verified in ChangeWaves.cs. Wave18_6 is defined and properly gated.

FullFrameworkFolders null-conditional removal is correct: VerifyThrowArgumentNull at line 961 guarantees value is non-null, making value?.Select(...)value.Select(...) safe.

?? [] defensive null handling is a good improvement: Prevents null assignment to array fields that are initialized as [], maintaining the non-null invariant.

_stateFile type change is sound: _stateFile is AbsolutePath (not Lazy<AbsolutePath>), so calling SerializeCache(_stateFile, Log) directly (dropping .Value) correctly matches the new AbsolutePath overload.

Verdict

No blocking issues. Two MAJOR defensive-coding concerns around GetAbsolutePath being called without null/empty guards on config-derived paths. Recommend adding guards to prevent ArgumentException from propagating in edge cases.

Note

🔒 Integrity filter blocked 2 items

The following items were blocked because they don't meet the GitHub integrity level.

  • #13319 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #13319 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by Expert Code Review (command) for issue #13319 · ● 13.9M

Comment thread src/Tasks/InstalledSDKResolver.cs Outdated
Comment thread src/Tasks.UnitTests/AssemblyDependency/Miscellaneous.cs Outdated
Comment thread src/Tasks/StateFileBase.cs
Comment thread src/Tasks.UnitTests/AssemblyDependency/Miscellaneous.cs Outdated
@AR-May
Copy link
Copy Markdown
Member Author

AR-May commented Apr 23, 2026

/review

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 23, 2026

Expert Code Review (command) completed successfully!

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RAR Multi-Threaded Task Migration Review

Overall this is a solid migration. The architecture of propagating TaskEnvironment through the resolver hierarchy, gating behavioral changes behind Wave 18.6, and adding connection pooling for out-of-proc RAR is well-designed. A few findings:

🔴 IMPORTANT (2 findings)

  1. Broken connections returned to pool (OutOfProcRarClient.cs line 199): If WritePacket/ReadPacket throws (broken pipe), the finally block returns the broken pipeClient to the pool. Next caller may dequeue it, see stale IsConnected == true, and fail. Fix: Only return to pool on success; dispose on failure.

  2. Unguarded GetCanonicalForm() calls (ReferenceTable.cs lines 481, 1370): GetCanonicalForm() calls Path.GetFullPath() which can throw ArgumentException on .NET Framework for invalid path characters. The PR introduces TryGetCanonicalForm() as a resilient alternative but doesn't use it consistently. These call sites lack protection, unlike MakeCanonicalPath in RAR (line 1170) which correctly uses TryGetCanonicalForm(Log).

🟡 MINOR (3 findings)

  1. LINQ in property getters (RAR lines 298, 407, 435, 947): .Select(path => path.OriginalValue).ToArray() allocates on every property access. Acceptable for setters (called once), but getters are called during serialization.

  2. AppConfigFile setter null handling (RAR line 699-713): The three-way branch (non-empty / empty / null) is correct but subtle — a comment would help maintainability.

  3. Broad exception catch in TryGetCanonicalForm (TaskEnvironmentExtensions.cs line 30): Not blocking; the fallback is safe.

✅ Verified correct

  • TaskEnvironment excluded from serialization is correctly handled — OutOfProcRarNodeEndpoint creates its own TaskEnvironment with MultiThreadedTaskEnvironmentDriver (line 103-105).
  • ChangeWave 18.6 guards are consistently applied for behavioral changes.
  • Resolver base class correctly propagates taskEnvironment to all subclasses.
  • Double-checked locking in Dispose() is correct (volatile + lock).
  • StateFileBase string overloads with ignoreRootedCheck: true are appropriate for backward compat.

Note

🔒 Integrity filter blocked 2 items

The following items were blocked because they don't meet the GitHub integrity level.

  • #13319 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
  • #13319 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by Expert Code Review (command) for issue #13319 · ● 12.1M

Comment on lines 699 to +710
public string AppConfigFile
{
get { return _appConfigFile; }
set { _appConfigFile = value; }
get => _appConfigFile.OriginalValue;
set
{
if (!string.IsNullOrEmpty(value))
{
_appConfigFile = MakeAbsolutePath(value);
}
else if (value == string.Empty && !ChangeWaves.AreFeaturesEnabled(ChangeWaves.Wave18_6))
{
_appConfigValueIsEmptyString = true;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MINOR — null value silently falls through both branches

When value is null:

  1. string.IsNullOrEmpty(null)true → first branch skipped
  2. value == string.Emptyfalse → second branch skipped
  3. Neither _appConfigFile nor _appConfigValueIsEmptyString is set

This appears intentional (null = "not set"), and the backward compat shim at line 2506 only fires for _appConfigValueIsEmptyString, which correctly only triggers for the empty-string case.

A brief comment documenting the null-falls-through behavior would be helpful for future maintainers, since this three-way branching (non-empty / empty / null) is subtle.

Comment on lines +1368 to +1370
if (ChangeWaves.AreFeaturesEnabled(ChangeWaves.Wave18_6))
{
resolvedPath = FileUtilities.FixFilePath(_taskEnvironment.GetAbsolutePath(resolvedPath).GetCanonicalForm()).Value;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMPORTANT — Unhandled exception from GetCanonicalForm() in the Wave 18.6 path

Same concern as NameAssemblyFileReference line 481: GetCanonicalForm() can throw ArgumentException on .NET Framework for invalid path characters, but there's no try-catch around this specific code path.

The try at line 1340 only catches BadImageFormatException.

Consider using TryGetCanonicalForm() here too:

resolvedPath = FileUtilities.FixFilePath(_taskEnvironment.GetAbsolutePath(resolvedPath).TryGetCanonicalForm()).Value;

This is especially important because resolvedPath comes from resolvers which may return paths with unusual characters (e.g., from registry-based AssemblyFoldersEx entries).

Comment on lines +193 to +199
}
finally
{
// Return the pipe client to the pool
if (pipeClient != null)
{
ErrorUtilities.ThrowInternalError($"Received unexpected packet type {packet.Type}");
ReleaseConnection(pipeClient);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMPORTANT — Broken connection returned to pool on exception

If WritePacket (line 152) or ReadPacket (lines 154/186) throws (e.g., IOException from a broken pipe, deserialization error), the finally block returns the broken pipeClient to the pool via ReleaseConnection.

The next caller that dequeues this client may see IsConnected == true (the property can be stale — the pipe stream hasn't been explicitly disconnected), attempt to write/read on a corrupted stream, and get another exception. This creates a cascading failure pattern.

Suggested fix: Track success and only return to pool on success. On failure, dispose the client:

bool success = false;
try
{
    pipeClient = AcquireConnection();
    // ... existing logic ...
    success = true;
    return response.Success;
}
finally
{
    if (pipeClient != null)
    {
        if (success)
            ReleaseConnection(pipeClient);
        else
            pipeClient.Dispose();
    }
}

This also handles the minor sub-case where ConnectToServer returns false — currently the never-connected client is pooled (harmless but wasteful).

{
return absolutePath.GetCanonicalForm();
}
catch (Exception e)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MINOR — Broad exception catch

Catching Exception is very broad. Consider narrowing to the specific exceptions Path.GetFullPath() can throw: ArgumentException, NotSupportedException, PathTooLongException, SecurityException.

That said, the XML doc comment explains the rationale well, and the fallback (returning the original path) is safe. The logging at MessageImportance.Low is appropriate for this resilience pattern. Not blocking.

Comment on lines 479 to +481
if (!Path.IsPathRooted(assemblyFileName))
{
reference.FullPath = Path.GetFullPath(assemblyFileName);
reference.FullPath = _taskEnvironment.GetAbsolutePath(assemblyFileName).GetCanonicalForm();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMPORTANT — Missing TryGetCanonicalForm error handling (unlike other call sites)

This call to GetCanonicalForm() (which calls Path.GetFullPath()) is unguarded and can throw ArgumentException on .NET Framework for paths with illegal characters (e.g., |, <, >).

In contrast, ResolveAssemblyReference.MakeCanonicalPath (line 1170) uses TryGetCanonicalForm(Log) which catches exceptions and falls back gracefully. Here, an exception would propagate up uncaught (the try block at line 488 only catches BadImageFormatException and UnauthorizedAccessException).

Consider using TryGetCanonicalForm() here for consistency and resilience:

reference.FullPath = _taskEnvironment.GetAbsolutePath(assemblyFileName).TryGetCanonicalForm();

Note: The old code Path.GetFullPath(assemblyFileName) had the same throw risk, so this isn't a regression per se — but since the PR introduces TryGetCanonicalForm as a resilient alternative, it should be used consistently.

/// </summary>
private AbsolutePath[] MakeAbsolutePaths(string[] paths)
{
return paths?.Select(path => MakeAbsolutePath(path)).ToArray() ?? [];
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MINOR — LINQ allocation on potentially large arrays

MakeAbsolutePaths uses .Select().ToArray(). While the setter is called once per task execution, CandidateAssemblyFiles can contain hundreds of items in large projects, and similar LINQ patterns appear in:

  • Property getters like CandidateAssemblyFiles (line 407): .Select(path => path.OriginalValue).ToArray()
  • MakeCanonicalPaths (line 1180)
  • TargetFrameworkDirectories, LatestTargetFrameworkDirectories, FullFrameworkFolders getters

For setters (called once), this is acceptable. For getters that allocate a new array on every access, consider caching the string[] result or using a simple loop instead of LINQ.

Not blocking, but worth noting since RAR runs for every project and these getters may be called during serialization (out-of-proc path).

@SimaTian
Copy link
Copy Markdown
Member

SimaTian commented Apr 23, 2026

Hello, I poked and proded at the PR, making use of the whole shebang I gathered while trying to migrate sdk tasks. I also noticed a drop in my ability to properly review this manually, which is concerning and vexing.

That being said - here is a group of tests that pin what the review considers breaking points:
https://github.com/dotnet/msbuild/tree/pr-13319-regression-tests
Single file: src/Tasks.UnitTests/AssemblyDependency/Node/PRBreakage_Tests.cs

Repro:
.\build.cmd -projects src\Tasks.UnitTests\Microsoft.Build.Tasks.UnitTests.csproj -c Debug
dotnet artifacts\bin\Microsoft.Build.Tasks.UnitTests\Debug\net10.0\Microsoft.Build.Tasks.UnitTests.dll --filter-class "*PRBreakage_Tests"

Deterministic tests coverage, each pins a behavior worth considering. I'm not saying all of these need to be addressed. But it would be nice to consider how much of an issue they are

NEW3 _appConfigValueIsEmptyString silently dropped across the OOP wire
-> with Wave18_6 OFF, the legacy error path on the node never fires.
MP raises an error; OOP succeeds silently. Behavioural divergence.

NEW4 StateFile / AppConfigFile no longer absolutized at the wire boundary
-> under Wave18_6 OFF the node receives a bare relative string and
resolves against node-process CWD instead of project directory.
Two tests: one for StateFile, one for AppConfigFile.
(relevant only in the (OOP-on ∧ Wave18_6-off) configuration )

NEW8 RawFilenameResolver guards only rawFileNameCandidate != null
-> empty ItemSpec hits AbsolutePath ctor and throws
ArgumentException("path"). Pre-PR this codepath silently no-op'd.

NEW10 MultiThreadedTaskEnvironmentDriver accepts unvalidated ProjectDirectory
from the wire. Two tests:
- null -> opaque ArgumentException with ParamName="path" rather
than a typed precondition on the public parameter.
- relative -> silently accepted via ignoreRootedCheck:true and the
ctor immediately writes the bogus value to
FileUtilities.CurrentThreadWorkingDirectory.

C5/N3 AppConfigFile setter has stale state on re-assignment to "":
with Wave18_6 OFF, _appConfigFile retains the prior absolute path
while _appConfigValueIsEmptyString flips to true - the instance
simultaneously claims "I have a config" and "the user cleared it".

SimaTian added a commit that referenced this pull request Apr 23, 2026
…13319

Adds a paired `_Control` test next to each `_Breakage` test where a
multi-process / single-threaded equivalent of the scenario exists. Each
Control test is expected to PASS on the PR head, demonstrating that the
breakage is specific to the new multi-threaded / out-of-proc path and
not present in the original `-multiprocess` paradigm.

Also adds the concurrency regression suite (PRBreakageConcurrency_Tests):
NEW-1 publication race, C1 broken-pipe pool poisoning (10s hard
timeout), NEW-2 client read deadlock on server exception, and NEW-6
GetInstance TOCTOU leak.

Findings without a meaningful MP equivalent (NEW-8, NEW-10, C1, NEW-2)
have an inline note explaining why no Control test exists.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@SimaTian
Copy link
Copy Markdown
Member

SimaTian commented Apr 23, 2026

Secondary findings - nondeterministic test breakages

https://github.com/dotnet/msbuild/tree/pr-13319-regression-tests-concurrency

Comments per concurrency test (PRBreakageConcurrency_Tests)

NEW1_StringsInitialize_HasPublicationRace
Strings (RAR.cs:74-170) is a private static class with private static bool initialized = false and a plain check-then-set in Initialize, followed by ~30 dependent field assignments. No volatile/lock/LazyInitializer.
Reader can observe initialized==true while one or more dependent fields are still null. The test resets state (nulls all dependent fields LAST after initialized=false) and races 32 threads × 200 iterations through new RAR().
Confidence:
high on ARM64 (weak memory ordering exposes the publication race directly)
medium on x64 (relies on TSO + thread interleaving).
Bump Iterations/Threads if false negative on x64 CI.
Fix shape: LazyInitializer.EnsureInitialized, a static ctor (CLR guarantees publication), or volatile bool + Interlocked.MemoryBarrier() after the field assignments.

NEW1_StringsInitialize_SingleThreaded_Control
The control spawns multiple fresh child test processes in parallel and verify the scenario succeeds under process isolation.

C1_FailedExecute_DoesNotReturnBrokenPipeToPool
OutOfProcRarClient.Execute finally block calls ReleaseConnection(pipeClient) unconditionally (lines 128-202), even after the pipe write/read just threw.
ReleaseConnection enqueues into _availablePipeClients.
Next Execute dequeues a broken pipe -> 100% failure rate downstream. The test starts a real endpoint, succeeds once (pipe pooled), kills the endpoint, fails the second Execute, then asserts _availablePipeClients.Count == 0 via reflection. Wrapped in Task.Run + 10s Wait because the broken-pipe symptom often manifests as a hang (NEW-2 is the same root
cause: no client read timeout). Without the timeout the test deadlocks the process. Fix shape: in the catch/finally, dispose the pipe instead of pooling it; only pool on the happy path.

NEW2_OopNodeException_LeavesClientReadBlocked
OutOfProcRarNodeEndpoint.RunAsync (line 129-132) wraps the entire packet-handling block in catch (Exception e) when (e is not OperationCanceledException)
that just traces and continues the accept loop - no response sent, no disconnect. The client's ReadPacket (NodePipeBase:106) has no timeout. The test spins a real endpoint, sends a RarNodeExecuteRequest
with _projectDirectory reflection-nulled (so MultiThreadedTaskEnvironmentDriver(null) throws server-side), and reads asynchronously with a 3s CancellationToken. A passing test = an OperationCanceledException from the client read = deadlock confirmed. This is an inverted assertion (catching the bug as a timeout) - clearly noted in the assertion message. Confidence: medium-to-high;
depends on the endpoint actually catching/swallowing the exception within 3 s. Fix shape: send a typed failure response packet on caught exception, OR disconnect the pipe on any unhandled error.

NEW6_GetInstance_TOCTOU_LeaksUnregisteredClients
OutOfProcRarClient.GetInstance (lines 61-76) does Get → new → RegisterTaskObject. RegisterTaskObject overwrites the cache entry, but the local-variable instance is what's returned. Concurrent winners each construct their own instance; only the LAST register survives; earlier instances leak (returned to caller, never disposed, hold pipe handles for the
lifetime of the MSBuild server). The test races 32 threads through GetInstance, counts distinct returned instances, asserts distinct == 1. High confidence - TOCTOU is reliably reproducible at 32 threads.
Fix shape: ConcurrentDictionary.GetOrAdd on EngineServices, or a lock around the get/register pair with re-read after add.

NEW6_GetInstance_SingleThreaded_Control
The control spawns multiple fresh child test processes in parallel and verify the scenario succeeds under process isolation.

Copy link
Copy Markdown
Member

@JanProvaznik JanProvaznik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review point 1 before I get to the juicy stuff:
pls remove everywhere t.TaskEnvironment = TaskEnvironmentHelper.CreateForTest(); which will make the PR also optically smaller

if (resolvedPath != null)
{
resolvedPath = FileUtilities.FixFilePath(_taskEnvironment.GetAbsolutePath(resolvedPath).GetCanonicalForm()).Value;
if (ChangeWaves.AreFeaturesEnabled(ChangeWaves.Wave18_6))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wave 18.7/8 depends on when we merge...

{
AllowOutOfProcNode = true,
BuildEngine = engine,
TaskEnvironment = TaskEnvironmentHelper.CreateForTest(),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obsolete, remove - is implicit in task what is fallback

ResolveAssemblyReference clientRar = new()
{
BuildEngine = new MockEngine(),
TaskEnvironment = TaskEnvironmentHelper.CreateForTest(),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obsolete remove (search for all occurrences in your changed files)

ResolveAssemblyReference t = new ResolveAssemblyReference();

t.BuildEngine = new MockEngine(_output);
t.TaskEnvironment = TaskEnvironmentHelper.CreateForTest();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are there any tests for the -mt path?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrate RAR task to the new task API

4 participants