Skip to content

Consolidate Helix shards for quick test assemblies via MethodLimitMultiplier#10

Closed
MichaelSimons wants to merge 373 commits intomainfrom
consolidate-helix-shards
Closed

Consolidate Helix shards for quick test assemblies via MethodLimitMultiplier#10
MichaelSimons wants to merge 373 commits intomainfrom
consolidate-helix-shards

Conversation

@MichaelSimons
Copy link
Copy Markdown
Owner

Summary

Adds a MethodLimitMultiplier metadata option for SDKCustomXUnitProject items to consolidate Helix test shards for assemblies that produce many very short work items. This reduces per-item overhead (payload download, setup, teardown) and makes us better citizens of the shared Helix queues.

How it works

AssemblyScheduler partitions each test assembly into shards based on a methodLimit threshold (default 16 methods per shard). MethodLimitMultiplier simply multiplies that threshold, producing fewer, larger shards. It is set via MSBuild item metadata in UnitTests.proj — no new config files.

Changes

  • SDKCustomCreateXUnitWorkItemsWithTestExclusion.cs — Reads optional MethodLimitMultiplier metadata; warns on invalid values
  • UnitTests.proj — Applies multipliers to three assembly groups

Data (averaged across 5 recent main builds)

NetAnalyzers (x5 multiplier)

Platform Builds Current Shards Avg/Shard Max/Shard Est. New Shards Est. Avg/Shard
Linux x64 5 197 42s 2m 45s ~39 ~3m 34s
Windows x64 4 197 1m 45s 4m 28s ~39 ~8m 46s

TemplateEngine / TemplateSearch (x10 multiplier)

Platform Builds Current Shards Avg/Shard Max/Shard Est. New Shards Est. Avg/Shard
Linux x64 5 50 23s 1m 14s ~5 ~3m 57s
Windows x64 4 50 49s 1m 53s ~5 ~8m 12s

Api* (ApiCompatibility, ApiDiff, ApiSymbolExtensions) (x5 multiplier)

Platform Builds Current Shards Avg/Shard Max/Shard Est. New Shards Est. Avg/Shard
Linux x64 5 19 19s 36s ~4 ~1m 35s
Windows x64 4 19 1m 46s 2m 37s ~4 ~8m 50s

Total estimated savings per build

Platform Current work items Estimated new Saved
Linux x64 ~266 ~48 ~218
Windows x64 ~266 ~48 ~218

Note: Estimated avg/shard times are rough upper bounds (current avg × multiplier). Actual times may be lower since total compute is redistributed, not increased. The 2-hour Helix work item timeout provides ample headroom.

Verification

This is a draft PR — will compare actual shard counts and durations from the CI run against the estimates above.

dotnet-maestro Bot and others added 30 commits April 1, 2026 12:47
Updated Dependencies:
Microsoft.AspNetCore.Mvc.Razor.Extensions.Tooling.Internal, Microsoft.CodeAnalysis.Razor.Tooling.Internal, Microsoft.NET.Sdk.Razor.SourceGenerators.Transport (Version 10.0.0-preview.26181.113 -> 10.0.0-preview.26201.103)
Microsoft.Build (Version 18.3.3 -> 18.3.3)
Microsoft.Build.Localization (Version 18.3.3-servicing-26181-113 -> 18.3.3-servicing-26201-103)
Microsoft.Build.NuGetSdkResolver, NuGet.Build.Tasks, NuGet.Build.Tasks.Console, NuGet.Build.Tasks.Pack, NuGet.CommandLine.XPlat, NuGet.Commands, NuGet.Common, NuGet.Configuration, NuGet.Credentials, NuGet.DependencyResolver.Core, NuGet.Frameworks, NuGet.LibraryModel, NuGet.Localization, NuGet.Packaging, NuGet.ProjectModel, NuGet.Protocol, NuGet.Versioning (Version 7.3.0-rc.18213 -> 7.3.0-rc.20203)
Microsoft.Build.Tasks.Git, Microsoft.SourceLink.AzureRepos.Git, Microsoft.SourceLink.Bitbucket.Git, Microsoft.SourceLink.Common, Microsoft.SourceLink.GitHub, Microsoft.SourceLink.GitLab, Microsoft.TemplateEngine.Abstractions, Microsoft.TemplateEngine.Authoring.TemplateVerifier, Microsoft.TemplateEngine.Edge, Microsoft.TemplateEngine.Orchestrator.RunnableProjects, Microsoft.TemplateEngine.Utils, Microsoft.TemplateSearch.Common (Version 10.0.202 -> 10.0.202)
Microsoft.CodeAnalysis, Microsoft.CodeAnalysis.BuildClient, Microsoft.CodeAnalysis.CSharp, Microsoft.CodeAnalysis.CSharp.CodeStyle, Microsoft.CodeAnalysis.CSharp.Features, Microsoft.CodeAnalysis.CSharp.Workspaces, Microsoft.CodeAnalysis.ExternalAccess.HotReload, Microsoft.CodeAnalysis.PublicApiAnalyzers, Microsoft.CodeAnalysis.Workspaces.Common, Microsoft.CodeAnalysis.Workspaces.MSBuild, Microsoft.Net.Compilers.Toolset, Microsoft.Net.Compilers.Toolset.Framework (Version 5.3.0-2.26181.113 -> 5.3.0-2.26201.103)
Microsoft.DotNet.Arcade.Sdk, Microsoft.DotNet.Build.Tasks.Installers, Microsoft.DotNet.Build.Tasks.Templating, Microsoft.DotNet.Build.Tasks.Workloads, Microsoft.DotNet.Helix.Sdk, Microsoft.DotNet.SignTool, Microsoft.DotNet.XliffTasks, Microsoft.DotNet.XUnitExtensions (Version 10.0.0-beta.26181.113 -> 10.0.0-beta.26201.103)
Microsoft.FSharp.Compiler (Version 15.2.202-servicing.26181.113 -> 15.2.202-servicing.26201.103)
Microsoft.NET.Test.Sdk, Microsoft.TestPlatform.Build, Microsoft.TestPlatform.CLI (Version 18.3.0-release-26181-113 -> 18.3.0-release-26201-103)
Microsoft.TemplateEngine.Mocks, Microsoft.TemplateEngine.TestHelper, Microsoft.TemplateSearch.TemplateDiscovery (Version 10.0.202-servicing.26181.113 -> 10.0.202-servicing.26201.103)
[[ commit created by automation ]]
Adds device selection to dotnet-watch for MAUI/mobile scenarios, mirroring the `dotnet-run` device selection flow from the spec (`documentation/specs/dotnet-run-for-maui.md`).

Merged `TargetFrameworkSelectionPrompt` and the new device prompt into a single `WatchSelectionPrompt` (library) / `SpectreWatchSelectionPrompt` (console app), keeping Spectre.Console isolated to the console app project.

After TFM selection, `HotReloadDotNetWatcher` calls the `ComputeAvailableDevices` MSBuild target via in-process MSBuild. A single device is auto-selected; multiple devices show an interactive Spectre prompt with search. The selected device and its `RuntimeIdentifier` are passed to `dotnet build` (`-p:Device`, `-p:RuntimeIdentifier`) and to the launched dotnet run subprocess (`--device`). A re-restore is performed when the device provides a `RuntimeIdentifier` not present in the original restore.

Adds `--device` CLI option to `dotnet-watch` for pre-specifying a device.

Tests:
- Unit tests for prompt selection, search, caching, and `FormatDevice`
- E2E tests using `DotnetRunDevices` test asset: interactive device selection, single-device auto-select

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Tomáš Matoušek <tmat@users.noreply.github.com>
…t#53648)

Fixes: dotnet#53634

When a mobile/WebSocket app is shut down (Ctrl+C) or restarted (Ctrl+R), the child process is killed which closes the TCP connection. Kestrel tears down the HTTP context, and then `RunningProject.DisposeAsync` tries to dispose the WebSocket — hitting an `ObjectDisposedException` on the already-disposed `IFeatureCollection`. The `ListenForResponsesAsync` loop also logs a spurious `WebSocketException` error.

**Fixes:**
- `RequestHandler.Dispose`: catch `ObjectDisposedException` when the underlying Kestrel HTTP context is already torn down
- `IsExpectedConnectionTermination`: treat `WebSocketException` as expected when the socket state is `Aborted` (remote disconnected)

**Tests:**
- `CtrlC_ShutsDownCleanly`: verifies clean shutdown with WebSocket transport
- `CtrlR_RestartsCleanly`: verifies restart without `WebSocketException` or `ObjectDisposedException` errors

Both tests fail without the fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Updated Dependencies:
Microsoft.AspNetCore.Mvc.Razor.Extensions.Tooling.Internal, Microsoft.CodeAnalysis.Razor.Tooling.Internal, Microsoft.NET.Sdk.Razor.SourceGenerators.Transport (Version 10.0.0-preview.26181.114 -> 10.0.0-preview.26201.115)
Microsoft.Build, Microsoft.Build.Localization (Version 18.6.0-preview-26181-114 -> 18.6.0-preview-26201-115)
Microsoft.Build.NuGetSdkResolver, NuGet.Build.Tasks, NuGet.Build.Tasks.Console, NuGet.Build.Tasks.Pack, NuGet.CommandLine.XPlat, NuGet.Commands, NuGet.Common, NuGet.Configuration, NuGet.Credentials, NuGet.DependencyResolver.Core, NuGet.Frameworks, NuGet.LibraryModel, NuGet.Localization, NuGet.Packaging, NuGet.ProjectModel, NuGet.Protocol, NuGet.Versioning (Version 7.6.0-rc.18214 -> 7.6.0-rc.20215)
Microsoft.Build.Tasks.Git, Microsoft.SourceLink.AzureRepos.Git, Microsoft.SourceLink.Bitbucket.Git, Microsoft.SourceLink.Common, Microsoft.SourceLink.GitHub, Microsoft.SourceLink.GitLab (Version 10.0.300-alpha.26181.114 -> 10.0.300-alpha.26201.115)
Microsoft.CodeAnalysis, Microsoft.CodeAnalysis.BuildClient, Microsoft.CodeAnalysis.CSharp, Microsoft.CodeAnalysis.CSharp.CodeStyle, Microsoft.CodeAnalysis.CSharp.Features, Microsoft.CodeAnalysis.CSharp.Workspaces, Microsoft.CodeAnalysis.ExternalAccess.HotReload, Microsoft.CodeAnalysis.PublicApiAnalyzers, Microsoft.CodeAnalysis.Workspaces.Common, Microsoft.CodeAnalysis.Workspaces.MSBuild, Microsoft.CodeAnalysis.Workspaces.MSBuild.BuildHost, Microsoft.Net.Compilers.Toolset, Microsoft.Net.Compilers.Toolset.Framework (Version 5.6.0-2.26181.114 -> 5.7.0-1.26201.115)
Microsoft.DotNet.Arcade.Sdk, Microsoft.DotNet.Build.Tasks.Installers, Microsoft.DotNet.Build.Tasks.Templating, Microsoft.DotNet.Build.Tasks.Workloads, Microsoft.DotNet.Helix.Sdk, Microsoft.DotNet.SignTool, Microsoft.DotNet.XliffTasks, Microsoft.DotNet.XUnitExtensions (Version 10.0.0-beta.26181.114 -> 10.0.0-beta.26201.115)
Microsoft.FSharp.Compiler (Version 15.2.300-servicing.26181.114 -> 15.2.300-servicing.26201.115)
Microsoft.NET.Test.Sdk, Microsoft.TestPlatform.Build, Microsoft.TestPlatform.CLI (Version 18.3.0-release-26181-114 -> 18.3.0-release-26201-115)
Microsoft.TemplateEngine.Abstractions, Microsoft.TemplateEngine.Authoring.TemplateVerifier, Microsoft.TemplateEngine.Edge, Microsoft.TemplateEngine.Mocks, Microsoft.TemplateEngine.Orchestrator.RunnableProjects, Microsoft.TemplateEngine.TestHelper, Microsoft.TemplateEngine.Utils, Microsoft.TemplateSearch.Common, Microsoft.TemplateSearch.TemplateDiscovery (Version 10.0.300-preview.26181.114 -> 10.0.300-preview.26201.115)
[[ commit created by automation ]]
…260401.3

On relative base path root
Microsoft.Testing.Platform From Version 2.2.1-preview.26180.5 -> To Version 2.2.1-preview.26201.3
MSTest From Version 4.2.1-preview.26180.5 -> To Version 4.2.1-preview.26201.3
…260401.3

On relative base path root
Microsoft.Testing.Platform From Version 2.2.1-preview.26180.5 -> To Version 2.2.1-preview.26201.3
MSTest From Version 4.2.1-preview.26180.5 -> To Version 4.2.1-preview.26201.3
Reset patterns:
- global.json
- NuGet.config
- eng/Version.Details.xml
- eng/Version.Details.props
- eng/common/*
…260401.3

On relative base path root
Microsoft.Testing.Platform From Version 2.2.1-preview.26180.5 -> To Version 2.2.1-preview.26201.3
MSTest From Version 4.2.1-preview.26180.5 -> To Version 4.2.1-preview.26201.3
Reset patterns:
- global.json
- NuGet.config
- eng/Version.Details.xml
- eng/Version.Details.props
- eng/common/*
Updated Dependencies:
Microsoft.AspNetCore.Mvc.Razor.Extensions.Tooling.Internal, Microsoft.CodeAnalysis.Razor.Tooling.Internal, Microsoft.NET.Sdk.Razor.SourceGenerators.Transport (Version 10.0.0-preview.26201.103 -> 10.0.0-preview.26202.103)
Microsoft.Build (Version 18.3.3 -> 18.3.3)
Microsoft.Build.Localization (Version 18.3.3-servicing-26201-103 -> 18.3.3-servicing-26202-103)
Microsoft.Build.NuGetSdkResolver, NuGet.Build.Tasks, NuGet.Build.Tasks.Console, NuGet.Build.Tasks.Pack, NuGet.CommandLine.XPlat, NuGet.Commands, NuGet.Common, NuGet.Configuration, NuGet.Credentials, NuGet.DependencyResolver.Core, NuGet.Frameworks, NuGet.LibraryModel, NuGet.Localization, NuGet.Packaging, NuGet.ProjectModel, NuGet.Protocol, NuGet.Versioning (Version 7.3.0-rc.20203 -> 7.3.0-rc.20303)
Microsoft.Build.Tasks.Git, Microsoft.SourceLink.AzureRepos.Git, Microsoft.SourceLink.Bitbucket.Git, Microsoft.SourceLink.Common, Microsoft.SourceLink.GitHub, Microsoft.SourceLink.GitLab, Microsoft.TemplateEngine.Abstractions, Microsoft.TemplateEngine.Authoring.TemplateVerifier, Microsoft.TemplateEngine.Edge, Microsoft.TemplateEngine.Orchestrator.RunnableProjects, Microsoft.TemplateEngine.Utils, Microsoft.TemplateSearch.Common (Version 10.0.202 -> 10.0.202)
Microsoft.CodeAnalysis, Microsoft.CodeAnalysis.BuildClient, Microsoft.CodeAnalysis.CSharp, Microsoft.CodeAnalysis.CSharp.CodeStyle, Microsoft.CodeAnalysis.CSharp.Features, Microsoft.CodeAnalysis.CSharp.Workspaces, Microsoft.CodeAnalysis.ExternalAccess.HotReload, Microsoft.CodeAnalysis.PublicApiAnalyzers, Microsoft.CodeAnalysis.Workspaces.Common, Microsoft.CodeAnalysis.Workspaces.MSBuild, Microsoft.Net.Compilers.Toolset, Microsoft.Net.Compilers.Toolset.Framework (Version 5.3.0-2.26201.103 -> 5.3.0-2.26202.103)
Microsoft.DotNet.Arcade.Sdk, Microsoft.DotNet.Build.Tasks.Installers, Microsoft.DotNet.Build.Tasks.Templating, Microsoft.DotNet.Build.Tasks.Workloads, Microsoft.DotNet.Helix.Sdk, Microsoft.DotNet.SignTool, Microsoft.DotNet.XliffTasks, Microsoft.DotNet.XUnitExtensions (Version 10.0.0-beta.26201.103 -> 10.0.0-beta.26202.103)
Microsoft.FSharp.Compiler (Version 15.2.202-servicing.26201.103 -> 15.2.202-servicing.26202.103)
Microsoft.NET.Test.Sdk, Microsoft.TestPlatform.Build, Microsoft.TestPlatform.CLI (Version 18.3.0-release-26201-103 -> 18.3.0-release-26202-103)
Microsoft.TemplateEngine.Mocks, Microsoft.TemplateEngine.TestHelper, Microsoft.TemplateSearch.TemplateDiscovery (Version 10.0.202-servicing.26201.103 -> 10.0.202-servicing.26202.103)
[[ commit created by automation ]]
Testing `dotnet-watch` end-to end on a `dotnet new maui` project...

I found the Spectre.Console input was just "stuck", neither up/down arrows or typing to search did anything!

`PhysicalConsole` runs `Console.ReadKey()` in a tight background loop, consuming all key presses and dispatching them via the `KeyPressed` event. When `SpectreBuildParametersSelectionPrompt` returned `AnsiConsole.Console` for non-redirected stdin, Spectre.Console would also call `Console.ReadKey()` internally, creating a race where `PhysicalConsole` steals every key press and the selection prompt appears completely stuck (arrow keys, typing, and search all do nothing).

Fix by always using `KeyPressedAnsiConsole`, which reads keys from `PhysicalConsole.KeyPressed` events instead of competing for `Console.ReadKey()`. This ensures a single key reader (`PhysicalConsole`) distributes keys to all consumers including Spectre.Console.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reset patterns:
- global.json
- NuGet.config
- eng/Version.Details.xml
- eng/Version.Details.props
- eng/common/*
Reset patterns:
- global.json
- NuGet.config
- eng/Version.Details.xml
- eng/Version.Details.props
- eng/common/*
…260402.5

On relative base path root
Microsoft.Testing.Platform From Version 2.2.1-preview.26201.3 -> To Version 2.3.0-preview.26202.5
MSTest From Version 4.2.1-preview.26201.3 -> To Version 4.3.0-preview.26202.5
…260402.5

On relative base path root
Microsoft.Testing.Platform From Version 2.2.1-preview.26201.3 -> To Version 2.3.0-preview.26202.5
MSTest From Version 4.2.1-preview.26201.3 -> To Version 4.3.0-preview.26202.5
MichaelSimons and others added 29 commits April 22, 2026 12:52
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reset patterns:
- global.json
- NuGet.config
- eng/Version.Details.xml
- eng/Version.Details.props
- eng/common/*
Resolve merge conflicts, fix auto-merge compilation errors:
- Accept main's versions, TFMs, and code style (nameof, TestAssetsManager, ValueTask)
- Add back release-only content (DOTNET_HOST_PATH, ProjectConvertDuplicateRefFolderName, MissingShebang analyzer, file-based program warnings)
- Fix file headers on MissingShebang analyzer files (IDE0073)
- Fix _testAssetsManager -> TestAssetsManager renames in test files
- Remove references to deleted experimental feature flags
- Fix CreateProjectInstance signature mismatch
- Regenerate xlf files

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Gate AOT fast path behind DOTNET_CLI_ENABLEAOT and fix hostfxr initialization

The AOT parser in NativeEntryPoint is now only invoked when DOTNET_CLI_ENABLEAOT=true. When unset, the bridge falls through to the managed CLI immediately.

ManagedHost.RunApp now configures the runtime exactly as the muxer would for an SDK command:

Sets host_path in hostfxr_initialize_parameters so hostfxr uses the correct host executable identity instead of the current process.
Sets the HOSTFXR_PATH runtime property via hostfxr_set_runtime_property_value, matching the muxer's is_sdk_command=true behavior so the SDK can locate hostfxr directly.
All debug configurations (launchSettings.json, vcxproj.user, debug-dn.cmd, tasks.code-workspace) updated to set the new env var. DESIGN.md updated to reflect the DOTNET_CLI_ENABLEAOT gate and the muxer's existing try_invoke_aot_sdk integration.
…e args

The fix from 344e433 on release/10.0.4xx (replacing Except() with Where()
to preserve duplicate command arguments) was lost during merge conflict
resolution with OTel refactoring on main. This restores the correct behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e_feed folder to those required for CI to run.
…ine.Orchestrator.RunnableProjects project reference to avoid build collisions.
…tiplier

Add per-assembly MethodLimitMultiplier metadata support to reduce wasted
Helix compute on assemblies whose shards all complete very quickly. The
multiplier increases the methods-per-shard threshold, producing fewer but
larger work items and reducing per-item setup overhead.

Applied to three assembly groups:
- Microsoft.CodeAnalysis.NetAnalyzers (x5)
- Microsoft.TemplateEngine.* / TemplateSearch.* (x10)
- Microsoft.DotNet.Api{Compatibility,Diff,SymbolExtensions} (x5)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.