Skip to content

Fix Helix tests not running after xUnit v3 migration#53757

Closed
MichaelSimons wants to merge 1 commit intodotnet:mainfrom
MichaelSimons:fix/helix-xunitv3-dotnet-exec
Closed

Fix Helix tests not running after xUnit v3 migration#53757
MichaelSimons wants to merge 1 commit intodotnet:mainfrom
MichaelSimons:fix/helix-xunitv3-dotnet-exec

Conversation

@MichaelSimons
Copy link
Copy Markdown
Member

Summary

After the xUnit v2 to v3 migration (#52930), all Helix work items fail with:

No test is available in {assembly}.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

This means zero tests are actually executing on Helix. The ~2k tests visible on the PR Tests tab come from non-Helix sources (FullFramework/AoT legs), down from ~71k tests pre-migration (e.g. PR #53613).

Root Cause

dotnet test {assembly}.dll relies on VSTest discovery to find tests. With xUnit v3, the VSTest adapter (xunit.v3.runner.visualstudio) is not resolvable by the test host on Helix agents — only the SDK's built-in Extensions directory is searched, and the xUnit v3 adapter is not there. The diagnostic log confirms "TestDiscoverers":[] and "ExecutedTests":0.

Fix

Switch from dotnet test to dotnet exec to run xUnit v3 tests as self-hosting executables, matching the approach used by the Arcade Helix SDK's CreateXUnitV3WorkItems task.

Changes

File Change
SDKCustomCreateXUnitWorkItemsWithTestExclusion.cs Replace dotnet test with dotnet exec --roll-forward Major. Replace VSTest options (--logger trx, --blame-hang, --filter) with xUnit v3 native options (-xml testResults.xml, -noAutoReporters, -class). Convert env var pass-through from -e to shell export/set.
AssemblyScheduler.cs Remove trailing . from class names in partition filter string. The dot was needed for VSTest FullyQualifiedName contains-matching; xUnit v3's native -class filter uses exact match.

Before (broken)

dotnet test X.dll --filter "Ns.Class1.|Ns.Class2." --logger trx --blame-hang ...

After (fixed)

dotnet exec --roll-forward Major --runtimeconfig X.runtimeconfig.json --depsfile X.deps.json X.dll -xml testResults.xml -noAutoReporters -nocolor -class Ns.Class1 -class Ns.Class2

Verification

Confirmed via Helix API console logs from PR #53680 (post-xUnit v3) that every work item produces No test is available and ExecutedTests: 0.

After the xUnit v2 to v3 migration (dotnet#52930), all Helix work items fail with
'No test is available in {assembly}.dll' because 'dotnet test' relies on VSTest
discovery which cannot find the xUnit v3 adapter on Helix agents.

Switch from 'dotnet test' to 'dotnet exec' to run xUnit v3 tests as
self-hosting executables, matching the approach used by the Arcade Helix SDK's
CreateXUnitV3WorkItems task.

Changes:
- SDKCustomCreateXUnitWorkItemsWithTestExclusion.cs: Replace 'dotnet test' with
  'dotnet exec --roll-forward Major --runtimeconfig/--depsfile'. Replace VSTest
  options (--logger trx, --blame-hang, --filter) with xUnit v3 native options
  (-xml testResults.xml, -noAutoReporters, -class). Convert env var pass-through
  from 'dotnet test -e' to shell export/set commands.
- AssemblyScheduler.cs: Remove trailing '.' from class names in partition filter
  string. The dot was needed for VSTest FullyQualifiedName contains-matching but
  xUnit v3's native -class filter uses exact match (with optional wildcards).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant