Fix compatibility test failures for legacy vstest.console and MSTest adapter by nohwnd · Pull Request #15534 · microsoft/vstest

nohwnd · 2026-03-20T10:27:33Z

Summary

Fix compatibility tests, by reducing them to console that is likely to be used with vstest.console wrapper (we are unlikely to use latest wrapper, with legacy console), and we don't encourage such configuration.

Add dotnet_rollforwards=latestmajor to use the latest tfm for the vstest.console that is used with wrapper, to avoid installing additional tfms for the console. Tfm incompatibilities are rare.

Add custom test adapter for compatibility tests, to avoid testing mstest compatibility rather than vstest.console compatibility.

Copilot

Pull request overview

This PR addresses internal compatibility test failures by enabling .NET runtime roll-forward for legacy vstest.console.dll executions and by replacing an MSTest-dependent mixed-TFM test asset with a minimal custom adapter + project to avoid adapter/runtime dependency conflicts.

Changes:

Set DOTNET_ROLL_FORWARD=LatestMajor when launching vstest.console.dll via both wrapper and direct execution paths.
Add SimpleTestAdapter and SimpleTestProject4 test assets and update the mixed-TFM translation layer test to use them.
Introduce SourceAssert/SourceNavigationParser utilities and update several integration tests to use source-based assertions instead of configuration-specific line numbers.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
test/TestAssets/TestAssets.sln	Adds new test asset projects and additional solution configurations.
test/TestAssets/SimpleTestProject4/UnitTest1.cs	Adds minimal tests used for mixed-TFM execution validation.
test/TestAssets/SimpleTestProject4/SimpleTestProject4.csproj	Adds a new test asset project referencing the minimal adapter.
test/TestAssets/SimpleTestAdapter/SimpleTestAdapter.csproj	Adds a minimal adapter project targeting netstandard2.0.
test/TestAssets/SimpleTestAdapter/SimpleTestAdapter.cs	Implements a minimal discoverer/executor to avoid MSTest dependency conflicts.
test/Microsoft.TestPlatform.TestUtilities/SourceAssert.cs	Adds utilities to map/assert source line numbers without Debug/Release branching.
test/Microsoft.TestPlatform.TestUtilities/IntegrationTestBase.cs	Sets roll-forward env var for legacy `vstest.console.dll` execution.
test/Microsoft.TestPlatform.Library.IntegrationTests/TranslationLayerTests/RunTests.cs	Updates mixed-TFM test to use `SimpleTestProject4` and bypass compat-hash pathing.
test/Microsoft.TestPlatform.Library.IntegrationTests/TranslationLayerTests/DiscoverTests.cs	Switches to source-based line assertions for navigation tests.
test/Microsoft.TestPlatform.Library.IntegrationTests/TranslationLayerTests/DifferentTestFrameworkSimpleTests.cs	Switches to source-based assertions for adapter line-number checks.
test/Microsoft.TestPlatform.Library.IntegrationTests/DiaSessionTests.cs	Switches DIA navigation line-number assertions to source-based assertions.
test/Microsoft.TestPlatform.Common.UnitTests/SourceNavigationParserTests.cs	Adds unit tests validating the new source navigation parser behavior.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

nohwnd · 2026-03-20T17:23:27Z

CI failure is a pre-existing build error on main — \TestServiceLocator\ not found in \TestRequestSender.cs:93. This is in \Microsoft.TestPlatform.CommunicationUtilities, a file not modified in this PR. The same error occurs when building the integration tests from the latest \main\ commit locally.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Copilot · 2026-03-20T19:09:56Z

        if (IsNetCoreRunner())
        {
            args = SetVSTestConsoleDLLPathInArgs(args);
+
+            // Old vstest.console.dll versions (e.g. 15.9.2 targeting netcoreapp2.0) need to roll forward
+            // to the current .NET runtime.
+            environmentVariables ??= new();
+            environmentVariables["DOTNET_ROLL_FORWARD"] = "LatestMajor";
        }


In the direct execution path, DOTNET_ROLL_FORWARD is now always set for .NET Core runner usage. If your process-launch code replaces (rather than merges) the environment when any variables are provided (as noted elsewhere in this file), this can cause reliability issues by dropping required inherited variables (e.g., TEMP, PATH, proxy config). Recommendation (mandatory): merge inherited environment variables when adding DOTNET_ROLL_FORWARD, or only override when you can guarantee merge semantics.

…adapter Two fixes for internal pipeline compatibility test failures: 1. Set DOTNET_ROLL_FORWARD=LatestMajor when launching vstest.console.dll Old vstest.console versions (e.g. 15.9.2 targeting netcoreapp2.0) cannot run on .NET 10+ without roll-forward. This is set in both the wrapper path (GetVsTestConsoleWrapper) and direct execution path (ExecuteVsTestConsole). 2. Replace MSTestProject1 with SimpleTestProject4 in mixed-TFM compat test MSTest 4.1.0 adapter fails to load System.Runtime.CompilerServices.Unsafe 4.0.4.1 when running in older vstest.console 17.12.0 testhost. Created a minimal test adapter (SimpleTestAdapter) with no external dependencies beyond Microsoft.TestPlatform.ObjectModel, and a test project using it. The RunAllTestsWithMixedTFMsWillRunTestsFromAllProvidedDllEvenWhenTheyMixTFMs test now uses this adapter-only project, avoiding MSTest version conflicts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

# Conflicts: # test/TestAssets/TestAssets.sln

Updated [Microsoft.NET.Test.Sdk](https://github.com/microsoft/vstest) from 18.5.1 to 18.6.0. <details> <summary>Release notes</summary> _Sourced from [Microsoft.NET.Test.Sdk's releases](https://github.com/microsoft/vstest/releases)._ ## 18.6.0 ## What's Changed * Revert removal of Video Recorder by @nohwnd in microsoft/vstest#15336 * Speed up blame by filtering non-.NET processes from dump collection by @nohwnd in microsoft/vstest#15518 * Add README.md to NuGet packages by @nohwnd in microsoft/vstest#15550 * Report child process info on connection timeout by @nohwnd in microsoft/vstest#15603 ### Changes to tests and infra * Brand as 18.6 by @nohwnd in microsoft/vstest#15423 * Upgrading code coverage version to 18.5.1, by @fhnaseer in microsoft/vstest#15422 * Updating System.Collections.Immutable to 9.0.11 by @MSLukeWest in microsoft/vstest#15425 * Fix attachVS when used for debugging integration tests by @nohwnd in microsoft/vstest#15451 * Replace dotnet.config, with global.json by @nohwnd in microsoft/vstest#15449 * Document debugging integration tests with AttachVS by @Copilot in microsoft/vstest#15452 * Fix stack overflow tests by @nohwnd in microsoft/vstest#15461 * Make TestAssets.sln buildable locally by @Youssef1313 in microsoft/vstest#15466 * Try filtering out tests by @nohwnd in microsoft/vstest#15463 * Build just once when tfms run in parallel by @nohwnd in microsoft/vstest#15465 * Review simplify compatibility sources, deduplicate tests by @nohwnd in microsoft/vstest#15472 * Cleanup dead TRX code by @Youssef1313 in microsoft/vstest#15474 * Update .NET runtimes to 8.0.25, 9.0.14, and 10.0.4 by @nohwnd in microsoft/vstest#15481 * Compat matrix checker by @nohwnd in microsoft/vstest#15480 * Add trx analysis skill by @nohwnd in microsoft/vstest#15486 * Split integration tests to single tfm and multi tfm project by @nohwnd in microsoft/vstest#15484 * Update matrix by @nohwnd in microsoft/vstest#15477 * Break infinite restore loop in VS by @nohwnd in microsoft/vstest#15503 * Use global package cache for build, and local for running integration tests by @nohwnd in microsoft/vstest#15500 * Update contributing by @nohwnd in microsoft/vstest#15505 * Reduce test wall-clock time by increasing minThreads by @drognanar in microsoft/vstest#15502 * Indicator flakiness by @nohwnd in microsoft/vstest#15513 * Fix ci build by @nohwnd in microsoft/vstest#15515 * Fix thread safety issues by @Evangelink in microsoft/vstest#15512 * Optimize DotnetSDKSimulation_PostProcessing test (163s → 61s) by @nohwnd in microsoft/vstest#15516 * Build isolated test assets for single TFM instead of 7 by @nohwnd in microsoft/vstest#15517 * Remove unused dependencies from Library.IntegrationTests by @nohwnd in microsoft/vstest#15527 * Remove printing _attachments content to console by @nohwnd in microsoft/vstest#15520 * Add Linux/macOS test filtering guide to CONTRIBUTING.md by @nohwnd in microsoft/vstest#15521 * Change integration test parallelization from ClassLevel to MethodLevel by @nohwnd in microsoft/vstest#15526 * Unify target framework checks with IsNetFrameworkTarget/IsNetTarget by @nohwnd in microsoft/vstest#15523 * Add unattended work instructions to copilot-instructions.md by @nohwnd in microsoft/vstest#15531 * Reduce code style rule severity from warning to suggestion by @nohwnd in microsoft/vstest#15522 * Remove Debug/Release line number branching from tests by @nohwnd in microsoft/vstest#15519 * Revise unattended work instructions in copilot-instructions.md by @nohwnd in microsoft/vstest#15532 * Improve CompatibilityRowsBuilder error message with diagnostic details by @nohwnd in microsoft/vstest#15529 * docs: add git worktree and upstream sync workflow to copilot-instructions.md by @nohwnd in microsoft/vstest#15538 * Add VSIX runner to smoke tests by @nohwnd in microsoft/vstest#15541 * Remove deprecated WebTest and TMI test methods by @nohwnd in microsoft/vstest#15525 * Fix compatibility test failures for legacy vstest.console and MSTest adapter by @nohwnd in microsoft/vstest#15534 * Convert TestPlatform.sln to slnx format by @nohwnd in microsoft/vstest#15551 * Convert test/TestAssets .sln files to .slnx format by @nohwnd in microsoft/vstest#15557 ... (truncated) Commits viewable in [compare view](microsoft/vstest@v18.5.1...v18.6.0). </details> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=Microsoft.NET.Test.Sdk&package-manager=nuget&previous-version=18.5.1&new-version=18.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 20, 2026 10:27

Copilot AI reviewed Mar 20, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 20, 2026 16:45

nohwnd force-pushed the fix-compat-test-failures branch from 9336b39 to 7222269 Compare March 20, 2026 16:45

Copilot AI reviewed Mar 20, 2026

View reviewed changes

nohwnd force-pushed the fix-compat-test-failures branch from 7222269 to 02547dc Compare March 20, 2026 18:34

Copilot AI review requested due to automatic review settings March 20, 2026 19:02

nohwnd force-pushed the fix-compat-test-failures branch from 02547dc to 9c92f7c Compare March 20, 2026 19:02

Copilot AI reviewed Mar 20, 2026

View reviewed changes

nohwnd force-pushed the fix-compat-test-failures branch from 9c92f7c to e71c1d7 Compare March 20, 2026 19:55

nohwnd commented Mar 21, 2026

View reviewed changes

Comment thread test/Microsoft.TestPlatform.Library.IntegrationTests/TranslationLayerTests/RunTests.cs Outdated

nohwnd added 2 commits March 21, 2026 14:54

fixes

313e387

Fix tests

9de50d5

Copilot AI review requested due to automatic review settings March 23, 2026 14:06

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Merge branch 'main' into fix-compat-test-failures

fa8094d

# Conflicts: # test/TestAssets/TestAssets.sln

nohwnd added the 🚢 Ship it! Add to PRs where owner approves automated PR, but cannot approve because they "wrote it". label Mar 23, 2026

nohwnd enabled auto-merge (squash) March 23, 2026 16:00

Evangelink approved these changes Mar 23, 2026

View reviewed changes

nohwnd merged commit c0edcb8 into microsoft:main Mar 23, 2026
6 checks passed

This was referenced Mar 25, 2026

[main] Source code updates from microsoft/vstest dotnet/dotnet#5511

Merged

[release/11.0.1xx-preview3] Source code updates from microsoft/vstest dotnet/dotnet#5743

Closed

dotnet-maestro Bot mentioned this pull request Apr 27, 2026

[release/10.0.4xx] Source code updates from microsoft/vstest dotnet/dotnet#6290

Merged

This was referenced May 28, 2026

Bump the test-packages group with 2 updates Ozymandros/Godot-Generator#36

Open

Bump Microsoft.NET.Test.Sdk from 18.5.1 to 18.6.0 IntelliTect/EssentialCSharp.ListingManager#257

Merged

Conversation

nohwnd commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nohwnd commented Mar 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nohwnd commented Mar 20, 2026 •

edited

Loading