Improve MTP dotnet test error reporting by Youssef1313 · Pull Request #52911 · dotnet/sdk

Youssef1313 · 2026-02-09T15:57:06Z

Fixes #51802

For all screenshots, left is "dogfood" (this PR - the new behavior)

Comparison when given a project that is not a test project

The case of passing VSTest project

The case of having a build error

Same experience, except that we capture the real exit code of MSBuild in case it's something other than "1".

Youssef1313 · 2026-02-09T16:01:01Z

            // be slowing us down unnecessarily.
            // Alternatively, if we can enqueue right after every project evaluation without waiting all evaluations to be done, we can enqueue early.
            actionQueue = new TestApplicationActionQueue(degreeOfParallelism, buildOptions, testOptions, _output, OnHelpRequested);
-            if (!msBuildHandler.EnqueueTestApplications(actionQueue))


This used to return false when we have any VSTest projects. After the changes in the PR, we will report the error having VSTest projects early in the RunMSBuild call above. So EnqueueTestApplications no longer need to return anything as it never fails.

Copilot

Pull request overview

This PR improves dotnet test (Microsoft Testing Platform / MTP) failure reporting by surfacing more specific MSBuild exit codes and adding a dedicated error when no test projects are discovered, along with corresponding localized strings.

Changes:

Propagate MSBuild/restore exit codes through MTP project discovery to improve error output accuracy.
Fail with a specific “No test projects were found.” error when project discovery yields zero test projects.
Add the new string to CliCommandStrings.resx and update all CliCommandStrings.*.xlf localization files.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs	Adjusts enqueue flow to rely on `RunMSBuild()` for failure conditions (no longer checks enqueue return value).
src/Cli/dotnet/Commands/Test/MTP/MSBuildUtility.cs	Changes build/restore result from `bool` to `int` exit code and propagates it to callers.
src/Cli/dotnet/Commands/Test/MTP/MSBuildHandler.cs	Adds “no test projects found” error path; updates error reporting to include actual MSBuild exit code; changes enqueue method signature.
src/Cli/dotnet/Commands/CliCommandStrings.resx	Adds `CmdTestNoTestProjectsFound` resource (and includes unrelated header whitespace reformat).
src/Cli/dotnet/Commands/xlf/CliCommandStrings.cs.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.de.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.es.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.fr.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.it.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ja.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ko.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.pl.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.pt-BR.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ru.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.tr.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.zh-Hans.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.zh-Hant.xlf	Adds localized entry for `CmdTestNoTestProjectsFound`.

Youssef1313 · 2026-02-09T16:12:41Z

    private readonly BuildOptions _buildOptions = buildOptions;

    private readonly ConcurrentBag<ParallelizableTestModuleGroupWithSequentialInnerModules> _testApplications = [];
-    private bool _areTestingPlatformApplications = true;


This used to track whether we have any VSTest test projects. Before the PR, this would be set to false during InitializeTestApplications which is called by RunMSBuild, and then checked in EnqueueTestApplications. Now all is aligned to happen in the same place.

Youssef1313 · 2026-02-09T16:13:05Z

+        (IEnumerable<ParallelizableTestModuleGroupWithSequentialInnerModules> projects, int buildExitCode) = isSolution ?
+            MSBuildUtility.GetProjectsFromSolution(projectOrSolutionFilePath, _buildOptions) :
+            MSBuildUtility.GetProjectsFromProject(projectOrSolutionFilePath, _buildOptions);


GetProjectsProperties was mostly this single line. So I inlined it.

Youssef1313 · 2026-02-09T16:13:12Z

+        (IEnumerable<ParallelizableTestModuleGroupWithSequentialInnerModules> projects, int buildExitCode) = isSolution ?
+            MSBuildUtility.GetProjectsFromSolution(projectOrSolutionFilePath, _buildOptions) :
+            MSBuildUtility.GetProjectsFromProject(projectOrSolutionFilePath, _buildOptions);


GetProjectsProperties was mostly this single line. So I inlined it.

Youssef1313 · 2026-02-09T16:15:30Z

+        if (buildExitCode != 0)
        {
-            Reporter.Error.WriteLine(string.Format(CliCommandStrings.CmdMSBuildProjectsPropertiesErrorDescription, ExitCode.GenericFailure));
+            Reporter.Error.WriteLine(string.Format(CliCommandStrings.CmdMSBuildProjectsPropertiesErrorDescription, buildExitCode));


We previously reported GenericFailure regardless of the actual build exit code because we only had a bool indicating success/failure. This is fixed now so that we flow the build exit code.

Youssef1313 · 2026-02-09T16:17:36Z

-        if (!_areTestingPlatformApplications)
+        if (_testApplications.IsEmpty)
        {
+            Reporter.Error.WriteLine(string.Format(CliCommandStrings.CmdTestNoTestProjectsFound));


Now we have a separate error for when there are no test projects.

Youssef1313 · 2026-02-10T12:32:36Z

Build analysis seems to be stuck. Let me close and re-open for a fresh build

Evangelink · 2026-02-10T12:38:52Z

Comparison when given a project that is not a test project

Shall we add some link to some doc to explain what is the condition for a project to be considered as test project?

The case of having a build error

We still see the GetProjects properties with MSBuild... message that seems quite technical, could we use different wording? cc @baronfel is this message something that we would expect to see?

Youssef1313 · 2026-02-10T12:53:51Z

Shall we add some link to some doc to explain what is the condition for a project to be considered as test project?

Let's consider in a follow-up. Some of the details of our detection are too technical for normal users. We definitely want to balance the level of technical details with what end users are concerned about, and I expect we will be doing major updates to our documentation. So we can revisit after.

We still see the GetProjects properties with MSBuild... message that seems quite technical, could we use different wording? cc @baronfel is this message something that we would expect to see?

I agree and I never liked this message. I'm going to simplify it to Build failed with exit code: {0}.

Copilot AI review requested due to automatic review settings February 9, 2026 15:57

Copilot started reviewing on behalf of Youssef1313 February 9, 2026 15:58 View session

Youssef1313 commented Feb 9, 2026

View reviewed changes

Copilot AI reviewed Feb 9, 2026

View reviewed changes

Comment thread src/Cli/dotnet/Commands/Test/MTP/MSBuildHandler.cs Outdated

Comment thread src/Cli/dotnet/Commands/Test/MTP/MSBuildHandler.cs

Comment thread src/Cli/dotnet/Commands/CliCommandStrings.resx

Youssef1313 commented Feb 9, 2026

View reviewed changes

Improve MTP dotnet test error reporting

1e14d90

Youssef1313 force-pushed the dev/ygerges/accurate-error branch from 4f0bdcb to 1e14d90 Compare February 9, 2026 16:18

This was referenced Feb 9, 2026

Unable to checkout dotnet/dnceng#4115

Open

Checkout failure: "Git fetch failed with exit code 128" dotnet/arcade#9009

Open

Evangelink approved these changes Feb 9, 2026

View reviewed changes

Youssef1313 closed this Feb 10, 2026

Youssef1313 reopened this Feb 10, 2026

Youssef1313 enabled auto-merge (squash) February 10, 2026 12:38

Youssef1313 added 2 commits February 10, 2026 13:54

Simplify message

0562d1a

Update comment

3d76a61

Youssef1313 merged commit 31812eb into release/10.0.3xx Feb 10, 2026
24 of 26 checks passed

Youssef1313 deleted the dev/ygerges/accurate-error branch February 10, 2026 14:12

dotnet-maestro Bot mentioned this pull request Feb 12, 2026

[release/10.0.3xx] Source code updates from dotnet/sdk dotnet/dotnet#4826

Merged

dotnet-maestro Bot mentioned this pull request Mar 6, 2026

[main] Source code updates from dotnet/sdk dotnet/dotnet#5268

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve MTP dotnet test error reporting#52911

Improve MTP dotnet test error reporting#52911
Youssef1313 merged 3 commits intorelease/10.0.3xxfrom
dev/ygerges/accurate-error

Youssef1313 commented Feb 9, 2026 •

edited

Loading

Uh oh!

Youssef1313 Feb 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Youssef1313 Feb 9, 2026

Uh oh!

Youssef1313 Feb 9, 2026

Uh oh!

Youssef1313 Feb 9, 2026

Uh oh!

Youssef1313 Feb 9, 2026

Uh oh!

Youssef1313 Feb 9, 2026

Uh oh!

Youssef1313 commented Feb 10, 2026

Uh oh!

Evangelink commented Feb 10, 2026

Uh oh!

Youssef1313 commented Feb 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Youssef1313 commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comparison when given a project that is not a test project

The case of passing VSTest project

The case of having a build error

Uh oh!

Youssef1313 Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Youssef1313 Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Youssef1313 Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Youssef1313 Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Youssef1313 Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Youssef1313 Feb 9, 2026

Choose a reason for hiding this comment

Uh oh!

Youssef1313 commented Feb 10, 2026

Uh oh!

Evangelink commented Feb 10, 2026

Uh oh!

Youssef1313 commented Feb 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Youssef1313 commented Feb 9, 2026 •

edited

Loading