Skip to content

Simplify retry extension logic to use built-in filters#7511

Merged
Evangelink merged 8 commits intomainfrom
dev/amauryleve/simpler-retry
Apr 24, 2026
Merged

Simplify retry extension logic to use built-in filters#7511
Evangelink merged 8 commits intomainfrom
dev/amauryleve/simpler-retry

Conversation

@Evangelink
Copy link
Copy Markdown
Member

Fixes #7500

Copilot AI review requested due to automatic review settings March 9, 2026 09:58
Comment thread src/Platform/Microsoft.Testing.Extensions.Retry/RetryExtensions.cs
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR simplifies the Retry extension by removing the custom test execution filter factory and instead relying on the platform’s built-in --filter-uid functionality when re-running failed tests.

Changes:

  • Update RetryOrchestrator to remove any existing test filter options and apply --filter-uid with the failed test UIDs on retry.
  • Remove the Retry extension’s custom ITestExecutionFilterFactory integration (and the default TestHostManager requirement).
  • Add/adjust acceptance coverage to validate retry behavior when an initial filter is present.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/RetryFailedTestsTests.cs Adds new acceptance scenarios for retry behavior with pre-existing filters and updates the embedded test asset to respect UID filtering.
src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs Replaces the previous in-process filter approach by rewriting command line filters on retry and introduces helper logic to remove existing options.
src/Platform/Microsoft.Testing.Extensions.Retry/RetryExtensions.cs Removes registration of the custom retry execution filter factory and the dependency on the default TestHostManager.
src/Platform/Microsoft.Testing.Extensions.Retry/RetryExecutionFilterFactory.cs Deletes the custom filter factory implementation (no longer needed with built-in filtering).

Comment thread src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs Outdated
Comment thread src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs Outdated
Copilot AI review requested due to automatic review settings March 9, 2026 11:56
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

Comment thread src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs Outdated
Comment thread src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs Outdated
@Evangelink Evangelink enabled auto-merge March 9, 2026 17:24
…r-retry

# Conflicts:
#	test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/RetryFailedTestsTests.cs
… netstandard2.0

- Write one UID per line in RSP file with quotes to handle whitespace/comments
- Fix StartsWith(char) in RemoveOption to use [0] != '-' for netstandard2.0 compat
Copilot AI review requested due to automatic review settings April 20, 2026 09:55
Comment thread src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs Outdated
Comment thread src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Comments suppressed due to low confidence (1)

src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs:220

  • The response file created for long UID lists is never cleaned up. Since it contains test UIDs (potentially sensitive) and is written under the results directory, consider deleting it after the retry attempt (or after the process has started), and/or placing it in a more appropriate temp location with restricted permissions to avoid leaving artifacts behind.
                    // Use a response file to avoid exceeding command-line length limits.
                    // Write to retryRootFolder (not the per-attempt folder) so it won't be included
                    // in the final results move.
                    string responseFilePath = Path.Combine(retryRootFolder, $"retry-filter-uids-{attemptCount}.rsp");
                    using (IFileStream stream = _fileSystem.NewFileStream(responseFilePath, FileMode.Create, FileAccess.Write))
                    using (var writer = new StreamWriter(stream.Stream))
                    {
                        // Write one UID per line. The RSP parser (ResponseFileHelper.SplitCommandLine) splits
                        // each line by whitespace and uses '"' for grouping. Wrapping each UID in quotes
                        // handles UIDs containing whitespace or starting with '#' (comment marker).
                        // Note: UIDs containing literal '"' characters cannot be safely round-tripped
                        // through the RSP parser because it strips all quote characters from tokens.
                        await writer.WriteLineAsync($"--{PlatformCommandLineProvider.FilterUidOptionKey}").ConfigureAwait(false);
                        foreach (string uid in lastListOfFailedId)
                        {
                            await writer.WriteLineAsync($"\"{uid}\"").ConfigureAwait(false);
                        }
                    }

                    finalArguments.Add($"@{responseFilePath}");
                }
  • Files reviewed: 18/18 changed files
  • Comments generated: 2

Comment thread src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs Outdated
…imation

- UIDs with literal '"' characters now skip RSP and use inline args,
  since ResponseFileHelper.SplitCommandLine strips all '"' from tokens.
- Adjusted command-line length estimation to account for PasteArguments
  quoting overhead on pre-.NET 8 targets (+3 per argument).
- Simplified RSP format: write all UIDs on a single line instead of one
  per line, which is clearer and equivalent for the RSP parser.
Copy link
Copy Markdown

@YuliiaKovalova YuliiaKovalova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me.

@Evangelink Evangelink merged commit a522185 into main Apr 24, 2026
10 checks passed
@Evangelink Evangelink deleted the dev/amauryleve/simpler-retry branch April 24, 2026 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Simplify retry extension

4 participants