perf: Bypass RetryHelper and TimeoutHelper when features are not used

[thomhurst]

Summary

Every test execution goes through RetryHelper → TimeoutHelper wrappers even when these features are not configured (RetryLimit=0, Timeout=null). This adds ~10% combined overhead per test for simple tests.

Profiling Evidence

From dotnet-trace on ScaleTests (150 simple sync tests):

RetryHelper.ExecuteWithRetry                         5.58% inclusive
TimeoutHelper+<ExecuteWithTimeoutAsync>d__1.MoveNext 5.16% inclusive

This is for tests with:

• No retry configured (RetryLimit = 0)
• No timeout configured (Timeout = null)

Root Cause

In TUnit.Engine/Services/TestExecution/TestCoordinator.cs lines 108-177, every test unconditionally wraps execution in RetryHelper → TimeoutHelper:

await RetryHelper.ExecuteWithRetry(test.Context, async () =>
{
    var testTimeout = test.Context.Metadata.TestDetails.Timeout;
    // ...
    await TimeoutHelper.ExecuteWithTimeoutAsync(
        async ct => { /* test execution */ },
        testTimeout,
        cancellationToken,
        timeoutMessage).ConfigureAwait(false);
}).ConfigureAwait(false);

Even when:

• RetryLimit is 0 (no retry needed)
• Timeout is null (no timeout needed)

The wrappers still add overhead:

• RetryHelper: Loop + try/catch + async state machine
• TimeoutHelper: TaskCompletionSource + Task.WhenAny (see perf: TimeoutHelper creates unnecessary allocations when no timeout is configured #4287)

Suggested Fix

Add a fast path in TestCoordinator to bypass wrappers when not needed:

var retryLimit = test.Context.Metadata.TestDetails.RetryLimit;
var testTimeout = test.Context.Metadata.TestDetails.Timeout;

if (retryLimit == 0 && !testTimeout.HasValue)
{
    // Fast path: direct execution without wrappers
    await ExecuteTestLifecycleAsync(test, cancellationToken).ConfigureAwait(false);
}
else
{
    // Slow path: use retry and timeout wrappers
    await RetryHelper.ExecuteWithRetry(test.Context, async () =>
    {
        await TimeoutHelper.ExecuteWithTimeoutAsync(...);
    }).ConfigureAwait(false);
}

Impact

• Reduces per-test overhead by ~10% for tests without retry/timeout (majority of tests)
• Eliminates allocations from async state machines and wrapper objects
• Will significantly improve TUnit's performance in:
  • DataDrivenTests benchmark (where TUnit is 8% slower than MSTest)
  • ScaleTests benchmark (where TUnit is 4% slower than xUnit)
  • Any test suite with many simple, fast tests

Related Issues

• perf: TimeoutHelper creates unnecessary allocations when no timeout is configured #4287 (TimeoutHelper allocations)

[claude]

Analysis

This performance optimization is well-researched with concrete profiling data. Key points:

Impact: ~10% overhead reduction for tests without retry/timeout (the majority case)

Relationship to #4287:

• perf: TimeoutHelper creates unnecessary allocations when no timeout is configured #4287 optimizes TimeoutHelper internals
• This issue (perf: Bypass RetryHelper and TimeoutHelper when features are not used #4288) goes further by bypassing both wrappers entirely when not needed
• Implementing this would provide cumulative benefits beyond perf: TimeoutHelper creates unnecessary allocations when no timeout is configured #4287

Implementation: The suggested fast path is straightforward and low-risk:

if (retryLimit == 0 && !testTimeout.HasValue)
{
    // Direct execution - no wrapper overhead
    await ExecuteTestLifecycleAsync(test, cancellationToken);
}
else
{
    // Use wrappers only when needed
    await RetryHelper.ExecuteWithRetry(...);
}

Priority Justification:

• Affects every test execution
• Directly impacts benchmark competitiveness vs MSTest/xUnit
• Simple, well-defined fix with clear success criteria

Recommend prioritizing this optimization given the measurable performance gains.