[Bug]: [After(TestDiscovery)] TestDiscoveryContext.AllTests is incorrect

[epotter2297]

Description

When you do the following:

• Group tests into different classes that have overlapping test method names
• Give those tests dependencies which have an overlapping test name
• Run specific subsets of those tests

It can occasionally trip up the Test Discovery phase and cause tests that do not execute to incorrectly appear in TestDiscoveryContext.AllTests.

Expected Behavior

1. Run tests ABCD.B2, ABCV.B1, ABCVC.B2, and VPCU.A1000
2. ABCV.B2 is not included in TestDiscoveryContext.AllTests
3. ABCV.B2 is not executed

Actual Behavior

1. Run tests ABCD.B2, ABCV.B1, ABCVC.B2, and VPCU.A1000
2. ABCV.B2 is included in TestDiscoveryContext.AllTests
3. ABCV.B2 is not executed

Steps to Reproduce

public class ABCD
{
    [Test]
    public async Task B1()
    {
        await Assert.That(true).IsEqualTo(true);
    }

    [Test]
    public async Task B2()
    {
        await Assert.That(true).IsEqualTo(true);
    }
}


public class VPCU
{
    [Test, DependsOn<ABCV>(nameof(ABCV.B1))]
    public async Task A1000()
    {
        await Assert.That(true).IsEqualTo(true);
    }
}

public class ABCV
{
    [Test]
    public async Task A1()
    {
        await Assert.That(true).IsEqualTo(true);
    }

    [Test, DependsOn(nameof(A1))]
    public async Task B1()
    {
        await Assert.That(true).IsEqualTo(true);
    }

    [Test, DependsOn(nameof(A1))]
    public async Task B2()
    {
        await Assert.That(true).IsEqualTo(true);
    }
}

public class ABCVC
{
    [Test]
    public async Task B0()
    {
        await Assert.That(true).IsEqualTo(true);
    }

    [Test, DependsOn(nameof(B0))]
    public async Task B1()
    {
        await Assert.That(true).IsEqualTo(true);
    }

    [Test, DependsOn(nameof(B0))]
    public async Task B2()
    {
        await Assert.That(true).IsEqualTo(true);
    }
}

public static class TestSetup
{
    [After(TestDiscovery)]
    public static void Setup(TestDiscoveryContext context) => ...;
}

TUnit Version

1.12.125

.NET Version

.NET 10

Operating System

Windows

IDE / Test Runner

Visual Studio

Error Output / Stack Trace

Additional Context

I played with this for a while to try and create the smallest reproduction, and this is what I came up with. I'm not entirely sure what triggers this situation to happen, but this example does reproduce it consistently.

IDE-Specific Issue?

• I've confirmed this issue occurs when running via dotnet test or dotnet run, not just in my IDE

[thomhurst]

Can you try v1.12.139 please

[epotter2297]

That does fix the example I put in here, although my main test suite is now not able to run any tests when I run them through the VS Test Explorer (TestDiscoveryContext.AllTests always has a count of 0). dotnet test still works though, so maybe that's just me?

The only differences I can think of are that there are test classes in different namespaces (with dependencies across namespaces), and I have a few unloaded projects still using xUnit v2. I noticed a few errors mentioning something about "projects not referencing any .net nuget adapter" which I don't remember running into before.

I'll keep playing around with it today and see if I can narrow it down any more.

[epotter2297]

Okay, it does seem to be something in the latest update that broke it... if I make no other changes besides reverting back to 1.12.125 then the VS Test Explorer works again

[epotter2297]

It looks like you already figured it out, but can confirm it was the constructor parameters (from DI) that broke it 👍

[thomhurst]

Will try to get a release out later for ya