TST-04 (CoverletCollectorRule): Incomplete validation — passes when coverlet is pinned but not actually referenced in test project

[davidnmbond]

Problem

TST-04 (CoverletCollectorRule) currently gives a false pass when coverlet.collector appears in Directory.Packages.props but is not referenced as a in any test project .csproj.

With central package management (ManagePackageVersionsCentrally=true), having a entry in Directory.Packages.props only pins the version — it does not cause the package to be included in any project. The test project must also contain:

<PackageReference Include="coverlet.collector">
  <PrivateAssets>all</PrivateAssets>
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>

Real-world example

This exact bug was found in Lifx.Api: coverlet.collector 8.0.1 was pinned in Directory.Packages.props for months, TST-04 would have passed, but coverage was never collected because the test project didn't reference it.

Current behaviour

// CoverletCollectorRule.EvaluateAsync — passes immediately if found in Directory.Packages.props
var dirPackages = context.GetFileContent("Directory.Packages.props");
if (Contains(dirPackages, "coverlet.collector"))
{
    return Task.FromResult(Pass("coverlet.collector is referenced."));
}

Finding coverlet.collector anywhere in Directory.Packages.props short-circuits to pass without checking the test project.

Recommended fix

1. Rule logic change

When central package management is in use, the rule should verify both:

• Directory.Packages.props contains <PackageVersion Include="coverlet.collector" ...>
• At least one .Test.csproj also contains <PackageReference Include="coverlet.collector" ...>

Suggested logic:

var usesCpm = Contains(dirPackages, "ManagePackageVersionsCentrally")
    && Contains(dirPackages, "true");

var pinnedInProps = Contains(dirPackages, "coverlet.collector");

var testProjects = context.FindFiles(".csproj")
    .Where(f => f.Contains(".Test", StringComparison.OrdinalIgnoreCase));

var referencedInTestProject = testProjects
    .Any(tp => Contains(context.GetFileContent(tp), "coverlet.collector"));

if (usesCpm)
{
    // Both pin AND reference required
    if (pinnedInProps && referencedInTestProject)
        return Pass("coverlet.collector is pinned and referenced in test project.");
    if (pinnedInProps && !referencedInTestProject)
        return Fail("coverlet.collector is pinned in Directory.Packages.props but not referenced in any test project .csproj.");
    // ... other cases
}

2. Missing unit tests

Add tests for these scenarios:

Scenario	Expected
In Directory.Packages.props only (CPM enabled)	Fail
In both Directory.Packages.props and .Test.csproj (CPM enabled)	Pass
In .Test.csproj only (no CPM)	Pass
Not present anywhere	Fail (existing test)

3. Consider a new companion rule (TST-05?)

Optionally, validate that the repo has a coverlet.runsettings file or that CI (.github/workflows/*.yml) invokes --collect:"XPlat Code Coverage". This would catch repos that reference coverlet but never actually collect coverage.

Impact

Without this fix, TST-04 provides false confidence across all repos using central package management — which is likely the majority.