perf: use reflection for attribute factories instead of source-generating attribute construction

[thomhurst]

Summary

Suggested by @teo-tsirpanis in dotnet/runtime#126541 (comment)

The source generator currently emits __Attributes(int groupIndex) methods that reconstruct attribute instances in code:

private static Attribute[] __Attributes(int groupIndex)
{
    switch (groupIndex)
    {
        case 0:
            return
            [
                new TestAttribute(),
                new CustomAttribute("Foo") { Property = "Bar" },
                // ... more attributes
            ];
    }
}

Since MethodInfo.GetCustomAttributes() is AOT-safe (Native AOT's reflection-free mode was deleted in dotnet/runtime#109857), we could replace the generated attribute factories with a reflection call:

// Instead of source-generating attribute construction, delegate to reflection
CreateAttributes = static (groupIndex) => methodInfo.GetCustomAttributes().ToArray(),

Benefits

• Reduced generated code size — eliminates per-class __Attributes switch methods entirely
• Fewer methods to JIT — one fewer generated method per test class
• Always correct — no risk of source generator failing to replicate complex attribute constructors or property setters
• Simpler generator code — removes PreGenerateAttributeFactoryBody and related generation logic

Current state

• TUnit's __Attributes generates only method-level test-relevant attributes (not assembly metadata — the bloated example in the runtime issue was from a different project)
• The reflection path (ReflectionAttributeExtractor.GetAllAttributes) already uses GetCustomAttributes() and caches results
• Attribute factories are lazy — only invoked during test execution, not discovery — so the reflection cost is deferred

Considerations

• Reflection has per-call overhead vs pre-built arrays, but attribute access is not a hot path (called once per test during setup)
• Would need MethodInfo available at factory call time — currently the source-gen path avoids storing MethodInfo
• The [DynamicallyAccessedMembers] annotations would need to cover attribute types
• Could be behind a feature switch if perf characteristics differ between JIT and AOT

Context

• Runtime issue: JIT and one-time source generated methods performance cost dotnet/runtime#126541
• Lazy registration PR: perf: defer per-class JIT via lazy test registration + parallel resolution #5395

[thomhurst]

Benchmark Results

Measured with BenchmarkDotNet on .NET 10.0.5, i7-12700K:

Method	Mean	Allocated	Notes
SourceGen_Simple (baseline)	5.9 ns	56 B	return [new TestAttr()]
Reflection_Simple_Cached	4.3 ns	0 B	ConcurrentDictionary hit
Reflection_Simple_FirstCall	6,523 ns	488 B	First GetCustomAttributes()
SourceGen_Complex (3 attrs)	12.2 ns	128 B	Attrs with ctor args + properties
Reflection_Complex_Cached	4.2 ns	0 B	ConcurrentDictionary hit
Reflection_Complex_FirstCall	21,287 ns	2,448 B
SourceGen_DataSource	8.7 ns	88 B	2 attrs with Type arg
Reflection_DataSource_Cached	4.3 ns	0 B	ConcurrentDictionary hit
Reflection_DataSource_FirstCall	15,479 ns	1,488 B
CustomAttributeData_Simple	196 ns	592 B	.NET Framework path
CustomAttributeData_Complex	1,048 ns	3,408 B	.NET Framework path

Analysis

First-call reflection is 1,000–3,500× slower than source-gen. Since attribute factories are invoked exactly once per test (then cached by TestMetadata.GetOrCreateAttributes()), the first-call cost is what matters.

For a 500-test suite across 50 classes:

• JIT saved by removing __Attributes: ~50 methods × ~5–10 μs = ~250–500 μs
• Reflection added: ~500 methods × ~10 μs = ~5,000 μs (5 ms)
• Net effect: ~10× worse startup for attribute materialization

Cached reflection does win slightly in steady-state (4.3 ns vs 5.9–12.2 ns, zero alloc vs small alloc), but that's irrelevant since both paths cache after the first call anyway.

.NET Framework blocker

The CustomAttributeData + Activator.CreateInstance path (required for .NET Framework where GetCustomAttributes() can't handle generic attributes) is 33–177× slower than source-gen and allocates 10–60× more memory.

TUnit uses generic attributes extensively (ClassDataSourceAttribute<T>, ClassConstructorAttribute<T>, MethodDataSourceAttribute<T>, DependsOnAttribute<T>, etc.) — these appear in 30+ generated source files. On .NET Framework, the existing GetCustomAttributesSafe fallback silently swallows failures via empty catch blocks. The source-gen path avoids this entirely since new ClassDataSourceAttribute<T>() is just regular C# that works on any runtime.

Conclusion

The proposal regresses runtime performance without meaningful JIT savings. The __Attributes method is already consolidated to 1 per class (not per test) and bodies are deduplicated across methods with identical attribute sets. Closing as won't-fix.

[thomhurst]

Closing based on benchmark results — reflection regresses startup by ~10× for attribute materialization with no meaningful JIT savings.