StackOverflowException with Unbound Generic Types and ArgumentNullException in Reflection Context

[RBDavidson-ABS]

Description

Summary

.NET 10.0.103 preview causes StackOverflowException when:

1. Using reflection on unbound generic type definitions (e.g., typeof(List<>))
2. Throwing ArgumentNullException in methods that process reflection types while running under the Visual Studio debugger

Reproduction Steps

Reproduction

Consistently Failing Test Case

[Fact]
public void GetMemberNameForType_WithGenericTypeDefinition_HandlesUnboundGenerics()
{
    // Arrange
    var type = typeof(List<>);

    // Act
    var result = XmlCommentsNodeNameHelper.GetMemberNameForType(type);

    // Assert
    Assert.StartsWith("T:", result);
    Assert.Contains("System.Collections.Generic.List", result);
}

Result: StackOverflowException with exit code -1

Method Being Tested

public static string GetMemberNameForType(Type type)
{
    if (type == null)
        throw new ArgumentNullException(nameof(type));

    return $"T:{GetFullTypeName(type)}";
}

private static string GetFullTypeName(Type type)
{
    if (type == null)
        return string.Empty;

    var sb = new StringBuilder();

    if (!string.IsNullOrEmpty(type.Namespace))
    {
        sb.Append(type.Namespace);
        sb.Append('.');
    }

    AppendTypeName(sb, type);

    return sb.ToString();
}

private static void AppendTypeName(StringBuilder sb, Type type)
{
    if (type.DeclaringType != null)
    {
        AppendTypeName(sb, type.DeclaringType);
        sb.Append('.');
    }

    if (type.IsGenericType)
    {
        var name = type.Name;
        var backtickIndex = name.IndexOf('`');
        sb.Append(backtickIndex > 0 ? name[..backtickIndex] : name);

        sb.Append('{');
        var genericArgs = type.GetGenericArguments();
        for (var i = 0; i < genericArgs.Length; i++)
        {
            if (i > 0)
                sb.Append(',');
            sb.Append(GetFullTypeName(genericArgs[i])); // Recursion point
        }
        sb.Append('}');
    }
    else
    {
        sb.Append(type.Name);
    }
}

ArgumentNullException Debugger Freeze

[Fact]
public void GetMemberNameForFieldOrProperty_WithNullMember_ThrowsArgumentNullException()
{
    // Act & Assert
    var exception = Assert.Throws<ArgumentNullException>(() =>
        XmlCommentsNodeNameHelper.GetMemberNameForFieldOrProperty(null!));

    Assert.Equal("member", exception.ParamName);
}

// Method under test:
public static string GetMemberNameForFieldOrProperty(MemberInfo member)
{
    if (member == null)
        throw new ArgumentNullException(nameof(member)); // <-- Freezes Visual Studio debugger
    
    // ... rest of method
}

Result:

• When debugging test and single stepping through code:
  • Visual Studio debugger freezes completely (becomes unresponsive)
  • After several minutes, Visual Studio error dialog appears
  • Test output has enormous highly repetitive nested exceptions
• When running in Debug mode by itself
  • Test method runs for perhaps a minute, then fails.
  • Test output shows "Stack Overflow Exception" and no other text
• When running in Debug mode with all tests in test class
  • All but 4 tests in the class run to completion.
  • The same 4 tests, including the one being reported, run for about a minute, then fail.
  • All failing tests show "Stack Overflow Exception" in their output window and no other text.
• When running all tests in test class in Release mode.
  • All tests fail after a few seconds.
  • All tests show "Stack Overflow Exception" in their output window.
• When the test being reported is skipped during test runs, all tests pass and complete in less than a second.

Expected behavior

Expected Behavior

1. typeof(List<>) should successfully return type information without StackOverflowException
2. The helper method should return: "T:System.Collections.Generic.List1"` or similar format
3. Throwing ArgumentNullException should not freeze the debugger
4. No state accumulation should occur between test executions
5. Debug and Release modes should behave consistently

Actual behavior

Observed Behavior Patterns

Pattern 1: Test Execution Mode Differences

Execution Mode	Behavior
Release mode (all tests)	ALL tests in the class fail with StackOverflowException
Debug mode (all tests)	Exactly 4 tests fail consistently: 1. GetMemberNameForType_WithGenericTypeDefinition_HandlesUnboundGenerics 2. GetMemberNameForFieldOrProperty_WithNullMember_ThrowsArgumentNullException 3. GetMemberNameForType_WithNullType_ThrowsArgumentNullException 4. [One additional test that runs after List<> test]
Debug mode (individual test)	3 of the 4 "failing" tests pass when run individually ONLY List<> test consistently fails
Debug mode (List<> test runs first)	Subsequent tests in execution order inherit corrupted state and fail

Pattern 2: State Accumulation

Evidence of static state corruption:

• Tests that execute AFTER the typeof(List<>) test fail even though they test unrelated functionality
• Running tests in different order changes which tests fail
• Each test class instance should be isolated (xUnit creates new instance per test), but static runtime state persists

Pattern 3: Debugger-Specific Behavior

When throw new ArgumentNullException(nameof(member)) executes:

1. Code successfully reaches the throw statement (verified via debugger)
2. Visual Studio debugger completely freezes (no UI response)
3. After 2-3 minutes, VS shows error dialog
4. Test process crashes with StackOverflowException

This suggests the issue is in:

• Exception object creation/formatting
• Debugger symbol resolution for exception display
• Type resolution during exception ToString() evaluation

Regression?

Regression testing not performed.

Known Workarounds

Workaround

The issue is avoided by skipping the test:

[Fact(Skip = "This test is skipped because it was causing stack overflow exception. This is likely related to a .NET 10 Preview bug.")]
public void GetMemberNameForType_WithGenericTypeDefinition_HandlesUnboundGenerics()
{
    var type = typeof(List<>);
    var result = XmlCommentsNodeNameHelper.GetMemberNameForType(type);
    Assert.StartsWith("T:", result);
    Assert.Contains("System.Collections.Generic.List", result);
}

After skipping this single test, ALL other tests pass successfully.

Configuration

Environment

• .NET SDK Version: 10.0.103 (confirmed via dotnet --version)
• Target Framework: .NET 10
• C# Language Version: 14.0
• OS: Windows Windows 11 Pro
  • Version: 25H2
  • Installed on: ‎10/‎21/‎2025
  • OS build: 26200.8037
  • Experience: Windows Feature Experience Pack 1000.26100.300.0
• Visual Studio: 18.3.2
  • Instance: 6134d8d1
• Test Framework: xUnit v3.2.2 (xunit.v3.core, Version=3.2.2.0)
• Project Type: ASP.NET Core API with Swagger/OpenAPI integration

Other information

Stack Trace

The stack trace shows only xUnit and runtime infrastructure - no user code. Exception:

Exit code is -1 (Output is too long. Showing the last 100 lines:

at Xunit.v3.TestCaseRunner`3+<RunTestCase>d__3[[System.__Canon, ...]]
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Xunit.v3.TestCaseRunner`3+<RunTestCase>d__3[[System.__Canon, ...]]
at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1[[Xunit.v3.RunSummary, ...]]
...
[Repeating pattern of System.__Canon generic type resolution]
...
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
at System.Threading.Thread.StartCallback()

Key observations:

• Massive number of System.__Canon references (runtime's canonical type for generics)
• No actual user code in stack trace (exception never propagated to test framework)
• Pattern suggests infinite recursion in generic type resolution
• Stack trace truncated due to length ("Output is too long")

Analysis

Root Cause Hypothesis

The issue appears to be in .NET 10 preview's handling of unbound generic type definitions during reflection operations.

When typeof(List<>) is processed:

1. type.IsGenericType returns true
2. type.GetGenericArguments() returns array of unbound type parameters (e.g., [T])
3. Recursive call to GetFullTypeName(T) where T is an unbound type parameter
4. .NET 10 bug: The unbound type parameter's properties cause infinite recursion:
   • Possibly T.IsGenericType incorrectly returns true
   • Or T.DeclaringType points back to List<> creating circular reference
   • Or type resolution for unbound parameters enters infinite loop in .NET 10

Static State Corruption

The runtime appears to cache type resolution results in static state. When typeof(List<>) corrupts this cache:

• Release mode: JIT optimizations trigger cache initialization early → all tests fail
• Debug mode: Cache initialized lazily → only tests after List<> fail
• Individual runs: Static state reset between runs → corruption isolated

ArgumentNullException Issue

The debugger freeze when throwing ArgumentNullException in reflection contexts suggests:

1. Debugger intercepts exception for display formatting
2. Attempts to call ToString() on exception properties
3. For MemberInfo parameter (even though null), debugger tries type resolution
4. .NET 10 bug triggers during type name resolution for debugger display
5. Infinite recursion in debugger/runtime communication layer
6. Visual Studio becomes unresponsive waiting for debugger response

Tests That Work Correctly

These related tests work without issues:

// Closed generic types work fine
typeof(List<int>)                          // Works ✓
typeof(Dictionary<string, int>)            // Works ✓
typeof(Dictionary<string, List<int>>)      // Works ✓

// Simple types work fine
typeof(string)                             // Works ✓
typeof(int)                                // Works ✓

// Array types work fine
typeof(string[])                           // Works ✓
typeof(int[,])                             // Works ✓

The issue is specific to unbound generic type definitions like:

• typeof(List<>)
• typeof(Dictionary<,>)
• Any generic type definition with unspecified type parameters

Impact

This bug affects:

• Unit testing of XML documentation helpers using reflection
• Swagger/OpenAPI schema generation that processes unbound generic types
• Any code using reflection on generic type definitions in .NET 10 preview
• Development workflow (debugger freezes require VS restart)

Additional Context

Use Case

The code is part of an enhanced Swagger/OpenAPI documentation system that reads XML documentation comments and applies them to API schemas. The helper methods generate XML documentation member names (e.g., "T:System.Collections.Generic.List1"`) to look up comments from generated XML documentation files.

Why Unbound Generics Matter

Swagger schema generation may need to process generic type definitions to:

• Document generic type constraints
• Generate schema for generic DTOs
• Handle generic collection types in API responses

Production Code Works

The production code (XmlCommentsNodeNameHelper) works correctly when run directly. Failure only happens when running automated tests.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-reflection
See info in area-owners.md if you want to be subscribed.