Test failure: JIT/opt/Casts/shared_Casts/shared_Casts.dll

[VincentBu]

Failed in: runtime-coreclr outerloop 20260421.3

Failed tests:

R2R-CG2 windows x86 Checked @ Windows.10.Amd64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 windows x64 Checked @ Windows.10.Amd64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 linux arm64 Checked @ (Alpine.323.Arm64.Open)AzureLinux.3.Arm64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.23-helix-arm64v8
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 linux arm64 Checked no_tiered_compilation @ (Alpine.323.Arm64.Open)AzureLinux.3.Arm64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.23-helix-arm64v8
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 linux arm64 Checked @ AzureLinux.3.Arm64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 linux arm64 Checked no_tiered_compilation @ AzureLinux.3.Arm64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 linux x64 Checked @ (Alpine.323.Amd64.Open)AzureLinux.3.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.23-helix-amd64
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 linux x64 Checked no_tiered_compilation @ (Alpine.323.Amd64.Open)AzureLinux.3.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.23-helix-amd64
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 linux x64 Checked @ AzureLinux.3.Amd64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 linux x64 Checked no_tiered_compilation @ AzureLinux.3.Amd64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 windows arm64 Checked @ Windows.11.Arm64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 windows arm64 Checked no_tiered_compilation @ Windows.11.Arm64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 osx x64 Checked @ OSX.15.Amd64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll
R2R-CG2 osx x64 Checked no_tiered_compilation @ OSX.15.Amd64.Open
- JIT/opt/Casts/shared_Casts/shared_Casts.dll

Error message:

Assert.Equal() Failure: Values differ
Expected: 100
Actual:   0

Stack trace:

   at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) in /_/src/arcade/src/Microsoft.DotNet.XUnitAssert/src/EqualityAsserts.cs:line 102
   at Xunit.Assert.Equal[T](T expected, T actual) in /_/src/arcade/src/Microsoft.DotNet.XUnitAssert/src/EqualityAsserts.cs:line 36
   at Program.<<Main>$>g__TestExecutor286|0_287(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[AndyAyersMS]

Might be related to #126930. So I'll take this one for now.

[AndyAyersMS]

Note

This analysis was generated with assistance from GitHub Copilot.

Root cause

This is a regression from #127146 ("Handle canonical types in casting logic").

The failing test (src/tests/JIT/opt/Casts/shared.cs) contains:

static bool F0<T>(I<T> i) => i is I<object>;
// ...
bool b01 = F0<string>(iNotObj); // expected: false

When crossgen2 compiles the shared generic body F0<__Canon>, it asks the JIT type-compare-for-cast API whether I<__Canon> can be cast to I<object>. This is the positive-result branch in CorInfoImpl.compareTypesForCast:

// Pass back positive results unfiltered.
if (canCast)
    result = TypeCompareState.Must;

fromType.CanCastTo(toType) is I<__Canon>.CanCastTo(I<object>). Before #127146 that returned false; after #127146 it returns true, because the new IsCanonEquivalent check in CanCastToNonVariantInterface treats any I<__Canon> as matching I<X> for any reference type X.

Returning Must here lets impExpandIsinst fold the isinst I<object> to "always true":

Considering optimization of isinst from I<__Canon> to I<object>
Cast will succeed, optimizing to simply return input

So at runtime F0<string>(iNotObj) (where iNotObj is I<string>) returns true instead of false, and the test's output accumulator becomes 15 instead of 9.

The canon-equivalence rule is correct for the dataflow/constraint-check scenario that #127146 targeted (#126604), but it violates the invariant compareTypesForCast depends on — namely that a true result from CanCastTo implies the cast must succeed for every concrete instantiation the shared code can see.

Note: the VM path (TypeHandle::CanCastTo in src/coreclr/vm/jitinterface.cpp) was not touched by #127146, so non-R2R runs are unaffected. The bug only reproduces under crossgen2/R2R, which matches the failure signature (all R2R platforms fail, jitstress does not).

Fix

In CorInfoImpl.compareTypesForCast, when casting from shared to unshared interface target, only trust a positive CanCastTo as Must when toType.HasInstantiation is false. Otherwise demote to May, which preserves the runtime test and avoids miscompilation:

if (canCast && !toType.HasInstantiation)
    result = TypeCompareState.Must;
else if (canCast)
    result = TypeCompareState.May;

This is a minimal, targeted fix. A broader/cleaner long-term option is to scope the canon-equivalence additions in CastingHelper.cs to only the reflection/constraint-check code paths that need them (e.g., via an explicit CanCastToWithCanonEquivalence entrypoint), since compareTypesForCast is not the only consumer of CanCastTo — DevirtualizationManager, MetadataVirtualMethodAlgorithm, and ILImporter.StackValue also call it and have the same trust assumption. Those haven't shown observable regressions yet, but auditing them (or narrowing the CastingHelper change) would reduce future risk. Happy to do that as a follow-up.

Verification

• Reproduced locally on Windows (not OSX-specific). Non-R2R: a=9 (pass). R2R: a=15 (fail, exit≠100).
• With the fix applied and crossgen2/JIT rebuilt: R2R run produces a=9 and exits 100.
• JIT dump for F0 confirms the isinst fold no longer occurs for I<__Canon> → I<object>.
• Full JIT/opt non-R2R merged suite: 0 failures.

/cc @MichalStrehovsky @dotnet/jit-contrib

[MichalStrehovsky]

I have a revert at #127301 (anyone can approve). Copilot lied to me that CoreCLR CanCastTo says yes for anything canonical and the managed implementation was actually more precise. I really didn't want to read C++. Looks like I still need to read C++ in the AI age.