Unexpected JIT helper in TreatAsHasRetBufArg

[AndyAyersMS]

I have a customer case that fails on 6.0 with checked CLR assert:

Assert failure(PID 20024 [0x00004e38], Thread: 21576 [0x5448]): Assertion failed '!"Unexpected JIT helper in TreatAsHasRetBufArg"' in 'Python.Runtime.Converter:ToPython(System.Object,System.Type):Python.Runtime.NewReference' during 'Morph - Global' (IL size 1175)

    File: C:\repos\runtime1\runtime6\src\coreclr\jit\gentree.cpp Line: 1025
    Image: C:\repos\runtime1\runtime6\artifacts\tests\coreclr\windows.x64.checked\Tests\Core_Root\corerun.exe

looking at the dump, we have the following tree

[000504] --CXG+------              *  RETURN    struct
[001448] --CXG+------              \--*  CALL help struct HELPER.CORINFO_HELP_PINVOKE_CALLI
[001436] -----+------ arg2 in rcx     +--*  CAST      long <- int
[001435] -----+------                 |  \--*  LCL_VAR   int    V114 tmp88       
[003159] -----+------ arg0 in r10     +--*  CNS_INT   long   0x7ff98ffb0b2c
[001449] H----+-N---- arg1 in r11     \--*  CNS_INT(h) long   0x1fa2d976648 pinvoke

and the code in TreatAsHasRetBufArg is not expecting to see this helper.

I'll see if I can come up with a simple repro. Seems like in release we'll do the right thing as the default case is to return false.

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I have a customer case that fails on 6.0 with checked CLR assert:

Assert failure(PID 20024 [0x00004e38], Thread: 21576 [0x5448]): Assertion failed '!"Unexpected JIT helper in TreatAsHasRetBufArg"' in 'Python.Runtime.Converter:ToPython(System.Object,System.Type):Python.Runtime.NewReference' during 'Morph - Global' (IL size 1175)

    File: C:\repos\runtime1\runtime6\src\coreclr\jit\gentree.cpp Line: 1025
    Image: C:\repos\runtime1\runtime6\artifacts\tests\coreclr\windows.x64.checked\Tests\Core_Root\corerun.exe

looking at the dump, we have the following tree

[000504] --CXG+------              *  RETURN    struct
[001448] --CXG+------              \--*  CALL help struct HELPER.CORINFO_HELP_PINVOKE_CALLI
[001436] -----+------ arg2 in rcx     +--*  CAST      long <- int
[001435] -----+------                 |  \--*  LCL_VAR   int    V114 tmp88       
[003159] -----+------ arg0 in r10     +--*  CNS_INT   long   0x7ff98ffb0b2c
[001449] H----+-N---- arg1 in r11     \--*  CNS_INT(h) long   0x1fa2d976648 pinvoke

and the code in TreatAsHasRetBufArg is not expecting to see this helper.

I'll see if I can come up with a simple repro. Seems like in release we'll do the right thing as the default case is to return false.

Author:	AndyAyersMS
Assignees:	-
Labels:	area-CodeGen-coreclr
Milestone:	-

[jakobbotsch]

It sounds strange that we have a TYP_STRUCT returning calli that does not return true for ShouldHaveRetBufArg. That means we didn't add GTF_CALL_M_RETBUFFARG correctly (or, didn't normalize the return type correctly?).

[AndyAyersMS]

JitDump for the above. This is using a build at latest release/6.0.

I'm actually looking at another issue with this customer case, so won't get back to this for a while.

[AndyAyersMS]

Here's a repro case (repros in both .NET 6 and .NET 7):

;; set COMPlus_TieredPGO=1

using System;
using System.Runtime.CompilerServices;
using System.Threading;

ref struct NewReference
{
    public IntPtr pointer;
}

static unsafe class Delegates
{
    static Delegates()
    {
        PyLong_FromLongLong = (delegate* unmanaged[Cdecl]<long, NewReference>)0xbadcab;
        z = 100;
    }

    public static delegate* unmanaged[Cdecl]<long, NewReference> PyLong_FromLongLong { get; }
    public static long z;
}

class P
{
    static NewReference ToPython(object value, Type type)
    {
        TypeCode tc = Type.GetTypeCode(type);
        
        switch (tc)
        {
            case TypeCode.Byte:
                return PyInt_FromInt32((byte)value);
            case TypeCode.Int16:
                return PyInt_FromInt32((short)value);
            case TypeCode.UInt16:
                return PyInt_FromInt32((ushort)value);
            case TypeCode.Int32:
                return PyInt_FromInt32((int)value);
            default:
                return new NewReference();
        }
    }

    static NewReference PyInt_FromInt32(int value) => PyLong_FromLongLong(value);

    unsafe static NewReference PyLong_FromLongLong(long value) => Delegates.PyLong_FromLongLong(value);

    [MethodImpl(MethodImplOptions.NoOptimization)]
    static int Main()
    {
        for (int i = 0; i < 100; i++)
        {
            _ =  ToPython(Delegates.z, typeof(long));
            Thread.Sleep(15);
        }

        Thread.Sleep(50);
        _ = ToPython(Delegates.z, typeof(long));
        return 100;

    }
}

[AndyAyersMS]

We have

               [000042] --CXG------                         *  RETURN    struct
               [000069] --CXG------                         \--*  CALL ind  struct
               [000057] ----------- arg0                       +--*  CAST      long <- int
               [000056] -----------                            |  \--*  LCL_VAR   int    V06 tmp2         
               [000068] ----------- calli tgt                  \--*  LCL_VAR   long   V07 tmp3         

and morph creates

               [000042] --CXG+-----                         *  RETURN    struct
               [000069] --CXG+-----                         \--*  CALL help struct HELPER.CORINFO_HELP_PINVOKE_CALLI
               [000057] -----+----- arg2 in rcx                +--*  CAST      long <- int
               [000056] -----+-----                            |  \--*  LCL_VAR   int    V06 tmp2         
               [000138] -----+----- pinv tgt in r10            +--*  CNS_INT   long   0xBADCAB
               [000070] H----+-N--- pinv cookie in r11         \--*  CNS_INT(h) long   0x25b70278290 pinvoke

and asserts later when trying to merge return blocks.