[clr-interp] Add support for unmanaged thiscall calling convention

[davidwrighton]

• Notably, on Windows platforms, the thiscall calling convention changes the argument order of the return buffer, and changes the rules where the return buffer pointer is placed
• This should also be good prep for the Swift calling convention work, which also needs to identify the swift calling convention at this point
• Also a drive by fix for the behavior of explicit this managed function pointers which return values in the return buffer, we were also missing a test cases for that scenario.

[dotnet-policy-service]

Tagging subscribers to this area: @BrzVlad, @janvorli, @kg
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @BrzVlad, @janvorli, @kg
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

[davidwrighton]

@kotlarmilos This change includes logic to discover the correct unmanaged calling convention in the call stub generator. Note that it handles the pinvoke, UnmanagedCallersOnlyAttribute, as well as the calli signature scenarios for specifying the calling convention. This will need to merge with the work you've done around the Swift calling convention.

[Copilot]

Pull request overview

This PR adds support for unmanaged thiscall calling convention to the CLR interpreter, addressing platform-specific calling conventions (particularly on Windows) where the thiscall convention changes argument order for return buffers. The changes also fix handling of explicit this managed function pointers that return values in a return buffer.

Key Changes:

• Added detection and handling of unmanaged calling conventions (thiscall, CMemberFunction, etc.) in the interpreter call stub generator
• Implemented signature rewriting from ExplicitThis to HasThis for proper return buffer handling on Windows
• Extended test coverage for explicit this instance method calls returning large value types

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/tests/JIT/Regression/JitBlue/GitHub_35384/GitHub_35384.il	Expands test to include large valuetype returns and verifies explicit this calling convention with return buffers
src/coreclr/vm/siginfo.hpp	Adds declaration for GetUnmanagedCallConvExtension helper function
src/coreclr/vm/siginfo.cpp	Implements GetUnmanagedCallConvExtension to extract calling convention from method signature
src/coreclr/vm/callstubgenerator.h	Updates ComputeCallStub signature to accept MethodDesc parameter
src/coreclr/vm/callstubgenerator.cpp	Core implementation: detects unmanaged calling conventions, rewrites signatures, and adds helper for native primitive struct detection
src/coreclr/vm/callconvbuilder.hpp	Declares new function to parse calling convention from return type position in signature
src/coreclr/vm/callconvbuilder.cpp	Refactors calling convention parsing to support starting from return type position

[kotlarmilos]

LGTM, thanks!