Skip to content

Conversation

@jevansaks
Copy link
Member

@jevansaks jevansaks commented Nov 11, 2025

COM interfaces with methods that return struct have never worked correctly -- see #167. With all the different marshalling modes that cswin32 supports this also gets fairly complex. This change applies fixes / workarounds to all the modes in order to get this to work:

1. In .NET Core + source generated COM

This just works, [GeneratedComInterface] takes care of this for us.

There is a risk that type def'd structs are improperly handled (see discussion in #1478). To handle this we add a custom marshaller to typedefs so that the native type is the pointer and it's wrapped in the struct during marshalling.

2. In .NET Core + blittable mode

a. We can change members with struct returns to use MemberFunction calling convention.
b. We apply this only to functions that have struct returns just so that we don't perturb the generated code too much. When the return type is non-struct, there should be no difference.

3. In .NET Framework + blittable mode

Sadly there is no MemberFunction calling convention we can use so we emulate it using the suggestion @tannergooding offered in #167:

public D3D12_RESOURCE_ALLOCATION_INFO GetResourceAllocationInfo(uint visibleMask, uint numResourceDescs, D3D12_RESOURCE_DESC* pResourceDescs)
{
    D3D12_RESOURCE_ALLOCATION_INFO result;
    return *((delegate* unmanaged<ID3D12Device*, D3D12_RESOURCE_ALLOCATION_INFO*, uint, uint, D3D12_RESOURCE_DESC*, D3D12_RESOURCE_ALLOCATION_INFO*>)(lpVtbl[25]))((ID3D12Device*)Unsafe.AsPointer(ref this), &result, visibleMask, numResourceDescs, pResourceDescs);
}

Which is to say, change the return type to pointer and add a new 1st parameter which is a pointer to the buffer to receive the return value. This is relatively contained in the DeclareInterfaceAsStruct path where we can keep the public method the same and only change the function pointer signature and implementation of the method call.

4. Built-in COM

Built-in COM for both .NET Framework and .NET Core mis-handles this scenario as well. We can use the same technique as blittable mode without MemberFunction and modify the ABI methods of the COM interface to be the modified signature. When modifying the signature we also rename the ABI member with ("_StructReturn") so it's obvious what happened:

[PreserveSig()]
unsafe winmdroot.Graphics.Direct3D12.D3D12_RESOURCE_ALLOCATION_INFO* GetResourceAllocationInfo_StructReturn(winmdroot.Graphics.Direct3D12.D3D12_RESOURCE_ALLOCATION_INFO*__retVal, uint visibleMask, uint numResourceDescs, winmdroot.Graphics.Direct3D12.D3D12_RESOURCE_DESC* pResourceDescs);

And then the friendly overloads help to smooth this over for callers:

internal static unsafe winmdroot.Graphics.Direct3D12.D3D12_RESOURCE_ALLOCATION_INFO GetResourceAllocationInfo(this winmdroot.Graphics.Direct3D12.ID3D12Device1 @this, uint numResourceDescs, ReadOnlySpan<winmdroot.Graphics.Direct3D12.D3D12_RESOURCE_DESC> pResourceDescs)
{
	fixed (winmdroot.Graphics.Direct3D12.D3D12_RESOURCE_DESC* pResourceDescsLocal = pResourceDescs)
	{
		winmdroot.Graphics.Direct3D12.D3D12_RESOURCE_ALLOCATION_INFO __retVal = default(winmdroot.Graphics.Direct3D12.D3D12_RESOURCE_ALLOCATION_INFO);
		winmdroot.Graphics.Direct3D12.D3D12_RESOURCE_ALLOCATION_INFO __result = *@this.GetResourceAllocationInfo_StructReturn(&__retVal, (uint )pResourceDescs.Length, numResourceDescs, pResourceDescsLocal);
		return __result;
	}
}

All of these modes have test coverage for not just the codegen but samples that call the functions and validate the correct values are returned. Previously these tests cases either crashed or returned garbage results.

Fixes #167, Fixes #1479, Fixes #1478, Fixes #1436.

@jevansaks jevansaks merged commit 51614b6 into main Nov 13, 2025
7 checks passed
@jevansaks jevansaks deleted the user/jevansaks/marshalreturn branch November 13, 2025 02:48
@tannergooding
Copy link
Member

  1. In .NET Framework + blittable mode
    Sadly there is no MemberFunction calling convention we can use so we emulate it using the suggestion @tannergooding offered in COM interfaces returning structs use wrong calling convention #167:

It's worth noting that this relies on impl details of the ABI a bit and it "may" still break for some scenario on Arm64, namely due to potentially different conventions in some edge cases involving multi-register returns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

5 participants