Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jul 28, 2025

The COM interface source generator was generating incorrect marshalling code for array parameters with SizeParamIndex when the native array pointer could be null, affecting both stateful and stateless marshalling strategies.

Problem

When generating unmanaged-to-managed stubs for COM interfaces with array parameters using [MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 0)], the generator would create code like:

__buffer2_native__numElements = bufferSize;
buffer2 = ArrayMarshaller<int, int>.AllocateContainerForManagedElements(__buffer2_native, __buffer2_native__numElements);

This caused issues when the native array pointer was null but the buffer size was non-zero, as it would attempt to allocate an array for a null pointer using the provided size.

Solution

Modified both StatefulLinearCollectionMarshalling and StatelessLinearCollectionSpaceAllocator to generate null-aware size calculation:

__buffer2_native__numElements = __buffer2_native == null ? 0 : bufferSize;

The fix:

  • Detects when marshalling direction is UnmanagedToManaged
  • Uses semantic NativeType is PointerTypeInfo check instead of fragile type name checking
  • Generates a conditional expression that sets numElements to 0 when the native pointer is null
  • Applied consistently across all relevant methods in both marshalling strategies

Testing

Added ArrayBufferMarshallingTests.cs with a test case that reproduces the scenario described in the issue, ensuring the generated code compiles and handles null pointer cases correctly.

Fixes #118135.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits July 28, 2025 20:29
Co-authored-by: AaronRobinsonMSFT <30635565+AaronRobinsonMSFT@users.noreply.github.com>
Co-authored-by: AaronRobinsonMSFT <30635565+AaronRobinsonMSFT@users.noreply.github.com>
Copilot AI changed the title [WIP] COM source generation generates unexpected buffer marshal logic Fix COM source generation for arrays with null pointers and SizeParamIndex Jul 28, 2025
Copilot AI requested a review from AaronRobinsonMSFT July 28, 2025 20:50
// we need to check if the native pointer is null before using the size parameter to avoid
// allocating arrays for null pointers. This applies to LPArray scenarios.
if (CodeContext.Direction == MarshalDirection.UnmanagedToManaged &&
innerMarshaller.NativeType.FullTypeName.EndsWith("*"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
innerMarshaller.NativeType.FullTypeName.EndsWith("*"))
innerMarshaller.NativeType.FullTypeName.EndsWith("*"))

@jkoritzinsky or @jtschuster This seems like a suspect check. Is there a better way to check for array scenarios?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be checking if NativeType is a PointerTypeInfo.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please try to apply the suggestion by @jtschuster

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right. I've replaced the fragile type name check with a more semantic approach that checks if we have CountInfo that uses a parameter (like SizeParamIndex). The new check countInfo is SizeAndParamIndexInfo { ParamAtIndex: not null } specifically targets scenarios where we have size parameter information, which is exactly when null pointer issues can occur. This is much more precise and maintainable than checking if the type name ends with "*". (2e2e628)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied @jtschuster's suggestion to use innerMarshaller.NativeType is PointerTypeInfo instead of the parameter-based check. This is much more semantic and type-safe than the previous approaches. (c2942df)

… checking

Co-authored-by: AaronRobinsonMSFT <30635565+AaronRobinsonMSFT@users.noreply.github.com>
Co-authored-by: AaronRobinsonMSFT <30635565+AaronRobinsonMSFT@users.noreply.github.com>
@AaronRobinsonMSFT
Copy link
Member

@copilot You've updated the Stateful marshaller. You need to also update the Stateless marshaller. See StatelessMarshallingStrategy starting on line 438.

Co-authored-by: AaronRobinsonMSFT <30635565+AaronRobinsonMSFT@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jul 29, 2025

@copilot You've updated the Stateful marshaller. You need to also update the Stateless marshaller. See StatelessMarshallingStrategy starting on line 438.

Applied the same null pointer fix to the stateless marshalling strategy. Updated three methods in StatelessLinearCollectionSpaceAllocator: GenerateUnmarshalStatements, GenerateGuaranteedUnmarshalStatements, and GenerateCleanupCallerAllocatedResourcesStatements to use the same semantic NativeType is PointerTypeInfo check and conditional null handling logic. (19dd954)

Copilot AI changed the title Fix COM source generation for arrays with null pointers and SizeParamIndex Fix COM source generation for arrays with null pointers and SizeParamIndex in both stateful and stateless marshalling Jul 29, 2025
// If the marshalling direction is unmanaged-to-managed and we have a native pointer type,
// we need to check if the native pointer is null before using the size parameter to avoid
// allocating arrays for null pointers.
if (CodeContext.Direction == MarshalDirection.UnmanagedToManaged &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to check the direction here as we're already under a "not ManagedToUnmanaged" check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right. Removed the redundant direction check since we're already under a "not ManagedToUnmanaged" condition at line 322. (265a03f)

// If the marshalling direction is unmanaged-to-managed and we have a native pointer type,
// we need to check if the native pointer is null before using the size parameter to avoid
// allocating arrays for null pointers.
if (CodeContext.Direction == MarshalDirection.UnmanagedToManaged &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change the direction checks in your changes to check for "not ManagedToUnmanaged" and add some tests for arrays passed by ref

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the direction check and added tests for arrays passed by ref. The GenerateGuaranteedUnmarshalStatements method only runs for unmarshal scenarios so the direction check was redundant. Added TestMethodWithRef to the test interface and implementation to ensure ref array scenarios are covered. (265a03f)

@jkoritzinsky
Copy link
Member

@copilot please address my review comments

…nd add ref array tests

Co-authored-by: jkoritzinsky <1571408+jkoritzinsky@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jul 29, 2025

@copilot please address my review comments

Addressed both review comments: removed redundant direction checks in stateless marshalling strategy and added tests for arrays passed by ref. The changes ensure consistency with the "not ManagedToUnmanaged" pattern and provide better test coverage. (265a03f)

@jtschuster
Copy link
Member

Superseded by #118190.

@jtschuster jtschuster closed this Jul 31, 2025
@AaronRobinsonMSFT
Copy link
Member

Valiant effort @copilot! Thanks.

@jkotas jkotas deleted the copilot/fix-118135 branch August 4, 2025 06:09
@github-actions github-actions bot locked and limited conversation to collaborators Sep 3, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

COM source generation generates unexpected buffer marshal logic

4 participants