[API Proposal]: Provide `BSTR` marshaller for `LibraryImport` source generator

[AaronRobinsonMSFT]

Background and motivation

Add BSTR marshaller in addition to new string marshallers in #67635. Since this marshaller is for adoption purposes it is similar to the existing ANSI marshaller and will require users to manually define it using MarshalUsing attribute or can rely on the UnmanagedType.BStr value in a MarshalAs.

API Proposal

Implementation PR - #69213

namespace System.Runtime.InteropServices.Marshalling
{
    /// <summary>
    /// Marshaller for BSTR strings
    /// </summary>
    [CLSCompliant(false)]
    [CustomTypeMarshaller(typeof(string), BufferSize = 0x100,
        Features = CustomTypeMarshallerFeatures.UnmanagedResources | CustomTypeMarshallerFeatures.TwoStageMarshalling | CustomTypeMarshallerFeatures.CallerAllocatedBuffer)]
    public unsafe ref struct BStrStringMarshaller
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="BStrStringMarshaller"/>.
        /// </summary>
        /// <param name="str">The string to marshal.</param>
        public BStrStringMarshaller(string? str);

        /// <summary>
        /// Initializes a new instance of the <see cref="BStrStringMarshaller"/>.
        /// </summary>
        /// <param name="str">The string to marshal.</param>
        /// <param name="buffer">Buffer that may be used for marshalling.</param>
        /// <remarks>
        /// The <paramref name="buffer"/> must not be movable - that is, it should not be
        /// on the managed heap or it should be pinned.
        /// <seealso cref="CustomTypeMarshallerFeatures.CallerAllocatedBuffer"/>
        /// </remarks>
        public BStrStringMarshaller(string? str, Span<ushort> buffer);

        /// <summary>
        /// Returns the native value representing the string.
        /// </summary>
        /// <remarks>
        /// <seealso cref="CustomTypeMarshallerFeatures.TwoStageMarshalling"/>
        /// </remarks>
        public void* ToNativeValue();

        /// <summary>
        /// Sets the native value representing the string.
        /// </summary>
        /// <param name="value">The native value.</param>
        /// <remarks>
        /// <seealso cref="CustomTypeMarshallerFeatures.TwoStageMarshalling"/>
        /// </remarks>
        public void FromNativeValue(void* value);

        /// <summary>
        /// Returns the managed string.
        /// </summary>
        /// <remarks>
        /// <seealso cref="CustomTypeMarshallerDirection.Out"/>
        /// </remarks>
        public string? ToManaged();

        /// <summary>
        /// Frees native resources.
        /// </summary>
        /// <remarks>
        /// <seealso cref="CustomTypeMarshallerFeatures.UnmanagedResources"/>
        /// </remarks>
        public void FreeNative();
    }
}

API Usage

The following is an example from aspnetcore where this new marshaller could be used.

[LibraryImport(AspNetCoreModuleDll)]
private static partial int http_get_server_variable(
    NativeSafeHandle pInProcessHandler,
    [MarshalAs(UnmanagedType.LPStr)] string variableName,
    [MarshalUsing(typeof(BStrStringMarshaller))] out string value);

// or

[LibraryImport(AspNetCoreModuleDll)]
private static partial int http_get_server_variable(
    NativeSafeHandle pInProcessHandler,
    [MarshalAs(UnmanagedType.LPStr)] string variableName,
    [MarshalAs(UnmanagedType.BStr)] out string value);

Alternative Designs

No response

Risks

None. This is a new opt-in API.

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

Add BSTR marshaller in addition to new string marshallers in #67635. Since this marshaller is for adoption purposes it is similar to the existing ANSI marshaller and will require users to manually define it using MarshalUsing attribute.

API Proposal

namespace   namespace System.Runtime.InteropServices.Marshalling
{
    /// <summary>
    /// Marshaller for BSTR strings
    /// </summary>
    [CustomTypeMarshaller(typeof(string),
        Features = CustomTypeMarshallerFeatures.UnmanagedResources | CustomTypeMarshallerFeatures.TwoStageMarshalling)]
    internal ref struct BstrStringMarshaller
    {
        private IntPtr _allocated;

        public BstrStringMarshaller(string? str)
        {
            _allocated = Marshal.StringToBSTR(str);
        }

        public IntPtr ToNativeValue() => _allocated;

        public void FromNativeValue(IntPtr value) => _allocated = value;

        public string? ToManaged() => Marshal.PtrToStringBSTR(_allocated);

        public void FreeNative()
        {
            if (_allocated != IntPtr.Zero)
            {
                Marshal.FreeBSTR(_allocated);
            }
        }
    }
}

API Usage

The following is an example from aspnetcore where this new marshaller could be used.

[LibraryImport(AspNetCoreModuleDll)]
private static partial int http_get_server_variable(
    NativeSafeHandle pInProcessHandler,
    [MarshalAs(UnmanagedType.LPStr)] string variableName,
    [MarshalUsing(typeof(BstrStringMarshaller))] out string value);

Alternative Designs

No response

Risks

None. This is a new opt-in API.

Author:	AaronRobinsonMSFT
Assignees:	-
Labels:	api-suggestion, area-System.Runtime.InteropServices, untriaged
Milestone:	-

[AaronRobinsonMSFT]

I am marking this blocking since there is other work the aspnetcore repo needs and getting all of the work done to fully support consumption of LibraryImport in one shot would be preferred to piecemeal.

See dotnet/aspnetcore#41573

[jkotas]

Does this need to take the stack-allocated buffer to enable optimization for passing small strings into the APIs (the built-in marshalling has this optimization today)? Or is this going to be an intentional take-back from the built-in marshalling?

[jkotas]

BstrStringMarshaller

Nit: This is new 3rd casing variant in the public APIs. We have Marshal.StringToBSTR and UnmanagedType.BStr. I do not think we have Bstr anywhere yet. Just wanted to make sure that we want to introduce 3rd casing.

[elinor-fung]

it is similar to the existing ANSI marshaller and will require users to manually define it using MarshalUsing attribute.
This is a new opt-in API.

If we think BSTR is important and prevalent enough to warrant adding a public marshaller, I'd argue we should also handle UnmanagedType.BStr - like we do for LPStr->AnsiStringMarshaller, LPWStr->Utf16StringMarshaller, LPUTF8Str->Utf8StringMarshaller.

[elinor-fung]

Does this need to take the stack-allocated buffer to enable optimization for passing small strings into the APIs (the built-in marshalling has this optimization today)?

I'd say so. I don't see a reason not to.

[AaronRobinsonMSFT]

BStrStringMarshaller proposal implementation has been updated with short string optimization.