[wasm]: JavaScript interop with [JSImport] and [JSExport] attributes and Roslyn

[pavelsavara]

Design

This evolved from #64493 and steals many ideas from #60765 and also from conversations with @kg
Related API proposal is #70133
Contributes to #46378
Contributes to #47909
Fixes #65959
Fixes #69394

• C# generated on compile time
  • improves trimming of managed code for JSImport
  • JSExport code is always retained as we don't have linker visibility to JavaScript usage.
  • only C# and JS, no generated C code.
• JS generated on runtime
  • improves download size and it's easier to implement.
  • but it's not CSP Compliant
  • buffer for method signature passed during method binding
• the "wire" format is bespoke, versioned
  • JS interop is not generic RPC:
  • WASM can't create JS objects, JS doesn't have natural notion of memory space
  • We handle 2 garbage collectors and optimize for speed.
  • Code generators like flatbuffers or protobuffers are not practical to use inside runtime.
  • structure of the buffers is opaque to the public API
  • use stackalloc for call arguments, both directions
    • because Mono has conservative GC, we use it to pin MonoString*
• As compared to existing private JS interop
  • It has public C# API
  • There are always generated stubs at both sides of the C#/JS boundary
  • Never pass MonoObject* to user javaScript
    • use GCHandle for C# objects
    • use JSHandle for JS objects
    • internally MonoString* is still used for strings and exception Message
  • JS boundary is not crossed to marshal individual argument
    • except for some Task/Promise scenarios
    • except for some String scenarios
  • primitive types are not boxed, calls could be zero allocation
  • generated C# code inline marshallers
  • It could bind functions both directions, only [JSExport] marked methods are accessible from JS side.
• Roslyn analyzer enforces that non-trivial parameters are annotated with [JSMarshalAs] attribute.
  • Roslyn analyzer enforces that only supported combinations of C# type and JSType could be used.
  • Explicit definition of intent for nontrivial arg types, will help with backward compatibility.

Examples

[JSImport("console.log")]
internal static partial void Log(string message);

[JSImport("INTERNAL.http_wasm_get_response_header_names")]
private static partial string[] _GetResponseHeaderNames(
    JSObject fetchResponse);

[JSImport("INTERNAL.ws_wasm_send")]
public static partial Task? WebSocketSend(
    JSObject webSocket,
    [JSMarshalAs<JSType.MemoryView>] ArraySegment<byte> buffer,
    int messageType,
    bool endOfMessage);

[JSImport("INTERNAL.create_function")]
[return: JSMarshalAs<JSType.Function<JSType.Number, JSType.Number, JSType.Number>>]
public static partial Func<double, double, double> CreateFunctionDoubleDoubleDouble(
    string arg1Name, 
    string arg2Name, 
    string code);

[JSExport("JavaScriptTestHelper.AwaitTaskOfObject")]
public static async Task<string> SlowFailure(Task<int> promisedNumber)
{
    var delayMs = await promisedNumber;
    if (promisedNumber<0) throw new ArgumentException("delayMs");

    await Task.Delay(delayMs);
    return "Slow hello";
}

In scope

• JSImportAttribute, JSExportAttribute
• Function.New(string code, out Action function) and few other Func<> signatures
• JSObject.GetPropertyAsXXX
• JSObject.SetProperty as extension method
• JSMarshalAsAttribute and JSType
  • Require that all parameters of exported and imported functions are explicitly annotated, so that we don't have to guess correct default behavior.
  • JSType.Bool - C# bool as JS boolean as opposed to possible but unusual Number in JS.
    • also Nullable<>
  • JSType.Number - it's actually like double inside
    • valid for C# number types, bool and char, decimal, float, double
    • could represent also IntPtr, void* for people who know what they are doing with it
    • also Nullable<>
  • JSType.BigInt
    • for Int64 and Nullable<Int64>
  • JSType.String - System.String
    • pinned by the buffer on stack
    • including interned strings
  • JSType.Date - System.DateTime, System.DateTimeOffset
    • also Nullable<>
  • JSType.Promise - Task, Task<>
    • including GC on both sides
  • JSType.Function - Func<> and Action<>
    • up to 3 generic arguments
  • JSType.MemoryView - ArraySegment<byte>, ArraySegment<int>, ArraySegment<double>
    • zero copy, Uint8Array-like view, with API protecting user from wasm memory resize
    • with underlying managed array Pinned GCHandle until the JS view is collected
    • could be only created on C# side
  • JSType.MemoryView - Span<byte>, Span<int>, Span<double>
    • useful for short term shared buffer, usually C# stack allocated
    • with caller responsible for pinning and de-allocation
  • JSType.Error - System.Exception and JSException
    • including lazy stack trace
  • JSType.Array
    • copy of the array, one direction only
    • for string[] as JavaScript Array
      • with _RegisterGCRoot/mono_wasm_register_root on the list buffer
    • for object[] as JavaScript Array
      • dynamic marshling of the elements by type
    • for JSObject[] as JavaScript Array
    • for byte[] as JavaScript Uint8Array
    • for int[] as JavaScript Int32Array
    • for double[] as JavaScript Float64Array
  • JSType.Object for JSObject proxy
  • JSType.Discard, JSType.Void
  • JSType.Any for System.Object - by dynamic type of parameter instance
    • JS->C#: string, double, bool, DateTime, JSObject, Exception, and C# proxy back get default mapping
    • JS->C#: otherwise JSObject proxy would be created
    • C# -> JS: string, bool, byte, char, short, int, float, double, DateTime, DateTimeOffset, JSObject, Exception, DateTime, Nullable<> get default mapping
    • C# -> JS: otherwise ManagedObject proxy would be created
    • Task/Promise only with Task<object> signature
    • byte[], int[], double[] <-> Uint8Array, Int32Array, Float64Array and back
    • string[], object[] -> Array
    • Array -> object[]
    • not supported: long, functions, Span, ArraySegment, will throw NotImplementedException
• JSMarshalAsAttribute on generic parameters with multiple params.
  • void Foo([JSMarshalAs<JSType.Promise<JSType.BigInt>>] Task<long> value)
• generate code instead of GetArgumentSignature
• convert WebSocket to [JSImport]
• convert BrowserHttpHandlerto[JSImport]`
• Blazor [draft] JavaScript interop via [JSImport] and [JSExport] aspnetcore#41665
• old JS API compatibility for JSHandle/GCHandle and interned strings
• large Int64 values could throw overflow when passed as System.Object, instead we throw NotImplementedException when we see Int64. Same for bigint.
• Good enough roslyn diagnostic messages
• test ref assembly compilation on non-wasm target
• make sure that it can AOT well!
• add to samples

Consider

• implement micro benchmark in Wasm.BrowserProfile.Sample
• JS side lazy stack trace
• interpreted JS side marshaling
• JSExport lazy binding

Possible Future

• enable bind_static_method to consume [JSExport]
• Tests for Roslyn diagnostic scenarios
• Roslyn Auto-correct analysis for missing [JSMarshalAs(JSType.XXX)]
• What is the error experience for things that are out-of-scope? And if they will become in-scope, what is the light-up experience?
• What analyzer/fixers might be added to help users do the ‘right’ thing when using the new APIs?
• measure number of ccalls
• measure interp/AOT transitions
• provide more detailed message about what failed why on runtime
• move old interop into separate .js optional file, get rid of private interop DLL
• make interpreted version of JS marshaling which is CSP friendly
• JSType.Array - [MarshalUsing] for with P/Invoke native custom marshaler
  • allocation managed by marshaler, copy bytes
  • JSImport only
• Uri as string as custom marshaler
• JSMarshalAsAttribute with JSType.Custom and JSCustomTypeMarshaller, JSMarshallingAttribute
  • with user provided JS code to un-marshal the value on JS side too.
• generate JS code on compile time and make it CSP Compliant
• JSMarshalAsAttribute with JSType.MemoryView
  • over [MarshalUsing]
  • over Memory<byte>
• JSMarshalAsAttribute with JSType.BigInt
  • over UInt64
• JSMarshalAsAttribute with JSType.String
  • over DateTime, DateTimeOffset
• JSMarshalAsAttribute with JSType.Number
  • over SByte, UInt16, UInt32, UInt64, UIntPtr
  • over Boolean, Char
  • over DateTime, DateTimeOffset
• [MarshalUsing] for JSExport
• JSMarshalAsAttribute and JSType
  • JSType.Int32Array for Span<int>
  • JSType.Int16Array for Span<short>
  • etc..
• old JS API compatibility for MonoString*
• old JS API compatibility for MonoObject*
• drop InFlight
• drop System.Private.Runtime.InteropServices.JavaScript assembly

Out of scope

• anything with nested generic parameters
• untyped Delegate
• C# arrays by reference
• C# arrays as delegate parameter/return or as task return
• P/Invoke MarshalAsAttribute and UnmanagedType behavior on method params
  • UnmanagedType.LPArray & UnmanagedType.LPStruct is not supported by generated marshalers
  • UnmanagedType.FunctionPtr - maybe for another wasm C function via JS ? What marshaler would be used to marshal the parameters of the function.
  • UnmanagedType.Bool, VariantBool - we could represent System.Boolean as number or as boolean in JS.
  • UnmanagedType.I1, I2, I4, I8, U1, U2, U4, U8, Currency, SysInt, SysUInt - all of that is always number or BigInt in JS
  • UnmanagedType.BStr, LPStr, LPWStr, LPTStr, ByValTStr, VBByRefStr, AnsiBStr, TBStr, HString, LPUTF8Str - all of this is string in JS.
  • UnmanagedType.IUnknown, IDispatch, Interface, SafeArray, AsAny, Error, IInspectable - not relevant
  • UnmanagedType.Struct, ByValArray - we could not map struct to JS call stack
  • UnmanagedType.IUnknown, IDispatch, Interface, SafeArray, AsAny, Error, IInspectable

[ghost]

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

[ghost]

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

just prototyping

Author:	pavelsavara
Assignees:	-
Labels:	NO-REVIEW, arch-wasm
Milestone:	7.0.0

[pavelsavara]

/azp run runtime-wasm

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[pavelsavara]

/azp run runtime-wasm

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[pavelsavara]

@kg @maraf @lewing, I rebased it and did another pass of cleanup.
I would like to merge it on Monday before preview 7 snap.
Please review and approve :)

[pavelsavara]

/azp run runtime-wasm

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[pavelsavara]

/azp run runtime-wasm

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[pavelsavara]

Test failure Fatal error in IL Linker Unhandled exception. System.NotImplementedException: switch is #71848

[radical]

Test failure Not found: args[0] = x is also known issue with long logs

#71887

[ViktorHofer]

I see that you are only removing code from this section. Should the newly added library not be exposed in the shared framework?

[pavelsavara]

The new ones are System.Runtime.InteropServices.JavaScript and JSImportGenerator.
Both of them should be visible and compile on any platform.
I don't fully understand the question nor the necessary action. Could you please elaborate ?

[ViktorHofer]

The new assembly System.Runtime.InteropServices.JavaScript is currently not exposed in the shared framework because the entry is missing from here:

runtime/src/libraries/NetCoreAppLibrary.props

Lines 28 to 176 in cf83273

	<NetCoreAppLibrary>
	$(NetFxReference)
	netstandard;
	Microsoft.CSharp;
	Microsoft.VisualBasic.Core;
	Microsoft.Win32.Primitives;
	Microsoft.Win32.Registry;
	System.AppContext;
	System.Buffers;
	System.Collections;
	System.Collections.Concurrent;
	System.Collections.Immutable;
	System.Collections.NonGeneric;
	System.Collections.Specialized;
	System.ComponentModel;
	System.ComponentModel.Annotations;
	System.ComponentModel.EventBasedAsync;
	System.ComponentModel.Primitives;
	System.ComponentModel.TypeConverter;
	System.Console;
	System.Data.Common;
	System.Data.DataSetExtensions;
	System.Diagnostics.Contracts;
	System.Diagnostics.Debug;
	System.Diagnostics.DiagnosticSource;
	System.Diagnostics.FileVersionInfo;
	System.Diagnostics.Process;
	System.Diagnostics.StackTrace;
	System.Diagnostics.TextWriterTraceListener;
	System.Diagnostics.Tools;
	System.Diagnostics.TraceSource;
	System.Diagnostics.Tracing;
	System.Drawing.Primitives;
	System.Dynamic.Runtime;
	System.Formats.Asn1;
	System.Formats.Tar;
	System.Globalization;
	System.Globalization.Calendars;
	System.Globalization.Extensions;
	System.IO;
	System.IO.Compression;
	System.IO.Compression.Brotli;
	System.IO.Compression.ZipFile;
	System.IO.FileSystem;
	System.IO.FileSystem.AccessControl;
	System.IO.FileSystem.DriveInfo;
	System.IO.FileSystem.Primitives;
	System.IO.FileSystem.Watcher;
	System.IO.IsolatedStorage;
	System.IO.MemoryMappedFiles;
	System.IO.Pipes;
	System.IO.Pipes.AccessControl;
	System.IO.UnmanagedMemoryStream;
	System.Linq;
	System.Linq.Expressions;
	System.Linq.Parallel;
	System.Linq.Queryable;
	System.Memory;
	System.Net.Http;
	System.Net.Http.Json;
	System.Net.HttpListener;
	System.Net.Mail;
	System.Net.NameResolution;
	System.Net.NetworkInformation;
	System.Net.Ping;
	System.Net.Primitives;
	System.Net.Quic;
	System.Net.Requests;
	System.Net.Security;
	System.Net.ServicePoint;
	System.Net.Sockets;
	System.Net.WebClient;
	System.Net.WebHeaderCollection;
	System.Net.WebProxy;
	System.Net.WebSockets;
	System.Net.WebSockets.Client;
	System.Numerics.Vectors;
	System.ObjectModel;
	System.Private.CoreLib;
	System.Private.DataContractSerialization;
	System.Private.Uri;
	System.Private.Xml;
	System.Private.Xml.Linq;
	System.Reflection;
	System.Reflection.DispatchProxy;
	System.Reflection.Emit;
	System.Reflection.Emit.ILGeneration;
	System.Reflection.Emit.Lightweight;
	System.Reflection.Extensions;
	System.Reflection.Metadata;
	System.Reflection.Primitives;
	System.Reflection.TypeExtensions;
	System.Resources.Reader;
	System.Resources.ResourceManager;
	System.Resources.Writer;
	System.Runtime;
	System.Runtime.CompilerServices.Unsafe;
	System.Runtime.CompilerServices.VisualC;
	System.Runtime.Extensions;
	System.Runtime.Handles;
	System.Runtime.InteropServices;
	System.Runtime.InteropServices.JavaScript;
	System.Runtime.InteropServices.RuntimeInformation;
	System.Runtime.Intrinsics;
	System.Runtime.Loader;
	System.Runtime.Numerics;
	System.Runtime.Serialization.Formatters;
	System.Runtime.Serialization.Json;
	System.Runtime.Serialization.Primitives;
	System.Runtime.Serialization.Xml;
	System.Security.AccessControl;
	System.Security.Claims;
	System.Security.Cryptography;
	System.Security.Cryptography.Algorithms;
	System.Security.Cryptography.Cng;
	System.Security.Cryptography.Csp;
	System.Security.Cryptography.Encoding;
	System.Security.Cryptography.OpenSsl;
	System.Security.Cryptography.Primitives;
	System.Security.Cryptography.X509Certificates;
	System.Security.Principal;
	System.Security.Principal.Windows;
	System.Security.SecureString;
	System.Text.Encoding;
	System.Text.Encoding.CodePages;
	System.Text.Encoding.Extensions;
	System.Text.Encodings.Web;
	System.Text.Json;
	System.Text.RegularExpressions;
	System.Threading;
	System.Threading.Channels;
	System.Threading.Overlapped;
	System.Threading.Tasks;
	System.Threading.Tasks.Dataflow;
	System.Threading.Tasks.Extensions;
	System.Threading.Tasks.Parallel;
	System.Threading.Thread;
	System.Threading.ThreadPool;
	System.Threading.Timer;
	System.Transactions.Local;
	System.ValueTuple;
	System.Web.HttpUtility;
	System.Xml.ReaderWriter;
	System.Xml.XDocument;
	System.Xml.XmlDocument;
	System.Xml.XmlSerializer;
	System.Xml.XPath;
	System.Xml.XPath.XDocument;
	</NetCoreAppLibrary>

The source generator is already exposed, so no action necessary for that one.

[pavelsavara]

It's already there from previous PR

runtime/src/libraries/NetCoreAppLibrary.props

Line 129 in cf83273

System.Runtime.InteropServices.JavaScript;

is that OK ?