Question: Linking components with Wasm modules

[skuzmich]

As far as I see, this proposal describes components interacting with each other and the host.

How would linking components with a non-component Wasm module look like? Could we use some sort of "relaxed component format", where some of the imports/exports use component types while others use core WebAssembly?

For example: a browser app that shares memory and references with JS and DOM might want to use a component to implement some of its functionality.

[alexcrichton]

If I understand your question correctly I believe this is more-or-less the intent of the canonical ABI. That provides a translation effectively between component model types and core WebAssembly. It currently is specified in the context of a component however so it's not possible yet to translate to core wasm in a vacuum (e.g. the canonical options all need to be specified in theory).

That being said though if you haven't seen it already jco is a project to run components on the web.

[skuzmich]

True, I see how we could use the same ABI, given that options are specified. I would be great if we could reuse component tooling for this case of "impure" components, the ones with some of their core imports and exports sticking out.

[lukewagner]

Part of what enables all the cross-language/language-agnostic tooling that makes the component model valuable in the first place is that there is no raw shared memory between a component and the outside world: this is what enables writing an interface once (in WIT) and generating bindings in N languages and a bunch of other scenarios as well. With raw public memory, there is no ABI and thus everyone would have to revert to what everyone does in wasm today, which is to roll your own custom thing and produce host-dependent code, which is the state of affairs that the Component Model is seeking to provide an alternative to.

Now, if your goal is to "mostly" reuse the Component Model, but do something browser-/JS-dependent "on the side" (e.g., by assuming certain special JS glue code), you can always run jco transpile and then manually edit the generated JS glue code to expose the otherwise-encapsulated ArrayBuffer that aliases core linear memory and then you can go to town. But that would very much be outside the Component Model and so all sorts of general ecosystem tooling (e.g., a lot of the composition and virtualization tooling that we're building) would simply break and thus these "impure" components would effectively be outside the ecosystem. But maybe that's fine for your use case.

My hope though is that with resources/handles in Preview 2 and futures/streams in Preview 3, the set of use cases that actually need raw publicly-shared memory would go away and the glue code you'd get from jco transpile (or, eventually, a native implementation) would be efficient enough. And if not, then I'd like to understand the concrete use case and design a new Component Model feature (say for a Preview 4 or v.1.1) that achieves the desired performance while maintaining the shared-nothing model that the ecosystem tooling depends on. This approach lifts up the whole ecosystem.

[alexcrichton]

Ah I see I think I misunderstood the question and it's instead sound like you want to have a wasm module that imports both component things and non-component things, and for that I would agree with what @lukewagner said.

[skuzmich]

@lukewagner, I like shared-nothing model! To me, it's a different design tradeoff that makes Wasm modules advantageous over components in some cases. Wasm GC was almost exclusively designed to accommodate efficient interop with JS and browser, but the component model chooses to disallow it. Should I propose adding a cross-component GC with cycle collection for Preview 4? :)

In the meantime, while some Wasm modules can't be components, they are a natural fit for using other components. A standard format for Canonical ABI and Post-MVP adapter functions applied to a subset of imports would help. If I remember it correctly, the interface types proposal allowed something like this.

[lukewagner]

Agreed that, when compiling a language to wasm with custom bindings tailored to the JS/browser environment, core wasm can still be the best fit in some cases; in such scenarios, the cross-language and host-agnostic benefits of the Component Model aren't relevant so that's not really a loss.

Based on the smiley, I think you're kidding about the cross-component-GC point but, just to be explicit on that topic: in a Component Model context where we're trying to keep the choice of programming language an internal impl choice of the component, it's crucial for components to never rely on cross-component GC, since this rules out a number of languages (including languages with GC that, for various reasons, are implemented in terms of linear memory). This is why the Component Model supports explicit acyclic ownership, which supports both wasm-gc (treating a handle as a root) and linear memory languages to interop. But given this acyclic ownership, I think components should still be able to do a decent job pointing at JS or DOM objects (via handle) and calling methods (noting that wasm-GC also requires opaque references and calling imports to access JS/DOM objects).

Incidentally, collecting cycles between wasm and JS/DOM is only one of the motivations for wasm-gc: another big one (that applies just as well in a Component Model context) is factoring the GC algorithm out of wasm and into the host (which can do a better job by being specialized to the environment). Here, we just need to add new Canonical ABI options for lifting and lowering to/from wasm-gc types which is very much in scope for a future Preview release.

[skuzmich]

Based on the smiley, I think you're kidding about the cross-component-GC point

Not quite, I was just glad you asked about use cases. :) Shared GC makes for a simpler programming model: you can exchange objects and closures freely between WasmGC language and JavaScript and they follow the same memory management rules as if they were your own. Lack this could become an obstacle preventing our toolchain from producing components when targeting JS hosts.

it's crucial for components to never rely on cross-component GC, since this rules out a number of languages

Could you elaborate on how GC would rule out linear memory languages? I could imagine them putting references into tables and using indices, similar to to how they currently handle externrefs. They would also want to have GC finalisers (Post-MVP) to free associated linear memory.

[rossberg]

@skuzmich:

Wasm GC was almost exclusively designed to accommodate efficient interop with JS and browser

I'm curious what gave you that idea, because that was at most a secondary or tertiary goal.

[skuzmich]

@rossberg I agree that it achieves other goals. But Wasm-host cycle collection was the main argument I heard used in favour of current design, as opposed to solving inefficient parts of GC in linear memory.

[lukewagner]

Could you elaborate on how GC would rule out linear memory languages? I could imagine them putting references into tables and using indices, similar to to how they currently handle externrefs.

The problem manifests when you have cycles that are successfully collected when all the components involved are implemented via wasm-gc but leak (due to the cycles being rooted by the tables) when some of the components involved are implemented via linear-memory+tables. What makes this failure mode especially insidious is that it won't show up early in design and testing; only when graphs of components are used at scale in certain compositions. And once you have one of these leaks, solving it is really hard, because now you need to reason about cross-language cycles. Closures are a particularly easy way for these cycles to appear since they close over their environment, which usually entrains containing scopes that often reference (via local or global variable) the foreign object to which the callback is being attached, creating a leak that's not obviously anyone's "fault" (which is the worst kind of leak). IIUC, Firefox 2 got the bad reputation for being leaky due to exactly this sort of problem, and the problem was only solved by adding a whole cycle collector (and cycle debugger) to FF along with a gnarly set of macros that all C++ classes that can possibly participate in a cycle have to use carefully. (Chrome also went through a number of iterations before settling on, iirc, having their C++ objects owned by the V8 GC and I think Safari does a fixpoint iteration sort of thing that also requires explicit participation from C++.) And this browser example is actually a nicer scenario than we have with components because there are only 2 "components" written in 2 fixed languages with tight manual integration in a single codebase.

Shared GC makes for a simpler programming model: you can exchange objects and closures freely between WasmGC language and JavaScript and they follow the same memory management rules as if they were your own.

Agreed on this point, though: if you only care about GC languages using wasm-gc as a substrate, it does open up new options. I could imagine a wholly different "GC-centric Component Model" that had a rather different technical design, and maybe backed off the whole "shared-nothing" model to allow sharing mutable (GC) memory and perhaps fixed some wasm-gc representations of common language concepts (like classes, vtables, etc). But now this sounds a lot more of a JVM- or CLR-like "language family", which isn't a bad thing and could be independently valuable and I expect someone will actually do at some point in time; it's just a very different goal statement than we have with the Component Model.

[skuzmich]

Thanks a bunch for breaking that down for me! Now I totally get why there's no cross-component garbage collection.

[skuzmich]

Sorry for derailing this into a GC discussion. Back to original question.

If we agree that component model will not be the best solution for all Wasm use cases, could we also agree that there should be a standard bridge mechanism between existing Wasm and emerging components for them to be interoperable in environments where both are supported? Since the limited ABI is the defining part of component model and it can't be changed, we are left with the option of using this ABI in regular Wasm for this kind of bridge.

[skuzmich]

I somehow missed the (exact?) thing I'm talking about proposed for post-MVP: adapter modules. Shouldn't we ensure that if this proposal receives first-class JS interop support in its MVP, it also gains equivalent first-class Wasm interop support at the MVP stage?

[lukewagner]

Oh hah, right; I totally forgot about writing that way way back. In the short term, given that components can't do anything in browsers that you can't do with core wasm + JS API and that you're mostly focused on doing special browser things with JS, I'd go with the route suggested above, which is to modify jco transpile output and use that as a way to prototype and flesh out the use case and expected developer experience for adding adapter modules in a post-MVP timeframe.