Current implementation of the shim is limiting

[whitequark]

I would like to ship, on npm, a few WASI commands that all need to use the virtualized filesystem. These commands are essentially pure functions that happen to be implemented in C++.

The browser shim implementation is perfect for this: I have a virtualized filesystem and arguments. The Node shim implementation seems to require the use of the host filesystem. It doesn't seem possible to use the virtualized filesystem from the browser implementation if I'm running on Node, limiting the usefulness of publishing an npm package.

It seems that the selection of the provided environment is done in a way that cannot be influenced by the code that uses the transpiled module. Am I missing anything?

(This seems related to #302)

[pchickey]

I believe that, if you want to provide your application with its own virtual (read-only) filesystem and environment vars, you want to use wasi-virt to produce a component providing those that it composes with.

wasi-virt is currently not compatible with jco because it is stuck on an older version of the WASI preview 2 wits. It needs to be updated but I believe everyone who has contributed to it is working on other things right this moment

[whitequark]

I believe that, if you want to provide your application with its own virtual (read-only) filesystem

I need it to be read-write because the output of the application is written to the filesystem.

wasi-virt is currently not compatible with jco because it is stuck on an older version of the WASI preview 2 wits. It needs to be updated but I believe everyone who has contributed to it is working on other things right this moment

Yep, I just hit this problem.

[pchickey]

My understanding is that a writable host filesystem can be overlaid on a read-only virtualized filesystem, but thats a question for over in wasi-virt...

[whitequark]

I settled on reimplementing the shim entirely. See also #302 (comment), which was the other half of the puzzle.

[whitequark]

@pchickey Take a look at YoWASP/yosys#28 if you'd like. I think my shim is fairly complete and in places might even do things the shim here doesn't. (I got a weird issue where scandir apparently passes .. to openAt which doesn't make a lot of sense...)

[guybedford]

@whitequark I really like your approach here - perhaps we should take what we have of a "browser implementation" and instead call it a "virtual JS implementation", and make it available always as an instantiation workflow as you have done. Obviously WASI-Virt is the ideal for this, because it saves implementing this in every ecosystem, but until we have that I definitely appreciate it's useful, and the browser implementation is kinda almost there. Does that sound like a sensible approach to you? Any other suggestions for the project very welcome.

[whitequark]

@guybedford That would be fantastic, and very useful to me.

Will WASI-Virt ever handle the case of a writable filesystem that is accessed by the embedder? If not the need for this feature will remain after WASI-Virt is done.

[guybedford]

WASI-Virt will get support for dynamic filesystems yes, it's just not been prioritized so far. Certainly this entire virtualization feature should be entirely subsumed by WASI-Virt though, since that offers the same virtualization system. But there's no reason we can't offer some flexibility by having options too.

[whitequark]

Got it! That would be nice.

[whitequark]

@pchickey @guybedford I've productized my JS runtime for YoWASP and published first non-experimental Yosys builds

• https://github.com/YoWASP/runtime-js
• https://www.npmjs.com/package/@yowasp/yosys
  Take a look :)