PoC of SQLite database access using OPFS #74
Conversation
This comment was marked as off-topic.
This comment was marked as off-topic.
|
🚨 Potential security issues detected. Learn more about Socket for GitHub ↗︎ To accept the risk, merge this PR and you will not be notified again.
Next stepsWhat is shell access?This module accesses the system shell. Accessing the system shell increases the risk of executing arbitrary code. Packages should avoid accessing the shell which can reduce portability, and make it easier for malicious shell access to be introduced. What is network access?This module accesses the network. Packages should remove all network access that is functionally unnecessary. Consumers should audit network access to ensure legitimate use. Take a deeper look at the dependencyTake a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev. Remove the packageIf you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency. Mark a package as acceptable riskTo ignore an alert, reply with a comment starting with
|
enabled by putting the kernel into a web worker.
FUDCo
force-pushed
the
chip/sqlite-experiment
branch
from
77ed5a8 to
b0cb68a
Compare
…ker (#100) This merges the work I've been doing on accessing SQLite from a web worker. This PR supercedes my earlier PR #74, since so much had changed in the main branch that it was easier to resync the changes in a new branch rather than to try to reconcile all the now irrelevant parts. Stuff that is still to do / stuff to note (a number of things are left undone in interest of moving things along, since they can be readily handled as separate issues/PRs): - This code creates a web worker that should eventually subsume the kernel, hence the file is called `kernel-worker.ts`. However, this does not replace the existing kernel (or `Kernel`) that's in the current source tree, but rather lives alongside it. The source file undoubtedly should be moved to the `kernel` package, and then split into the parts that go into the `Kernel` class and the parts that go into the script that launches the kernel (#57). - Messaging to and from the web worker is not yet integrated with our streams abstraction; for now I'm just using `postMessage` in the stupidest possible way. Integrating with streams proved cross-cutting enough with what I was already doing that I figured that was best handled in a separate pass. (#todo) - In the same vein, this would really like to have the rationalization of the messaging layer that @grypez is in the midst of, most especially more proper handling of reply messages. In particular, I found that the way we handle parameters is kind of sketchy, mostly because virtually the only operation we have that really uses parameters is `CapTpCall` and it is handled as a special case. Consequently I ended up having to add a case to the `isCommand` type guard to accept an array as one of the valid parameter types in order to accommodate the `kvSet` and `kvGet` kernel commands that I added. This is kind of a hack; while those two operations will go away, I expect there to be numerous commands to come that will really want proper parameter handling. (#94) - The database usage right now is just a simple key/value store that can be accessed via the console "API". This k/v thing is strictly for testing and validation purposes and should go away. However, inside `kernel-worker.ts` you'll find a bundle of preliminary TypeScript type defs for the state that the kernel should actually be storing. My plan is to generate corresponding database schemata as part of putting in the vat-to-vat message routing infrastructure (assuming nobody else gets to this first, which y'all have been amazingly good at doing). - In keeping with this being a PoC, there are no tests. Those will come when there's something worth testing.
Ref: #76
This is a snapshot of the things I did to get basic SQLite database access (via OPFS) working.
This includes beginning the relocation of the nominal kernel from the offscreen page into a web worker, along with some of the attendant message plumbing alterations. Note that while the message wiring to enable exercise and testing of the database functionality is present (in the form of a simple key/value store interface), the remainder of the console<->kernel interaction is incomplete as the pieces required to spawn and interact with vats in iframes involve additional offscreen page functionality to implement ICaaS (Iframe Creation as a Service). In the meantime, there remains some iframe-interacting kernel code copied from the offscreen page into the worker code and then commented out because the things it does don't work there. I'm committing these changes (as a draft) in this ugly intermediate state so people can get a look at the database functionality.
If we go the SQLite route I'd also like to provide a vastly more developer friendly SQLite API (where the developer I want to be most friendly to is me), most likely adopting wholesale the API provided by the npm
better-sqlite3package, whose innards I'm familiar with after porting it from Node to XS, but that's for later.