refactor: Begin using JSON-RPC for internal messages

[rekmarks]

Ref: #281 #369

This is the first step toward simplifying our internal message types. We will accomplish this simplifications by making our transports dumber, in that their only expectation is that every message is a valid JSON-RPC message. All other validation will be relegated to the RPC call and method implementation sites.

MetaMask itself uses JSON-RPC for all of its internal (and most external) messages. To this end, it has a mature stack for handling JSON-RPC messages and requests/responses, centered around @metamask/json-rpc-engine. Here, we introduce JsonRpcEngine and JSON-RPC for UI <-> kernel messages only. This will grow to include all internal messages.

JsonRpcEngine is somewhat idiosyncratic (read: has some dumb stuff we'd like to remove), but is ultimately pretty straightforward to work with. See its documentation for details. One of the main implications is for our message types. Where they previously looked e.g. like this:

// Request
{
  "method": "ping",
  "params": null,
}

// Response
{
  "method": "ping",
  "params": "pong", // or error
}

They will now look like this:

// Request
{
  "id": "0xdeadbeef", // string or number, we'll always use strings for simplicity
  "jsonrpc": "2.0", // required by spec
  "method": "ping",
  "params": [], // "params" must be a "structured value", i.e. object or array
}

// Success response
{
  "id": "0xdeadbeef", // or whatever
  "jsonrpc": "2.0", // required by spec
  "result": "pong",
}

// Error response
{
  "id": "0xdeadbeef", // or whatever
  "jsonrpc": "2.0", // required by spec
  "error": { ... },
}

Ultimately, callers and method implementers will never have to concern themselves with anything other than the method and params for calls, and result / error for responses. The biggest differences from our existing types are that:

• params must be either an array or an object—think positional or named—and can therefore not be null.
• responses must have a result or error property, never params.

This PR will be followed immediately by the next pipeline to receive the JSON-RPC treatment, and culminate in a rewrite of our RPC method implementations and their types.

[FUDCo]

If there are no bundles indirectly referenced in any of the individual vat configs (which is actually a special case that doesn't happen that often), the bundles property should simply be absent, but I don't think there is any case where it is appropriate for it to have the value null. I noticed that you changed the schema from optional to nullable, but optional is really the correct thing. Since it's almost always absent, requiring it would be an ergonomic irritation for most use cases. What prompted this?

[rekmarks]

I agree it's an ergonomic irritation, but this is another consequence of old TypeScript limitations around optional properties that we live with to this day via superstruct. Specifically, if you have a struct with property foo: optional(string), the resulting type will be foo?: string | undefined as opposed to foo?: string. Because JSON has no notion of undefined, our Json type forbids it, and optional properties on objects that will extend or be constrained to Json have to use nullable() instead of optional().

[rekmarks]

It's possible that this can be improved by adding a strictOptional struct to @metamask/superstruct. I'll try independently of this PR and undo the unergonomic-ness if so.

[FUDCo]

I genuinely don't understand the constraint with optional properties. It shouldn't require you to JSON encode a value of undefined (which you correctly point out that JSON can't do) but simply omit the property entirely from the object.

[rekmarks]

It used to be the case that TypeScript's notion of optional properties amounted to "the property can either be absent or set to undefined". superstruct has inherited this limitation, and does not in its current form support truly optional properties; whenever you use optional(), it permits the property to either be absent or set to undefined. Our own Json type is not so permissive; it does not permit the value undefined in any form. Because ClusterConfig is used in our messages, and our messages are as of this PR constrained to Json, ClusterConfig can no longer use optional(). However, I am trying to fix this upstream.

[FUDCo]

JS's builtin JSON serialization is totally happy with undefined properties. If you JSON.stringify an object with a property whose value is undefined, it simply leaves the property out of the resulting JSON string. So it sounds to me like our Json type is just buggy. Is correcting that the upstream fix you're talking about?

[FUDCo]

My vat restart PR #448 gets rid of this ping entirely.

[rekmarks]

Word, I'll delete it here too then.

[sirtimid]

LGTM