Use Case: Blockchain Data

[mikeal]

One critical use case is the replication of blockchain data.

Because of the way the data is encoded (each new block is added as a new root node, adding a layer of depth every cycle) the structure ends up being a very deep tree with fairly little data at each depth. Therefor, the bottleneck tends to be round-trip time as the requester doesn't have knowledge of the links at each depth until they receive each block.

A proposed solution has been asking for a indefinite number of blocks from a peer, potentially using an advanced selector interface (see IPLD Selectors).

Additional Concerns:

• Asking for an indefinite amount of data is always risky. At the very least there will need to be some kind of cancellation mechanism.
• Pulling the data from multiple peers is very difficult if you don't know the set of data you're asking for. Note: @jbenet commented that this complexity should be pushed up to the consumer of the API to deal with parallelism.
• Any local caches are irrelevant as the remote end won't know the cache state of the requester.

Action Items:

• Collect a few sample data sets for testing and modeling.
• Prioritize current blockers:
  • RPC interface for adding new replication methods.
  • IPLD Selector parser.

/cc @jbenet @Stebalien @vmx Did I miss anything?

[vmx]

Just to clarify that. A typical query would be like parent/parent/parent/.... So you know where the links are, but you don't know how deep you want to go. So the query is like parent{n} where n is the number of parents you want to traverse.

[mikeal]

So you know where the links are, but you don't know how deep you want to go.

Not necessarily. There's three I can see right away:

• /...(indefinite parent nodes)/Qmt(specific CID)
• /...(replicate the entire structure)
• /./././ replicate the top 4 parent paths

[vmx]

• /...(indefinite parent nodes)/Qmt(specific CID)

Is that one an "all parent up to the specific CID?

• /./././ replicate the top 4 parent paths

Isn't this just parent/parent/parent/parent resp. parent{4}?

[mikeal]

@vmx yes and yes :)

[mikeal]

In order to ship a working implementation quickly I'm starting to worry about using a full selector specification. The parser will take some time to write, but the really hard part is going to be writing logic in the requester to validate that each returned block is actually in the bounds of the selector provided.

If we can clearly identify the queries that need to be made then we could reduce the implementation work significantly.

[whyrusleeping]

A query I want to be able to make is: <blkhash>{/parent}100: Give me the chain of 100 blocks from this particular block hash. Note that in my specific usecase, a block may have multiple parents, but all of those blocks will have the same set of parents. The number should be (obviously) a parameter, and 'infinity' should be an option.

Another usecase (also for blockchain) that i'd really like supported, but is less critical, is a diff request:
"Give me the blocks that exist in X but not Y". This would be used to get updates to a state tree between blocks.

[vmx]

In order to ship a working implementation quickly I'm starting to worry about using a full selector specification. The parser will take some time to write, but the really hard part is going to be writing logic in the requester to validate that each returned block is actually in the bounds of the selector provided.

As outlined in the IPLD Selectors proposal, there's a binary and a human usable part of it. If we concentrate on the binary part, it shouldn't be too hard. The selectors mentioned here are already almost all of the ones outlined in the IPLD Selectors proposal anyway.

[Stebalien]

Pulling the data from multiple peers is very difficult if you don't know the set of data you're asking for. Note: @jbenet commented that this complexity should be pushed up to the consumer of the API to deal with parallelism.

So, the protocol shouldn't have to handle parallelism but neither should the user (ideally). One should be able to write a smart driver that either (a) splits a larger selector into bounded chunks or (b) simply cancels/kills requests for sub-trees from one peer when it finds a new peer able to serve that sub-tree.

Any local caches are irrelevant as the remote end won't know the cache state of the requester.

This isn't necessarily true.

1. The sender can try to send send sibling blocks between a parent and a child block. This gives the requester a chance to cancel the child.
2. The requester may know which subdags that are likely to appear within the dag they're requesting. They can encode this information in the selector.

In order to ship a working implementation quickly I'm starting to worry about using a full selector specification. The parser will take some time to write, but the really hard part is going to be writing logic in the requester to validate that each returned block is actually in the bounds of the selector provided.

The selector proposal is supposed to be extensible and upgradable (or should have been). We should be able to start with a small subset and add more features later (as long as peers advertise which selectors they support).

---

Other related issues we should keep in mind:

• IPLD Selector Thoughts ipfs/notes#272
• IPLD Selector ipfs/notes#12
• Session Proposal: Designing IPLD Selectors ipfs-inactive/2016-IPFS-Workshop-Lisbon#5

[mikeal]

Does cancellation need to be part of the RPC interface or is it sufficient to just disconnect the transport as a cancellation mechanism?

[mikeal]

The selector proposal is supposed to be extensible and upgradable (or should have been). We should be able to start with a small subset and add more features later (as long as peers advertise which selectors they support).

This is getting quite complicated as we're now adding a mechanism by which peers broadcast the parts of the selector spec they support and other peers can discover and alter their behavior based on what is supported.

If we wanted to ship this quickly we could just agree on the subset we are supporting for a specific RPC interface. If we add support for more later we could choose a new name for that interface and potentially add in the service discovery parts you're suggesting to avoid RPC method proliferation.

[Stebalien]

Does cancellation need to be part of the RPC interface or is it sufficient to just disconnect the transport as a cancellation mechanism?

By transport, do you mean close the stream or close the connection? We can't just kill the entire connection.

Note, an RPC interface may not play well with:

1. Streamed responses.
2. Returning blocks that match multiple selectors.

I wouldn't be hung up on an RPC interface. There's a reason we're using a "wantlist" approach in bitswap. Really, most "advanced" replication protocols

If we wanted to ship this quickly we could just agree on the subset we are supporting for a specific RPC interface. If we add support for more later we could choose a new name for that interface and potentially add in the service discovery parts you're suggesting to avoid RPC method proliferation.

The tricky part is that some selectors are compossible.

[mikeal]

By transport, do you mean close the stream or close the connection? We can't just kill the entire connection.

Sorry, I meant the stream.

Note, an RPC interface may not play well with:
Streamed responses.
Returning blocks that match multiple selectors.

Good point. We basically need a request -> stream of blocks as response, which is not a typical RPC system at all.

[rvagg]

closing for archival