Proposal: Replication RPC

[mikeal]

This is a proposal to separate some of the concerns in replication and give a new path towards building more replicators for IPLD.

Currently, we are proposing dueling specifications for a new replicator. ipld/specs#78 ipld/specs#66 ipld/specs#75. Each specification contains requirements or implications for transport changes, new APIs, new graph traversal logic, and in some cases new storage level changes or at the very least addition indexes on existing storage.

Instead, I'd like to break this problem into more re-usable parts that we can implement incrementally.

Proposal

1. Define simple requirements for an RPC system.
   • This should only lay out our requirements, not an entire implementation. People should be able to take existing RPC systems that meet these requirements and implement the interfaces we specify.
2. Using the language defined above we propose specific RPC interfaces for improving replication.
   • Examples:
     • Proposal: Graphsync (C) specs#78 would be written as a proposal for two new interfaces: get-merkle-proof-for-path and get-multiple-blocks.
     • Proposal: GraphSync (B) specs#75 would be written as a proposal for a single new interface: get-ipld-selector and would likely reference an additional specification for IPLD Selectors.

This would allow us then build and iterate on replication logic much faster using a combination of new and old interfaces as well as any enhancements we might be able to make on top of different storage engines or even by layering on entirely new transports and discovery schemes.

[rvagg]

closing for archival