From d4f87ffc8285fc6edd4c97c66f25be4f500bc104 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Sun, 4 Dec 2022 23:07:21 -0500 Subject: [PATCH 01/12] add content for rendezvous --- content/concepts/discovery/rendezvous.md | 64 ++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 content/concepts/discovery/rendezvous.md diff --git a/content/concepts/discovery/rendezvous.md b/content/concepts/discovery/rendezvous.md new file mode 100644 index 00000000..4bc95c00 --- /dev/null +++ b/content/concepts/discovery/rendezvous.md @@ -0,0 +1,64 @@ +--- +title : "Rendezvous" +description: "The rendezvous protocol can facilitate the routing and discovery of nodes in a peer-to-peer network using a common location." +weight: 223 +--- + +## What is Rendezvous? + +A rendezvous protocol is a routing protocol that enables nodes and resources +in a peer-to-peer network to discover each other. Rendezvous are used +as a common location to route between two routes. + +Rendezvous points are typically nodes that are well-connected and stable in the +network and can handle large amounts of traffic and data. They +serve as a hub for nodes to discover and connect. Rendezvous points +relay packets between other nodes by acting as a routing intermediary. + +## Rendezvous in LibP2P + +{{< alert icon="💡" context="info" text="The current rendezvous implementation replaces the initial ws-star-rendezvous implementation with rendezvous daemons and a fleet of p2p-circuit relays." />}} + +The LibP2P rendezvous protocol can be used for different use cases. It is used during bootstrap to discover +circuit relays that provide connectivity for browser nodes. Generally, a peer can use known rendezvous points +to find peers that provide critical network services. Rendezvous is also utilized throughout the lifetime +of an application for real-time peer discovery by registering and polling rendezvous points in a decentralized +manner. In an application-specific setting, rendezvous points are used to progressively discover peers that can +answer specific queries or host shards of content. + +The rendezvous protocol allows peers to connect to a rendezvous point and register their +presence by sending a `REGISTER` message containing their serialized peer record in one or more +namespaces. Any node implementing the rendezvous protocol can act as a rendezvous point, and +any peer can connect to a rendezvous point. However, only peers initiating a registration can +register themselves at the rendezvous point. + +By registering with the rendezvous point, peers allow for their discovery by other peers who query +the rendezvous point. The query may: + +- provide namespace(s), such as `test-app`; +- optionally provide a maximum number of peers to return; +- can include a cookie that is obtained from the response to a previous query which would only + contain registrations that weren't part of the previous response. + > This simplifies real-time discovery as it reduces the overhead of queried peers and allows for + > the pagination of query responses. + +There is a default peer registration lifetime of 2 hours. Peers can optionally specify the lifetime using a +TTL parameter in the `REGISTER` message, with an upper bound of 72hrs. + +The rendezvous protocol runs over LibP2P streams using the protocol ID `/rendezvous/1.0.0`. + + + +### Rendezvous and publish-subscribe + +For effective real-time discovery, rendezvous can be combined with [LibP2P pubsub](../messaging/pubsub/overview). At a basic level, rendezvous can bootstrap pubsub by discovering peers +subscribed to a topic. The rendezvous would be responsible for publishing packets, subscribing, or +unsubscribing from packet shapes. + +Pubsub can also be used as a mechanism for building rendezvous services, where a number +of rendezvous points can federate using pubsub for internal real-time distribution while still +providing a simple interface to clients. + + + +{{< alert icon="💡" context="note" text="See the rendezvous technical specification technical specification for more details." />}} From 9335edb4a51df9b317b3718db2438193c3f78901 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 5 Dec 2022 04:17:22 -0500 Subject: [PATCH 02/12] edits --- content/concepts/discovery/rendezvous.md | 56 ++++++++++++------------ 1 file changed, 29 insertions(+), 27 deletions(-) diff --git a/content/concepts/discovery/rendezvous.md b/content/concepts/discovery/rendezvous.md index 4bc95c00..c0534f06 100644 --- a/content/concepts/discovery/rendezvous.md +++ b/content/concepts/discovery/rendezvous.md @@ -7,33 +7,35 @@ weight: 223 ## What is Rendezvous? A rendezvous protocol is a routing protocol that enables nodes and resources -in a peer-to-peer network to discover each other. Rendezvous are used -as a common location to route between two routes. +in a peer-to-peer network to discover each other. Rendezvous is used +as a common location (point) to route between two routes. -Rendezvous points are typically nodes that are well-connected and stable in the -network and can handle large amounts of traffic and data. They -serve as a hub for nodes to discover and connect. Rendezvous points -relay packets between other nodes by acting as a routing intermediary. +Rendezvous points are typically nodes that are well-connected and stable in +a network and can handle large amounts of traffic and data. They +serve as a hub for nodes to discover and connect with the primary +responsibility of relaying packets between other nodes. -## Rendezvous in LibP2P +## Rendezvous in libp2p {{< alert icon="💡" context="info" text="The current rendezvous implementation replaces the initial ws-star-rendezvous implementation with rendezvous daemons and a fleet of p2p-circuit relays." />}} -The LibP2P rendezvous protocol can be used for different use cases. It is used during bootstrap to discover -circuit relays that provide connectivity for browser nodes. Generally, a peer can use known rendezvous points -to find peers that provide critical network services. Rendezvous is also utilized throughout the lifetime -of an application for real-time peer discovery by registering and polling rendezvous points in a decentralized -manner. In an application-specific setting, rendezvous points are used to progressively discover peers that can -answer specific queries or host shards of content. +The libp2p rendezvous protocol can be used for different use cases. It is used +during bootstrap to discover circuit relays that provide connectivity for browser +nodes. Generally, a peer can use known rendezvous points to find peers that provide +critical network services. Rendezvous is also used throughout the lifetime of +an application for real-time peer discovery by registering and polling rendezvous points +in a decentralized manner. In an application-specific setting, rendezvous points are +used to progressively discover peers that can answer specific queries or host shards of +content. -The rendezvous protocol allows peers to connect to a rendezvous point and register their -presence by sending a `REGISTER` message containing their serialized peer record in one or more -namespaces. Any node implementing the rendezvous protocol can act as a rendezvous point, and -any peer can connect to a rendezvous point. However, only peers initiating a registration can -register themselves at the rendezvous point. +The libp2p rendezvous protocol allows peers to connect to a rendezvous point and register +their presence by sending a `REGISTER` message containing their serialized peer record in +one or more namespaces. Any node implementing the rendezvous protocol can act as a rendezvous +point, and any peer can connect to a rendezvous point. However, only peers initiating a +registration can register themselves at a rendezvous point. -By registering with the rendezvous point, peers allow for their discovery by other peers who query -the rendezvous point. The query may: +By registering with a rendezvous point, peers allow for their discovery by other peers who +query the rendezvous point. The query may: - provide namespace(s), such as `test-app`; - optionally provide a maximum number of peers to return; @@ -42,18 +44,18 @@ the rendezvous point. The query may: > This simplifies real-time discovery as it reduces the overhead of queried peers and allows for > the pagination of query responses. -There is a default peer registration lifetime of 2 hours. Peers can optionally specify the lifetime using a -TTL parameter in the `REGISTER` message, with an upper bound of 72hrs. +There is a default peer registration lifetime of 2 hours. Peers can optionally specify the +lifetime using a TTL parameter in the `REGISTER` message, with an upper bound of 72 hours. -The rendezvous protocol runs over LibP2P streams using the protocol ID `/rendezvous/1.0.0`. +The rendezvous protocol runs over libp2p streams using the protocol ID `/rendezvous/1.0.0`. ### Rendezvous and publish-subscribe -For effective real-time discovery, rendezvous can be combined with [LibP2P pubsub](../messaging/pubsub/overview). At a basic level, rendezvous can bootstrap pubsub by discovering peers -subscribed to a topic. The rendezvous would be responsible for publishing packets, subscribing, or -unsubscribing from packet shapes. +For effective real-time discovery, rendezvous can be combined with [libp2p publish/subscribe](../messaging/pubsub/overview). At a basic level, rendezvous can bootstrap pubsub by discovering peers +subscribed to a topic. The rendezvous would be responsible for publishing packets, subscribing, +or unsubscribing from packet shapes. Pubsub can also be used as a mechanism for building rendezvous services, where a number of rendezvous points can federate using pubsub for internal real-time distribution while still @@ -61,4 +63,4 @@ providing a simple interface to clients. -{{< alert icon="💡" context="note" text="See the rendezvous technical specification technical specification for more details." />}} +{{< alert icon="💡" context="note" text="See the rendezvous technical specification for more details." />}} From 2b9ff86188aba52c98ed36fea5dec225175f7867 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 5 Dec 2022 04:25:26 -0500 Subject: [PATCH 03/12] init discovery section --- content/concepts/discovery/_index.md | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 content/concepts/discovery/_index.md diff --git a/content/concepts/discovery/_index.md b/content/concepts/discovery/_index.md new file mode 100644 index 00000000..ad808c78 --- /dev/null +++ b/content/concepts/discovery/_index.md @@ -0,0 +1,5 @@ +--- +title : "Discovery" +description: "Peer discovery protocols are used to discover and announce services to other peers." +weight: 7 +--- From 431d3b7a3b45beffc0ce553affa067b1055ef2a9 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 19 Dec 2022 17:25:37 -0500 Subject: [PATCH 04/12] Apply suggestions from code review Co-authored-by: Max Inden --- content/concepts/discovery/rendezvous.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/content/concepts/discovery/rendezvous.md b/content/concepts/discovery/rendezvous.md index c0534f06..d686099d 100644 --- a/content/concepts/discovery/rendezvous.md +++ b/content/concepts/discovery/rendezvous.md @@ -12,8 +12,7 @@ as a common location (point) to route between two routes. Rendezvous points are typically nodes that are well-connected and stable in a network and can handle large amounts of traffic and data. They -serve as a hub for nodes to discover and connect with the primary -responsibility of relaying packets between other nodes. +serve as a hub for nodes to discover. ## Rendezvous in libp2p @@ -22,7 +21,7 @@ responsibility of relaying packets between other nodes. The libp2p rendezvous protocol can be used for different use cases. It is used during bootstrap to discover circuit relays that provide connectivity for browser nodes. Generally, a peer can use known rendezvous points to find peers that provide -critical network services. Rendezvous is also used throughout the lifetime of +network services. Rendezvous is also used throughout the lifetime of an application for real-time peer discovery by registering and polling rendezvous points in a decentralized manner. In an application-specific setting, rendezvous points are used to progressively discover peers that can answer specific queries or host shards of From 2959233129c754063ec6c3aa3d51c95b8fb5837d Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Sun, 15 Jan 2023 13:31:32 -0500 Subject: [PATCH 05/12] rename dir and delete index --- .../concepts/{discovery => discovery-routing}/rendezvous.md | 0 content/concepts/discovery/_index.md | 5 ----- 2 files changed, 5 deletions(-) rename content/concepts/{discovery => discovery-routing}/rendezvous.md (100%) delete mode 100644 content/concepts/discovery/_index.md diff --git a/content/concepts/discovery/rendezvous.md b/content/concepts/discovery-routing/rendezvous.md similarity index 100% rename from content/concepts/discovery/rendezvous.md rename to content/concepts/discovery-routing/rendezvous.md diff --git a/content/concepts/discovery/_index.md b/content/concepts/discovery/_index.md deleted file mode 100644 index ad808c78..00000000 --- a/content/concepts/discovery/_index.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title : "Discovery" -description: "Peer discovery protocols are used to discover and announce services to other peers." -weight: 7 ---- From d5a180d32677bde89ec0efb0f79e43baf63121df Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Sun, 15 Jan 2023 13:35:30 -0500 Subject: [PATCH 06/12] incorporate PR feedback, rmv concept of real-time --- content/concepts/discovery-routing/rendezvous.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/content/concepts/discovery-routing/rendezvous.md b/content/concepts/discovery-routing/rendezvous.md index d686099d..cc363327 100644 --- a/content/concepts/discovery-routing/rendezvous.md +++ b/content/concepts/discovery-routing/rendezvous.md @@ -22,7 +22,7 @@ The libp2p rendezvous protocol can be used for different use cases. It is used during bootstrap to discover circuit relays that provide connectivity for browser nodes. Generally, a peer can use known rendezvous points to find peers that provide network services. Rendezvous is also used throughout the lifetime of -an application for real-time peer discovery by registering and polling rendezvous points +an application for peer discovery by registering and polling rendezvous points in a decentralized manner. In an application-specific setting, rendezvous points are used to progressively discover peers that can answer specific queries or host shards of content. @@ -40,7 +40,7 @@ query the rendezvous point. The query may: - optionally provide a maximum number of peers to return; - can include a cookie that is obtained from the response to a previous query which would only contain registrations that weren't part of the previous response. - > This simplifies real-time discovery as it reduces the overhead of queried peers and allows for + > This simplifies discovery as it reduces the overhead of queried peers and allows for > the pagination of query responses. There is a default peer registration lifetime of 2 hours. Peers can optionally specify the @@ -52,14 +52,14 @@ The rendezvous protocol runs over libp2p streams using the protocol ID `/rendezv ### Rendezvous and publish-subscribe -For effective real-time discovery, rendezvous can be combined with [libp2p publish/subscribe](../messaging/pubsub/overview). At a basic level, rendezvous can bootstrap pubsub by discovering peers -subscribed to a topic. The rendezvous would be responsible for publishing packets, subscribing, -or unsubscribing from packet shapes. +For effective discovery, rendezvous can be combined with [libp2p publish/subscribe](../messaging/pubsub/overview). +At a basic level, rendezvous can bootstrap pubsub by discovering peers subscribed to a topic. The rendezvous would +be responsible for publishing packets, subscribing, or unsubscribing from packet shapes. Pubsub can also be used as a mechanism for building rendezvous services, where a number -of rendezvous points can federate using pubsub for internal real-time distribution while still +of rendezvous points can federate using pubsub for internal distribution while still providing a simple interface to clients. - + {{< alert icon="💡" context="note" text="See the rendezvous technical specification for more details." />}} From d0d2bb1b026397512c2bb441a7c4f5fa297e2184 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Sun, 15 Jan 2023 13:49:07 -0500 Subject: [PATCH 07/12] incorporate PR feedback; indicate rendez not being decentralized --- .../concepts/discovery-routing/rendezvous.md | 24 +++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/content/concepts/discovery-routing/rendezvous.md b/content/concepts/discovery-routing/rendezvous.md index cc363327..534c7984 100644 --- a/content/concepts/discovery-routing/rendezvous.md +++ b/content/concepts/discovery-routing/rendezvous.md @@ -1,19 +1,39 @@ --- title : "Rendezvous" -description: "The rendezvous protocol can facilitate the routing and discovery of nodes in a peer-to-peer network using a common location." +description: "The rendezvous protocol can facilitate the routing and discovery of nodes in a P2P network using a common location." weight: 223 --- ## What is Rendezvous? A rendezvous protocol is a routing protocol that enables nodes and resources -in a peer-to-peer network to discover each other. Rendezvous is used +in a P2P network to discover each other. Rendezvous is used as a common location (point) to route between two routes. Rendezvous points are typically nodes that are well-connected and stable in a network and can handle large amounts of traffic and data. They serve as a hub for nodes to discover. +
+ Rendezvous is not decentralized + + It is important to note that Rendezvous is not decentralized but rather + federated. While this has its use cases, it also introduces a single + point of failure into the network. This can be contrasted with fully decentralized + solutions like DHT and Gossipsub. DHT (Distributed Hash Table) and Gossipsub are + decentralized alternatives to Rendezvous. + + [DHT](kaddht.md) is a distributed network protocol used to store and + retrieve data in a P2P network efficiently. It uses a hash table to map keys + to values, allowing for fast lookups and efficient data distribution across the network. + + [Gossipsub](pubsub.md), on the other hand, is a pub-sub (publish-subscribe) protocol + that is used to distribute messages and data across a network. It uses a gossip-based + mechanism to propagate messages throughout the network, allowing fast and efficient + distribution without relying on a central control point. + +
+ ## Rendezvous in libp2p {{< alert icon="💡" context="info" text="The current rendezvous implementation replaces the initial ws-star-rendezvous implementation with rendezvous daemons and a fleet of p2p-circuit relays." />}} From 79b66360c12bd51658c182af666e07e0ce90c711 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 13 Feb 2023 09:34:05 +0100 Subject: [PATCH 08/12] Remove "decentralized manner" as it is contrasted to it above --- content/concepts/discovery-routing/rendezvous.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/content/concepts/discovery-routing/rendezvous.md b/content/concepts/discovery-routing/rendezvous.md index 534c7984..a63402b0 100644 --- a/content/concepts/discovery-routing/rendezvous.md +++ b/content/concepts/discovery-routing/rendezvous.md @@ -39,12 +39,12 @@ serve as a hub for nodes to discover. {{< alert icon="💡" context="info" text="The current rendezvous implementation replaces the initial ws-star-rendezvous implementation with rendezvous daemons and a fleet of p2p-circuit relays." />}} The libp2p rendezvous protocol can be used for different use cases. It is used -during bootstrap to discover circuit relays that provide connectivity for browser -nodes. Generally, a peer can use known rendezvous points to find peers that provide -network services. Rendezvous is also used throughout the lifetime of -an application for peer discovery by registering and polling rendezvous points -in a decentralized manner. In an application-specific setting, rendezvous points are -used to progressively discover peers that can answer specific queries or host shards of +during bootstrap to discover circuit relays that provide connectivity for +browser nodes. Generally, a peer can use known rendezvous points to find peers +that provide network services. Rendezvous is also used throughout the lifetime +of an application for peer discovery by registering and polling rendezvous +points. In an application-specific setting, rendezvous points are used to +progressively discover peers that can answer specific queries or host shards of content. The libp2p rendezvous protocol allows peers to connect to a rendezvous point and register From 35e2012798591d4e58d2b3bef42b2b8e82650d8d Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 13 Feb 2023 09:34:35 +0100 Subject: [PATCH 09/12] Compare dht to hash map, don't state that they are equal --- content/concepts/discovery-routing/rendezvous.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/concepts/discovery-routing/rendezvous.md b/content/concepts/discovery-routing/rendezvous.md index a63402b0..2a6e0c07 100644 --- a/content/concepts/discovery-routing/rendezvous.md +++ b/content/concepts/discovery-routing/rendezvous.md @@ -24,7 +24,7 @@ serve as a hub for nodes to discover. decentralized alternatives to Rendezvous. [DHT](kaddht.md) is a distributed network protocol used to store and - retrieve data in a P2P network efficiently. It uses a hash table to map keys + retrieve data in a P2P network efficiently. It is like a hash table mapping keys to values, allowing for fast lookups and efficient data distribution across the network. [Gossipsub](pubsub.md), on the other hand, is a pub-sub (publish-subscribe) protocol From de3ce72f9442d9fe199ddc8da56dc76099b7fcb5 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 13 Feb 2023 09:36:22 +0100 Subject: [PATCH 10/12] Stress that rendezvous CAN do something, not that it does --- content/concepts/discovery-routing/rendezvous.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/content/concepts/discovery-routing/rendezvous.md b/content/concepts/discovery-routing/rendezvous.md index 2a6e0c07..2678b89b 100644 --- a/content/concepts/discovery-routing/rendezvous.md +++ b/content/concepts/discovery-routing/rendezvous.md @@ -38,14 +38,14 @@ serve as a hub for nodes to discover. {{< alert icon="💡" context="info" text="The current rendezvous implementation replaces the initial ws-star-rendezvous implementation with rendezvous daemons and a fleet of p2p-circuit relays." />}} -The libp2p rendezvous protocol can be used for different use cases. It is used -during bootstrap to discover circuit relays that provide connectivity for -browser nodes. Generally, a peer can use known rendezvous points to find peers -that provide network services. Rendezvous is also used throughout the lifetime -of an application for peer discovery by registering and polling rendezvous -points. In an application-specific setting, rendezvous points are used to -progressively discover peers that can answer specific queries or host shards of -content. +The libp2p rendezvous protocol can be used for different use cases. E.g. it can +be used during bootstrap to discover circuit relays that provide connectivity +for browser nodes. Generally, a peer can use known rendezvous points to find +peers that provide network services. Rendezvous is also used throughout the +lifetime of an application for peer discovery by registering and polling +rendezvous points. In an application-specific setting, rendezvous points can be +used to progressively discover peers that can answer specific queries or host +shards of content. The libp2p rendezvous protocol allows peers to connect to a rendezvous point and register their presence by sending a `REGISTER` message containing their serialized peer record in From 18cf692734ee7c9fe849cd8f5a2f47f950c976a4 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 13 Feb 2023 09:37:06 +0100 Subject: [PATCH 11/12] Serialized peer record is irrelevant here --- content/concepts/discovery-routing/rendezvous.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/concepts/discovery-routing/rendezvous.md b/content/concepts/discovery-routing/rendezvous.md index 2678b89b..0e8c722b 100644 --- a/content/concepts/discovery-routing/rendezvous.md +++ b/content/concepts/discovery-routing/rendezvous.md @@ -47,11 +47,11 @@ rendezvous points. In an application-specific setting, rendezvous points can be used to progressively discover peers that can answer specific queries or host shards of content. -The libp2p rendezvous protocol allows peers to connect to a rendezvous point and register -their presence by sending a `REGISTER` message containing their serialized peer record in -one or more namespaces. Any node implementing the rendezvous protocol can act as a rendezvous -point, and any peer can connect to a rendezvous point. However, only peers initiating a -registration can register themselves at a rendezvous point. +The libp2p rendezvous protocol allows peers to connect to a rendezvous point and +register their presence by sending a `REGISTER` message in one or more +namespaces. Any node implementing the rendezvous protocol can act as a +rendezvous point, and any peer can connect to a rendezvous point. However, only +peers initiating a registration can register themselves at a rendezvous point. By registering with a rendezvous point, peers allow for their discovery by other peers who query the rendezvous point. The query may: From c9537ca09b1a84d06858de18656cff00c238d60a Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 13 Feb 2023 09:38:37 +0100 Subject: [PATCH 12/12] Reword cookie --- content/concepts/discovery-routing/rendezvous.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/content/concepts/discovery-routing/rendezvous.md b/content/concepts/discovery-routing/rendezvous.md index 0e8c722b..ed7a75df 100644 --- a/content/concepts/discovery-routing/rendezvous.md +++ b/content/concepts/discovery-routing/rendezvous.md @@ -58,8 +58,9 @@ query the rendezvous point. The query may: - provide namespace(s), such as `test-app`; - optionally provide a maximum number of peers to return; -- can include a cookie that is obtained from the response to a previous query which would only - contain registrations that weren't part of the previous response. +- can include a cookie that is obtained from the response to a previous query, + thus the current query only contain registrations that weren't part of the + previous response. > This simplifies discovery as it reduces the overhead of queried peers and allows for > the pagination of query responses.