From 0e090a658fdb0966d5f743a945046d520d05933a Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Sun, 15 Jan 2023 10:40:33 -0500 Subject: [PATCH 01/12] populate kaddht doc --- .../concepts/introduction/protocols/dht.md | 80 ++++++++++++++++++- 1 file changed, 77 insertions(+), 3 deletions(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index 74ce9109..59351cf7 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -1,9 +1,83 @@ --- title: "Kademlia DHT" -description: "" +description: "Kad-DHT is subsystem is an implementation of the Kademlia +DHT, a distributed hash table that is designed for peer-to-peer networks." weight: 25 --- -Coming soon! +## Overview -For now, check out the following [Launchpad guide on the DHT](https://curriculum.pl-launchpad.io/curriculum/libp2p/dht/). +The Kademlia Distributed Hash Table (DHT), or Kad-DHT, is a distributed hash table +that is designed for peer-to-peer networks. + +Kad-DHT in libp2p is a subsystem based on the +[Kademlia whitepaper](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf) +and is augmented with other systems, such as +[S/Kademlia](https://attachment.victorlampcdn.com/article/content/20220705/SKademlia_2007.pdf), +[Coral](https://www.cs.princeton.edu/~mfreed/docs/coral-iptps03.pdf), and the +[BitTorrent DHT](https://www.bittorrent.org/beps/bep_0005.html). + +Kad-DHT offers a way to find and manage nodes and data on the network by using a +[routing table](https://en.wikipedia.org/wiki/Routing_table) that organizes peers based on how similar +their keys are. + +
+ A look deeper + + The routing table is organized based on a prefix length and a distance metric. The prefix length helps + to group similar keys, and the distance metric helps to find the closest peers to a specific key + in the routing table. The table maintains a list of k closest peers for each possible prefix length between + 0 and L-1, where L is the length of the keyspace, determined by the length of the hash function used. + Kad-DHT uses SHA-256, meaning the keyspace is 256 bits. In other words, the routing table will try to + maintain k peers with a shared key prefix for every possible prefix length between 0 and 255. + + The prefix length measures how similar two keys are in the routing table. It is determined by + counting the number of leading bits that are the same in the two keys' SHA-256 hash. The more leading + bits that are the same, the shorter the prefix length and the more similar the keys are considered to be. + + The distance metric is a way to calculate the distance between two keys by taking the bitwise + exclusive-or (XOR) of the SHA-256 hash of the two keys. The resulting hash is a measure of the distance + between the two keys, where a distance of 0 means the keys are identical, and a distance of 1 means that + only one bit is different. + + This allows for efficient and effective lookups in the routing table when trying to find nodes or data that + share similar prefixes. + +
+ +## Peer discovery + +The Kad-DHT uses a process called "peer routing" to discover new nodes in the network. This process starts by generating +a random peer ID and looking it up via the routing table. The node then contacts the k closest nodes to the peer ID and +repeats the process until it finds the peer or determines that it is not in the network. Nodes also adds +any new nodes they discovers to their routing table to improve its awareness of the network. This process can also +find nodes with a specific key prefix. Check out the Kad-DHT discovery document for more information. + +## Content provider discovery + +Kad-DHT also includes a feature for content provider discovery, where nodes can look up providers for a given key. This is +done by sending an RPC message to the k closest nodes to the key and collecting the responses. The node then returns the list +of providers it has discovered. Check out the Kad-DHT content routing document for more information. + +## Bootstrap process + +To maintain a healthy routing table and discover new network nodes, the Kad-DHT includes a bootstrap process that +runs periodically. The process starts by generating a random peer ID and looking it up via the peer routing process. The +node then adds the closest peers it discovers to its routing table and repeats the process multiple times. This process +also includes a mechanism to look up its peer ID to improve its awareness of nodes close to itself. + +## Client and server mode + +The Kad-DHT in libp2p has a concept of a "client" and "server" mode. A node can operate in one of the modes, depending on +the characteristics of the network topology and the properties of the DHT. For example, publicly routable nodes can +operate in server mode, while non-publicly routable nodes can operate in client mode. The distinction allows restricted +nodes to utilize the DHT, i.e., query the DHT, without decreasing the quality of the distributed hash table. + +## RPC messages + +The Kad-DHT in libp2p also includes various RPC messages for performing operations on the DHT, such as `PUT_VALUE`, `GET_VALUE`, +`ADD_PROVIDER`, and `GET_PROVIDERS`. These messages are used for storing and retrieving key-value pairs and finding providers +for a given key. These messages are sent over a stream, and all messages sent over a stream are prefixed with the message length +in bytes. + +{{< alert icon="💡" context="note" text="See the Kademlia DHT technical specification for more details." />}} From 56aa21da724bdffb5f5bde98d2cc0c33b08e2139 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Tue, 17 Jan 2023 05:55:18 -0500 Subject: [PATCH 02/12] edits --- .../concepts/introduction/protocols/dht.md | 98 +++++++++++-------- 1 file changed, 57 insertions(+), 41 deletions(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index 59351cf7..de36c289 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -1,14 +1,14 @@ --- title: "Kademlia DHT" description: "Kad-DHT is subsystem is an implementation of the Kademlia -DHT, a distributed hash table that is designed for peer-to-peer networks." +DHT, a distributed hash table that is designed for P2P networks." weight: 25 --- ## Overview The Kademlia Distributed Hash Table (DHT), or Kad-DHT, is a distributed hash table -that is designed for peer-to-peer networks. +that is designed for P2P networks. Kad-DHT in libp2p is a subsystem based on the [Kademlia whitepaper](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf) @@ -18,66 +18,82 @@ and is augmented with other systems, such as [BitTorrent DHT](https://www.bittorrent.org/beps/bep_0005.html). Kad-DHT offers a way to find and manage nodes and data on the network by using a -[routing table](https://en.wikipedia.org/wiki/Routing_table) that organizes peers based on how similar -their keys are. +[routing table](https://en.wikipedia.org/wiki/Routing_table) that organizes peers based +on how similar their keys are.
A look deeper - The routing table is organized based on a prefix length and a distance metric. The prefix length helps - to group similar keys, and the distance metric helps to find the closest peers to a specific key - in the routing table. The table maintains a list of k closest peers for each possible prefix length between - 0 and L-1, where L is the length of the keyspace, determined by the length of the hash function used. - Kad-DHT uses SHA-256, meaning the keyspace is 256 bits. In other words, the routing table will try to - maintain k peers with a shared key prefix for every possible prefix length between 0 and 255. - - The prefix length measures how similar two keys are in the routing table. It is determined by - counting the number of leading bits that are the same in the two keys' SHA-256 hash. The more leading - bits that are the same, the shorter the prefix length and the more similar the keys are considered to be. - - The distance metric is a way to calculate the distance between two keys by taking the bitwise - exclusive-or (XOR) of the SHA-256 hash of the two keys. The resulting hash is a measure of the distance - between the two keys, where a distance of 0 means the keys are identical, and a distance of 1 means that - only one bit is different. - - This allows for efficient and effective lookups in the routing table when trying to find nodes or data that - share similar prefixes. + The routing table is organized based on a prefix length and a distance metric. + The prefix length helps to group similar keys, and the distance metric helps to + find the closest peers to a specific key in the routing table. The table maintains + a list of `k` closest peers for each possible prefix length between `0` and `L-1`, + where `L` is the length of the keyspace, determined by the length of the hash + function used. **Kad-DHT uses SHA-256**, with a keyspace of 256 bits, maintaining + `k` peers with a shared key prefix for every prefix length between `0` and `255` in + its routing table. + + The prefix length measures the proximity of two keys in the routing table and + divides the keyspace into smaller subspaces, called "buckets", each containing nodes + that share a common prefix of bits in their SHA-256 hash. The prefix length is the + number of bits that are the same in the two keys' SHA-256 hash. The more leading bits + that are the same, the shorter the prefix length and the closer the proximity of the + two keys are considered to be. + + The distance metric is a way to calculate the distance between two keys by taking + the bitwise exclusive-or (XOR) of the SHA-256 hash of the two keys. The resulting + hash is a measure of the distance between the two keys, where a distance of `0` means + the keys are identical, and a distance of `1` means that only one bit is different, + meaning the two keys are close to each other (i.e. their SHA-256 hashes are similar). + + This allows for efficient and effective lookups in the routing table when trying + to find nodes or data that share similar prefixes.
## Peer discovery -The Kad-DHT uses a process called "peer routing" to discover new nodes in the network. This process starts by generating -a random peer ID and looking it up via the routing table. The node then contacts the k closest nodes to the peer ID and -repeats the process until it finds the peer or determines that it is not in the network. Nodes also adds -any new nodes they discovers to their routing table to improve its awareness of the network. This process can also -find nodes with a specific key prefix. Check out the Kad-DHT discovery document for more information. +The Kad-DHT uses a process called "peer routing" to discover new nodes in the network. +This process starts by generating a random peer ID and looking it up via the routing +table. The node then contacts the k closest nodes to the peer ID and repeats the process +until it finds the peer or determines that it is not in the network. Nodes also adds any +new nodes they discovers to their routing table to improve its awareness of the network. + +This process can also find nodes with a specific key prefix. Check out the Kad-DHT +discovery document for more information. + + ## Content provider discovery -Kad-DHT also includes a feature for content provider discovery, where nodes can look up providers for a given key. This is -done by sending an RPC message to the k closest nodes to the key and collecting the responses. The node then returns the list -of providers it has discovered. Check out the Kad-DHT content routing document for more information. +Kad-DHT also includes a feature for content provider discovery, where nodes can look up +providers for a given key. This is done by sending an RPC message to the k closest nodes +to the key and collecting the responses. The node then returns the list of providers it +has discovered. Check out the [Kad-DHT content routing document](../../content-routing/kaddht.md) +for more information. ## Bootstrap process -To maintain a healthy routing table and discover new network nodes, the Kad-DHT includes a bootstrap process that -runs periodically. The process starts by generating a random peer ID and looking it up via the peer routing process. The -node then adds the closest peers it discovers to its routing table and repeats the process multiple times. This process -also includes a mechanism to look up its peer ID to improve its awareness of nodes close to itself. +To maintain a healthy routing table and discover new network nodes, the Kad-DHT includes +a bootstrap process that runs periodically. The process starts by generating a random peer +ID and looking it up via the peer routing process. The node then adds the closest peers it +discovers to its routing table and repeats the process multiple times. This process also +includes a mechanism to look up its peer ID to improve its awareness of nodes close to itself. ## Client and server mode -The Kad-DHT in libp2p has a concept of a "client" and "server" mode. A node can operate in one of the modes, depending on -the characteristics of the network topology and the properties of the DHT. For example, publicly routable nodes can -operate in server mode, while non-publicly routable nodes can operate in client mode. The distinction allows restricted -nodes to utilize the DHT, i.e., query the DHT, without decreasing the quality of the distributed hash table. +The Kad-DHT in libp2p has a concept of a "client" and "server" mode. A node can operate in +one of the modes, depending on the characteristics of the network topology and the properties +of the DHT. For example, publicly routable nodes can operate in server mode, while non-publicly +routable nodes can operate in client mode. The distinction allows restricted nodes to utilize +the DHT, i.e., query the DHT, without decreasing the quality of the distributed hash table. ## RPC messages -The Kad-DHT in libp2p also includes various RPC messages for performing operations on the DHT, such as `PUT_VALUE`, `GET_VALUE`, -`ADD_PROVIDER`, and `GET_PROVIDERS`. These messages are used for storing and retrieving key-value pairs and finding providers -for a given key. These messages are sent over a stream, and all messages sent over a stream are prefixed with the message length +The Kad-DHT in libp2p also includes various RPC messages for performing operations on the DHT, +such as `PUT_VALUE`, `GET_VALUE`, `ADD_PROVIDER`, and `GET_PROVIDERS`. These messages are used +for storing and retrieving key-value pairs and finding providers for a given key. These messages +are sent over a stream, and all messages sent over a stream are prefixed with the message length in bytes. {{< alert icon="💡" context="note" text="See the Kademlia DHT technical specification for more details." />}} From 32da302166f8dbcc786b071dc6dd0b5fddbd8336 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 23 Jan 2023 06:39:38 -0500 Subject: [PATCH 03/12] Apply suggestions from code review Co-authored-by: Max Inden --- content/concepts/introduction/protocols/dht.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index de36c289..a8e8d6a9 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -1,6 +1,6 @@ --- title: "Kademlia DHT" -description: "Kad-DHT is subsystem is an implementation of the Kademlia +description: "Kad-DHT subsystem is an implementation of the Kademlia DHT, a distributed hash table that is designed for P2P networks." weight: 25 --- @@ -56,8 +56,8 @@ on how similar their keys are. The Kad-DHT uses a process called "peer routing" to discover new nodes in the network. This process starts by generating a random peer ID and looking it up via the routing table. The node then contacts the k closest nodes to the peer ID and repeats the process -until it finds the peer or determines that it is not in the network. Nodes also adds any -new nodes they discovers to their routing table to improve its awareness of the network. +until it finds the peer or determines that it is not in the network. Nodes also add any +new nodes they discover to their routing table to improve its awareness of the network. This process can also find nodes with a specific key prefix. Check out the Kad-DHT discovery document for more information. From ff44c0fb0c24abecb6a1a1bbeff9a4477be89ac3 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 23 Jan 2023 08:34:24 -0500 Subject: [PATCH 04/12] incorporate PR feedback; edits --- .../concepts/introduction/protocols/dht.md | 30 +++++++------------ 1 file changed, 11 insertions(+), 19 deletions(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index a8e8d6a9..4a7e30d1 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -1,6 +1,6 @@ --- title: "Kademlia DHT" -description: "Kad-DHT subsystem is an implementation of the Kademlia +description: "The libp2p Kad-DHT subsystem is an implementation of the Kademlia DHT, a distributed hash table that is designed for P2P networks." weight: 25 --- @@ -13,8 +13,7 @@ that is designed for P2P networks. Kad-DHT in libp2p is a subsystem based on the [Kademlia whitepaper](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf) and is augmented with other systems, such as -[S/Kademlia](https://attachment.victorlampcdn.com/article/content/20220705/SKademlia_2007.pdf), -[Coral](https://www.cs.princeton.edu/~mfreed/docs/coral-iptps03.pdf), and the +[Coral](https://www.cs.princeton.edu/~mfreed/docs/coral-iptps03.pdf) and the [BitTorrent DHT](https://www.bittorrent.org/beps/bep_0005.html). Kad-DHT offers a way to find and manage nodes and data on the network by using a @@ -22,7 +21,7 @@ Kad-DHT offers a way to find and manage nodes and data on the network by using a on how similar their keys are.
- A look deeper + A deeper look The routing table is organized based on a prefix length and a distance metric. The prefix length helps to group similar keys, and the distance metric helps to @@ -46,8 +45,8 @@ on how similar their keys are. the keys are identical, and a distance of `1` means that only one bit is different, meaning the two keys are close to each other (i.e. their SHA-256 hashes are similar). - This allows for efficient and effective lookups in the routing table when trying - to find nodes or data that share similar prefixes. + This design allows for efficient and effective lookups in the routing table when + trying to find nodes or data that share similar prefixes.
@@ -59,18 +58,13 @@ table. The node then contacts the k closest nodes to the peer ID and repeats the until it finds the peer or determines that it is not in the network. Nodes also add any new nodes they discover to their routing table to improve its awareness of the network. -This process can also find nodes with a specific key prefix. Check out the Kad-DHT -discovery document for more information. - - - ## Content provider discovery Kad-DHT also includes a feature for content provider discovery, where nodes can look up -providers for a given key. This is done by sending an RPC message to the k closest nodes -to the key and collecting the responses. The node then returns the list of providers it -has discovered. Check out the [Kad-DHT content routing document](../../content-routing/kaddht.md) -for more information. +providers for a given key. This is done by sending an [RPC message](#rpc-messages) (which uses a +key/value API) to the `k` closest nodes to the key and collecting the responses. The node then +returns the list of providers it has discovered. Check out the +[Kad-DHT content routing document](../../content-routing/kaddht.md) for more information. ## Bootstrap process @@ -90,10 +84,8 @@ the DHT, i.e., query the DHT, without decreasing the quality of the distributed ## RPC messages -The Kad-DHT in libp2p also includes various RPC messages for performing operations on the DHT, +There are various RPC messages for performing operations on the DHT, such as `PUT_VALUE`, `GET_VALUE`, `ADD_PROVIDER`, and `GET_PROVIDERS`. These messages are used -for storing and retrieving key-value pairs and finding providers for a given key. These messages -are sent over a stream, and all messages sent over a stream are prefixed with the message length -in bytes. +for storing and retrieving key-value pairs and finding providers for a given key. {{< alert icon="💡" context="note" text="See the Kademlia DHT technical specification for more details." />}} From beb9021acb9dc3001d13b47e9848c62245c27fd9 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 10 Feb 2023 15:34:06 +0100 Subject: [PATCH 05/12] Name as result not hash --- content/concepts/introduction/protocols/dht.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index 4a7e30d1..a1767a88 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -39,11 +39,12 @@ on how similar their keys are. that are the same, the shorter the prefix length and the closer the proximity of the two keys are considered to be. - The distance metric is a way to calculate the distance between two keys by taking - the bitwise exclusive-or (XOR) of the SHA-256 hash of the two keys. The resulting - hash is a measure of the distance between the two keys, where a distance of `0` means - the keys are identical, and a distance of `1` means that only one bit is different, - meaning the two keys are close to each other (i.e. their SHA-256 hashes are similar). + The distance metric is a way to calculate the distance between two keys by + taking the bitwise exclusive-or (XOR) of the SHA-256 hash of the two keys. The + result is a measure of the distance between the two keys, where a distance of + `0` means the keys are identical, and a distance of `1` means that only one + bit is different, meaning the two keys are close to each other (i.e. their + SHA-256 hashes are similar). This design allows for efficient and effective lookups in the routing table when trying to find nodes or data that share similar prefixes. From 4231b62cd49b675c0d145a3b265cea3f11eae7dc Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 10 Feb 2023 15:37:21 +0100 Subject: [PATCH 06/12] Drop reference to other DHTs The majority of our implementations is still based on vanilla Kademlia. Thus mentioning the others might just confuse users. --- content/concepts/introduction/protocols/dht.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index a1767a88..868bf1c6 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -11,10 +11,7 @@ The Kademlia Distributed Hash Table (DHT), or Kad-DHT, is a distributed hash tab that is designed for P2P networks. Kad-DHT in libp2p is a subsystem based on the -[Kademlia whitepaper](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf) -and is augmented with other systems, such as -[Coral](https://www.cs.princeton.edu/~mfreed/docs/coral-iptps03.pdf) and the -[BitTorrent DHT](https://www.bittorrent.org/beps/bep_0005.html). +[Kademlia whitepaper](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf). Kad-DHT offers a way to find and manage nodes and data on the network by using a [routing table](https://en.wikipedia.org/wiki/Routing_table) that organizes peers based From fe88713ca7d541a279ccd806f842c5064d0645cb Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 10 Feb 2023 15:38:43 +0100 Subject: [PATCH 07/12] Drop the "management" of nodes --- content/concepts/introduction/protocols/dht.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index 868bf1c6..b88e5ac0 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -13,7 +13,7 @@ that is designed for P2P networks. Kad-DHT in libp2p is a subsystem based on the [Kademlia whitepaper](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf). -Kad-DHT offers a way to find and manage nodes and data on the network by using a +Kad-DHT offers a way to find nodes and data on the network by using a [routing table](https://en.wikipedia.org/wiki/Routing_table) that organizes peers based on how similar their keys are. From 73747d27a255a7a90411792e28ad226d50c572ee Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 10 Feb 2023 15:48:46 +0100 Subject: [PATCH 08/12] Reword peer and provider routing and remove client/server mode The latter is an advanced topic. I don't think it is useful for newcomers. --- .../concepts/introduction/protocols/dht.md | 35 +++++++------------ 1 file changed, 13 insertions(+), 22 deletions(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index b88e5ac0..e7630039 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -48,37 +48,28 @@ on how similar their keys are. -## Peer discovery +## Peer routing -The Kad-DHT uses a process called "peer routing" to discover new nodes in the network. -This process starts by generating a random peer ID and looking it up via the routing -table. The node then contacts the k closest nodes to the peer ID and repeats the process -until it finds the peer or determines that it is not in the network. Nodes also add any -new nodes they discover to their routing table to improve its awareness of the network. +The Kad-DHT uses a process called "peer routing" to discover nodes in the +network. When looking for a peer, the local node contacts the `k` closest nodes to +the remote peer's ID asking them for closer nodes. The local node repeats the +process until it finds the peer or determines that it is not in the network. -## Content provider discovery +## Content provider routing -Kad-DHT also includes a feature for content provider discovery, where nodes can look up -providers for a given key. This is done by sending an [RPC message](#rpc-messages) (which uses a -key/value API) to the `k` closest nodes to the key and collecting the responses. The node then -returns the list of providers it has discovered. Check out the -[Kad-DHT content routing document](../../content-routing/kaddht.md) for more information. +Kad-DHT also includes a feature for content provider discovery, where nodes can +look up providers for a given key. The local node again contacts the `k` closest +nodes to the key asking them for either providers of the key and/or closer nodes +to the key. The local node repeats the process until it finds providers for the +key or determines that it is not in the network. ## Bootstrap process -To maintain a healthy routing table and discover new network nodes, the Kad-DHT includes +To maintain a healthy routing table and discover new nodes, the Kad-DHT includes a bootstrap process that runs periodically. The process starts by generating a random peer ID and looking it up via the peer routing process. The node then adds the closest peers it discovers to its routing table and repeats the process multiple times. This process also -includes a mechanism to look up its peer ID to improve its awareness of nodes close to itself. - -## Client and server mode - -The Kad-DHT in libp2p has a concept of a "client" and "server" mode. A node can operate in -one of the modes, depending on the characteristics of the network topology and the properties -of the DHT. For example, publicly routable nodes can operate in server mode, while non-publicly -routable nodes can operate in client mode. The distinction allows restricted nodes to utilize -the DHT, i.e., query the DHT, without decreasing the quality of the distributed hash table. +includes looking up its own peer ID to improve awareness of nodes close to itself. ## RPC messages From 0cc16971ae71552687b775aea71b42f7ae9ec197 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 10 Feb 2023 15:49:34 +0100 Subject: [PATCH 09/12] Remove RPC message section I would consider this a technical detail not interesting for newcomers. --- content/concepts/introduction/protocols/dht.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index e7630039..2acde010 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -71,10 +71,4 @@ ID and looking it up via the peer routing process. The node then adds the closes discovers to its routing table and repeats the process multiple times. This process also includes looking up its own peer ID to improve awareness of nodes close to itself. -## RPC messages - -There are various RPC messages for performing operations on the DHT, -such as `PUT_VALUE`, `GET_VALUE`, `ADD_PROVIDER`, and `GET_PROVIDERS`. These messages are used -for storing and retrieving key-value pairs and finding providers for a given key. - {{< alert icon="💡" context="note" text="See the Kademlia DHT technical specification for more details." />}} From 61b701168d0925c1c6f7f9f7d9296675afdd0ede Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 10 Feb 2023 15:50:44 +0100 Subject: [PATCH 10/12] Stress that the majority of buckets is always empty --- content/concepts/introduction/protocols/dht.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index 2acde010..a7c2638d 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -25,7 +25,7 @@ on how similar their keys are. find the closest peers to a specific key in the routing table. The table maintains a list of `k` closest peers for each possible prefix length between `0` and `L-1`, where `L` is the length of the keyspace, determined by the length of the hash - function used. **Kad-DHT uses SHA-256**, with a keyspace of 256 bits, maintaining + function used. **Kad-DHT uses SHA-256**, with a keyspace of 256 bits, trying to maintain `k` peers with a shared key prefix for every prefix length between `0` and `255` in its routing table. From 1b5de71c50a1c2e0f084f99d8c2a7db9efe493d7 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 10 Feb 2023 15:52:31 +0100 Subject: [PATCH 11/12] Typo on prefix length being disproportional to closeness --- content/concepts/introduction/protocols/dht.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index a7c2638d..42adc1d4 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -33,7 +33,7 @@ on how similar their keys are. divides the keyspace into smaller subspaces, called "buckets", each containing nodes that share a common prefix of bits in their SHA-256 hash. The prefix length is the number of bits that are the same in the two keys' SHA-256 hash. The more leading bits - that are the same, the shorter the prefix length and the closer the proximity of the + that are the same, the longer the prefix length and the closer the proximity of the two keys are considered to be. The distance metric is a way to calculate the distance between two keys by From 8bbabcc4e56327399534596c711cd40574aa1a5e Mon Sep 17 00:00:00 2001 From: Max Inden Date: Fri, 10 Feb 2023 15:54:23 +0100 Subject: [PATCH 12/12] I would argue all DHTs are build for p2p systems --- content/concepts/introduction/protocols/dht.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/concepts/introduction/protocols/dht.md b/content/concepts/introduction/protocols/dht.md index 42adc1d4..0fd20cee 100644 --- a/content/concepts/introduction/protocols/dht.md +++ b/content/concepts/introduction/protocols/dht.md @@ -1,7 +1,7 @@ --- title: "Kademlia DHT" description: "The libp2p Kad-DHT subsystem is an implementation of the Kademlia -DHT, a distributed hash table that is designed for P2P networks." +DHT, a distributed hash table." weight: 25 ---