From f6b4642d1c96a3af127557f3e4988363c40f9e95 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 7 Dec 2022 04:51:28 -0500 Subject: [PATCH 1/9] populate muxer overview --- content/concepts/multiplex/overview.md | 45 +++++++++++++++----------- 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/content/concepts/multiplex/overview.md b/content/concepts/multiplex/overview.md index 9a0b8eaa..622b8dfc 100644 --- a/content/concepts/multiplex/overview.md +++ b/content/concepts/multiplex/overview.md @@ -9,24 +9,33 @@ aliases: ## Overview -Stream Multiplexing (_stream muxing_) is a way of sending multiple streams of data over one -communication link. It combines multiple signals into one unified signal so it can be transported -'over the wires', then it is demulitiplexed (_demuxed_) so it can be output and used by separate -applications. This is done to share a single TCP connection using unique port numbers to distinguish -streams, between the multiple proceeses (such as kademlia and gossipsub) used by applications (such as IPFS) -to make connection and transmission more efficient. - -With muxing, libp2p applications may have many separate streams of communication between peers, as well as -have multiple concurrent streams open at the same time with a peer. Stream multiplexing allows us to initialize -and use the same [transport](../../transports/overview) connection across the lifetime of our interaction with a -peer. With muxing, we also only need to deal with [NAT traversal](../../nat/overview) once to be able to open as -many streams as we need, since they will all share the same underlying transport connection. Applications can enable -support for multiple multiplexers, which will allow you to fall back to a widely-supported multiplexer if a preferred -choice is not supported by a remote peer. +Stream multiplexing (or stream muxing) is a method to send multiple streams +of data over a single communication link. It combines multiple signals into one +unified signal. The unifed signal can then be demultiplexed (or demuxed), where +the output becomes available to use in an application. + +Stream multiplexing is done to share a single TCP connection using unique port numbers +that distinguish streams between multiple processes (such as kademlia and gossipsub) +used by applications (such as IPFS) to make connection and transmission more efficient. + +## Muxers in libp2p + +Stream muxers are pluggable in libp2p. A given libp2p application can support several +muxers simultaneously and choose one as the default, and allowing an application to +fall back to a widely-supported multiplexer if a remote peer does not support a preferred +choice. + +With muxing, libp2p applications can have separate communication streams between peers +and multiple concurrent streams open simultaneously with a peer. Stream muxing +allows peers to initialize and use the same [transport](../../transports/overview) +connection across the lifetime of their interaction with another peer. {{< alert icon="💡" context="info">}} -libp2p's multiplexing happens at the application layer, meaning it's not provided by the -operating system's network stack. However, developers writing libp2p applications rarely need to -interact with stream multiplexers directly, except during initial configuration to control which -modules are enabled. +libp2p's multiplexing happens at the application layer, meaning the +operating system's network stack does not provide it. However, developers writing libp2p +applications rarely need to interact with stream multiplexers directly, except during +initial configuration to control which modules are enabled. {{< /alert >}} + +libp2p supports two muxers, [mplex](mplex) and [yamux](yamux). libp2p also supports +transports that support native stream muxing, like [QUIC](quic). From 1a28c0b85a71b71d8ca9385dda5e35387d63e15c Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 12 Dec 2022 16:59:02 -0500 Subject: [PATCH 2/9] Apply suggestions from code review Co-authored-by: Marten Seemann --- content/concepts/multiplex/overview.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/concepts/multiplex/overview.md b/content/concepts/multiplex/overview.md index 622b8dfc..c0f3853a 100644 --- a/content/concepts/multiplex/overview.md +++ b/content/concepts/multiplex/overview.md @@ -20,10 +20,10 @@ used by applications (such as IPFS) to make connection and transmission more eff ## Muxers in libp2p -Stream muxers are pluggable in libp2p. A given libp2p application can support several -muxers simultaneously and choose one as the default, and allowing an application to -fall back to a widely-supported multiplexer if a remote peer does not support a preferred -choice. +Stream muxers are pluggable in libp2p. A given libp2p host can support several +muxers simultaneously. The stream muxer is negotiated between the two nodes during the handshake. +Having such a negotiation protocol in place allows libp2p to adopt new muxers in the future, while at +the same time keeping backwards-compatibility with currently deployed muxers. With muxing, libp2p applications can have separate communication streams between peers and multiple concurrent streams open simultaneously with a peer. Stream muxing From a030a8b94d92b301d900eaf26ff66509384676df Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 12 Dec 2022 18:18:38 -0500 Subject: [PATCH 3/9] incorporate pr feedback --- content/concepts/multiplex/overview.md | 32 +++++++++++++------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/content/concepts/multiplex/overview.md b/content/concepts/multiplex/overview.md index c0f3853a..cd5f5b61 100644 --- a/content/concepts/multiplex/overview.md +++ b/content/concepts/multiplex/overview.md @@ -11,31 +11,31 @@ aliases: Stream multiplexing (or stream muxing) is a method to send multiple streams of data over a single communication link. It combines multiple signals into one -unified signal. The unifed signal can then be demultiplexed (or demuxed), where -the output becomes available to use in an application. +unified signal. The unified signal can then be demultiplexed (or demuxed), where +the output becomes available in an application. This allows +peers to initialize and use the same [transport](../../transports/overview) +connection across the lifetime of their interaction with another peer. -Stream multiplexing is done to share a single TCP connection using unique port numbers +In particular, stream muxing is done to share a single connection using unique port numbers that distinguish streams between multiple processes (such as kademlia and gossipsub) used by applications (such as IPFS) to make connection and transmission more efficient. ## Muxers in libp2p -Stream muxers are pluggable in libp2p. A given libp2p host can support several -muxers simultaneously. The stream muxer is negotiated between the two nodes during the handshake. -Having such a negotiation protocol in place allows libp2p to adopt new muxers in the future, while at -the same time keeping backwards-compatibility with currently deployed muxers. +Establishing a libp2p connection can be expensive and take a few round-trips. Peers +should aim to establish a connection once, then reuse that connection by opening a stream, +which has a low overhead and does not impose any latency penalty. -With muxing, libp2p applications can have separate communication streams between peers -and multiple concurrent streams open simultaneously with a peer. Stream muxing -allows peers to initialize and use the same [transport](../../transports/overview) -connection across the lifetime of their interaction with another peer. +Stream muxers are pluggable in libp2p. A given libp2p host can support several +muxers simultaneously. The stream muxer is negotiated between the two nodes during the +handshake. Having such a negotiation protocol in place allows libp2p to adopt new muxers in +the future while simultaneously keeping backward compatibility with currently deployed +muxers. {{< alert icon="💡" context="info">}} -libp2p's multiplexing happens at the application layer, meaning the -operating system's network stack does not provide it. However, developers writing libp2p -applications rarely need to interact with stream multiplexers directly, except during -initial configuration to control which modules are enabled. +Developers writing libp2p applications rarely need to interact with stream multiplexers +directly, except during initial configuration to control which modules are enabled. {{< /alert >}} libp2p supports two muxers, [mplex](mplex) and [yamux](yamux). libp2p also supports -transports that support native stream muxing, like [QUIC](quic). +transports that support native stream muxing, like [QUIC](../transports/quic). From ff5cb07b3e17261649a722db029949191813a4e8 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Tue, 17 Jan 2023 06:08:34 -0500 Subject: [PATCH 4/9] incorporate PR feedback + enhancements --- content/concepts/multiplex/overview.md | 39 ++++++++++++++------------ 1 file changed, 21 insertions(+), 18 deletions(-) diff --git a/content/concepts/multiplex/overview.md b/content/concepts/multiplex/overview.md index cd5f5b61..2268b1bf 100644 --- a/content/concepts/multiplex/overview.md +++ b/content/concepts/multiplex/overview.md @@ -9,32 +9,35 @@ aliases: ## Overview -Stream multiplexing (or stream muxing) is a method to send multiple streams -of data over a single communication link. It combines multiple signals into one -unified signal. The unified signal can then be demultiplexed (or demuxed), where -the output becomes available in an application. This allows -peers to initialize and use the same [transport](../../transports/overview) -connection across the lifetime of their interaction with another peer. +Stream multiplexing (or stream muxing) is a method to send multiple streams of +data over a single communication link. It allows peers to use the same connection +across the lifetime of their interaction with another peer. -In particular, stream muxing is done to share a single connection using unique port numbers -that distinguish streams between multiple processes (such as kademlia and gossipsub) -used by applications (such as IPFS) to make connection and transmission more efficient. +In particular, stream muxing is done to share a single connection between multiple +protocols used by applications to make connection and transmission more efficient. +This is especially useful when doing NAT traversal or hole punching, +as it allows for efficient use of limited resources. This is also used outside of +libp2p, for example, HTTP/2 introduced streams into HTTP, allowing for many HTTP +requests in parallel on the same connection. ## Muxers in libp2p -Establishing a libp2p connection can be expensive and take a few round-trips. Peers -should aim to establish a connection once, then reuse that connection by opening a stream, -which has a low overhead and does not impose any latency penalty. +Establishing a libp2p connection can be expensive and take a few round-trips. +Peers therefore aim to establish a connection once, then run many streams over +the same connection. Opening has a low resource overhead and does not impose any +latency penalty. Stream muxers are pluggable in libp2p. A given libp2p host can support several -muxers simultaneously. The stream muxer is negotiated between the two nodes during the -handshake. Having such a negotiation protocol in place allows libp2p to adopt new muxers in -the future while simultaneously keeping backward compatibility with currently deployed -muxers. +muxers simultaneously. The stream muxer is negotiated between the two nodes during +the handshake. Having such a negotiation protocol in place allows libp2p to adopt +new muxers in the future while simultaneously keeping backward compatibility with +currently deployed muxers. {{< alert icon="💡" context="info">}} -Developers writing libp2p applications rarely need to interact with stream multiplexers -directly, except during initial configuration to control which modules are enabled. +Developers writing libp2p applications rarely need to interact with stream +multiplexers directly, except during initial configuration to control which modules +are enabled. Some protocols have streams natively, and only pluggable muxers are +needed for protocols that don't. {{< /alert >}} libp2p supports two muxers, [mplex](mplex) and [yamux](yamux). libp2p also supports From 6c32fe582b15e6264bac8f1f4a380c6d2cbffe45 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 1 Feb 2023 09:06:59 -0500 Subject: [PATCH 5/9] Apply suggestions from code review Co-authored-by: Marten Seemann --- content/concepts/multiplex/overview.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/content/concepts/multiplex/overview.md b/content/concepts/multiplex/overview.md index 2268b1bf..60de8826 100644 --- a/content/concepts/multiplex/overview.md +++ b/content/concepts/multiplex/overview.md @@ -13,14 +13,14 @@ Stream multiplexing (or stream muxing) is a method to send multiple streams of data over a single communication link. It allows peers to use the same connection across the lifetime of their interaction with another peer. -In particular, stream muxing is done to share a single connection between multiple -protocols used by applications to make connection and transmission more efficient. -This is especially useful when doing NAT traversal or hole punching, -as it allows for efficient use of limited resources. This is also used outside of +libp2p uses stream muxing to share a single connection between multiple +libp2p and application protocols. +This is especially useful when establishing the connection required a lot of effort, +e.g. when NAT hole punching was necessary to establish a direct connection to a peer. libp2p, for example, HTTP/2 introduced streams into HTTP, allowing for many HTTP requests in parallel on the same connection. -## Muxers in libp2p +## Stream Multiplexers in libp2p Establishing a libp2p connection can be expensive and take a few round-trips. Peers therefore aim to establish a connection once, then run many streams over @@ -30,14 +30,14 @@ latency penalty. Stream muxers are pluggable in libp2p. A given libp2p host can support several muxers simultaneously. The stream muxer is negotiated between the two nodes during the handshake. Having such a negotiation protocol in place allows libp2p to adopt -new muxers in the future while simultaneously keeping backward compatibility with +new muxers in the future, while simultaneously keeping backward compatibility with currently deployed muxers. {{< alert icon="💡" context="info">}} Developers writing libp2p applications rarely need to interact with stream -multiplexers directly, except during initial configuration to control which modules -are enabled. Some protocols have streams natively, and only pluggable muxers are -needed for protocols that don't. +multiplexers directly, except during initial configuration. A libp2p connection always +is a stream-multiplexed connection, and the libp2p stack takes care of negotiating and +setting up a stream multiplexer. {{< /alert >}} libp2p supports two muxers, [mplex](mplex) and [yamux](yamux). libp2p also supports From 1461ad22863e36f3281283543ea331f5318a7bca Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 2 Feb 2023 15:17:34 -0500 Subject: [PATCH 6/9] address feedback + edits --- content/concepts/multiplex/overview.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/concepts/multiplex/overview.md b/content/concepts/multiplex/overview.md index 60de8826..95998f9c 100644 --- a/content/concepts/multiplex/overview.md +++ b/content/concepts/multiplex/overview.md @@ -11,11 +11,11 @@ aliases: Stream multiplexing (or stream muxing) is a method to send multiple streams of data over a single communication link. It allows peers to use the same connection -across the lifetime of their interaction with another peer. +over the lifetime of their interaction with another peer. libp2p uses stream muxing to share a single connection between multiple libp2p and application protocols. -This is especially useful when establishing the connection required a lot of effort, +This is especially useful when establishing the connection required a lot of effort, e.g. when NAT hole punching was necessary to establish a direct connection to a peer. libp2p, for example, HTTP/2 introduced streams into HTTP, allowing for many HTTP requests in parallel on the same connection. From 806b232a543bec8cfaa163c2c60a09dfbd2cf73d Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Thu, 2 Feb 2023 20:24:23 -0500 Subject: [PATCH 7/9] incorporate PR feedback and rewrite --- content/concepts/multiplex/overview.md | 61 +++++++++++++++----------- 1 file changed, 35 insertions(+), 26 deletions(-) diff --git a/content/concepts/multiplex/overview.md b/content/concepts/multiplex/overview.md index 95998f9c..191cd447 100644 --- a/content/concepts/multiplex/overview.md +++ b/content/concepts/multiplex/overview.md @@ -9,36 +9,45 @@ aliases: ## Overview -Stream multiplexing (or stream muxing) is a method to send multiple streams of -data over a single communication link. It allows peers to use the same connection -over the lifetime of their interaction with another peer. - -libp2p uses stream muxing to share a single connection between multiple -libp2p and application protocols. -This is especially useful when establishing the connection required a lot of effort, -e.g. when NAT hole punching was necessary to establish a direct connection to a peer. -libp2p, for example, HTTP/2 introduced streams into HTTP, allowing for many HTTP -requests in parallel on the same connection. +Network protocols often use bi-directional message streams built on top of a stream +abstraction to transmit data between nodes. However, relying on a single message +stream can result in scalability issues and bottlenecks, as one application would +need to wait for another to finish utilizing the stream before being able to send +and receive its own messages. + +To overcome this issue, network connections employ stream multiplexing, which allows +for the creation of multiple virtual connections within a single connection. This +enables nodes to send multiple streams of messages over separate virtual connections, +providing a scalable solution that eliminates the bottleneck created by a +single stream. An example is when HTTP/2 introduced streams into HTTP, +allowing for many HTTP requests in parallel on the same connection. + +In libp2p, stream muxing is used to share a single connection between various protocols, +providing a more efficient solution, particularly when establishing the connection is +resource-intensive, such as when NAT hole punching is necessary. By establishing a +connection once and running multiple streams over the same connection, libp2p can reduce +the resource overhead and latency penalty associated with frequent connection establishment. ## Stream Multiplexers in libp2p -Establishing a libp2p connection can be expensive and take a few round-trips. -Peers therefore aim to establish a connection once, then run many streams over -the same connection. Opening has a low resource overhead and does not impose any -latency penalty. - -Stream muxers are pluggable in libp2p. A given libp2p host can support several -muxers simultaneously. The stream muxer is negotiated between the two nodes during -the handshake. Having such a negotiation protocol in place allows libp2p to adopt -new muxers in the future, while simultaneously keeping backward compatibility with -currently deployed muxers. +Stream muxers are a key component of the libp2p stack, providing pluggable multiplexing +capabilities for peers. libp2p hosts can support multiple muxers simultaneously, and the +choice of muxer is negotiated between the nodes during the initial connection handshake. +This negotiation protocol allows libp2p to adopt new muxers in the future while +maintaining backward compatibility with existing muxers. {{< alert icon="💡" context="info">}} -Developers writing libp2p applications rarely need to interact with stream -multiplexers directly, except during initial configuration. A libp2p connection always -is a stream-multiplexed connection, and the libp2p stack takes care of negotiating and -setting up a stream multiplexer. +For developers building libp2p applications, interaction with stream muxers is typically +limited to the initial configuration phase. The libp2p stack automatically handles the +negotiation and setup of the muxer, ensuring that all connections are stream-multiplexed +and allowing for the seamless transmission of multiple streams of data over a single +connection. {{< /alert >}} -libp2p supports two muxers, [mplex](mplex) and [yamux](yamux). libp2p also supports -transports that support native stream muxing, like [QUIC](../transports/quic). +Currently, libp2p supports two stream muxers, [mplex](/concepts/multiplex/mplex) +and [yamux](/concepts/multiplex/yamux). However, many of the +[transport protocols](/concepts/transports/overview) available in the libp2p stack +come with native streams, such as [QUIC](/concepts/transports/quic), +[WebTransport](/concepts/transports/webtransport), and +[WebRTC](/concepts/transports/webrtc), and in these cases, libp2p +**does not need to perform stream multiplexing** as the protocol already provides it. From 6e20ff498a7ae220dd113fe30b19735ca301acc7 Mon Sep 17 00:00:00 2001 From: Prithvi Shahi <50885601+p-shahi@users.noreply.github.com> Date: Mon, 6 Feb 2023 18:59:17 -0800 Subject: [PATCH 8/9] Apply suggestions from code review --- content/concepts/multiplex/overview.md | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/content/concepts/multiplex/overview.md b/content/concepts/multiplex/overview.md index 191cd447..14adaef2 100644 --- a/content/concepts/multiplex/overview.md +++ b/content/concepts/multiplex/overview.md @@ -9,20 +9,27 @@ aliases: ## Overview -Network protocols often use bi-directional message streams built on top of a stream -abstraction to transmit data between nodes. However, relying on a single message -stream can result in scalability issues and bottlenecks, as one application would +libp2p is built on top of a stream abstraction and uses a bi-directional message stream to send data between peers. +However, relying on a single message +stream over a connection between two peers can result in scalability issues and bottlenecks. +Each peer on either side of the connection may run multiple applications sending and waiting for data over the stream. +A single stream would block applications on one another, as one application would need to wait for another to finish utilizing the stream before being able to send and receive its own messages. -To overcome this issue, network connections employ stream multiplexing, which allows -for the creation of multiple virtual connections within a single connection. This +To overcome this issue, libp2p enables applications to employ stream multiplexing. +Multiplexing allows +for the creation of multiple "virtual" connections within a single connection. This enables nodes to send multiple streams of messages over separate virtual connections, providing a scalable solution that eliminates the bottleneck created by a -single stream. An example is when HTTP/2 introduced streams into HTTP, +single stream. +Two libp2p peers may have a single TCP connection and use different port numbers to distinguish streams. +Then different applications/processes like Kademlia or GossipSub used by an application like IPFS would get their own stream of data and make transmission more efficient. +Stream multiplexing makes it so that applications or protocols running on top of libp2p think that they’re the only ones running on that connection. +Another example is when HTTP/2 introduced streams into HTTP, allowing for many HTTP requests in parallel on the same connection. -In libp2p, stream muxing is used to share a single connection between various protocols, +In summary, stream muxing can be used by applications on top of libp2p to share a single connection between various protocols, providing a more efficient solution, particularly when establishing the connection is resource-intensive, such as when NAT hole punching is necessary. By establishing a connection once and running multiple streams over the same connection, libp2p can reduce From 7fa5a042626444a24fcf14f5aa18e2a5d4318106 Mon Sep 17 00:00:00 2001 From: Prithvi Shahi Date: Mon, 6 Feb 2023 19:02:50 -0800 Subject: [PATCH 9/9] fix lint --- content/concepts/multiplex/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/concepts/multiplex/overview.md b/content/concepts/multiplex/overview.md index 14adaef2..e76dd888 100644 --- a/content/concepts/multiplex/overview.md +++ b/content/concepts/multiplex/overview.md @@ -22,7 +22,7 @@ Multiplexing allows for the creation of multiple "virtual" connections within a single connection. This enables nodes to send multiple streams of messages over separate virtual connections, providing a scalable solution that eliminates the bottleneck created by a -single stream. +single stream. Two libp2p peers may have a single TCP connection and use different port numbers to distinguish streams. Then different applications/processes like Kademlia or GossipSub used by an application like IPFS would get their own stream of data and make transmission more efficient. Stream multiplexing makes it so that applications or protocols running on top of libp2p think that they’re the only ones running on that connection.