Compact blinded path creation #3011

jkczyz · 2024-04-22T19:59:28Z

Add support for creating a compact BlindedPath, which consists of:

using an SCID instead of a node id in BlindedHop::encrypted_payload
using IntroductionNode::DirectedShortChannelId

The first is accomplished by specifying SCIDs when calling BlindedPath::new_for_message using a new message::ForwardNode struct. The second is through calling BlindedPath::use_compact_introduction_node. Both are called by DefaultMessageRouter.

codecov-commenter · 2024-04-22T20:07:35Z

Codecov Report

Attention: Patch coverage is 92.19858% with 11 lines in your changes are missing coverage. Please review.

Project coverage is 90.29%. Comparing base (a95338a) to head (e4661fe).
Report is 7 commits behind head on main.

Files	Patch %	Lines
lightning/src/ln/offers_tests.rs	81.25%	0 Missing and 6 partials ⚠️
lightning/src/blinded_path/mod.rs	86.20%	3 Missing and 1 partial ⚠️
lightning/src/util/test_utils.rs	50.00%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3011      +/-   ##
==========================================
+ Coverage   89.87%   90.29%   +0.42%     
==========================================
  Files         117      117              
  Lines       96952   100069    +3117     
  Branches    96952   100069    +3117     
==========================================
+ Hits        87134    90357    +3223     
+ Misses       7273     7130     -143     
- Partials     2545     2582      +37

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

valentinewallace

Basically LGTM

lightning/src/onion_message/messenger.rs

valentinewallace · 2024-04-23T19:13:26Z

lightning/src/blinded_path/mod.rs

+
+	/// Attempts to a use a compact representation for the [`IntroductionNode`] by using a directed
+	/// short channel id from a channel in `network_graph` leading to the introduction node.
+	pub fn compact_introduction_node(&mut self, network_graph: &ReadOnlyNetworkGraph) {


Pedantic but the convention is for setters to use the set_ prefix: https://github.com/rust-lang/rfcs/blob/master/text/0344-conventions-galore.md#gettersetter-apis and I think it would read a bit cleaner in this case.

Hmmm... this is not a setter in the normal sense, though, as we don't pass a value to set a field with. Nor is the field private, which would necessitate a setter. It also may not update the field if a value cannot be found in the network graph. Happy to go with a better name. Just not sure if a set_ prefix is appropriate here.

Still, it does seem a bit strange that we're mutating the blinded path with a fn name that sounds like it could just as well be a straight getter.

Ok, added a use_ prefix. I can see how the docs make it less clear that compact was a verb in the name.

TheBlueMatt · 2024-05-09T19:16:08Z

lightning/src/onion_message/messenger.rs

 		T: secp256k1::Signing + secp256k1::Verification
 	>(
-		&self, recipient: PublicKey, peers: Vec<PublicKey>, secp_ctx: &Secp256k1<T>,
+		&self, recipient: PublicKey, peers: Vec<PublicKey>, scid_lookup: &SL,


Rather than passing a list of our peers and a trait so that create_blinded_paths can make a callback to the caller to ask for an SCID for a given peer, shouldn't we just pass in the peers as a ForwardNode or (PublicKey, u64)?

Yeah, come to think of it that is probably cleaner. That would also allow the caller to use the compact hop representation only when desired. (e.g., in offer/refund but not in reply paths).

TheBlueMatt

Looks like this needs a rebase.

TheBlueMatt · 2024-05-13T20:14:35Z

lightning/src/ln/channelmanager.rs

+				node_id: *node_id,
+				short_channel_id: peer.channel_by_id
+					.iter()
+					.find(|(_, channel)| channel.context().is_usable())


Do we want to sort by oldest or biggest or something?

Good idea! Oldest is probably best as this is only for onion messages, so we just want to make sure the channel announcement has been propagated in the gossip.

jkczyz · 2024-05-13T21:48:30Z

Rebased

TheBlueMatt

Basically LGTM, but IMO we really should make this optional somehow. It could happen in a followup if we want, cause we should also change how many hops we include based on if an offfer is long-lived.

TheBlueMatt · 2024-05-14T14:58:49Z

lightning/src/blinded_path/mod.rs

+
+	/// Attempts to a use a compact representation for the [`IntroductionNode`] by using a directed
+	/// short channel id from a channel in `network_graph` leading to the introduction node.
+	pub fn compact_introduction_node(&mut self, network_graph: &ReadOnlyNetworkGraph) {


Still, it does seem a bit strange that we're mutating the blinded path with a fn name that sounds like it could just as well be a straight getter.

TheBlueMatt · 2024-05-14T14:59:26Z

lightning/src/blinded_path/mod.rs

 	}
+
+	/// Attempts to a use a compact representation for the [`IntroductionNode`] by using a directed
+	/// short channel id from a channel in `network_graph` leading to the introduction node.


We should probably have some kind of discussion of how this makes paths shorter but if a channel closes will invalidate it.

TheBlueMatt · 2024-05-14T14:59:48Z

lightning/src/blinded_path/mod.rs

+				if let Some((scid, channel_info)) = node_info
+					.channels
+					.iter()
+					.find_map(|scid| network_graph.channel(*scid).map(|info| (*scid, info)))


Similar here, should we sort by size/age?

Yeah, good catch. Using the block height from the scid as it seems the timestamp in the update is really a counter specific to the channel.

TheBlueMatt · 2024-05-14T15:01:15Z

lightning/src/ln/channelmanager.rs

+			.filter(|(_, peer)| peer.latest_features.supports_onion_messages())
+			.map(|(node_id, peer)| ForwardNode {
+				node_id: *node_id,
+				short_channel_id: peer.channel_by_id


Should we make setting this optional somehow? I feel like if I'm building a super long-term offer I may have a different preference from something being scanned right now.

Yeah... arguably we shouldn't bother with it for reply paths, either. Just not sure exactly how we want to convey it through the MessageRouter trait. Currently, the caller makes the decision for the penultimate hop using ForwardNode, but when adding more hops the MessageRouter makes the decision since it needs a NetworkGraph to find more hops. Similarly for the introduction node.

So right now it's partly an implementation concern given you need a NetworkGraph. I guess we can just add a bool parameter and it say it is best effort? Any other ideas?

I see what you mean... if there's an obvious "default behavior" that stands out, we could have a separate method, e.g. create_long_term_blinded_paths, or have a Config struct. That way we could also have a config setting for compact offers vs offers that don't need to be QR-scanned, or privacy-oriented offers that want longer blinded paths.

Opened #3080 with separate MessageRouter methods, using compact paths for Offer::paths and Refund::paths and non-compact paths for onion message reply paths. Basically, the trait allows either for the caller to decide.

As for how this is exposed in ChannelManager utilities, I'm not sure how we should handle short- vs long-lived offers. One thought was to chose the type of path based on the expiry. But for offers, the expiry is set by the user after the builder is returned and path already set. For refunds created via ChannelManager, we require an expiration (even though the spec does not), though, so we could infer there.

Alternatives would be:

making a special purpose create_offer_builder for long-lived offers

adding a parameter to create_offer_builder indicating if short- or long-lived

adding a Optional absolute expiry parameter to create_offer_builder used to infer which type of path to create

Any preferences other alternatives?

I'm good with whichever of those options you think is best!

TheBlueMatt · 2024-05-14T15:04:21Z

lightning/src/onion_message/messenger.rs

 			},
+		}?;
+		for path in &mut paths {
+			path.compact_introduction_node(&network_graph);


Similar here, can we make this optional on a per-offer/message basis?

#3080 refactors MessageRouter to have two different methods.

jkczyz

Oops, forgot to publish these comments.

jkczyz · 2024-05-14T16:23:53Z

lightning/src/blinded_path/mod.rs

+
+	/// Attempts to a use a compact representation for the [`IntroductionNode`] by using a directed
+	/// short channel id from a channel in `network_graph` leading to the introduction node.
+	pub fn compact_introduction_node(&mut self, network_graph: &ReadOnlyNetworkGraph) {


Ok, added a use_ prefix. I can see how the docs make it less clear that compact was a verb in the name.

jkczyz · 2024-05-14T16:47:40Z

lightning/src/blinded_path/mod.rs

+				if let Some((scid, channel_info)) = node_info
+					.channels
+					.iter()
+					.find_map(|scid| network_graph.channel(*scid).map(|info| (*scid, info)))


Yeah, good catch. Using the block height from the scid as it seems the timestamp in the update is really a counter specific to the channel.

jkczyz · 2024-05-14T17:10:44Z

lightning/src/ln/channelmanager.rs

+			.filter(|(_, peer)| peer.latest_features.supports_onion_messages())
+			.map(|(node_id, peer)| ForwardNode {
+				node_id: *node_id,
+				short_channel_id: peer.channel_by_id


Yeah... arguably we shouldn't bother with it for reply paths, either. Just not sure exactly how we want to convey it through the MessageRouter trait. Currently, the caller makes the decision for the penultimate hop using ForwardNode, but when adding more hops the MessageRouter makes the decision since it needs a NetworkGraph to find more hops. Similarly for the introduction node.

So right now it's partly an implementation concern given you need a NetworkGraph. I guess we can just add a bool parameter and it say it is best effort? Any other ideas?

valentinewallace

LGTM pending maybe making the compact encoding optional. Looks like it needs another rebase as well.

lightning/src/blinded_path/mod.rs

jkczyz · 2024-05-14T22:49:09Z

LGTM pending maybe making the compact encoding optional. Looks like it needs another rebase as well.

Rebased now. Yeah, maybe best to do that in a follow-up as it may be a little more involved. Let me know if you have any thoughts on #3011 (comment).

TheBlueMatt · 2024-05-17T16:09:17Z

Feel free to squash and we can land this IMO. We should open an issue and block the next release on the followup.

jkczyz · 2024-05-22T22:29:25Z

Feel free to squash and we can land this IMO. We should open an issue and block the next release on the followup.

Rebased and opened #3080. Looking for feedback on the ChannelManager API (see #3011 (comment)).

lightning/src/blinded_path/message.rs

lightning/src/blinded_path/mod.rs

valentinewallace · 2024-05-23T18:50:46Z

lightning/src/routing/router.rs

 		T: secp256k1::Signing + secp256k1::Verification
 	> (
-		&self, recipient: PublicKey, peers: Vec<PublicKey>, secp_ctx: &Secp256k1<T>,
+		&self, recipient: PublicKey, peers: Vec<message::ForwardNode>, secp_ctx: &Secp256k1<T>,


If we're going to split this into two methods, do we still need message::ForwardNode? Can't one method take pubkeys and the other take scids?

We alwasys need the pubkeys when creating the blinded path, so we'd need to either make a new struct or use a tuple. It's kinda nice having it use an Option for short_channel_id, though. It makes it easy in ChannelManager::create_blinded_path to fallback to None if a usable channel can't be found.

lightning/src/blinded_path/message.rs

When sending an onion message to a blinded path, the short channel id between hops isn't need in each hop's encrypted_payload since it is not a payment. However, using the short channel id instead of the node id gives a more compact representation. Update BlindedPath::new_for_message to allow for this.

Instead of passing Vec<PublicKey> to MessageRouter::crate_blinded_path, pass Vec<ForwardNode>. This way callers can include a short_channel_id for a more compact BlindedPath encoding.

Add a method to BlindedPath that given a network graph will compact the IntroductionNode as the DirectedShortChannelId variant. Call this method from DefaultMessageRouter so that Offer paths use the compact representation (along with reply paths). This leaves payment paths in Bolt12Invoice using the NodeId variant, as the compact representation isn't as useful there.

jkczyz · 2024-05-28T21:43:45Z

Added derives to ForwardNode.

jkczyz mentioned this pull request Apr 22, 2024

Compact BlindedPath representation for onion messaging #2921

Closed

9 tasks

valentinewallace reviewed Apr 23, 2024

View reviewed changes

TheBlueMatt reviewed May 9, 2024

View reviewed changes

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch from 084d070 to a74e84f Compare May 9, 2024 22:55

TheBlueMatt reviewed May 13, 2024

View reviewed changes

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch 2 times, most recently from 5d08b1f to f3726a7 Compare May 13, 2024 21:48

TheBlueMatt reviewed May 14, 2024

View reviewed changes

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch from f3726a7 to b295e98 Compare May 14, 2024 17:17

jkczyz commented May 14, 2024

View reviewed changes

valentinewallace reviewed May 14, 2024

View reviewed changes

lightning/src/blinded_path/mod.rs Outdated Show resolved Hide resolved

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch 2 times, most recently from 2dd24ed to 7d85abd Compare May 14, 2024 22:48

jkczyz mentioned this pull request May 22, 2024

Optional compact blinded path creation #3080

Merged

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch 2 times, most recently from e7f5035 to bb5dc03 Compare May 22, 2024 22:28

valentinewallace previously approved these changes May 23, 2024

View reviewed changes

lightning/src/blinded_path/message.rs Show resolved Hide resolved

lightning/src/blinded_path/mod.rs Show resolved Hide resolved

valentinewallace reviewed May 23, 2024

View reviewed changes

jkczyz dismissed valentinewallace’s stale review via f0d81ea May 23, 2024 20:53

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch from bb5dc03 to f0d81ea Compare May 23, 2024 20:53

TheBlueMatt previously approved these changes May 28, 2024

View reviewed changes

lightning/src/blinded_path/message.rs Show resolved Hide resolved

jkczyz added 4 commits May 28, 2024 16:35

Pass ForwardNode when creating BlindedPath

e553a71

Instead of passing Vec<PublicKey> to MessageRouter::crate_blinded_path, pass Vec<ForwardNode>. This way callers can include a short_channel_id for a more compact BlindedPath encoding.

Unwrap reply_path in extract_invoice_request

4956ade

jkczyz added 2 commits May 28, 2024 16:35

Compact introduction node test vector

e8354b2

Add missing copyright statement

e4661fe

jkczyz dismissed TheBlueMatt’s stale review via e4661fe May 28, 2024 21:42

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch from f0d81ea to e4661fe Compare May 28, 2024 21:42

TheBlueMatt approved these changes May 28, 2024

View reviewed changes

valentinewallace approved these changes May 28, 2024

View reviewed changes

TheBlueMatt merged commit df01208 into lightningdevkit:main May 29, 2024

Compact blinded path creation #3011

Compact blinded path creation #3011

Uh oh!

Conversation

jkczyz commented Apr 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Apr 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

valentinewallace left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TheBlueMatt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jkczyz commented May 13, 2024

Uh oh!

TheBlueMatt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jkczyz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

valentinewallace left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jkczyz commented May 14, 2024

Uh oh!

TheBlueMatt commented May 17, 2024

Uh oh!

jkczyz commented May 22, 2024

Uh oh!

jkczyz commented Apr 22, 2024 •

edited

Loading

codecov-commenter commented Apr 22, 2024 •

edited

Loading