BOLT7: How to sync channel updates ?

[sstone]

To route messages, LN nodes use a local routing table that includes channel announcements and updates. There is one announcement per channel, but nodes can publish many updates, and should use the most recent updates when they compute routes.

Keeping their local routing table up to date is difficult for nodes which are often offline, and can create UX issues where the first payments that users will try will fail because the local routing table is using old updates. It may also alter the computing of routes by ignoring nodes that have published new updates with lower fees.

The current "channel range queries sync" scheme works by reconciling lists of channel ids:

• A asks for a list a short channel ids for a given block height range
• B replies with a list of short channel ids
• A compares with its own routing table and asks for missing channel ids
• B sends back channel announcements and updates

It works very well when nodes have missed channel announcements, but not when they've missed channel updates:
Suppose A has been offline for some time, and has the same channels as B but many of its channel updates are too old.
If is only starts for a short time, it won't get new updates through the channel query scheme, and will not be online long enough to receive channel updates broadcast.

To fix this is propose that we add channel update timestamps to our channel range queries:

• reply_channel_range will now include a list of short channel ids (8 bytes) + timestamps (4 bytes)
• query_short_channel_ids will include a flag. If set to 1 the receiving node should send back a channel announcement + updates, otherwise it should just send back updates.

[rustyrussell]

s/32 bytes/4 bytes/.

We need 2 channel update timestamps though, right? Maybe 0 meaning "no update received"?

[sstone]

Yes 4 bytes (we could use the timestamp of he most recent updates that we have).

There is already a gossip_timestamp_filter that can be used to sync updates but I believe that it is not enough: the receiving node may differ sending back updates until the next "gossip flush", and even if they do send back matching updates right away we still don't know how to set our timestamp filter:

Suppose a node stores a "last online at" timestamp. When it becomes online again, it uses this timestamp (minus a few hours for example) to send a gossip_timestamp_filter. Then it goes offline again after a few minutes: does it update its "last online at" timestamp ? If it does not it's going to receive the same updates again, and if it does it will never receives the ones it missed (because it went offline before it has received them). It could also set its filter to "everything in the last 2 weeks" and ask for every update but then we're back to the original problem.

[sstone]

I've implemented and tested several variations and this is what I propose:

query_channel_range remains unchanged.

reply_channel_range now includes a list of:

• short_channel_id (8 bytes)
• `channel_update timestamp bolt07: Adding P2P routing gossiping #1' (4 bytes)
• `channel_update timestamp Pick a licence for the specs #2' (4 bytes)

There are 2 channel_update messages for each short_channel_id, by convention the first one is the one for which direction is 0.

query_short_channel_ids now includes a list of:

• short_channel_id (8 bytes)
• query_flag (1 byte)

query_flag specifies what the sender wants and can be a combination of:

• INCLUDE_CHANNEL_ANNOUNCEMENT = 4
• INCLUDE_CHANNEL_UPDATE_1 = 1
• INCLUDE_CHANNEL_UPDATE_2 = 2

When a node receive a reply_channel_range message, it compares the short_channel_id and timestamps that it contains to its routing table and reply with query_short_channel_ids messages to query the channel_announcement and/or channel_updates that its does not have or are more recent than its own.

It's very basic but not that different from the existing messages and so should be fairly easy to implement if you already support channel range queries.

[t-bast]

Closing this old issue, we are fortunately able to sync channel_updates now (even though the gossip mechanism should be improved!).