Add an option to subscribe to every Hold Invoice at once

[sergioabril]

Background

There is no easy way to get Hold Invoice events with one rpc subscription. You have to track them individually, and whenever you start tracking a bunch of them, it doesn't make sense keeping so many rpc subscriptions open, when you could just subscribe once and get all the events there (Just like it happens with regular invoices). This could be a problem for any service that could want to provide Hold Invoices as a solution for their users, and needed to track lots of them at once.

Your environment

• Lnd 0.6.1
• (Any environment or setup)

Steps to reproduce

1. Subscribe to subscribeInvoices

2. You will start getting standard events for invoices (even for hold invoices), such as OPEN and SETTLE, but not those advanced events used on Hold specific ones, such as ACCEPT or CANCEL.

3. You could then try subscribing to SubscribeSingleInvoice using the Sub-Server Invoices, but you should do that for each of the Hold invoices you are tracking.

Problem

Whenever you are tracking a bunch of hold invoices, doesn't make sense to subscribe to each of them individually.

Possible solutions

1. One option could be including an advanced SubscribeInvoices on the invoicesrpc Sub-Server, that gets every STATE change for every invoice. I'm not sure if it's possible to filter it to get Hold-Invoices-only here, but if it's not, then could be used as an advanced replacement of the regular SubscribeInvoices we already have.

2. Other option might be including Hold Invoice events on the regular subscribeInvoices rpc call. To avoid the hassle of receiving too many updates, it could be an optional parameter whenever you subscribe, set to false by default.

I'm not sure how this works under the hood, but I guess that maybe you would be getting all the state events for every invoice, not just for hold ones, and that's why you haven't included it as an option yet. If that's the case, and is not possible to filter out invoices by its kind before firing the events, then at least would be great to have a "get all the states for every invoice" as an advanced option (whether option 1 or option 2)

Of course, would be great to have a subscribeHoldInvoices on the sub-server invoices, that gets every STATE event, but for hold invoices only

[Roasbeef]

Can't this be emulated with the existing calls by making individual subscriptions for each open invoice you know of? Depending on your language, you can likely combine all those streams into one (say using something like a python generator that takes other generators and yields them all).

[sergioabril]

I guess they could probably be combined, and for a small amount of them it would be probably fine (using Node.js). However, if you have a few hundred open hold invoices, I can imagine things would get messy; specially if you lose connection at any point and have to reconnect every stream for every (still) open invoice; I mean, it's doable, but much easier with a single subscription, just like happens with regular Invoices. I also understand it is not a common case for regular wallets, but will probably be for any LN service using Hold invoices (That's why it could maybe be added to the Sub Server `invoices` as an advanced version, leaving the regular `subscribeInvoices` as it is)

…

On 27 May 2019, at 21:57, Olaoluwa Osuntokun ***@***.***> wrote: Can't this be emulated with the existing calls by making individual subscriptions for each open invoice you know of? Depending on your language, you can likely combine all those streams into one (say using something like a python generator that takes other generators and yields them all). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[joostjager]

I think that a variation on Subscribe(All)Invoices that sends out updates for every event is the way to go. Not only are the Accepted and Canceled events currently not sent, but the stream also does not include updates to the invoice htlc list that don't necessarily change the invoice state. Especially for multi-part receives, it may be useful to get some insight in how the total set is coming together.

One question is how to signal to SubscribeInvoices what the starting point is from which backlogged events need to be sent. Currently there is add_index and settle_index, but with the increased state space it may be better to introduce something new. Either an always increasing global update counter (goes up for every commit to the invoice db) or alternatively a timestamp. Most data in the invoice db already includes a timestamp, so that could possibly be implemented without a db migration. A timestamp also has more meaning than just a counter value.

Feedback on these kind of design aspects and/or other ideas are welcome.