Skip to content

htlcswitch+invoices: allow settling invoices via multi-path payments#3415

Merged
joostjager merged 4 commits into
lightningnetwork:masterfrom
joostjager:mpp
Dec 11, 2019
Merged

htlcswitch+invoices: allow settling invoices via multi-path payments#3415
joostjager merged 4 commits into
lightningnetwork:masterfrom
joostjager:mpp

Conversation

@joostjager
Copy link
Copy Markdown
Contributor

@joostjager joostjager commented Aug 19, 2019

Before this PR, MPP fields in the payload were parsed but not processed. This PR adds processing of those fields in the invoice registry. This means that it is now possible to settle an invoice using multiple partial payments.

@joostjager joostjager changed the title htlcswitch+invoices: multi-path payments [wip] [no review] htlcswitch+invoices: multi-path payments [wip] [don't review] Aug 23, 2019
@joostjager joostjager force-pushed the mpp branch 3 times, most recently from 5f1966a to 45a27a3 Compare August 30, 2019 20:30
@joostjager joostjager force-pushed the mpp branch 2 times, most recently from 9e0b20f to a4b9fa2 Compare October 11, 2019 09:16
@joostjager joostjager added incomplete WIP PR, not fully finalized, but light review possible blocked labels Oct 11, 2019
@joostjager joostjager self-assigned this Oct 23, 2019
Copy link
Copy Markdown
Contributor

@cfromknecht cfromknecht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just did a quick pass to refresh my memory of this pr. i know you said don't review, but just leaving some initial comments tho i'm sure the underlying pr has changed a bit

Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread cmd/lncli/commands.go Outdated
Comment thread cmd/lncli/commands.go Outdated
Comment thread cmd/lncli/commands.go Outdated
Comment thread lnrpc/rpc.proto Outdated
@joostjager joostjager changed the title htlcswitch+invoices: multi-path payments [wip] [don't review] htlcswitch+invoices: allow settling invoices via multi-path payments Nov 5, 2019
@joostjager joostjager added this to the 0.9.0 milestone Nov 5, 2019
@joostjager joostjager force-pushed the mpp branch 3 times, most recently from 9c81a17 to c03862c Compare November 6, 2019 11:44
Copy link
Copy Markdown
Contributor

@cfromknecht cfromknecht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so far so good, changes coming together :) should be pretty close to being able to do a full e2e!

Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
@halseth halseth self-requested a review November 7, 2019 09:45
@joostjager joostjager force-pushed the mpp branch 3 times, most recently from 9a8df74 to b75c07f Compare November 12, 2019 16:40
Comment thread channeldb/db.go Outdated
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imho it's cleaner if this is not passed as a closure but an interface. See #3694 for details (clock package).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already do it like this in two other locations. I didn't notice much problems with a closure really. As you say, if it is extended with additional functions like time.After it probably makes sense to have an interface

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nice thing about using the interface that once it's spread, refactor will be simpler as well as it may lead to better code hygene (common test clock implementation maybe?)

Comment thread channeldb/db.go Outdated
Comment thread channeldb/invoice_test.go Outdated
Comment thread channeldb/invoice_test.go Outdated
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Now closure or if you choose to instead use the clock.Clock interface as suggested earlier should be an incomit parameter of makeTestDB() function. Otherwise all other tests will depend on the real time.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to make that better, but it is also pre-existing.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or other option if you choose to stay with the Now closure is to still inject it in makeTestDB() or create a default Now() function for all tests and modfity makeTestDB() instead.

Comment thread channeldb/invoices.go Outdated
Comment thread channeldb/invoices.go Outdated
Comment thread invoices/update.go Outdated
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

invoices/update.go needs unit tests

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is covered by invoiceregistry_test.go. There is indeed an opportunity to now test this code in isolation, but this PR also didn't introduce this code, just moved it.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, missed that this is a dependent PR. Discussed this offline too, but to restate I'm on the opinion that separate units should be tested separately if possible.

Comment thread queue/min_time_heap.go Outdated
Comment thread invoices/invoiceregistry_test.go Outdated
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's quiet a bit of refactor in #3694 to move all test data generation to a separate compulation unit. I'd suggest building on that. Not sure how to do this properly without merging that first though.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is mostly rename/move it seems. I think it can be done independently?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine merging this if this PR is merged to master earlier. Optionally we could somehow make it part of this PR too (it's in commit: f663b33).

Comment thread cmd/lncli/commands.go Outdated
Comment thread go.mod Outdated
Comment thread invoices/invoiceregistry.go Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure what a good value would be. Maybe on the order of 1-2 min?

Comment thread invoices/invoiceregistry.go Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then why skip the HTLC if htlc.State != channeldb.HtlcStateSettled ? 🤔

Comment thread invoices/invoiceregistry.go Outdated
@joostjager joostjager force-pushed the mpp branch 7 times, most recently from f47ed9b to f3b071a Compare December 4, 2019 14:31
@joostjager
Copy link
Copy Markdown
Contributor Author

Itest fail, problem is that PaymentAddr doesn't match

@joostjager
Copy link
Copy Markdown
Contributor Author

Rebased on top of @cfromknecht's commits that create the payment address.

Copy link
Copy Markdown
Contributor

@halseth halseth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I think this one is nearly there 👍 Only a few comments.

Comment thread queue/priority_queue.go Outdated
Comment thread queue/priority_queue.go Outdated
Comment thread queue/priority_queue.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry_test.go Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could have a channel signal it is ready instead

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modified so that tick channels also tick immediately when the trigger time is already there.

Comment thread invoices/update.go Outdated
@joostjager
Copy link
Copy Markdown
Contributor Author

Note to self: don't forget to remove debug commits

Copy link
Copy Markdown
Contributor

@cfromknecht cfromknecht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did another pass, just one minor nit. is it time to remove debug commits?

Comment thread invoices/invoiceregistry.go Outdated
@joostjager
Copy link
Copy Markdown
Contributor Author

debug commits removed

Copy link
Copy Markdown
Contributor

@cfromknecht cfromknecht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🍾

@joostjager
Copy link
Copy Markdown
Contributor Author

@cfromknecht brought up the mpp timeout failure. Will address that in a follow up in coordination with @carlaKC.

Without this error returned, senders that actually support sending multiple shards and interpret this new failure will not function optimally. But seems acceptable for where we are right now.

Copy link
Copy Markdown
Contributor

@halseth halseth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All aboard the mpptrain! 🚂 LGTM

Comment thread queue/priority_queue.go Outdated
Comment thread invoices/invoiceregistry_test.go Outdated
Comment thread queue/priority_queue.go Outdated
Comment thread invoices/clock_test.go Outdated
Comment thread invoices/clock_test.go Outdated
Comment thread invoices/clock_test.go Outdated
Comment thread invoices/clock_test.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Comment thread invoices/invoiceregistry.go Outdated
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we use either signed or unsigned types for block height everywhere?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is inconsistent at the moment. But if you start changing, you will expand into many different areas and also replace one case by the other sometimes.

Comment thread invoices/clock_test.go Outdated
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice! May I suggest you to extend clock.Clock interface and use that as a common ground instead? Maybe also move this test clock in that package such that others can use it?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to delay this PR with a discussion on where to put that package/file. Given the time those took in past prs I'd rather avoid it (reference class forecasting).

Comment thread invoices/invoiceregistry_test.go Outdated
Comment thread invoices/invoiceregistry_test.go Outdated
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the file changed during review. The above comment meant to be here: "I think it's important to test cases where settlement may fail".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a case where the partial payment times out. Which other cases do you mean? Of course the total number of possible test cases is large, but I tried to cover at least the ones that usually occur.

Comment thread invoices/invoiceregistry.go Outdated
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this ever happen though?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. Perhaps we will do invoice cleanup in the future?

bhandras and others added 4 commits December 11, 2019 16:08
This commit introduces PriorityQueue, which is a general, heap
based priority queue, and PriorityQueueItem which is an interface
that concrete priority queue items must implement.
This implementation is encapsulated, users do not need to use any
other package for full functionality.
PriorityQueue exports the usual public methids: Push, Pop, Top,
Empty and Len. For full documentaton consult the priority_queue.go,
for usage: priority_queue_test.go
Copy link
Copy Markdown
Collaborator

@bhandras bhandras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

amp HTLC payments Related to invoices/payments

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants