Skip to content

[KILO]: Quarantine paymentDescriptor to the lnwallet package#8981

Merged
guggero merged 18 commits into
lightningnetwork:masterfrom
ProofOfKeags:refactor/payment-descriptor-quarantine
Sep 12, 2024
Merged

[KILO]: Quarantine paymentDescriptor to the lnwallet package#8981
guggero merged 18 commits into
lightningnetwork:masterfrom
ProofOfKeags:refactor/payment-descriptor-quarantine

Conversation

@ProofOfKeags
Copy link
Copy Markdown
Collaborator

Change Description

This PR's main contribution is quarantining the paymentDescriptor type to the lnwallet package. This is in preparation for upcoming changes where we try and change the fundamental unit of the update log to a LogUpdate, as opposed to the former PaymentDescriptor. This is because we will functionally eliminate the PaymentDescriptor as a meaningful idea and instead just have a linear sequence of log updates accompanied by some sparse indices for other metadata. To effectively accomplish this, however, we have to reduce the scope of making significant changes to this data structure and that begins by making sure it does not leak outside the package.

The primary observation that allows us to do this is to realize that the only other place this is currently used is in the processRemote[Adds|SettleFails] functions within the ChannelLink. What is notable about this is that this information is already duplicated in the forwarding package itself. A forwarding package is already a sequence of updates that is committed by a given commitment transaction. Since the link does not make use of the metadata beyond that we simply just make it use the LogUpdates inside the forwarding package rather than the accompanying payment descriptors.

In my best assessment this is completely safe to do and so far none of the testing I have done has revealed that we cannot do this. However, it is worth paying special attention to what I am doing here and ensuring that it does not miss an important detail on which we rely.

Steps to Test

make unit pkg=lnwallet
make itest

Pull Request Checklist

Testing

  • Your PR passes all CI checks.
  • Tests covering the positive and negative (error paths) are included.
  • Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

📝 Please see our Contribution Guidelines for further guidance.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Aug 6, 2024

Important

Review skipped

Auto reviews are limited to specific labels.

Labels to auto review (1)
  • llm-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@ProofOfKeags ProofOfKeags added the size/kilo medium, proper context needed, less than 1000 lines label Aug 6, 2024
@ProofOfKeags ProofOfKeags changed the title Quarantine paymentDescriptor to the lnwallet package [KILO]: Quarantine paymentDescriptor to the lnwallet package Aug 6, 2024
@ProofOfKeags ProofOfKeags force-pushed the refactor/payment-descriptor-quarantine branch 2 times, most recently from 509032f to 32cdf57 Compare August 6, 2024 06:20
@ProofOfKeags ProofOfKeags requested a review from carlaKC August 6, 2024 17:09
@ProofOfKeags ProofOfKeags force-pushed the refactor/payment-descriptor-quarantine branch 2 times, most recently from cc6272f to 548c6a5 Compare August 6, 2024 18:05
Copy link
Copy Markdown
Member

@yyforyongyu yyforyongyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice - left some comments re how we can break the large commit 05ac04d since this refactor is touching the most critical part.

Comment thread lnwallet/channel.go Outdated
Comment thread lnwallet/payment_descriptor.go Outdated
Comment thread htlcswitch/link.go
Comment thread lnwallet/channel.go
Comment thread lnwallet/channel.go
Comment thread lnwallet/channel.go Outdated
Comment thread lnwallet/payment_descriptor.go Outdated
Copy link
Copy Markdown
Collaborator

@carlaKC carlaKC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beautiful cleanup, lowkey blows my mind that this works so cleanly. cACK, just did a first pass to familiarize myself - still need to do a more thorough run through of the details!

+1 for YY's commit restructuring comments.

This is in preparation for upcoming changes where we try and change the fundamental unit of the update log to a LogUpdate, as opposed to the former PaymentDescriptor.

Is there a PR/issue where we attempt this? Would like to add some context for myself.

Comment thread htlcswitch/link.go Outdated
Comment thread htlcswitch/link.go Outdated
Comment thread htlcswitch/link.go
@ProofOfKeags ProofOfKeags force-pushed the refactor/payment-descriptor-quarantine branch from 548c6a5 to aebec95 Compare August 9, 2024 19:16
@ProofOfKeags ProofOfKeags added this to the v0.19.0 milestone Aug 15, 2024
@ProofOfKeags ProofOfKeags added the P1 MUST be fixed or reviewed label Aug 15, 2024
@ProofOfKeags ProofOfKeags force-pushed the refactor/payment-descriptor-quarantine branch 2 times, most recently from 5096e9e to dd4e90e Compare August 15, 2024 18:55
@lightninglabs-deploy
Copy link
Copy Markdown
Collaborator

@ProofOfKeags, remember to re-request review from reviewers when ready

@ProofOfKeags ProofOfKeags force-pushed the refactor/payment-descriptor-quarantine branch from dd4e90e to fb3f3dc Compare August 16, 2024 18:37
@ProofOfKeags
Copy link
Copy Markdown
Collaborator Author

ProofOfKeags commented Aug 16, 2024

+1 for YY's commit restructuring comments.

I will do that but I don't want to hear any complaints about diff churn. There was a reason this was done as one commit originally.

image

@ProofOfKeags ProofOfKeags force-pushed the refactor/payment-descriptor-quarantine branch 6 times, most recently from 2c0b8a3 to baf8f4f Compare August 16, 2024 21:39
Here we add a function that is capable of recovering LogUpdates from
paymentDescriptors and we refactor the lnwallet code to use this
rather than doing JIT inline construction of the LogUpdates.
This is done as part of a systematic removal of PaymentDescriptor from
the mechanics of the htlcswitch package.
This is done as part of a systematic removal of PaymentDescriptor from
the mechanics of the htlcswitch package.
…ignature

This is part of a systematic removal of PaymentDescriptor from the mechanics
of the htlcswitch package.
This is part of a systematic removal of PaymentDescriptor from the
mechanics of the htlcswitch package.
…ature

This is part of a systematic removal of PaymentDescriptor from the mechanics
of the htlcswitch package.
…ll signature

This is part of a systematic removal of PaymentDescriptor from the mechanics
of the htlcswitch package.
…returns

This is part of a systematic removal of PaymentDescriptor from the public
API of the lnwallet package. This marks the last change needed before we
make the PaymentDescriptor structure private.
…ails

This is part of a systematic removal of PaymentDescriptor from the mechanics
of the htlcswitch package.
This function is no longer used as of the last commit and it is the
last remaining leak of the PaymentDescriptor type through the public
API.
@ProofOfKeags ProofOfKeags force-pushed the refactor/payment-descriptor-quarantine branch from 4a2a693 to f3f2b4f Compare September 10, 2024 03:00
Copy link
Copy Markdown
Member

@yyforyongyu yyforyongyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM🦾

In the future it may be nice to have some docs explaining the relationship among htlcPacket, paymentDescriptor and LogUpdate, which I believe are the core data structures used in different subsystems.

In case others wanna review it, my summary is: overall this PR is a refactoring effort for lnwallet, with the only functional change in this line.

  • Before this PR, the log updates in the fwdPkg are pointing to a list of unfiltered updates, with fwdPkg.Add pointing to all the adds and fwdPkg.SettleFails pointing to all the settles/fails we've received and stored in remoteLogUpdate.
  • After this PR, these updates are now filtered using the remote/local chain tail, which I believe is the right change.

@ProofOfKeags
Copy link
Copy Markdown
Collaborator Author

Looks like the only test failures are from something going on with the sweeper?

@yyforyongyu
Copy link
Copy Markdown
Member

Looks like the only test failures are from something going on with the sweeper?

yeah flakes

The objective of this commit is to make paymentDescriptor a private
data structure so we can quarantine it to the lnwallet package.
To accomplish this we had to prevent it from leaking out via the
arguments or return values of the public functions in lnwallet.
This naturally had consequences for the htlcswitch package as we
choose other mechanisms for tracking the data that paymentDescriptor
was responsible for.

Astoundingly, this was highly successful and allowed us to remove
a ton of redundant code. The diff for this commit represents a
substantial reduction in total lines of code as well as extraneous
arguments and return values from key functions.

This also sets the stage for future commits where we actually will
be attempting to rid lnwallet of paymentDescriptor completely.
@ProofOfKeags ProofOfKeags force-pushed the refactor/payment-descriptor-quarantine branch from f3f2b4f to 93d17a4 Compare September 10, 2024 20:57
@ProofOfKeags
Copy link
Copy Markdown
Collaborator Author

This PR is ready for merge if @Roasbeef or @Crypt-iQ do not object by EOD Wednesday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dynamic commitments no-changelog P1 MUST be fixed or reviewed size/kilo medium, proper context needed, less than 1000 lines

Projects

Status: Merged
Status: Done

Development

Successfully merging this pull request may close these issues.

6 participants