Skip to content

Revamp KeychainTxOutIndex API to be safer#1269

Merged
danielabrozzoni merged 7 commits intobitcoindevkit:masterfrom
evanlinjin:issue/1268
Jan 18, 2024
Merged

Revamp KeychainTxOutIndex API to be safer#1269
danielabrozzoni merged 7 commits intobitcoindevkit:masterfrom
evanlinjin:issue/1268

Conversation

@evanlinjin
Copy link
Copy Markdown
Member

@evanlinjin evanlinjin commented Jan 12, 2024

Closes #1268

Description

Previously SpkTxOutIndex methods can be called from KeychainTxOutIndex due to the DeRef implementation. However, the internal SpkTxOut will also contain lookahead spks resulting in an error-prone API.

SpkTxOutIndex methods are now not directly callable from KeychainTxOutIndex. Methods of KeychainTxOutIndex are renamed for clarity. I.e. methods that return an unbounded spk iter are prefixed with unbounded.

In addition to this, I also optimized the peek-address logic of bdk::Wallet using the optimized <SpkIterator as Iterator>::nth implementation.

Notes to the reviewers

This is mostly refactoring, but can also be considered a bug-fix (as the API before was very problematic).

Changelog notice

Changed

  • Wallet's peek-address logic is optimized by making use of <SpkIterator as Iterator>::nth.
  • KeychainTxOutIndex API is refactored to better differentiate between methods that return unbounded vs stored spks.
  • KeychainTxOutIndex no longer directly exposes SpkTxOutIndex methods via DeRef. This was problematic because SpkTxOutIndex also contains lookahead spks which we want to hide.

Added

  • SpkIterator::descriptor method which gets a reference to the internal descriptor.

Checklists

All Submissions:

  • I've signed all my commits
  • I followed the contribution guidelines
  • I ran cargo fmt and cargo clippy before committing

New Features:

* [ ] I've added tests for the new feature

  • I've added docs for the new feature

Bugfixes:

  • This pull request breaks the existing API
  • I've added tests to reproduce the issue which are now passing
  • I'm linking the issue being fixed by this PR

@evanlinjin evanlinjin self-assigned this Jan 12, 2024
This allows us to pass this to chain sources without calling
`Iterator::collect` first.
@evanlinjin evanlinjin added this to the 1.0.0 milestone Jan 13, 2024
@evanlinjin evanlinjin marked this pull request as ready for review January 13, 2024 12:55
@evanlinjin evanlinjin added bug Something isn't working api A breaking API change labels Jan 16, 2024
@notmandatory notmandatory mentioned this pull request Jan 16, 2024
23 tasks
Copy link
Copy Markdown
Member

@danielabrozzoni danielabrozzoni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments. I still haven't finished my review, but I wanted to post this in the meantime :)

Comment thread crates/bdk/src/wallet/mod.rs
Comment thread example-crates/example_esplora/src/main.rs
Comment thread crates/bdk/src/wallet/mod.rs
Comment thread crates/chain/src/keychain/txout_index.rs Outdated
Comment thread crates/chain/src/keychain/txout_index.rs Outdated
Previously `SpkTxOutIndex` methods can be called from
`KeychainTxOutIndex` due to the `DeRef` implementation. However, the
internal `SpkTxOut` will also contain lookahead spks resulting in an
error-prone API.

`SpkTxOutIndex` methods are now not directly callable from
`KeychainTxOutIndex`. Methods of `KeychainTxOutIndex` are renamed for
clarity. I.e. methods that return an unbounded spk iter are prefixed
with `unbounded`.
Otherwise there will be no way to view the descriptor of the
`SpkIterator`.
Changes the peek address logic to use the optimized `Iterator::nth`
implementation of `SpkIterator`.

Additionally, docs are added for panics that will occur when the caller
requests for addresses with out-of-bound derivation indices (BIP32).
Copy link
Copy Markdown
Collaborator

@LLFourn LLFourn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 83e7b7e

Copy link
Copy Markdown
Member

@danielabrozzoni danielabrozzoni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding, this PR is making some SpkTxOutIndex methods slightly more difficult to call (you need to use KeychainTxOutIndex::inner instead of just using Deref).

These are SpkTxOutIndex methods that are not directly exposed from KeychainTxOutIndex:

  • all_spks
  • insert_spk
  • outputs_in_range
  • txouts
  • txouts_in_tx

It makes sense to me that insert_spk and all_spks are not exposed: you should let the KeychainTxOutIndex insert the spks, and all_spks exposes the lookahead spks.

But I just can't figure out why outputs_in_range, txouts, and txouts_in_tx shouldn't be exposed as well - is there a particular reason?

/// Return a reference to the internal [`SpkTxOutIndex`].
///
/// **WARNING:** The internal index will contain lookahead spks. Refer to
/// [struct-level docs](KeychainTxOutIndex) for more about `lookahead`.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we will eventually avoid exposing inner, or is it useful for some use cases, and so exposing it with a warning is good enough?

Copy link
Copy Markdown
Member Author

@evanlinjin evanlinjin Jan 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would assume in some cases getting lookahead spks will be handy.

I.e. we haven't finalized how a CBF chain-source will look like and how it will interact with bdk_chain structures. However, it might make sense to match lookahead spks against GCS (filters). Or maybe using an unbounded spk iterator makes more sense?

Maybe later on, we determine that it will make sense to use helper methods on KeychainTxOutIndex instead. However, for most usecases now, outputting lookahead spks is almost always non-intentional.

Copy link
Copy Markdown
Member Author

@evanlinjin evanlinjin Jan 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In summary, I think we should keep .inner with a warning (for now).

Comment thread example-crates/example_esplora/src/main.rs
@evanlinjin
Copy link
Copy Markdown
Member Author

But I just can't figure out why outputs_in_range, txouts, and txouts_in_tx shouldn't be exposed as well - is there a particular reason?

@danielabrozzoni I didn't include them because they weren't used in the examples. I guess it doesn't hurt to include?

`txouts` and `txouts_in_tx` are exposed from `SpkTxOutIndex`, but
modified to remove nested unions.

Add `keychain_outpoints_in_range` that iterates over outpoints of a
given keychain derivation range.
Copy link
Copy Markdown
Member

@danielabrozzoni danielabrozzoni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 71fff16

@danielabrozzoni danielabrozzoni merged commit 60abd87 into bitcoindevkit:master Jan 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api A breaking API change bug Something isn't working

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Sync command grabs histories from lookahead spks.

3 participants