Sync command grabs histories from lookahead spks.

[evanlinjin]

Describe the bug

There is a problem with the sync command in the esplora/electrum examples. Because of the 1000 lookahead default, we are trying to fetch txs for each spk (inc. lookahead).

We are fetching spks using SpkTxOutIndex::all_spks() and SpkTxOutIndex::unused_spks() to pass into {Electrum|Esplora}::sync (which includes lookahead spks).

To Reproduce

Run sync command with esplora/electrum cli examples (not wallet).

Expected behavior

We only fetch histories of revealed spks.

Suggested implementation

We probably need methods on KeychainTxOutIndex for non-lookahead versions for all_spks and unused_spks.

Maybe it is a bad idea to expose methods of SpkTxOutIndex from KeychainTxOutIndex?

[evanlinjin]

I think methods on KeychainTxOutIndex are rather confusing. This is my proposal:

    /// Get an unbounded spk iterator over a given `keychain`.
    pub fn unbounded_keychain_spk_iter(&self, keychain: &K) -> SpkIterator<Descriptor<DescriptorPublicKey>> { todo!() }
    /// Iterate over revealed spks of all keychains.
    pub fn revealed_spks(&self) -> impl Iterator<Item = (K, u32, &Script)> { core::iter::empty() }
    /// Iterate over revealed spks of a given `keychain`.
    pub fn revealed_keychain_spks(&self, keychain: &K) -> impl Iterator<Item = (u32, &Script)> { core::iter::empty() }
    /// Iterate over revealed and unused spks of all keychains.
    pub fn unused_spks(&self) -> impl Iterator<Item = (K, u32, &Script)> { core::iter::empty() }
    /// Iterate over revealed and unused spks of a given `keychain`.
    pub fn unused_keychain_spks(&self, keychain: &K) -> impl Iterator<Item = (u32, &Script)> { core::iter::empty() }

[LLFourn]

We probably need methods on KeychainTxOutIndex for non-lookahead versions for all_spks and unused_spks.

What is a non look ahead version? It should be impossible to see the values of the stored lookahead'd SPKs in the API of KeyChainTxOutIndex.

Maybe it is a bad idea to expose methods of SpkTxOutIndex from KeychainTxOutIndex?

Yes the lookahead feature has means that access to SpkTxOutIndex is a bug.

I think methods on KeychainTxOutIndex are rather confusing. This is my proposal:

    /// Get an unbounded spk iterator over a given `keychain`.
    pub fn unbounded_keychain_spk_iter(&self, keychain: &K) -> SpkIterator<Descriptor<DescriptorPublicKey>> { todo!() }
    /// Iterate over revealed spks of all keychains.
    pub fn revealed_spks(&self) -> impl Iterator<Item = (K, u32, &Script)> { core::iter::empty() }
    /// Iterate over revealed spks of a given `keychain`.
    pub fn revealed_keychain_spks(&self, keychain: &K) -> impl Iterator<Item = (u32, &Script)> { core::iter::empty() }
    /// Iterate over revealed and unused spks of all keychains.
    pub fn unused_spks(&self) -> impl Iterator<Item = (K, u32, &Script)> { core::iter::empty() }
    /// Iterate over revealed and unused spks of a given `keychain`.
    pub fn unused_keychain_spks(&self, keychain: &K) -> impl Iterator<Item = (u32, &Script)> { core::iter::empty() }

LGTM except that you'll probably need one to get the unbounded BTreeMap of iterators

[evanlinjin]

LGTM except that you'll probably need one to get the unbounded BTreeMap of iterators

@LLFourn What do you mean by this? Thanks!

[LLFourn]

I mean we still need this method: https://docs.rs/bdk_chain/latest/bdk_chain/keychain/struct.KeychainTxOutIndex.html#method.spks_of_all_keychains