Skip to content

Signing support for Miniscript Descriptors#24149

Merged
fanquake merged 12 commits into
bitcoin:masterfrom
darosior:miniscript_wallet_signing
Feb 16, 2023
Merged

Signing support for Miniscript Descriptors#24149
fanquake merged 12 commits into
bitcoin:masterfrom
darosior:miniscript_wallet_signing

Conversation

@darosior
Copy link
Copy Markdown
Member

@darosior darosior commented Jan 25, 2022

This makes the Miniscript descriptors solvable.

Note this introduces signing support for much more complex scripts than the wallet was previously able to solve, and the whole tooling isn't provided for a complete Miniscript integration in the wallet. Particularly, the PSBT<->Miniscript integration isn't entirely covered in this PR.

@DrahtBot
Copy link
Copy Markdown
Contributor

DrahtBot commented Jan 26, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK sipa, achow101
Stale ACK instagibbs

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #26756 (wallet: Replace GetBalance() logic with AvailableCoins() by w0xlt)
  • #26627 (wallet: Migrate non-HD keys with single combo containing a list of keys by achow101)
  • #26626 (descriptors: Add a KEY expression representing a list of individual keys by achow101)
  • #26573 (Wallet: don't underestimate the fees when spending a Taproot output by darosior)
  • #26567 (Wallet: estimate the size of signed inputs using descriptors by darosior)
  • #26101 (script: create V1SigVersion for functions which should only accept taproot/tapscript by theuni)
  • #26076 (Switch hardened derivation marker to h (in normalized descriptors and new wallets) by Sjors)
  • #22838 (descriptors: Be able to specify change and receiving in a single descriptor string by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Copy Markdown
Member

@achow101 achow101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Particularly, the PSBT<->Miniscript integration isn't part of this PR.

I don't think it would be particularly hard considering that you're already using SignatureData for non-wallet data lookups. I think all that would really need to be done is to update FillSignatureData to include the preimages from the PSBT.

Edit: This diff does the trick

diff --git a/src/psbt.cpp b/src/psbt.cpp
index 8248609ba6..9e09ccf15d 100644
--- a/src/psbt.cpp
+++ b/src/psbt.cpp
@@ -113,6 +113,18 @@ void PSBTInput::FillSignatureData(SignatureData& sigdata) const
     for (const auto& key_pair : hd_keypaths) {
         sigdata.misc_pubkeys.emplace(key_pair.first.GetID(), key_pair);
     }
+    for (const auto& [hash, preimage] : ripemd160_preimages) {
+        sigdata.hash_preimages.emplace(std::vector<unsigned char>(hash.begin(), hash.end()), preimage);
+    }
+    for (const auto& [hash, preimage] : sha256_preimages) {
+        sigdata.hash_preimages.emplace(std::vector<unsigned char>(hash.begin(), hash.end()), preimage);
+    }
+    for (const auto& [hash, preimage] : hash160_preimages) {
+        sigdata.hash_preimages.emplace(std::vector<unsigned char>(hash.begin(), hash.end()), preimage);
+    }
+    for (const auto& [hash, preimage] : hash256_preimages) {
+        sigdata.hash_preimages.emplace(std::vector<unsigned char>(hash.begin(), hash.end()), preimage);
+    }
 }
 
 void PSBTInput::FromSignatureData(const SignatureData& sigdata)

Comment thread src/script/miniscript.h Outdated
Comment thread src/script/miniscript.cpp Outdated
Comment thread src/script/miniscript.h Outdated
Comment thread src/test/descriptor_tests.cpp Outdated
@darosior darosior mentioned this pull request Feb 18, 2022
4 tasks
@darosior darosior force-pushed the miniscript_wallet_signing branch from c226ee3 to 8e268cc Compare February 21, 2022 14:29
@darosior
Copy link
Copy Markdown
Member Author

Rebased, addressed Andrew's comments, added some cleanups from @sipa, and added a miniscript_random fuzz target which generates a random Miniscript node based on a binary encoding. We might eventually add another one which tries to be smart, either here or in a follow-up.

I don't think it would be particularly hard considering that you're already using SignatureData for non-wallet data lookups. I think all that would really need to be done is to update FillSignatureData to include the preimages from the PSBT.

Yeah, but i figured it would be cleaner to have this as part of a PR which'd also update the various PSBT RPCs and test it using those.

This is a workaround for Miniscript descriptors containing hash
challenges. For those we can't mock the signature creator without making
OP_EQUAL mockable in the interpreter, so CalculateMaximumInputSize will
always return -1 and outputs for these descriptors would appear
unsolvable while they actually are.
We'll add more of them in the next commit, let's keep it bearable.
We'll need a better integration of the hash preimages PSBT fields to
satisfy Miniscript with such challenges from the RPC.

Thanks to Greg Sanders for his examples and suggestions to improve this
test.
@darosior
Copy link
Copy Markdown
Member Author

@instagibbs thanks for the suggestions and the examples. This functional test was clearly lacking imagination. :) However what you suggest is already thoroughly tested by the unit tests (and to a lesser extent by the fuzz tests). But it's inexpensive to test also in the functional test so i included your examples and added a couple more things to test for each descriptor: the number of signatures expected (always equal to the number of private keys we have in the descriptor), whether this "completes" the transaction or not, and the expected witness stack size of the satisfaction (to assert we used the expected satisfaction path).

@sipa thanks, i think it'll be helpful to reviewers now and to future readers. I squashed Simplify InputStack constructor calls into the first commit of this PR. I cherry-picked all your commits adding comments and explanations about the satisfaction logic, that i squashed in a single documentation commit (now the second commit of this PR). I was going to say the two commits aligning the properties with the website belong in another PR.. But it's a tiny diff and the e property is related to signing, so i just cherry-picked them as well (squashed in one commit).

@darosior darosior force-pushed the miniscript_wallet_signing branch from 33d30d1 to 3c6e3e0 Compare February 11, 2023 13:16
darosior and others added 2 commits February 11, 2023 16:51
This is a "dumb" way of randomly generating a Miniscript node from
fuzzer input. It defines a strict binary encoding and will always generate
a node defined from the encoding without "helping" to create valid nodes.
It will cut through as soon as it encounters an invalid fragment so
hopefully the fuzzer can tend to learn the encoding and generate valid
nodes with a higher probability.

On a valid generated node a number of invariants are checked, especially
around the satisfactions and testing them against the Script
interpreter.

The node generation and testing is modular in order to later introduce
other ways to generate nodes from fuzzer inputs with minimal code.

Co-Authored-By: Pieter Wuille <pieter@wuille.net>
At the expense of more complexity, this target generates a valid
Miniscript node at every iteration.

This target will at first run populate a list of recipe (a map from
desired type to possible ways of creating such type) and curate it
(remove the unavailable or redundant recipes).
Then, at each iteration it will pick a type, choose a manner to create a
node of such type from the available recipes, and then
pseudo-recursively do the same for the type constraints of the picked
recipe.

For instance, if it is instructed based on the fuzzer output to create a
Miniscript node of type 'Bd', it could choose to create an 'or_i(subA, subB)'
nodes with type constraints 'B' for subA and 'Bd' for subB. It then
consults the recipes for creating subA and subB, etc...

Here is the list of all the existing recipes, by type constraint:

B: 0()
B: 1()
B: older()
B: after()
B: sha256()
B: hash256()
B: ripemd160()
B: hash160()
B: c:(K)
B: d:(Vz)
B: j:(Bn)
B: n:(B)
B: and_v(V,B)
B: and_b(B,W)
B: or_b(Bd,Wd)
B: or_d(Bdu,B)
B: or_i(B,B)
B: andor(Bdu,B,B)
B: thresh(Bdu)
B: thresh(Bdu,Wdu)
B: thresh(Bdu,Wdu,Wdu)
B: multi()

V: v:(B)
V: and_v(V,V)
V: or_c(Bdu,V)
V: or_i(V,V)
V: andor(Bdu,V,V)

K: pk_k()
K: pk_h()
K: and_v(V,K)
K: or_i(K,K)
K: andor(Bdu,K,K)

W: a:(B)
W: s:(Bo)

Bz: 0()
Bz: 1()
Bz: older()
Bz: after()
Bz: n:(Bz)
Bz: and_v(Vz,Bz)
Bz: or_d(Bzdu,Bz)
Bz: andor(Bzdu,Bz,Bz)
Bz: thresh(Bzdu)

Vz: v:(Bz)
Vz: and_v(Vz,Vz)
Vz: or_c(Bzdu,Vz)
Vz: andor(Bzdu,Vz,Vz)

Bo: sha256()
Bo: hash256()
Bo: ripemd160()
Bo: hash160()
Bo: c:(Ko)
Bo: d:(Vz)
Bo: j:(Bon)
Bo: n:(Bo)
Bo: and_v(Vz,Bo)
Bo: and_v(Vo,Bz)
Bo: or_d(Bodu,Bz)
Bo: or_i(Bz,Bz)
Bo: andor(Bzdu,Bo,Bo)
Bo: andor(Bodu,Bz,Bz)
Bo: thresh(Bodu)

Vo: v:(Bo)
Vo: and_v(Vz,Vo)
Vo: and_v(Vo,Vz)
Vo: or_c(Bodu,Vz)
Vo: or_i(Vz,Vz)
Vo: andor(Bzdu,Vo,Vo)
Vo: andor(Bodu,Vz,Vz)

Ko: pk_k()
Ko: and_v(Vz,Ko)
Ko: andor(Bzdu,Ko,Ko)

Bn: sha256()
Bn: hash256()
Bn: ripemd160()
Bn: hash160()
Bn: c:(Kn)
Bn: d:(Vz)
Bn: j:(Bn)
Bn: n:(Bn)
Bn: and_v(Vz,Bn)
Bn: and_v(Vn,B)
Bn: and_b(Bn,W)
Bn: multi()

Vn: v:(Bn)
Vn: and_v(Vz,Vn)
Vn: and_v(Vn,V)

Kn: pk_k()
Kn: pk_h()
Kn: and_v(Vz,Kn)
Kn: and_v(Vn,K)

Bon: sha256()
Bon: hash256()
Bon: ripemd160()
Bon: hash160()
Bon: c:(Kon)
Bon: d:(Vz)
Bon: j:(Bon)
Bon: n:(Bon)
Bon: and_v(Vz,Bon)
Bon: and_v(Von,Bz)

Von: v:(Bon)
Von: and_v(Vz,Von)
Von: and_v(Von,Vz)

Kon: pk_k()
Kon: and_v(Vz,Kon)

Bd: 0()
Bd: sha256()
Bd: hash256()
Bd: ripemd160()
Bd: hash160()
Bd: c:(Kd)
Bd: d:(Vz)
Bd: j:(Bn)
Bd: n:(Bd)
Bd: and_b(Bd,Wd)
Bd: or_b(Bd,Wd)
Bd: or_d(Bdu,Bd)
Bd: or_i(B,Bd)
Bd: or_i(Bd,B)
Bd: andor(Bdu,B,Bd)
Bd: thresh(Bdu)
Bd: thresh(Bdu,Wdu)
Bd: thresh(Bdu,Wdu,Wdu)
Bd: multi()

Kd: pk_k()
Kd: pk_h()
Kd: or_i(K,Kd)
Kd: or_i(Kd,K)
Kd: andor(Bdu,K,Kd)

Wd: a:(Bd)
Wd: s:(Bod)

Bzd: 0()
Bzd: n:(Bzd)
Bzd: or_d(Bzdu,Bzd)
Bzd: andor(Bzdu,Bz,Bzd)
Bzd: thresh(Bzdu)

Bod: sha256()
Bod: hash256()
Bod: ripemd160()
Bod: hash160()
Bod: c:(Kod)
Bod: d:(Vz)
Bod: j:(Bon)
Bod: n:(Bod)
Bod: or_d(Bodu,Bzd)
Bod: or_i(Bz,Bzd)
Bod: or_i(Bzd,Bz)
Bod: andor(Bzdu,Bo,Bod)
Bod: andor(Bodu,Bz,Bzd)
Bod: thresh(Bodu)

Kod: pk_k()
Kod: andor(Bzdu,Ko,Kod)

Bu: 0()
Bu: 1()
Bu: sha256()
Bu: hash256()
Bu: ripemd160()
Bu: hash160()
Bu: c:(K)
Bu: d:(Vz)
Bu: j:(Bnu)
Bu: n:(B)
Bu: and_v(V,Bu)
Bu: and_b(B,W)
Bu: or_b(Bd,Wd)
Bu: or_d(Bdu,Bu)
Bu: or_i(Bu,Bu)
Bu: andor(Bdu,Bu,Bu)
Bu: thresh(Bdu)
Bu: thresh(Bdu,Wdu)
Bu: thresh(Bdu,Wdu,Wdu)
Bu: multi()

Bzu: 0()
Bzu: 1()
Bzu: n:(Bz)
Bzu: and_v(Vz,Bzu)
Bzu: or_d(Bzdu,Bzu)
Bzu: andor(Bzdu,Bzu,Bzu)
Bzu: thresh(Bzdu)

Bou: sha256()
Bou: hash256()
Bou: ripemd160()
Bou: hash160()
Bou: c:(Ko)
Bou: d:(Vz)
Bou: j:(Bonu)
Bou: n:(Bo)
Bou: and_v(Vz,Bou)
Bou: and_v(Vo,Bzu)
Bou: or_d(Bodu,Bzu)
Bou: or_i(Bzu,Bzu)
Bou: andor(Bzdu,Bou,Bou)
Bou: andor(Bodu,Bzu,Bzu)
Bou: thresh(Bodu)

Bnu: sha256()
Bnu: hash256()
Bnu: ripemd160()
Bnu: hash160()
Bnu: c:(Kn)
Bnu: d:(Vz)
Bnu: j:(Bnu)
Bnu: n:(Bn)
Bnu: and_v(Vz,Bnu)
Bnu: and_v(Vn,Bu)
Bnu: and_b(Bn,W)
Bnu: multi()

Bonu: sha256()
Bonu: hash256()
Bonu: ripemd160()
Bonu: hash160()
Bonu: c:(Kon)
Bonu: d:(Vz)
Bonu: j:(Bonu)
Bonu: n:(Bon)
Bonu: and_v(Vz,Bonu)
Bonu: and_v(Von,Bzu)

Bdu: 0()
Bdu: sha256()
Bdu: hash256()
Bdu: ripemd160()
Bdu: hash160()
Bdu: c:(Kd)
Bdu: d:(Vz)
Bdu: j:(Bnu)
Bdu: n:(Bd)
Bdu: and_b(Bd,Wd)
Bdu: or_b(Bd,Wd)
Bdu: or_d(Bdu,Bdu)
Bdu: or_i(Bu,Bdu)
Bdu: or_i(Bdu,Bu)
Bdu: andor(Bdu,Bu,Bdu)
Bdu: thresh(Bdu)
Bdu: thresh(Bdu,Wdu)
Bdu: thresh(Bdu,Wdu,Wdu)
Bdu: multi()

Wdu: a:(Bdu)
Wdu: s:(Bodu)

Bzdu: 0()
Bzdu: n:(Bzd)
Bzdu: or_d(Bzdu,Bzdu)
Bzdu: andor(Bzdu,Bzu,Bzdu)
Bzdu: thresh(Bzdu)

Bodu: sha256()
Bodu: hash256()
Bodu: ripemd160()
Bodu: hash160()
Bodu: c:(Kod)
Bodu: d:(Vz)
Bodu: j:(Bonu)
Bodu: n:(Bod)
Bodu: or_d(Bodu,Bzdu)
Bodu: or_i(Bzu,Bzdu)
Bodu: or_i(Bzdu,Bzu)
Bodu: andor(Bzdu,Bou,Bodu)
Bodu: andor(Bodu,Bzu,Bzdu)
Bodu: thresh(Bodu)

Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
@darosior darosior force-pushed the miniscript_wallet_signing branch from 3c6e3e0 to 36478ac Compare February 11, 2023 15:52
Copy link
Copy Markdown
Member

@instagibbs instagibbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 36478ac

reviewed tests only

Comment thread test/functional/wallet_miniscript.py Outdated
@DrahtBot DrahtBot requested a review from achow101 February 13, 2023 14:36
Co-Authored-By: Andrew Chow <github@achow101.com>
@darosior darosior force-pushed the miniscript_wallet_signing branch from 36478ac to 6c7a17a Compare February 13, 2023 14:39
Copy link
Copy Markdown
Member

@sipa sipa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 6c7a17a (to the extent that it's not my own code).

}
BOOST_CHECK(SignSignature(keystore, *txPrev, tx, 0, SIGHASH_ALL));
SignatureData empty;
BOOST_CHECK(SignSignature(keystore, *txPrev, tx, 0, SIGHASH_ALL, empty));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the need for all these dummy SignatureData objects that need to be created in calls, would it make sense to add another overload of SignSignature that has no SignaturaData& argument?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do if i need to retouch.

@DrahtBot DrahtBot requested a review from instagibbs February 13, 2023 21:07
@fanquake fanquake added this to the 25.0 milestone Feb 14, 2023
@achow101
Copy link
Copy Markdown
Member

ACK 6c7a17a

Reviewed code and non-fuzz tests.

@Sjors
Copy link
Copy Markdown
Member

Sjors commented Feb 15, 2023

@darosior do I understand correctly that you need #26567 in order for PSBT stuff to work? Can I just combine both PR's to continue the test I was doing, or are there more changes required?

@darosior
Copy link
Copy Markdown
Member Author

@Sjors see #24149 (comment). A minimal PSBT integration is part of this PR now. We treat Miniscript-related data when we are given a PSBT. However to fill Miniscript-related data to a PSBT from our wallet will need more work than just #26567, and also some design decisions to be taken.

@fanquake fanquake merged commit fb82d91 into bitcoin:master Feb 16, 2023
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Feb 16, 2023
sipa added a commit to sipa/miniscript that referenced this pull request Feb 16, 2023
930e2f2 Update the Miniscript fuzz tests from Bitcoin Core master (Antoine Poinsot)
f62abcf Update the Miniscript unit tests from Bitcoin master (Antoine Poinsot)
fd9bed0 miniscript: update the source from Bitcoin Core master branch (Antoine Poinsot)

Pull request description:

  The PRs introducing Miniscript to Bitcoin Core contained the most up to date version of the code (bitcoin/bitcoin#24148 and bitcoin/bitcoin#24149).

  Now they are both merged, update the code here to reflect the updates made in these PRs.

ACKs for top commit:
  sipa:
    ACK 930e2f2. Matches the Bitcoin Core master branch code, and compiles.

Tree-SHA512: 0ef587d005dc472cada86b99e62f37aa6bbfa09a8d1db95726a0103c0e827880d557ebce39c87eeb281c952b77fcf462c9a779008eed5b2d0f2d63478f4fab4c
using miniscript::operator"" _mst;

//! Construct a miniscript node as a shared_ptr.
template<typename... Args> NodeRef MakeNodeRef(Args&&... args) { return miniscript::MakeNodeRef<CPubKey>(KEY_COMP, std::forward<Args>(args)...); }
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm looks like this checks for duplicate keys on every single node created whereas we could just check that on the top-level node in GenNode()?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

domob1812 added a commit to domob1812/namecoin-core that referenced this pull request Feb 21, 2023
Some of the new miniscript signing tests in wallet_miniscript.py
are disabled for now as they fail on Namecoin, this needs debugging
and fixing in the future.  These were introduced upstream in
bitcoin/bitcoin#24149.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants