no_std support

[justinmoon]

Hardware wallets should be able to use BDK.

This will need to happen in a few steps:

• rust-bitcoin (no_std support rust-bitcoin/rust-bitcoin#409 | no_std support, keeping MSRV rust-bitcoin/rust-bitcoin#603 and std -> core rust-bitcoin/rust-bitcoin#614)
• rust-miniscript (no_std support rust-bitcoin/rust-miniscript#201 | Add no_std support rust-bitcoin/rust-miniscript#277)
• bdk

I have an ugly prototype for rust-bitcoin using bare-io to polyfill std::io stuff that's missing from no_std. Going to make a draft PR shortly and get some feedback.

[mcroad]

rust-bitcoin/rust-bitcoin#603 added no_std support

[mcroad]

Small update

• rust-bitcoin added support for no_std back in no_std support, keeping MSRV rust-bitcoin/rust-bitcoin#603 but no_std did not work with their 1.29 MSRV. BUG: Does not work with no_std under 1.29 (MSRV) rust-bitcoin/rust-bitcoin#690 aims to resolve that.
• rust-miniscript is close to having no_std except that I'm waiting for the rust-bitcoin MSRV issue to be resolved and a new version release. After that I'll cleanup the PR and hopefully get it merged. Track that here Add no_std support rust-bitcoin/rust-miniscript#277
• bdk no_std PR Soon™ after

To the BDK team

There are 2 main approaches to implementing no_std support. One, using the no-std-compat crate, and the other is using core directly.

Using no-std-compat is the simpler approach as it would leave most of the code unchanged. no-std-compat will continue using the std library in std environments but replaces std and polyfills any necessary features in no_std environments using core under the hood.

The second approach is to use the core crate directly. This apprach would modify most files as imports would be taken from core instead of std. This is what rust-bitcoin did and rust-miniscript will do. See https://github.com/rust-bitcoin/rust-miniscript/pull/277/files for an example of what it might look like.

The main downside that from using no-std-compat is taking on one more dependency with the upside being that the code would remain looking like regular std code. Using core directly keeps dependencies as is, but slightly increases code complexity and maintenance burden.

[notmandatory]

Thanks for the no_std update! one thing the team is working on, started in #535 is to decouple the bdk Wallet from the Blockchain and Database modules, this will make it possible to have a Wallet with less dependencies and maybe older MSRV and should make it easier to support no_std with something like core. Plus to support a full wallet we'll need at least a Database that supports no_std even if it's only in memory.

[afilini]

I would first give it a shot with the core approach and see what happens. If it turns out to be too messy we can try with something else, but ideally I would avoid extra dependencies and I would try to stay aligned with the rest of the rust-bitcoin environment.

Plus to support a full wallet we'll need at least a Database that supports no_std even if it's only in memory.

I'm assuming most hardware devices won't have permanent storage for UTXOs, so it's probably fine to release with just the MemoryDatabase initially.

[LLFourn]

I am writing about this for the blog post which should drop this week but here's my current vision on how this would work with bdk_core:

1. I'll introduce something called DescriptorTracker which will be something that has a descriptor and a bunch of in-memory stores of addresses, transactions, txouts and so on.
2. You update the descriptor tracker with data from blockchain (which it checkpoints based on block height/header)
3. You can then write out its state to persistent storage. In particular write out checkpoints, transactions and address index. The persistent storage doesn't index stuff like spks, txouts etc, rather this can be re-indexed when re-loaded into a DescriptorTracker.

So bdk_core app developer (which uses rust's core lib) would only really have the job of storing lists of transactions which could just be dumped to a file. So think MemoryDatabase except with better indexing and it is designed to be written out to different persistent storage options. Hopefully that makes sense. If it doesn't I'll try and do better in the blog post.

[notmandatory]

I like this approach, makes sense to me.

[mcroad]

rust-bitcoin resolved the MSRV issue in 0.28.1 and rust-miniscript no_std support added in rust-bitcoin/rust-miniscript#277. Will begin working on the BDK no_std PR Soon™.

[notmandatory]

Here's another request from @BitcoinComfy for no_std support and to be able to create a static build for thumbv4t-none-eabi and run on a feature phone.

twitter thread: https://twitter.com/BitcoinComfy/status/1611332656570191873