Skip to content

proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values#242

Open
Sjors wants to merge 2 commits intobitcoin-core:masterfrom
Sjors:2026/02/serialize-null
Open

proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values#242
Sjors wants to merge 2 commits intobitcoin-core:masterfrom
Sjors:2026/02/serialize-null

Conversation

@Sjors
Copy link
Member

@Sjors Sjors commented Feb 20, 2026

Taken from: bitcoin/bitcoin#34020 (comment)

Let applications override CustomHasField so they can decide to treat certain capnproto values as being unset. For example, when converting List(Data) to vector<shared_ptr<CTransaction>>, mapping empty Data fields to null pointers.

This safe to do in special cases, like in this example because serialized CTransaction representations are never empty. It is also useful to do in this case because Cap'n Proto doesn't currently provide any API for distinguishing between unset and empty data values in a list (although they can be distinguished on the wire).

@DrahtBot
Copy link

DrahtBot commented Feb 20, 2026

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK ryanofsky

If your review is incorrectly listed, please copy-paste <!--meta-tag:bot-skip--> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #243 (mpgen: support primitive std::optional struct fields by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@Sjors
Copy link
Member Author

Sjors commented Feb 20, 2026

So far I just took bitcoin/bitcoin@b147783 verbatim, minus the IPC tests which go into Bitcoin Core. Let me know if you need more adjustments. Looks like the LLM found some typos.

@Sjors Sjors force-pushed the 2026/02/serialize-null branch from fb2622f to e8bcca3 Compare February 20, 2026 09:48
@Sjors
Copy link
Member Author

Sjors commented Feb 20, 2026

Whipped up a test inspired by the one in bitcoin/bitcoin@b147783 on the Bitcoin Core side.

I fixed the typos and also added a commit to fix two missing includes, that would otherwise need to be added in the test.

@Sjors Sjors force-pushed the 2026/02/serialize-null branch from e8bcca3 to 423a789 Compare February 20, 2026 10:06
@Sjors
Copy link
Member Author

Sjors commented Feb 20, 2026

Added missing #include <vector> to mp/gen.cpp.

Copy link
Collaborator

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 413f915. Code here looks good. I think PR title & description need to be updated (also title & description of main commit) because they are referencing CTransactionRef which is not in this repository and also not affected by this change (it requires the CustomHasField(TypeList<CTransaction>...) overload in the original commit).

A good summary of changes here might be "Add CustomHasValue overload to allow mapping non-null Cap'n Proto values to null C++ values" and probably the LLM can turn that into a good title & description not referring to CTransactionRef

Also wondering if you'd want this added to bitcoin/bitcoin#34422 if merged (seems reasonable)

Comment on lines 196 to 197
//! requires function parameters, there's no way to call the function
//! constructing values for each of the parameters. Similarly there's no way to
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "ipc: Serialize null CTransactionRef as empty Data" (413f915)

I think there is a missing word and this supposed to say "call the function without constructing values"

@Sjors
Copy link
Member Author

Sjors commented Feb 27, 2026

Also wondering if you'd want this added to bitcoin/bitcoin#34422 if merged (seems reasonable)

That's not necessary. I need this for bitcoin/bitcoin#34020 which can wait for v32.

@Sjors Sjors changed the title ipc: Serialize null CTransactionRef as empty Data proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values Feb 27, 2026
Add a CustomHasField(TypeList<...>, InvokeContext&, const Input&) extension
point used by ReadField/CustomReadField to decide whether an input should
materialize a C++ value. The default behavior remains input.has().

This enables targeted mappings from specific non-null Cap'n Proto values to
null C++ values (for example empty Data in List(Data), where Cap'n Proto C++
cannot currently distinguish null vs empty), without CTransactionRef-specific
logic in libmultiprocess.

Apply the hook across nullable read paths and add a test covering
round-tripping data pointers including null values.

Originally proposed here:
bitcoin/bitcoin#34020 (comment)

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
@Sjors Sjors force-pushed the 2026/02/serialize-null branch from 413f915 to 2d3cc77 Compare February 27, 2026 18:05
@Sjors
Copy link
Member Author

Sjors commented Feb 27, 2026

Tweaked the commit message and add the missing "without".

@ryanofsky
Copy link
Collaborator

Code review ACK 2d3cc77, with commit message rewrite and comment fix since last review.

FWIW I think new commit message is clear, but new PR description is pretty random and doesn't say what the change does. Not important, but something like this might be better:

Let applications override CustomHasField so they can decide to treat certain capnproto values as being unset. For example, when converting List(Data) to vector<shared_ptr<CTransaction>>, mapping empty Data fields to null pointers.
This safe to do in special cases, like in this example because serialized CTransaction representations are never empty. It is also useful to do in this case because Cap'n Proto doesn't currently provide any API for distinguishing between unset and empty data values in a list (although they can be distinguished on the wire).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants