Skip to content

Conversation

@timothee-haudebourg
Copy link
Contributor

@timothee-haudebourg timothee-haudebourg commented Sep 23, 2025

Description

In the data-integrity library we currently use a FlatMapDeserializer to deserialize proof/configuration types that is part of serde private API. That type is used by serde's derive macro to implement #[derive(flatten)]. However because Data Integrity proofs/configurations are so hard to deserialize, we can't use serde's derive macro, we have to implement flatten "by hand". That's because the deserialization behavior depends on the crypto suite type, which is unknown when the deserialization starts. The result is a fairly complex deserialization function that requires a FlatMapDeserializer implementation. Unfortunately serde recently made it impossible to use their private API, and provides no public equivalent to FlatMapDeserializer. This PR implements a weaker but simpler version of FlatMapDeserializer based on serde_json::Value (to store arbitrary values deserialized during flattening).

Close #681
Close #682
Close #683

Other changes

I moved all the deserialization utility types in a dedicated de::utils module.

Tested

  • Test suite still passes.

};

pub struct FlatMapDeserializer<'a, E> {
entries: &'a mut Vec<Option<(String, serde_json::Value)>>,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the original FlatMapDeserializer, entries would be of type (Content, Content), accepting arbitrary values as key and values. Here I settled for a weaker (String, serde_json::Value) which should be enough considering Data-Integrity values are JSON documents.

}

unsupported! {
deserialize_option()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original implementation can flatten options, but it uses a private API. For our use case, rejecting options should be fine.

V: serde::de::Visitor<'de>,
{
visitor.visit_map(FlatMapAccess {
iter: self.entries.iter_mut(),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the original implementation, flattened maps would not consume entries. They would stay in the entry pool for subsequent flattened fields. Unfortunately we would need &serde_json::Value (reference) to implement Deserializer to preserve the same behavior, which is not the case. However I don't think that matters in our case.

Type,
Cryptosuite,
Other(serde::__private::de::Content<'de>),
Other(String),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only accept string keys now. It's fine since Data-Integrity documents are JSON documents.

@sbihel sbihel merged commit 507bdb1 into main Sep 24, 2025
3 checks passed
@sbihel sbihel deleted the serde-json-based-flat-map-deserializer branch September 24, 2025 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

serde > 1.0.219 breaking compilation due to use of private deserializer methods

3 participants