serdect: is it actually constant-time?

[fjarri]

The binary serializer uses serializer.serialize_tuple() and serialize_element() which, in some formats at least, makes it data-dependent. E.g. MessagePack prepends every element greater than 127 with 0xCC.

Also, this contradicts the documentation claim:

When using a binary format, the data is serialized as-is into binary.

What was the reason behind not using serialize_bytes()? Seems like it would provide better constant-time guarantees?

[tarcieri]

The README describes it as follows:

This crate provides "best effort" constant-time helper methods for reducing the amount of timing variability involved in serializing/deserializing data when using serde, Rust's standard serialization framework.

Really the goal is to be less noisy than e.g. a JSON serialization of bytes as an integer array. PRs accepted to improve the wording in the README.

What was the reason behind not using serialize_bytes()? Seems like it would provide better constant-time guarantees?

The slice serializer just uses the Serialize::serialize impl on [u8]. I'm not familiar with how that differs from serialize_bytes, but if you think there's a compelling case I'd suggest opening a PR. We test with a number of different serde format impls so perhaps you can point out specific deficiencies with a particular one.

serialize_tuple is used by the array serializer, which avoids length prefixing the data because it's known a priori by the type system. That implementation avoids a problematic deficiency with serde which prevents it from working with const generics: serde-rs/serde#1937

[fjarri]

The slice serializer just uses the Serialize::serialize impl on [u8]. I'm not familiar with how that differs from serialize_bytes

It seems that it is different, since serialize_bytes serializes bytes verbatim in MessagePack, while serialize doesn't.

serialize_tuple is used by the array serializer, which avoids length prefixing the data because it's known a priori by the type system.

That's true, but then binary formats instead prefix separate elements, which kind of defeats the point.

[tarcieri]

Can you point to a specific implementation, and ideally, write some test cases which capture the issue?

[tarcieri]

cc @daxpedda

[fjarri]

Can you point to a specific implementation, and ideally, write some test cases which capture the issue?

Let me make an MRE.

Also, speaking of serialize_tuple(), I would argue that while it would be nice to have an array serialization with no length specifier, it is secondary to the intent of serdect which is to a) strive for constant-timeness, and b) serialize bytes unchanged.

[tarcieri]

Again, we provide both options. Use the slice serializer if you're okay with a length prefix.

[fjarri]

They are not completely interchangeable. slice::deserialize_hex_or_bin() doesn't fail if the buffer is larger than what's being deserialized, which is not what I want if I'm actually deserializing an array.

[tarcieri]

I am loathe to change it without an actual empirical demonstration of timing variability or other performance issue. Can you put one together?

[tarcieri]

Also note there is a long, long history of anecdotal reports like this with the various serde serializers across the @RustCrypto project, which is why we put this crate together and why it tests against various format implementations, so I would really like to see running code as the motivation for any changes.

[fjarri]

I am loathe to change it without an actual empirical demonstration of timing variability or other performance issue. Can you put one together?

On it now. But if the length of the serialized bytestring is data-dependent, doesn't timing variability naturally follow? Not to mention that it's quite unexpected to get a variable-sized serialized data when you thought you were serializing a constant-sized array.

[tarcieri]

If the length varies, then the timing will as well, yes.

However, that would seem like a sharp edge and counterclaim to your argument that using a length-prefixed serialization would be better than one guaranteed fixed-length by the type system, if anything.

[fjarri]

use serde::{Serialize, Serializer};

#[derive(Serialize)]
struct A(
    #[serde(serialize_with = "serdect::slice::serialize_hex_lower_or_bin")]
    [u8; 8]
    );

#[derive(Serialize)]
struct B(
    #[serde(serialize_with = "serdect::array::serialize_hex_lower_or_bin")]
    [u8; 8]
    );

struct C([u8; 8]);

impl Serialize for C {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        serializer.serialize_bytes(&self.0)
    }
}


fn main() {
    let a = A([1, 2, 254, 255, 3, 4, 252, 253]);

    let b = rmp_serde::encode::to_vec(&a);
    println!("messagepack + slice: {:?}", b);

    let b = bincode::serialize(&a).unwrap();
    println!("bincode + slice: {:?}", b);

    let a = B([1, 2, 254, 255, 3, 4, 252, 253]);

    let b = rmp_serde::encode::to_vec(&a);
    println!("messagepack + array: {:?}", b);

    let b = bincode::serialize(&a).unwrap();
    println!("bincode + array: {:?}", b);

    let a = C([1, 2, 254, 255, 3, 4, 252, 253]);

    let b = rmp_serde::encode::to_vec(&a);
    println!("messagepack + serialize_bytes: {:?}", b);

    let b = bincode::serialize(&a).unwrap();
    println!("bincode + serialize_bytes: {:?}", b);
}

Output:

messagepack + slice: Ok([152, 1, 2, 204, 254, 204, 255, 3, 4, 204, 252, 204, 253])
bincode + slice: [8, 0, 0, 0, 0, 0, 0, 0, 1, 2, 254, 255, 3, 4, 252, 253]
messagepack + array: Ok([152, 1, 2, 204, 254, 204, 255, 3, 4, 204, 252, 204, 253])
bincode + array: [1, 2, 254, 255, 3, 4, 252, 253]
messagepack + serialize_bytes: Ok([196, 8, 1, 2, 254, 255, 3, 4, 252, 253])
bincode + serialize_bytes: [8, 0, 0, 0, 0, 0, 0, 0, 1, 2, 254, 255, 3, 4, 252, 253]

So Bincode behaves well, MessagePack doesn't.

However, that would seem like a sharp edge and counterclaim to your argument that using a length-prefixed serialization would be better than one guaranteed fixed-length by the type system, if anything.

The type system guarantees that the format will receive a constant-sized tuple, but not what it will do with it. Of course, giving it a byte slice instead does not guarantee it either, but it seems to me that it is a better "best effort" for constant-timeness.

[tarcieri]

I'm not sure I see anything actionable there?

[fjarri]

The "204" elements are not in the actual data and are added before any byte with the value above 127, making the serialization length (and time) data-dependent.

Also it shows the difference between serialize() (used in serdect::slice) and serialize_bytes