ml-dsa: some valid signature values don't successfully round trip through encode + decode

[json420]

I'm hitting this issue when testing v0.1.0-pre.0 from crates.io and and when testing git master.

The failure happens when calling Signature::<MlDsa65>::decode(). Lot's of valid signatures fail (like ~1% ish), so problematic ones aren't hard to find. Here's an example value that is failing for me:

// Minimal demonstration of a valid signature that wont round-trip through encode/decode.

use ml_dsa::{EncodedSignature, KeyGen, MlDsa65, Signature, B32};
use signature::Signer;

static SEED: [u8; 32] = [
    197, 185, 159, 59, 216, 233, 208, 40, 244, 4, 182, 73, 109, 244, 205, 113, 116, 55, 206, 145,
    214, 205, 247, 130, 41, 113, 93, 14, 140, 194, 191, 232,
];
static MESSAGE: &[u8] = b"There seems to be a round tripping issue somewhere in here";

fn main() {
    let mut seed = B32::default();
    seed.0.copy_from_slice(&SEED);
    let kp = MlDsa65::key_gen_internal(&seed);
    let sig = kp.signing_key().sign(MESSAGE);

    let mut sig_buf = [0; 3309];
    sig_buf.copy_from_slice(sig.encode().as_slice());
    let sig_enc = EncodedSignature::<MlDsa65>::try_from(&sig_buf[..]).unwrap();
    let _sig = Signature::<MlDsa65>::decode(&sig_enc).unwrap(); // This will fail
}

Further code demonstrating the bug is here: https://github.com/json420/ml_dsa_bug

[json420]

Oh, FYI, writing/reading to/from the sig_buf above isn't needed to trigger the bug. This simplified version of main() triggers it as well (with problematic values that is):

fn main() {
    let mut seed = B32::default();
    seed.0.copy_from_slice(&SEED);
    let kp = MlDsa65::key_gen_internal(&seed);
    let sig = kp.signing_key().sign(MESSAGE);
    let sig_enc = sig.encode();
    let _sig = Signature::<MlDsa65>::decode(&sig_enc).unwrap(); // This will fail
}

[bifurcation]

Thanks @json420, I have confirmed that your example fails for me locally. I will look into this.

[json420]

@bifurcation @tarcieri thank you for your awesome, quick response!

Seriously, all I want to do is hybrid sign with ed25519-dalek and ml-dsa in 🦀

I'll test whenever there is a branch ready!

[tarcieri]

@json420 you can take a look at #895 and see if it fixes your problem