chacha20: get ChaCha8 keystream blocks without applying them to a plaintext

[nazar-pc]

Chia Proof-of-Space implementation uses ChaCha8 cipher internally as PRNG instead of encrypting anything. This means that with current API it is necessary to apply it to zeroed vector first, which is an extra zeroing, memory copy and pointless XOR.

I'm wondering if it would be possible to somehow access internal keystream directly while doing less work, possibly by accepting &mut [MaybeUninit<u8>] as an argument.

[tarcieri]

If you just want a PRNG you can use the rand_core-based PRNG API that’s in the latest prereleases.

It’s unlikely we’ll add an unsafe API for writing into uninitialized memory. This is a micro-optimization that’s often unnecessary because LLVM can elide zero-initialization if it can prove it will be overwritten by a subsequent write.

[newpavlov]

You also could use the block-level trait directly. It still uses mutable references over blocks without MaybeUninit, but it's usually trivial for the compiler to elimination zero initialization of the buffer after inlining the methods.

[nazar-pc]

I found write_keystream_blocks in public API, which is closer to what I need, but not quite it. It is also a bit awkward to use, especially because there is no ChaCha8::get_core_mut() or similar, and there is no ChaChaBlock type alias. I ended up with this:

struct PartialYs {
    buffer: Vec<GenericArray<u8, U64>>,
    len: usize,
}

impl Deref for PartialYs {
    type Target = [u8];

    #[inline(always)]
    fn deref(&self) -> &Self::Target {
        unsafe { slice::from_raw_parts(self.buffer.as_ptr().cast::<u8>(), self.len) }
    }
}


fn partial_ys<const K: u8>(seed: Seed) -> PartialYs {
    let output_len_bits = usize::from(K) * (1 << K);
    let output_len = output_len_bits.div_ceil(u8::BITS as usize);
    let mut output = vec![
        Block::<ChaChaCore<U4>>::default();
        output_len.div_ceil(ChaChaCore::<U4>::block_size())
    ];

    let key = Key::from(seed);
    let nonce = Nonce::default();

    let mut cipher = ChaChaCore::<U4>::new(&key, &nonce);

    cipher.write_keystream_blocks(&mut output);

    PartialYs {
        buffer: output,
        len: output_len,
    }
}

StreamCipherCoreWrapper::write_keystream() (alternative to ``StreamCipherCoreWrapper::apply_keystream()`) would have been nice here. But it didn't make a meaningful difference.

I'll give rand_chacha a try to, though not sure how optimized it is when compared to chacha20.

[newpavlov]

I'll give rand_chacha a try to, though not sure how optimized it is when compared to chacha20.

As @tarcieri wrote, the chacha20 crate implements the rand_core traits in v0.10 pre-releases (see the *Rng types). For example, you can use RngCore::fill_bytes on ChaCha8Rng.

[tarcieri]

As far as I am aware, chacha20 is faster than rand_chacha on all platforms, has optimized backends for more platforms, and has fewer (mandatory) dependencies, while supporting the same rand_core API.

See specifically https://docs.rs/chacha20/0.10.0-rc.0/chacha20/struct.ChaCha8Rng.html

[nazar-pc]

I see, I thought it was the pre-release of rand crates initially.

chacha20::ChaCha8Rng doesn't seem to produce the same effective keystream though. I'm effectively doing something like this:

let mut chacha8 = ChaCha8Rng::from_seed(seed);
for _ in 0..N {
    let mut chunk = [0u8; 20];
    chacha8.fill_bytes(&mut chunk);

    // ...
}

And concatenation of chunks is not the original keystream. rand_chacha appears to have the same behavior.

[newpavlov]

Should be fixed by RustCrypto/traits#1907

[nazar-pc]

Thanks, I'm also still curious why ChaCha8Rng didn't produce the same bytes, I must misunderstand something.

[tarcieri]

@nazar-pc if you can generate a test vector which diverges between rand_chacha and chacha20, that would be important for us to know

[nazar-pc]

No, it does appear to be the same between the two, the difference is between the output of ChaCha8Rng and ChaCha8, but I didn't find the description of how ChaCha8Rng is supposed to behave.

[tarcieri]

@nazar-pc can you file an issue with a reproduction?