aead: stream module

[tarcieri]

Implementation of the STREAM construction as described in the paper "Online Authenticated-Encryption and its Nonce-Reuse Misuse-Resistance":

https://eprint.iacr.org/2015/189.pdf

The implementation is generic over AEAD ciphers and is factored into a low-level StreamPrimitive trait (permitting different "flavors" of STREAM) as well as higher-level stateful Encryptor and Decryptor objects which are generic over StreamPrimitive types.

Includes one concrete implementation of STREAM: StreamLE31, which uses a 31-bit counter and 1-bit last block flag. Note that this implementation differs slightly from the one described in the paper, which uses a 1-byte last block flag.

Using little endian provides better performance on commonly used CPU architectures, and using a 1-bit last block flag ensures the user-facing STREAM nonce is even numbered in terms of bytes (e.g. for a 96-bit nonce it'd be 64-bits or 8-bytes instead of a 7-byte nonce using the construction described in the paper) and also avoids wasting bits.

It would probably make sense to provide a concrete implementation of STREAM as described in the paper as well, especially for compatibility with existing deployments of this construction, however I wanted to both make sure we provide a useful deployed STREAM variant as such, and also wanted to keep the PR smaller for initial review.

[tarcieri]

Note: I left allocating APIs out of this PR as well to make it easier to review, but they're easily added in a follow-up.

[tarcieri]

Note that this implementation precludes calling encrypt_next_in_place/decrypt_next_in_place with the maximum counter value.

That's deliberate: it ensures any segment encrypted under the maximum counter value MUST have the "last block" flag set.

[str4d]

Consuming self here is annoying in async encryptors, where we may need to leave poll_close after encrypting the last chunk but before we've finished writing the encrypted chunk. It can be managed by storing an Option<aead::stream::Encryptor>, but it's a bit of a hassle.

More problematic is that consuming self here is incompatible with seeking decryptors, as we may need to decrypt the last chunk, read part of it, then seek back earlier than the last chunk. Decryptor doesn't implement Clone, so we can't clone before consuming. Instead, someone trying to implement Seek would need to save the key and nonce themselves, and then reconstruct the Decryptor every time a seek is requested.

[tarcieri]

The Encryptor and Decryptor objects are high-level misuse resistant APIs designed to ensure the STREAM is encoded/decoded correctly.

The lower-level StreamPrimitive trait is intended for the use cases you're describing.

[str4d]

There is currently no way to set position manually to a specific chunk, which completely prevents Decryptor from being used in a seeking context.

[str4d]

I tried migrating the age crate to this (which uses ChaCha20Poly1305 with a nonce structured as an 11-byte big-endian counter and 1-byte last block flag), and encountered a few issues.

[tarcieri]

@str4d I think for something like age, you'll want to use StreamPrimitive directly. Encryptor and Decryptor are higher-level APIs intended to make it easy to do the right thing at the cost of flexibility.

They could potentially include seeking behavior similar to SyncStreamCipherSeek, but if you are intending to do anything in parallel the fact they keep state at all seems problematic and I think you should just use StreamPrimitive instead (which is deliberately immutable/stateless to handle such cases).

[str4d]

As I noted in Discord, StreamPrimitive doesn't actually offer me much for age at present, because I'm implementing both StreamPrimitive (to use age's nonce structure) and the things that would use StreamPrimitive (the most I could do was to replace my own enc/dec wrappers with Encryptor and Decryptor, but per above they don't completely meet my use case).

If we could move (what is currently) age's StreamReader and StreamWriter into a generic crate (abstracted so they can also e.g. support Tink's more general use-case), then it would make sense for age to use StreamPrimitive. I'm not sure whether it makes sense for those structs to belong in aead::stream, or whether they should be in another module or crate.

[tarcieri]

@str4d it might make sense to spike things out in another crate initially, then circle back on upstreaming the parts that make sense into aead.

[tarcieri]

Added a StreamBE32 flavor of StreamPrimitive which implements the original construction as described in the paper, and marked this PR as ready for review

I'm wondering if we might reduce StreamPrimitive to just computing the nonce, rather than having an instance of the cipher, i.e. providing the aead_nonce method which is presently private.

That would reduce duplication of code between StreamBE32 and StreamLE31, and would allow StreamPrimitive to work with AeadMut, among other things.

[tarcieri]

I'm going to go ahead and land this as I think a stream module is both a valuable thing to have and something that comes up quite frequently.

I plan on submitting a follow-up PR to simplify the API as described above.

I'm also not in a rush to cut another release of aead and think we might consider making some breaking changes before the next release (e.g. #273), so I think at the very least it will have time to bake before that.