API proposal for SP800-108-CTR-HMAC cryptographic algorithm

[GrabYourPitchforks]

Motivation

A key derivation function (KDF) is a building block operation of any cryptosystem, alongside cipher algorithms and MAC algorithms. Within Microsoft, many of our partners use SP800-108-CTR-HMAC, and it is commonly recommended by the Crypto Board. (See NIST SP 800-108, §5.1.) ASP.NET's DataProtection API also uses this KDF under the covers.

.NET does not currently provide a managed wrapper around the SP800-108-CTR-HMAC algorithm, forcing all consumers to implement it themselves from spec. Representatives from the Crypto Board have asked us to create a supported .NET implementation that can be used by our mutual customers. Like the work we did to add System.IO.Hashing (see #53623), we should ensure we produce a netstandard2.0-consumable package for customers who target .NET Framework.

API Proposal

namespace System.Security.Cryptography;

public sealed class SP800108HmacCounterKdf : IDisposable
{
    //
    // CREATE A REUSABLE INSTANCE WRAPPED AROUND A FIXED KDK
    //
    public SP800108HmacCounterKdf(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm);
    public SP800108HmacCounterKdf(byte[] key, HashAlgorithmName hashAlgorithm);

    //
    // REUSABLE INSTANCE METHODS
    //
    public byte[] DeriveKey(byte[] label, byte[] context, int derivedKeyLengthInBytes);
    public byte[] DeriveKey(ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, int derivedKeyLengthInBytes);
    public byte[] DeriveKey(string label, string context, int derivedKeyLengthInBytes);
    public byte[] DeriveKey(ReadOnlySpan<char> label, ReadOnlySpan<char> context, int derivedKeyLengthInBytes);
    public void DeriveKey(ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, Span<byte> destination);
    public void DeriveKey(ReadOnlySpan<char> label, ReadOnlySpan<char> context, Span<byte> destination);

    //
    // STATIC ONE-SHOTS
    //
    public static byte[] DeriveBytes(byte[] key, HashAlgorithmName hashAlgorithm, byte[] label, byte[] context, int derivedKeyLengthInBytes);
    public static byte[] DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, int derivedKeyLengthInBytes);
    public static byte[] DeriveBytes(byte[] key, HashAlgorithmName hashAlgorithm, string label, string context, int derivedKeyLengthInBytes);
    public static byte[] DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<char> label, ReadOnlySpan<char> context, int derivedKeyLengthInBytes);
    public static void DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, Span<byte> destination);
    public static void DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<char> label, ReadOnlySpan<char> context, Span<byte> destination);

    public void Dispose();
}

Terminology

• key derivation key (KDK): the master key from which subkeys (derived keys) will be generated; taken as the key parameter in these APIs.
• pseudo-random function (PRF): the HMAC algorithm which is used to generate the subkey; taken as the hashAlgorithm parameter in these APIs.
• label and context: a descriptive tuple which changes how the subkey is generated and allows generation of multiple unique subkeys from a single KDK. (ASP.NET's DataProtection APIs combine these into a single concept called a purpose string.)

Usage notes

Security note: Callers must ensure the KDK input to these APIs consists of strong entropy. If the input key contains low entropy (e.g., a user-specified password), a key-stretching KDF like PBKDF2 (via RFC2898DeriveBytes) must be used instead. This is the caller's responsibility; our APIs will not fail if an empty or low entropy KDK is provided.

Per spec, the hashAlgorithm parameter must be one of: SHA1, SHA256, SHA384, or SHA512. (We wrap it within an HMAC construction.) Any other algorithm selection will result in ArgumentException.

Discussion

Two patterns of APIs are provided. The easiest pattern is to specify the desired length of the subkey, and the system will allocate a new array and return it to the caller. If the caller wishes to be in full control over memory allocation, a "fill the entire output buffer" overload is available. For each of these three patterns, instance methods and static one-shots are available. If the Secret<T> type is introduced in the future (see dotnet/designs#147), we should introduce overloads which take the kdk as Secret<byte> and which return the derived subkey as a new Secret<byte>, which follows ASP.NET's existing usage.

Since the label and context parameters could be binary or text, we should for convenience allow the caller to pass in char spans / strings. These overloads will convert the input to UTF-8 and call the binary overloads. If the input contains malformed UTF-16 text, ArgumentException is thrown. Callers should take care not to allow embedded nulls within the label parameter. However, our APIs will not check for embedded nulls and will not fail should they be encountered.

The KDK (key) parameter is expected to be a cryptographic key with strong entropy. This is raw binary data, not textual data, so there is no need for UTF-16 overloads for the key parameter.

Unlike RFC2898DeriveBytes, the instance methods proposed here are stateless. Constructing an instance and calling DeriveBytes over and over will - assuming constant label and context inputs and subkey lengths - produce the same output byte-for-byte. If the caller wishes to generate multiple subkeys from a single KDK, they should: (a) generate a single long subkey and slice pieces off as needed; or (b) vary the label or context parameters and run the operation again.

The type is sealed to match the designs of the recent cryptography primitives AesGcm and ChaCha20Poly1305. The SP800-108-CTR-HMAC algorithm is itself an opaque primitive and no extensibility points are considered per spec. Once the KDK and PRF are captured by the constructor, there is no public API to retrieve them.

Thread safety

Instances of this type shall be thread-safe for multiple callers. Instance methods shall not mutate internal state or persist state between calls. This is because applications might retain a single KDK for multiple operations, and they'll need to generate subkeys on-demand from arbitrary threads. (ASP.NET relies on this behavior.)

Packaging

Like System.IO.Hashing, we have considerable interest in this API from consumers on downlevel runtimes. We should create a new netstandard2.0-compatible package System.Security.Cryptography.SP800108 and expose the APIs from there. If needed, this package can have a different implementation on .NET 6+ or .NET 7, including any appropriate type-forwarding.

Alternative proposals

The names SP800108DeriveBytes and SP800108CtrHmac were proposed, with the former mirroring the RFC2898DeriveBytes naming. However, the *Hmac suffix was deemed to be misleading, since this algorithm isn't itself an HMAC. And leaving the "counter / HMAC" description out entirely isn't descriptive enough of what the API does.

This type would be one of the few thread-safe types in the System.Security.Cryptography namespace. If this asymmetry is not desired, we could consider making the type ICloneable, which would allow spawning a new instance with its own dedicated lifetime, and with the captured KDK and PRF duplicated. We could potentially get away with saying that callers who need thread safety should use the static one-shots. However, that would impact the performance of ASP.NET, which has its own implementation of this type which makes thread-safety guarantees.

The existing type HKDF takes its salt and info parameters at the end of the APIs because they're considered optional. I opt not to do that here, placing the label and context parameters upfront as required inputs (which can be empty). This is because SP800-108 very strongly recommends callers pass meaningful data for these parameters (see §§7.5 - 7.6), which allows a single KDK to generate an arbitrary number of subkeys. If the application instead has a fixed number of uses, they can use a static one-shot and pass empty arrays / strings / spans for these APIs, slicing subkeys as needed.

Tagging subscribers to this area: @dotnet/area-system-security, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Motivation

A key derivation function (KDF) is a building block operation of any cryptosystem, alongside cipher algorithms and MAC algorithms. Within Microsoft, many of our partners use SP800-108-CTR-HMAC, and it is commonly recommended by the Crypto Board. (See NIST SP 800-108, §5.1.) ASP.NET's DataProtection API also uses this KDF under the covers.

.NET does not currently provide a managed wrapper around the SP800-108-CTR-HMAC algorithm, forcing all consumers to implement it themselves from spec. Representatives from the Crypto Board have asked us to create a supported .NET implementation that can be used by our mutual customers. Like the work we did to add System.IO.Hashing (see #53623), we should ensure we produce a netstandard2.0-consumable package for customers who target .NET Framework.

API Proposal

namespace System.Security.Cryptography;

public sealed class SP800108HmacCounterKdf : IDisposable
{
    //
    // CREATE A REUSABLE INSTANCE WRAPPED AROUND A FIXED KDK
    //
    public SP800108HmacCounterKdf(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm);

    //
    // REUSABLE INSTANCE METHODS
    //
    public byte[] DeriveKey(ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, int derivedKeyLengthInBytes);
    public byte[] DeriveKey(ReadOnlySpan<char> label, ReadOnlySpan<char> context, int derivedKeyLengthInBytes);
    public void DeriveKey(ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, Span<byte> derivedKey);
    public void DeriveKey(ReadOnlySpan<char> label, ReadOnlySpan<char> context, Span<byte> derivedKey);

    //
    // STATIC ONE-SHOTS
    //
    public static byte[] DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, int derivedKeyLengthInBytes);
    public static byte[] DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<char> label, ReadOnlySpan<char> context, int derivedKeyLengthInBytes);
    public static void DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, Span<byte> derivedKey);
    public static void DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<char> label, ReadOnlySpan<char> context, Span<byte> derivedKey);

    public void Dispose();
}

Terminology

• key derivation key (KDK): the master key from which subkeys (derived keys) will be generated; taken as the key parameter in these APIs.
• pseudo-random function (PRF): the HMAC algorithm which is used to generate the subkey; taken as the hashAlgorithm parameter in these APIs.
• label and context: a descriptive tuple which changes how the subkey is generated and allows generation of multiple unique subkeys from a single KDK. (ASP.NET's DataProtection APIs combine these into a single concept called a purpose string.)

Usage notes

Security note: Callers must ensure the KDK input to these APIs consists of strong entropy. If the input key contains low entropy (e.g., a user-specified password), a key-stretching KDF like PBKDF2 (via RFC2898DeriveBytes) must be used instead. This is the caller's responsibility; our APIs will not fail if an empty or low entropy KDK is provided.

Per spec, the hashAlgorithm parameter must be one of: SHA1, SHA256, SHA384, or SHA512. (We wrap it within an HMAC construction.) Any other algorithm selection will result in ArgumentException.

Discussion

Two patterns of APIs are provided. The easiest pattern is to specify the desired length of the subkey, and the system will allocate a new array and return it to the caller. If the caller wishes to be in full control over memory allocation, a "fill the entire output buffer" overload is available. For each of these three patterns, instance methods and static one-shots are available. If the Secret<T> type is introduced in the future (see dotnet/designs#147), we should introduce overloads which take the kdk as Secret<byte> and which return the derived subkey as a new Secret<byte>, which follows ASP.NET's existing usage.

Since the label and context parameters could be binary or text, we should for convenience allow the caller to pass in char spans / strings. These overloads will convert the input to UTF-8 and call the binary overloads. If the input contains malformed UTF-16 text, ArgumentException is thrown. Callers should take care not to allow embedded nulls within the label parameter. However, our APIs will not check for embedded nulls and will not fail should they be encountered.

The KDK (key) parameter is expected to be a cryptographic key with strong entropy. There is no need for UTF-16 overloads for the key parameter.

Unlike RFC2898DeriveBytes, the instance methods proposed here are stateless. Constructing an instance and calling DeriveBytes over and over will - assuming constant label and context inputs and subkey lengths - produce the same output byte-for-byte. If the caller wishes to generate multiple subkeys from a single KDK, they should: (a) generate a single long subkey and slice pieces off as needed; or (b) vary the label or context parameters and run the operation again.

The type is sealed to match the designs of the recent cryptography primitives AesGcm and ChaCha20Poly1305. The SP800-108-CTR-HMAC algorithm is itself an opaque primitive and no extensibility points are considered per spec. Once the KDK and PRF are captured by the constructor, there is no public API to retrieve them.

Thread safety

Instances of this type shall be thread-safe for multiple callers. Instance methods shall not mutate internal state or persist state between calls. This is because applications might retain a single KDK for multiple operations, and they'll need to generate subkeys on-demand from arbitrary threads. (ASP.NET relies on this behavior.)

Packaging

Like System.IO.Hashing, we have considerable interest in this API from consumers on downlevel runtimes. We should create a new netstandard2.0-compatible package System.Security.Cryptography.SP800108 and expose the APIs from there. If needed, this package can have a different implementation on .NET 6+ or .NET 7, including any appropriate type-forwarding.

Alternative proposals

The names SP800108DeriveBytes and SP800108CtrHmac were proposed, with the former mirroring the RFC2898DeriveBytes naming. However, the *Hmac suffix was deemed to be misleading, since this algorithm isn't itself an HMAC. And leaving the "counter / HMAC" description out entirely isn't descriptive enough of what the API does.

This type would be one of the few thread-safe types in the System.Security.Cryptography namespace. If this asymmetry is not desired, we could consider making the type ICloneable, which would allow spawning a new instance with its own dedicated lifetime, and with the captured KDK and PRF duplicated. We could potentially get away with saying that callers who need thread safety should use the static one-shots. However, that would impact the performance of ASP.NET, which has its own implementation of this type which makes thread-safety guarantees.

The existing type HKDF takes its salt and info parameters at the end of the APIs because they're considered optional. I opt not to do that here, placing the label and context parameters upfront as required inputs (which can be empty). This is because SP800-108 very strongly recommends callers pass meaningful data for these parameters (see §§7.5 - 7.6), which allows a single KDK to generate an arbitrary number of subkeys. If the application instead has a fixed number of uses, they can use a static one-shot and pass empty spans for these APIs, slicing subkeys as needed.

Author:	GrabYourPitchforks
Assignees:	-
Labels:	api-suggestion, area-System.Security
Milestone:	7.0.0

[vcsjones]

Span<byte> derivedKey

We've typically called these writable inputs destination in many other recently added APIs in the S.S.Cryptography namespaces¹². I can see why derivedKey might be better, and derivedKeyDestination is maybe a bit too long, but destination is consistent with other API surface.

public SP800108HmacCounterKdf(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm);

I would consider byte[] and char[] (or string) APIs, at least for the allocating-return ones and constructor, i.e.

public SP800108HmacCounterKdf(byte[] key!!, HashAlgorithmName hashAlgorithm);
public byte[] DeriveKey(byte[] label!!, byte[] context!!, int derivedKeyLengthInBytes);

I realize the OOB package will probably just include a reference to System.Memory and "work" with just a span, but I'm keen to know if this had been considered. Using the previously cited ChaCha20Poly1305 and HKDF as an example, they too have array overloads where they accepts spans.

Perhaps the array ones are specifically more useful for the .NET Standard users (presumably .NET Framework) where arrays are more likely to be seen.

Footnotes

1. https://github.com/dotnet/runtime/issues/62489 ↩

2. https://github.com/dotnet/runtime/issues/24897 ↩

[GrabYourPitchforks]

Updated proposal with early API feedback, marking ready for review.

[terrajobst]

Video

• Looks good as proposed
  • For OOBing, System.Security.Cryptography.SP800108 would be reasonable package/assembly name.

namespace System.Security.Cryptography;

public sealed class SP800108HmacCounterKdf : IDisposable
{
    public SP800108HmacCounterKdf(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm);
    public SP800108HmacCounterKdf(byte[] key, HashAlgorithmName hashAlgorithm);

    public byte[] DeriveKey(byte[] label, byte[] context, int derivedKeyLengthInBytes);
    public byte[] DeriveKey(ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, int derivedKeyLengthInBytes);
    public byte[] DeriveKey(string label, string context, int derivedKeyLengthInBytes);
    public byte[] DeriveKey(ReadOnlySpan<char> label, ReadOnlySpan<char> context, int derivedKeyLengthInBytes);
    public void DeriveKey(ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, Span<byte> destination);
    public void DeriveKey(ReadOnlySpan<char> label, ReadOnlySpan<char> context, Span<byte> destination);

    public static byte[] DeriveBytes(byte[] key, HashAlgorithmName hashAlgorithm, byte[] label, byte[] context, int derivedKeyLengthInBytes);
    public static byte[] DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, int derivedKeyLengthInBytes);
    public static byte[] DeriveBytes(byte[] key, HashAlgorithmName hashAlgorithm, string label, string context, int derivedKeyLengthInBytes);
    public static byte[] DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<char> label, ReadOnlySpan<char> context, int derivedKeyLengthInBytes);
    public static void DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<byte> label, ReadOnlySpan<byte> context, Span<byte> destination);
    public static void DeriveBytes(ReadOnlySpan<byte> key, HashAlgorithmName hashAlgorithm, ReadOnlySpan<char> label, ReadOnlySpan<char> context, Span<byte> destination);

    public void Dispose();
}