It looks like we got an 0.7.1 release on crates.io (yay!), but there's no tag on the git repo. The version was updated in 87659b14, but was the release made from that, or from a subsequent commit?

We should compare the crates.io tarball against the last four commits, and add a hkdf-0.7.1 tag to the right one.

We're trying to complete RustCrypto upgrades against some of our projects which use the new releases of things like digest and hmac and this is one of the crates we're blocked on (for crates we'd like to publish ASAP as it were, so we can't rely on a git dependency).

I'm not sure what additional work you'd like to do before a final release (e.g. landing #34), but hopefully it's OK to cut an alpha release with the upgrades in the interim?

Background

It is not uncommon to use concatenation in the ikm parameter of HKDF-Extract and info parameter of HKDF-Expand to achieve context binding. See for example, the use of the LabeledExtract and LabeledExpand functions in the HPKE specfication and HKDF-Expand-Label function in the MLS and TLS 1.3 specifications.

Problem

Some of these labeled methods take inputs of arbitrary length, which requires me to allocate on every call, given the current Hkdf API. Concretely, if I wanted to HKDF-Extract with ikm = "mycontextstr" || input where input is some user-inputted, arbitrary-length string, I would have to compute Hkdf::extract(None, &[b"mycontextstr", input].concat()), which is an allocation of unbounded size.

Obviously, this is extremely undesirable if I want to make HPKE work on embedded devices. In fact, as of right now, this issue is the only thing preventing rust-hpke from being alloc-free.

Solution

Implement streaming support for HKDF-Extract and HKDF-Expand.

For HKDF-Extract, we can do proper iterative streaming, since the underlying MAC supports this kind of streaming. This is implemented using the new HkdfExtract struct.

Due to the structure of HKDF-Expand, streaming with iterators isn't possible. At the very least, one needs an iterator that can be cycled an arbitrary number of times. I chose to represent this as a simple slice over bytestrings, but you may want to generalize to an input that is Iterator<Item = &[u8]> + Clone. If you'd prefer this, let me know and I'll make the change.

It would be nice if your API made it easier to perform the expand without the initial extract phase. This is what I do now:

    let hk = Hkdf::<Sha256>{
        prk: *GenericArray::from_slice(prk),
    };

Possibly there should be a constructor that does this from_slice method call on the u8 slice.

Add support for HKDF with PRK lengths >= Digest::OutputSize. This is allowed in RFC 5869 (see section 2.3 for Extend operation input requirements and section 3.3 WRT using PRKs that were not generated from the Extract step). Unfortunately the current API does not support this (since Hkdf.prk is a fixed sized array) so breaking changes were required.

We've seen one (brief) bug in https://github.com/warner/magic-wormhole.rs in which our protocol requires calling HKDF without a salt, and the initial code used Hkdf::<Sha256>::extract(&[], key) to express this. But the HKDF definition (RFC 5869) says that when the salt is omitted, "it is set to a string of HashLen zeros". Passing an empty salt string is not the same as omitting the salt (i.e. passing 32 zero bytes), causing a protocol mismatch.

To properly omit the salt, you must use something like extract(&[0; Sha256::OutputSize], key) or something that I'm not even sure would compile. Manually setting the salt size feels error-prone.

The python HKDF library lets you pass None as the salt value, and it will do the right thing.

Should we consider changing the extract() API to take an Option<&[u8]> for the salt value?

Can Rust do some kind of polymorphic thing that might let us have two different extract() methods, with different type signatures (one with &[u8], the other with Option) to enable backwards compability?

This improves usability and marginally improves performance:

Before:

test sha256_10 ... bench:       4,176 ns/iter (+/- 88) = 2 MB/s
test sha256_1k ... bench:      68,458 ns/iter (+/- 2,005) = 14 MB/s
test sha256_8k ... bench:     520,298 ns/iter (+/- 14,572) = 15 MB/s

After:

test sha256_10 ... bench:       4,113 ns/iter (+/- 535) = 2 MB/s
test sha256_1k ... bench:      66,892 ns/iter (+/- 8,179) = 15 MB/s
test sha256_8k ... bench:     506,935 ns/iter (+/- 57,681) = 15 MB/s

This involves a few internal API changes. Applications or libraries which use hkdf will almost certainly need to depend on e.g. sha2-0.7 instead of sha2-0.6, so when this change is released, we should bump the hkdf version number (from the current 0.2.1 to something like 0.3.0).

This moves away from rust-crypto to RustCrypto, which my rust+crypto friends tell me is the preferred approach these days. It modifies the HKDF API to instantiate the Hkdf object around a Digest trait, so instead of passing in "sha256" as a string, you use sha2::Sha256 and then create Hkdf::<Sha256:>>new(&ikm, &salt).

It also adds test vectors for SHA1.

Due to the API change, this should not be landed until #3 is done, and an old-API 0.1.1 has been released.

Initially reported in Zulip.

Right now if function uses HKDF and wants to be generic over hash functions, trait bounds look quite scary:

fn hkdf<H: CoreProxy>()
where
    <H as CoreProxy>::Core: HashMarker
        + UpdateCore
        + FixedOutputCore
        + BufferKindUser<BufferKind = Eager>
        + Default
        + Clone,
    <<H as CoreProxy>::Core as BlockSizeUser>::BlockSize: IsLess<U256>,
    Le<<<H as CoreProxy>::Core as BlockSizeUser>::BlockSize, U256>: NonZero,
{
    Hkdf::<H>::new(None, &[]);
}

It could be worth to simplify the bounds somehow. Also it may be worth to allow hkdf to use the SimpleHmac in addition to Hmac. It probably will involve introducing a helper trait or two in hmac.

There shouldn't be any changes that actually impact hkdf.

This version uses a newer crypto-mac release which has a new optional cipher feature that uses the new cipher crate.