This is the "hazardous" part of the secreat stream API proposed in #94.

Things that are still missing are:

• Documentation
• More tests, especially with wrong inputs
• An user friendly, "safe" API

See #227.

TODOs

• [ ] Hashing
  • [x] implement from_reader for BLAKE2b
  • [ ] implement from_reader for sha2 (if we feel like we need to)
    • [ ] sha256
    • [ ] sha384
    • [ ] sha512
• [ ] AEAD
  • [ ] implement seal_copy for XChaCha20Poly1305
  • [ ] implement open_copy for XChaCha20Poly1305
  • [ ] Other, non-high-level-interface methods (?)
• [x] General
  • [x] decide whether to keep or change all these function names, like from_reader, seal_copy, open_copy

EDIT: The scope of this PR has changed. We were going to try to fit the Read/Write impls for all of Orion's types into this PR, but then decided to split it up. #227 will keep track of all the impls.

Now this PR implements std::io::Write for Blake2b and adds a convenience function orion::hash::digest_from_reader.

Closes #103 and closes #94

There is one link in the documentation that i can not get to work, i have marked it with a TODO.

I'm a bit unsure if it is better to have a get_nonce method or if it is better to return Self and the Nonce in the new method.

Tests a rather minimal for now, but i think this should be fine since its only a thin wrapper around the hazardous API

Is it useful to expose the option to authenticate additional data? The "normal" AEAD seal function does not do it

Description Tests currently fail with cargo test --no-default-features.

Additional information It seems like this is happening because lots of the tests generate a random key. We could probably just feature-gate all those tests behind getrandom or maybe safe_api. Thoughts?

• Orion features selected: default-features = false
• Orion version: 723d9ef11c2b4e5f2f7746ffa9392c7cea5d8dbf
• Rust version: $ rustc -V : rustc 1.56.0 (09c42c458 2021-10-18)

We should probably also make sure the CI is testing no "no enabled features" case.

Summary

We use fiat-crypto code to implement various low-level cryptographic primitives. If the generated code from fiat-rust changes, it could mean there's a security vulnerability in it. An easy way to ensure we're notified of these changes would be to fail CI tests if the hash of the file(s) we use changes.

Alternatives

• We could set up some kind of Matrix bot that detects if the fiat-rust hash changes and posts to the Orion room. This would require maintaining the bot, though.

Resolves #192.

This is still a draft PR. The following items should be addressed before merging.

• [x] No tests are currently implemented. I still have to think about what the best test cases would be. Suggestions are welcome.
• [x] When it was available for the given type, serialization implementations use unprotected_as_bytes over the AsRef<[u8]> trait impl. Should we prefer using AsRef for consistency with types that don't have unproected_as_bytes, or is there some reason to prefer unprotected_as_bytes for its semantics? I'm leaning toward just using AsRef for everything that converts to/from byte slices. (Decided to go with AsRef.)
• [x] All the serde-related logic is implemented here in a separate module. I've taken to doing it this way in my own projects to keep the boilerplate out of the modules for each type, but I'm happy to break it up and move the four serde implementations nearer to where the types themselves are defined. (Decided to move everything to the typedefs module.
• [x] Another style point: I prefixed most of the types with their module names instead of importing them directly. For example, kdf::Salt instead of just Salt. I was concerned that other types may implement Salt in the future and we may add them to the serde module. This is a downside of keeping everything in one module; the names may have to be differentiated further. (Resolved by moving everything to typedefs module.)
• [x] There's no documentation. Not even a comment in sight. I'm not sure what the usual convention is for serde documentation. I'll see what other crates do.

Brian Smith has done some pretty interesting investigations into the default security of GitHub Actions at https://github.com/briansmith/untrusted/issues/50 and how to harden these. I'd like for us to re-trace some (if not all) of the steps there and see which can be implemented for our CI/CD.

I was surprised to see that default permissions for GitHub Actions are read+write for a repository. We should be able to change this to read-only without breaking the current CI/CD.

The official GitHub documentation for this: https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions

Eg. setting read-only for an Action:

name: action

permissions:
  contents: read

TODO:

• [X] Delete the stored token once the changes are applied and if it's not used anymore

Summary

The serde::Serialize and serde::Deserialize traits are used pervasively throughout the Rust ecosystem to enable flexible (de)serialization of data. Many authors derive these types on structs using macros provided by serde. If orion's types don't support these traits, users won't be able to derive these serialization traits, and will have to resort to converting orion types before storing them in a struct for serialization.

By implementing serde's standard traits, we enable authors to skip the mental burden and increased complexity of serializing orion types without serde.

Scope

At least for a first implementation, I would suggest only implementing serialization traits for types that are expected to either be stored in a database or sent to another application (e.g. over the network). This would include the following types.

• pwhash::PasswordHash
• hash::Digest
• auth::Tag
• kdf::Salt (added after suggestion below)

Motivating Example

My motivating example is storing a User type in a database. It would be nice to simply define a User struct, and then have it serialize to one long string using, for example, the bincode crate. This is what I would like the basic code to look like (using a fictional database API).

use serde::{Serialize, Deserialize};
use orion::pwhash::PasswordHash;

#[derive(Serialize, Deserialize)]
struct User {
    username: String,
    pwhash: PasswordHash,
    last_login: chrono::DateTime<Utc>,
}

fn store_user_in_db(user: &User, db: &Db) {
    db.write_bytes(&user.username, bincode::serialize(user));
}

fn retrieve_user_from_db(db: &Db) -> User {
    let bytes = db.get_bytes(&user.username);
    bincode::deserialize(bytes)
}

The PasswordHash type would serialize as a string by delegating to PasswordHash::unprotected_as_encoded.

Without serde's types implemented, however, we would have to do one of the following.

1. Create a separate User type that can be serialized.
2. Implement serde manually on User and delegate the PasswordHash serialization to its unprotected_as_encoded method. Neither solution is particularly attractive.

A separate, serializable User struct is probably the easiest of the the alternatives. It would look like this.

// previous definitions omitted

#[derive(Serialize, Deserialize)]
struct UserSerialize {
    username: String,
    pwhash: String,
    last_login: chrono::DateTime<Utc>,
}

fn store_user_in_db(user: &User {username, pwhash, last_login}, db: &Db) {
    let user_serialize = UserSerialize {username, last_login, pwhash: pwhash.unprotected_as_encoded()};
    db.write_bytes(&user.username, bincode::serialize(to_serialize));
}

fn retrieve_user_from_db(db: &Db) -> User {
    let bytes = db.get_bytes(&user.username);
    let UserSerialize {username, pwhash, last_login} = bincode::deserialize(bytes);
    User {username, last_login, pwhash: PasswordHash::from_encoded(pwhash)}
}

It isn't terrible in terms of boilerplate, but it could start to get annoying and possibly error-prone to repeat struct definitions for several different types instead of just User.

Implementing Serialize and Deserialize manually can be… unpleasant. Especially for anything complex, and definitely more boilerplate in any case.

Pros of implementing Serialize/Deserialize

• There is significant ergonomic benefit to supporting this nearly ubiquitous pair of serialization traits.
• Users may be less likely to mistakenly reach for non-constant-time operations if they never actually (de)serialize the type themselves and just let serde do it for them. The common workflow is to deserialize some bytes or a string into the relevant type (would be User here, for example), and then to do operations on the fully featured type. This is beneficial because many of orion's types use constant-time operations for things like comparisons. It's anecdotal, but in my first example, the PasswordHash type spends very little time in its serialized state because it's so easy to deserialize into the fully featured type. Making deserialization the fast path could save users from security mistakes.

Cons

• The serialization functions in orion are helpfully named unprotected_* to signify that by using them, authors are relinquishing the constant-time features that orion provides. If we hide these functions in a serde::Serialize implementation, users may be more likely to mistakenly serialize a sensitive type and perform insecure operations on it. I realize that this somewhat contradicts the last "Pro" above, but they both seem like legitimate possibilities to consider.

Additional Considerations

• Serialization should conform to constant-time operations already in place. For example, PasswordHash should serialize as a string using the already-implemented PasswordHash::unprotected_as_encoded function.
• We may have to add additional mechanisms for testing to ensure that all tests pass with or without the hypothetical serde feature enabled.

Currently, for a given streaming context Ctx in module foo we have:

let _: Ctx = foo::init();

This PR changes this to be more explicit:

let _: Ctx = foo::Ctx::init();

Bumps EmbarkStudios/cargo-deny-action from 1.3.1 to 1.3.2.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

The following changes in performance of ChaCha20 and related functions have been observed:

rustc 1.41.0 (5e1a79984 2020-01-27), orion 0.15.0:

• ChaCha20Poly1305: Input 128KiB, Throughput 419.22 MiB/s
• XChaCha20Poly1305: Input 128KiB, Throughput 419.53 MiB/s

rustc 1.56.0 (09c42c458 2021-10-18), orion 0.16.1:

• ChaCha20Poly1305: Input 128KiB, Throughput 275.66 MiB/s
• XChaCha20Poly1305: Input 128KiB, Throughput 275.19 MiB/s

The change in performance was only observed on an Intel(R) Core(TM) i7-4790K CPU machine. No regression was found on the Raspberry Pi 2 Model B V1.1 used to benchmark. When running cargo bench with orion v0.15.0 and rustc 1.56.0 (09c42c458 2021-10-18), the same regressions are shown.

It doesn't seem like any major changes were introduced in the ChaCha20 implementation between these versions, when inspecting git log. This, and that regression persists between different versions of Orion, tells us its most likely due to changes outside of this crate.

Further things to investigate:

• If the regression persists between rustc versions 1.41.0 to 1.56.0 (regression due to changes in rustc, maybe LLVM upgrades?)
• If any changes in dependencies, could have caused this

criterion relies on atty which is unsound and also seems as if it's unmaintained. Let's see if criterion (https://github.com/bheisler/criterion.rs/issues/629) moves away from this, and if not, how else we should handle this.

We use actions-rs in our CI workflows. It is possible this is no longer fully maintained and also I have read talk about, that there exist more simple alternatives such as https://github.com/dtolnay/rust-toolchain

When #[warn(clippy::derive_partial_eq_without_eq)] is enabled, clippy will suggest to add Eq to some types. We should make sure this makes sense before we add this, because removing them later will be a breaking change.

This is still a WIP. The basic method was to define an inner type that behaves the way we want it to, e.g. with special Drop impls, etc. Then for each newtype we want to create, we create a small wrapper over the inner type, like struct Digest(Blake2bArray), and implement two small traits, essentially From and AsRef, to convert to that inner type.

In another module, we provide more complex blanket implementations for any types that implement From and AsRef for our inner type. This allow us to, for example, auto-implement len, from_slice, etc for the newtypes without exposing the inner type (which could be hazardous for inner types like the planned PrivateArray type.

Portable SIMD has landed in nightly Rust core::simd[1]. We should investigate if using SIMD in Orion is a viable option. For example, ChaCha20 can be vectorized[2], which could provide quite substantial performance benefits.

What needs to be figured out:

• What is, if there is any, the timeline for stabilization of core::simd and potentially std::simd (?)
• Will the portable SIMD API require use of unsafe code if utilized within Orion?
• What other primitives could benefit from vectorization? BLAKE2 is a candidate as well IIRC
• If simd would require std for some unknown reason, do we want to consider providing vectorized implementations for safe-api only?
• Will the portable SIMD offered by Rust be portable enough in the sense that we'd be able to remove the current portable non-vectorized implementations?
• Would the portable SIMD be usable on all platforms we test on currently, including the non-std targets? What would the impact on WASM support be here?

[1]: Most of the work seems to happen over here https://github.com/rust-lang/portable-simd [2]: https://eprint.iacr.org/2013/759.pdf

Since the introduction of serde support, most newtypes implement TryFrom<&[u8]> which simply delegates to from_slice(). This seems a bit redundant, especially if/when moving to Rust 2021 Edition, where TryFrom/TryInto are part of prelude. We could consider removing from_slice in the future and simply provide TryFrom only instead.