Description

Keypair serializes, but does not deserialize when trying to test it out in MystenLabs/narwhal#557. Does not break Narwhal nor Sui so continuing with MystenLabs/narwhal#557 anyways.

To replicate this issue, run the following test:

#[test]
fn test_kp_serde() {
    let kp = keys().pop().unwrap();
    let serialized = bincode::serialize(&kp).unwrap();
    let deserialized: Ed25519KeyPair = bincode::deserialize(&serialized).unwrap();
}

We get this error:

thread 'bls12377_tests::test_kp_serde' panicked at 'called `Result::unwrap()` on an `Err` value: Custom("invalid Base64 encoding")', crypto/src/tests/bls12377_tests.rs:375:75

Upon further digging, it looks like removing this line fixes the issue:

#[serde(tag = "type")] // REMOVE THIS LINE
pub struct Ed25519KeyPair {
    name: Ed25519PublicKey,
    secret: Ed25519PrivateKey,
}

Rationale

1. while we do use the BLS12-377 curve for proving (for good reason), we do not use it for signing, and are unlikely to do so in the near to medium term (due to strict performance requirements),
2. @benr-ml is about to introduce a completely distinct instance & implementation of this curve as a dependency, in the context of threshold signing (see repo)
3. The implementations of BLS12-377 as a signing scheme are not as mature as other instances, notably in the area of hashing to the curve. Indeed, BLS12-377 does not yet have a suite in the hash-to-curve RFC.

Conclusion

It seems best to spend a little bit of time thinking about BLS12-377, a scheme we don't really use, before adopting & maintaining a version of it. Once we sort out the map_to_curve part, it may be suitable to improve & build on the implementation we're set to receive from (2.) above (or to an upstream impl) rather than continuing with this specific variant of the code.

Fixes #242.

Hi I'm trying to build the sui repo with a change in the Cargo.toml file

fastcrypto = { path = "../fastcrypto/fastcrypto"}

So that I can reference a local repo o ffastcrypto reset to the revision

git reset --hard bbb2d02a7a64c27314721748cc4d015b00490dbe

I get the following error. Any ideas on what I am doing wrong? Thanks. error[E0277]: the trait bound fastcrypto::bls12381::min_sig::BLS12381PublicKey: MallocSizeOf` is not satisfied --> narwhal/types/src/primary.rs:148:48 | 148 | #[derive(Builder, Clone, Default, Deserialize, MallocSizeOf, Serialize)] | ^^^^^^^^^^^^ the trait `MallocSizeOf` is not implemented for `fastcrypto::bls12381::min_sig::BLS12381PublicKey

Fixes #14, at least partially! For the public keys, base64 will have to do until #1 is resolved.

For consideration: do you think a Secret derive macro that:

1. derives Zeroize
2. implements safe Display and Debug
3. implements unsafe_write_secret: &self -> String (for debugging)

makes sense? I could slot it into this PR. This would make the code nicer:

#[derive(SecretMaterial)]
 pub struct BLS12381PrivateKey {
     pub privkey: blst::SecretKey,
     pub bytes: OnceCell<[u8; BLS_PRIVATE_KEY_LENGTH]>,
     ...
}

Closing #181. PR #209 should be merged first.

This PR adds benchmarking to the CI.

• Running all benchmarks takes ~1 hour, so we only do it on publish, but it may also be triggered manually.
• We use cargo criterion to run benchmarks and generate reports:
  • In order to keep history of benchmarks, we first checkout the old reports from gh-pages
  • Then we use cargo criterion to run all benchmarks with the most recent version of fastcrypto and generate reports,
  • The new reports (including history) are pushed to gh-pages. See https://mystenlabs.github.io/fastcrypto/benchmarks/criterion/reports/ for an incomplete report (not all benchmarks have been run yet), and https://mystenlabs.github.io/fastcrypto/benchmarks/criterion/reports/Verify/Ed25519/history.html for an example of how the history of a benchmark is shown.
  • The results are stored as a JSON-file and stored under benchmarks/history with the latest commit hash as the filename. This allows analysis of historic data and comparison of versions using various analysis tools.

This restructures the gh-pages such that documentation is put under "docs" and benchmarks under "benchmarks".

As our backends evolve, we might realize that down the road the efficiency of some schemes might be updated, perhaps also have additional parallelization improvements etc. Let's maintain a timestamped bench-results page with diagrams to track improvement. Then protocol designers (and crypto paper authors) can really have a fair (and evolving) primitive comparison matrixes.

TL;DR We receive incredibly positive feedback that fastcrypto gradually starts being a de facto reference implementation for future research papers and protocol designers due to providing a well-defined comparison matrix between some of the fastest implementations in the space + for batch/aggregated (in the future w precomputed table) + parallelization-friendly versions etc. Our results already:

• changed the perception on how modern BLS impl. perform compared to the fastest ECDSA and EdDSA crates.
• identified unfair or outdated comparisons reported by some lib authors.
• some implementations perform extra serialization operations and/or others provide different security guarantees. for instance: -- BLS fast_aggregation_verify can be applied when rogue key attack protection already took place (ie via PoP sigs) -- another case in ed25519 batch-verify where two different libs use as inputs Objects Vs raw bytes (and thus extra serialization steps are required in the 2nd, skewing the comparison results a bit). -- another case is base64 malleable vs non-malleable and/or constant vs variable time implementations.
• how can someone combine different backends for different setups and input sizes (ie see the results for base64 encoders where for different input sizes we have another winner impl).
• what are the sweet points when we compare the two fastest Rust implementations (or ports). Example: Many ask about "When aggregated BLS starts winning against batched ed25519?", "what if we used a parallelized version?" etc.
• we already have evidence that some cryptographic protocols, which were supposed to be hanging behind re performance, are in practice a lot faster than expected and vice versa.
• how parallel and/or vectorized implementations perform against each other (ie AVX2, no-threads BLST etc)?
• easily compare against optimized and experimental versions (i.e., with precomputed tables, removing one half-scalar in random combinations, half-aggregation etc).

This will help to avoid surprises in Sui or Narwhal re using different sha2/sha3/keccak deps in different places. Ideally we should control every crypto algorithm in fastcrypto. After having these wrappers, we should update both Narwhal and Sui repos by invoking fastcrypto's api.

I was thinking that the same applies for base64, but let's wait for this, as Rust's popular base64 crate is working on the malleability fix.

Actually this should be the default mode, as our benchmarks say this is faster + sigs are reused, while pks are usually static for BFT committees (at least per epoch).

The test_sk_zeroization_on_drop test for BLS12377 fails sometimes because one of the bytes from the secret key not changing before this check will cause the test to fail. We instead check that not all the bytes are still equal.

Bumps signature from 1.6.3 to 1.6.4.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot will merge this PR once CI passes on it, as requested by @mystenadmin.

Added support for AES with 128 and 256 bit keys using either CTR, CBC, GCM with PKCS#7 padding. All modes are backed by crates from RustCrypto (https://github.com/RustCrypto/block-modes).

• Removed PublicKeyBytes and it dependencies (verified they are not used in sui/). That struct could have stored invalid "serialized" objects, thus it was not always safe to use it.
• Changed Base64Representation to serialize differently depending on is_human_readable and renamed it to BytesRepresentation. (I couldn't find a nicer way to support both options unfortunately). We may use BytesRepresentation everywhere we intended to use PublicKeyBytes. Also added a macro for deriving such type for a given type generate_bytes_representation.
• Added a macro serialize_deserialize_with_to_from_byte_array for simplifying adding ser/des for objects without caching.

moving this from sui to fastcrypto since both sui and narwhal depends on it. see https://github.com/MystenLabs/sui/pull/6927 for usage

intent_tests needed struct defined in sui, so the tests are kept there.

This is better hygiene, to also defend against the private - public key mismatching we found on ed25519 libs. The pub key can be derived during deserialization. Creating a similar issue in Sui repo.