the official Rust and C implementations of the BLAKE3 cryptographic hash function

Related tags

Cryptography BLAKE3
Overview

BLAKE3

BLAKE3 is a cryptographic hash function that is:

  • Much faster than MD5, SHA-1, SHA-2, SHA-3, and BLAKE2.
  • Secure, unlike MD5 and SHA-1. And secure against length extension, unlike SHA-2.
  • Highly parallelizable across any number of threads and SIMD lanes, because it's a Merkle tree on the inside.
  • Capable of verified streaming and incremental updates, again because it's a Merkle tree.
  • A PRF, MAC, KDF, and XOF, as well as a regular hash.
  • One algorithm with no variants, which is fast on x86-64 and also on smaller architectures.

The chart below is an example benchmark of 16 KiB inputs on modern server hardware (a Cascade Lake-SP 8275CL processor). For more detailed benchmarks, see the BLAKE3 paper.

performance graph

BLAKE3 is based on an optimized instance of the established hash function BLAKE2 and on the original Bao tree mode. The specifications and design rationale are available in the BLAKE3 paper. The default output size is 256 bits. The current version of Bao implements verified streaming with BLAKE3.

This repository is the official implementation of BLAKE3. It includes:

  • The blake3 Rust crate, which includes optimized implementations for SSE2, SSE4.1, AVX2, AVX-512, and NEON, with automatic runtime CPU feature detection on x86. The rayon feature provides multithreading.

  • The b3sum Rust crate, which provides a command line interface. It uses multithreading by default, making it an order of magnitude faster than e.g. sha256sum on typical desktop hardware.

  • The C implementation, which like the Rust implementation includes SIMD code and runtime CPU feature detection on x86. Unlike the Rust implementation, it's not currently multithreaded. See c/README.md.

  • The reference implementation, which is discussed in Section 5.1 of the BLAKE3 paper. This implementation is much smaller and simpler than the optimized ones above. If you want to see how BLAKE3 works, or you're writing a port that doesn't need multithreading or SIMD optimizations, start here.

  • A set of test vectors that covers extended outputs, all three modes, and a variety of input lengths.

  • Actions Status

BLAKE3 was designed by:

The development of BLAKE3 was sponsored by Electric Coin Company.

NOTE: BLAKE3 is not a password hashing algorithm, because it's designed to be fast, whereas password hashing should not be fast. If you hash passwords to store the hashes or if you derive keys from passwords, we recommend Argon2.

Usage

The b3sum utility

The b3sum command line utility prints the BLAKE3 hashes of files or of standard input. Prebuilt binaries are available for Linux, Windows, and macOS (requiring the unidentified developer workaround) on the releases page. If you've installed Rust and Cargo, you can also build b3sum yourself with:

cargo install b3sum

If rustup didn't configure your PATH for you, you might need to go looking for the installed binary in e.g. ~/.cargo/bin. You can test out how fast BLAKE3 is on your machine by creating a big file and hashing it, for example:

# Create a 1 GB file.
head -c 1000000000 /dev/zero > /tmp/bigfile
# Hash it with SHA-256.
time openssl sha256 /tmp/bigfile
# Hash it with BLAKE3.
time b3sum /tmp/bigfile

The blake3 crate docs.rs

To use BLAKE3 from Rust code, add a dependency on the blake3 crate to your Cargo.toml. Here's an example of hashing some input bytes:

// Hash an input all at once.
let hash1 = blake3::hash(b"foobarbaz");

// Hash an input incrementally.
let mut hasher = blake3::Hasher::new();
hasher.update(b"foo");
hasher.update(b"bar");
hasher.update(b"baz");
let hash2 = hasher.finalize();
assert_eq!(hash1, hash2);

// Extended output. OutputReader also implements Read and Seek.
let mut output = [0; 1000];
let mut output_reader = hasher.finalize_xof();
output_reader.fill(&mut output);
assert_eq!(&output[..32], hash1.as_bytes());

// Print a hash as hex.
println!("{}", hash1);

Besides hash, BLAKE3 provides two other modes, keyed_hash and derive_key. The keyed_hash mode takes a 256-bit key:

// MAC an input all at once.
let example_key = [42u8; 32];
let mac1 = blake3::keyed_hash(&example_key, b"example input");

// MAC incrementally.
let mut hasher = blake3::Hasher::new_keyed(&example_key);
hasher.update(b"example input");
let mac2 = hasher.finalize();
assert_eq!(mac1, mac2);

The derive_key mode takes a context string and some key material (not a password). The context string should be hardcoded, globally unique, and application-specific. A good default format for the context string is "[application] [commit timestamp] [purpose]":

// Derive a couple of subkeys for different purposes.
const EMAIL_CONTEXT: &str = "BLAKE3 example 2020-01-07 17:10:44 email key";
const API_CONTEXT: &str = "BLAKE3 example 2020-01-07 17:11:21 API key";
let input_key_material = b"usually at least 32 random bytes, not a password";
let email_key = blake3::derive_key(EMAIL_CONTEXT, input_key_material);
let api_key = blake3::derive_key(API_CONTEXT, input_key_material);
assert_ne!(email_key, api_key);

The C implementation

See c/README.md.

Other implementations

We post links to third-party bindings and implementations on the @BLAKE3team Twitter account whenever we hear about them. Some highlights include an optimized Go implementation, Wasm bindings for Node.js and browsers, binary wheels for Python, .NET bindings, and JNI bindings.

Contributing

Please see CONTRIBUTING.md.

Intellectual property

The Rust code is copyright Jack O'Connor, 2019-2020. The C code is copyright Samuel Neves and Jack O'Connor, 2019-2020. The assembly code is copyright Samuel Neves, 2019-2020.

This work is released into the public domain with CC0 1.0. Alternatively, it is licensed under the Apache License 2.0.

Miscellany

Comments
  • impl digest::* for Blake3?

    impl digest::* for Blake3?

    It'd be really nice for BLAKE3 to follow RustCrypto's digest crate to allow for BLAKE3 to substitute other functions in generic code.

    ~~Hasher is a poor name here, Blake3 would be cleaner. I expect to see many instances of use blake3::Hasher as Blake3. A type alias suffices to avoid a breaking change type Hasher = Blake3~~

    These traits can be implemented without a breaking change. However, it'd be cleaner to only implement digest's traits and any other breaking changes could be batched. The release is early and much more usage will come with the digest traits implemented.

    Great work BLAKE3-team!

    opened by WildCryptoFox 31
  • Hasher::new_derive_key accepts a str, why not [u8]?

    Hasher::new_derive_key accepts a str, why not [u8]?

    There is no reason for UTF-8 here. Please s/str/[u8]/ here. Alternatively provide another function to avoid the breaking change. Though, as I've mentioned in #11 I'm not against a breaking change for consistency with other crates, especially this early.

    https://docs.rs/blake3/0.1.0/blake3/struct.Hasher.html#method.new_derive_key

    pub fn new_derive_key(context: &str) -> Self
    
    opened by WildCryptoFox 21
  • Compilation error on Windows

    Compilation error on Windows

    Similar to issue https://github.com/BLAKE3-team/BLAKE3/issues/79, I'm getting compilation errors when building blake3 v0.3.7 on Windows:

    [2021-01-13T09:05:58.008Z] The following warnings were emitted during compilation:
    [2021-01-13T09:05:58.008Z] 
    [2021-01-13T09:05:58.008Z] warning: The C compiler "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\cl.exe" does not support /arch:AVX512.
    [2021-01-13T09:05:58.008Z] 
    [2021-01-13T09:05:58.008Z] error: failed to run custom build command for `blake3 v0.3.7`
    [2021-01-13T09:05:58.008Z] 
    [2021-01-13T09:05:58.008Z] Caused by:
    [2021-01-13T09:05:58.008Z]   process didn't exit successfully: `C:\Users\ci\AppData\Local\Temp\cargo-installMihItV\release\build\blake3-98f585247d671863\build-script-build` (exit code: 1)
    [2021-01-13T09:05:58.008Z]   --- stdout
    [2021-01-13T09:05:58.008Z]   cargo:rerun-if-env-changed=CARGO_FEATURE_PURE
    [2021-01-13T09:05:58.008Z]   TARGET = Some("x86_64-pc-windows-msvc")
    [2021-01-13T09:05:58.008Z]   HOST = Some("x86_64-pc-windows-msvc")
    [2021-01-13T09:05:58.008Z]   CC_x86_64-pc-windows-msvc = None
    [2021-01-13T09:05:58.008Z]   CC_x86_64_pc_windows_msvc = None
    [2021-01-13T09:05:58.008Z]   HOST_CC = None
    [2021-01-13T09:05:58.008Z]   CC = None
    [2021-01-13T09:05:58.008Z]   CFLAGS_x86_64-pc-windows-msvc = None
    [2021-01-13T09:05:58.008Z]   CFLAGS_x86_64_pc_windows_msvc = None
    [2021-01-13T09:05:58.008Z]   HOST_CFLAGS = None
    [2021-01-13T09:05:58.008Z]   CFLAGS = None
    [2021-01-13T09:05:58.008Z]   CRATE_CC_NO_DEFAULTS = None
    [2021-01-13T09:05:58.008Z]   CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
    [2021-01-13T09:05:58.008Z]   OPT_LEVEL = Some("3")
    [2021-01-13T09:05:58.008Z]   CC_x86_64-pc-windows-msvc = None
    [2021-01-13T09:05:58.008Z]   CC_x86_64_pc_windows_msvc = None
    [2021-01-13T09:05:58.008Z]   HOST_CC = None
    [2021-01-13T09:05:58.008Z]   CC = None
    [2021-01-13T09:05:58.008Z]   CFLAGS_x86_64-pc-windows-msvc = None
    [2021-01-13T09:05:58.008Z]   CFLAGS_x86_64_pc_windows_msvc = None
    [2021-01-13T09:05:58.008Z]   HOST_CFLAGS = None
    [2021-01-13T09:05:58.008Z]   CFLAGS = None
    [2021-01-13T09:05:58.008Z]   CRATE_CC_NO_DEFAULTS = None
    [2021-01-13T09:05:58.009Z]   CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
    [2021-01-13T09:05:58.009Z]   DEBUG = Some("false")
    [2021-01-13T09:05:58.009Z]   cargo:warning=The C compiler "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\cl.exe" does not support /arch:AVX512.
    [2021-01-13T09:05:58.009Z]   cargo:rerun-if-env-changed=BLAKE3_CI
    [2021-01-13T09:05:58.009Z]   cargo:rerun-if-env-changed=CARGO_FEATURE_PREFER_INTRINSICS
    [2021-01-13T09:05:58.009Z]   cargo:rerun-if-env-changed=CARGO_FEATURE_PURE
    [2021-01-13T09:05:58.009Z]   cargo:rustc-cfg=blake3_sse2_ffi
    [2021-01-13T09:05:58.009Z]   cargo:rustc-cfg=blake3_sse41_ffi
    [2021-01-13T09:05:58.009Z]   cargo:rustc-cfg=blake3_avx2_ffi
    [2021-01-13T09:05:58.009Z]   TARGET = Some("x86_64-pc-windows-msvc")
    [2021-01-13T09:05:58.009Z]   OPT_LEVEL = Some("3")
    [2021-01-13T09:05:58.009Z]   HOST = Some("x86_64-pc-windows-msvc")
    [2021-01-13T09:05:58.009Z]   CC_x86_64-pc-windows-msvc = None
    [2021-01-13T09:05:58.009Z]   CC_x86_64_pc_windows_msvc = None
    [2021-01-13T09:05:58.009Z]   HOST_CC = None
    [2021-01-13T09:05:58.009Z]   CC = None
    [2021-01-13T09:05:58.009Z]   CFLAGS_x86_64-pc-windows-msvc = None
    [2021-01-13T09:05:58.009Z]   CFLAGS_x86_64_pc_windows_msvc = None
    [2021-01-13T09:05:58.009Z]   HOST_CFLAGS = None
    [2021-01-13T09:05:58.009Z]   CFLAGS = None
    [2021-01-13T09:05:58.009Z]   CRATE_CC_NO_DEFAULTS = None
    [2021-01-13T09:05:58.009Z]   CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
    [2021-01-13T09:05:58.009Z]   DEBUG = Some("false")
    [2021-01-13T09:05:58.009Z]   running: "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\ml64.exe" "-nologo" "-FoC:\\Users\\ci\\AppData\\Local\\Temp\\cargo-installMihItV\\release\\build\\blake3-86dd5a2a633273e8\\out\\c/blake3_sse2_x86-64_windows_msvc.o" "-c" "c/blake3_sse2_x86-64_windows_msvc.asm"
    [2021-01-13T09:05:58.009Z]    Assembling: c/blake3_sse2_x86-64_windows_msvc.asm
    [2021-01-13T09:05:58.009Z]   c/blake3_sse2_x86-64_windows_msvc.asm(2057)pecified size
    [2021-01-13T09:05:58.009Z]   c/blake3_sse2_x86-64_windows_msvc.asm(2057) : error A2070:invalid instrucc/blake3_sse2_x86-64_windows_msvc.asm(2058)pecified size
    [2021-01-13T09:05:58.009Z]   c/blake3_sse2_x86-64_windows_msvc.asm(2058) : error A2070:invalid instrucc/blake3_sse2_x86-64_windows_msvc.asm(2189)pecified size
    [2021-01-13T09:05:58.009Z]   c/blake3_sse2_x86-64_windows_msvc.asm(2189) : error A2070:invalid instrucc/blake3_sse2_x86-64_windows_msvc.asm(2190)pecified size
    [2021-01-13T09:05:58.009Z]   c/blake3_sse2_x86-64_windows_msvc.asm(2190) : error A2070:invalid instrucexit code: 1
    [2021-01-13T09:05:58.009Z]   running: "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64\\ml64.exe" "-nologo" "-FoC:\\Users\\ci\\AppData\\Local\\Temp\\cargo-installMihItV\\release\\build\\blake3-86dd5a2a633273e8\\out\\c/blake3_sse41_x86-64_windows_msvc.o" "-c" "c/blake3_sse41_x86-64_windows_msvc.asm"
    [2021-01-13T09:05:58.009Z]    Assembling: c/blake3_sse41_x86-64_windows_msvc.asm
    [2021-01-13T09:05:58.009Z]   c/blake3_sse41_x86-64_windows_msvc.asm(1820)specified size
    [2021-01-13T09:05:58.009Z]   c/blake3_sse41_x86-64_windows_msvc.asm(1820) : error A2070:invalid instruc/blake3_sse41_x86-64_windows_msvc.asm(1821)specified size
    [2021-01-13T09:05:58.009Z]   c/blake3_sse41_x86-64_windows_msvc.asm(1821) : error A2070:invalid instruc/blake3_sse41_x86-64_windows_msvc.asm(1941)specified size
    [2021-01-13T09:05:58.009Z]   c/blake3_sse41_x86-64_windows_msvc.asm(1941) : error A2070:invalid instruc/blake3_sse41_x86-64_windows_msvc.asm(1942)specified size
    [2021-01-13T09:05:58.009Z]   c/blake3_sse41_x86-64_windows_msvc.asm(1942) : error A2070:invalid instruexit code: 1
    

    I'm actually trying to build sccache 0.2.15, and blake3 is a dependency (or transient dependency?) of that, so forgive me for my unfamiliarity with this project. I've checked the open issues/PRs and didn't see this reported yet.

    opened by nre-ableton 18
  • AVX-512 compile error on macOS Clang

    AVX-512 compile error on macOS Clang

    Describe the bug

    Clean clone of Wasmer fails to compile on MacOS due to AVX-512 error when compiling assembly code.

    The actual compilation command that fails (full compilation log below):

    "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-std=c11" "-o" "wasmer/target/debug/build/blake3-da163d92031ce8e8/out/c/blake3_avx512_x86-64_unix.o" "-c" "c/blake3_avx512_x86-64_unix.S"

    cargo:warning=c/blake3_avx512_x86-64_unix.S:2315:9: error: instruction requires: AVX-512 ISA AVX-512 VL ISA
    cargo:warning=        vpxord xmm3, xmm3, xmm0
    

    Compilation works if -march=skylake-avx512 is added to command above, the error happens because Clang by default does not enable AVX-512 intrinsics since they are not supported by my CPU.

    Additional context

    Full compilation log:

    error: failed to run custom build command for `blake3 v0.3.1`
    
    Caused by:
      process didn't exit successfully: `wasmer/target/debug/build/blake3-32d62a0f85ffa978/build-script-build` (exit code: 1)
    --- stdout
    cargo:rerun-if-env-changed=CARGO_FEATURE_PURE
    cargo:rerun-if-env-changed=CARGO_FEATURE_PURE
    TARGET = Some("x86_64-apple-darwin")
    HOST = Some("x86_64-apple-darwin")
    CC_x86_64-apple-darwin = None
    CC_x86_64_apple_darwin = None
    HOST_CC = None
    CC = None
    CFLAGS_x86_64-apple-darwin = None
    CFLAGS_x86_64_apple_darwin = None
    HOST_CFLAGS = None
    CFLAGS = None
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2,sse3,ssse3")
    CC_x86_64-apple-darwin = None
    CC_x86_64_apple_darwin = None
    HOST_CC = None
    CC = None
    CFLAGS_x86_64-apple-darwin = None
    CFLAGS_x86_64_apple_darwin = None
    HOST_CFLAGS = None
    CFLAGS = None
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2,sse3,ssse3")
    cargo:rerun-if-env-changed=CARGO_FEATURE_PREFER_INTRINSICS
    TARGET = Some("x86_64-apple-darwin")
    HOST = Some("x86_64-apple-darwin")
    CC_x86_64-apple-darwin = None
    CC_x86_64_apple_darwin = None
    HOST_CC = None
    CC = None
    CFLAGS_x86_64-apple-darwin = None
    CFLAGS_x86_64_apple_darwin = None
    HOST_CFLAGS = None
    CFLAGS = None
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2,sse3,ssse3")
    CC_x86_64-apple-darwin = None
    CC_x86_64_apple_darwin = None
    HOST_CC = None
    CC = None
    CFLAGS_x86_64-apple-darwin = None
    CFLAGS_x86_64_apple_darwin = None
    HOST_CFLAGS = None
    CFLAGS = None
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2,sse3,ssse3")
    cargo:rerun-if-env-changed=CARGO_FEATURE_PURE
    cargo:rerun-if-env-changed=CARGO_FEATURE_PREFER_INTRINSICS
    cargo:rustc-cfg=blake3_sse41_ffi
    cargo:rustc-cfg=blake3_avx2_ffi
    cargo:rustc-cfg=blake3_avx512_ffi
    TARGET = Some("x86_64-apple-darwin")
    OPT_LEVEL = Some("0")
    HOST = Some("x86_64-apple-darwin")
    CC_x86_64-apple-darwin = None
    CC_x86_64_apple_darwin = None
    HOST_CC = None
    CC = None
    CFLAGS_x86_64-apple-darwin = None
    CFLAGS_x86_64_apple_darwin = None
    HOST_CFLAGS = None
    CFLAGS = None
    CRATE_CC_NO_DEFAULTS = None
    DEBUG = Some("true")
    CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2,sse3,ssse3")
    running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-std=c11" "-o" "wasmer/target/debug/build/blake3-da163d92031ce8e8/out/c/blake3_sse41_x86-64_unix.o" "-c" "c/blake3_sse41_x86-64_unix.S"
    exit code: 0
    running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-std=c11" "-o" "wasmer/target/debug/build/blake3-da163d92031ce8e8/out/c/blake3_avx2_x86-64_unix.o" "-c" "c/blake3_avx2_x86-64_unix.S"
    exit code: 0
    running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-std=c11" "-o" "wasmer/target/debug/build/blake3-da163d92031ce8e8/out/c/blake3_avx512_x86-64_unix.o" "-c" "c/blake3_avx512_x86-64_unix.S"
    cargo:warning=c/blake3_avx512_x86-64_unix.S:28:9: error: instruction requires: AVX-512 ISA
    cargo:warning=        kmovw k1, r9d
    cargo:warning=        ^
    cargo:warning=c/blake3_avx512_x86-64_unix.S:38:9: error: instruction requires: AVX-512 ISA AVX-512 VL ISA
    cargo:warning=        vpcmpltud k2, ymm2, ymm0
    cargo:warning=        ^
    
    rustc 1.42.0 (b8cedc004 2020-03-09) | x86_64
    

    Output of sysctl -a | grep machdep.cpu.features:

    machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C
    

    To note, no support for AVX-512.

    Related issue from Wasmer: https://github.com/wasmerio/wasmer/issues/1373

    opened by tritao 18
  • Excessive futex syscalling with no-mmap

    Excessive futex syscalling with no-mmap

    Problem: b3sum --no-mmap is using a tiny read buffer size (8KB) and syscalls read(2) and futex(2) aggressively. I find that hashing at only 110 MB/s maxes out all 24 cores, doing about 10k syscalls per second (tail -f strace.log | pv -l >/dev/null).

    top

    CPU% MEM%   TIME+  Command                                                                                                                                                                                                           
    2025  0.0 22:58.81 b3sum --no-mmap idsab.tar.zst
    

    strace.log

    futex(0x563438b1804c, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
    futex(0x563438b17ff0, FUTEX_WAKE_PRIVATE, 1) = 0
    read(3, "\3\352\t\225\213r\371\3421\231\361\303\306\237\252\362\274\212>\33\t\f&\240u\31\252\331\310\242\265T"..., 8192) = 8192
    futex(0x563438b14028, FUTEX_WAKE_PRIVATE, 2147483647) = 2
    futex(0x563438b13fd0, FUTEX_WAKE_PRIVATE, 1) = 1
    read(3, "9\22\373\5\17\357\35\0364\35\371{\324\220\275J~B0$\273\332\264\20\v\352\304\23\301l\316\320"..., 8192) = 8192
    futex(0x563438b18048, FUTEX_WAIT_PRIVATE, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
    futex(0x563438b17ff0, FUTEX_WAKE_PRIVATE, 1) = 0
    read(3, "\225/\247\216\367Q\375,E\357\305\367\323\321\2\223f\266\n\222\257\305\260qLJ\f\247\370\202\6\306"..., 8192) = 8192
    futex(0x563438b1402c, FUTEX_WAKE_PRIVATE, 2147483647) = 1
    futex(0x563438b13fd0, FUTEX_WAKE_PRIVATE, 1) = 1
    read(3, "7\365\30n\370K\f\265\200\363r7\35E\321\347_\226\351-\331\24\232\200\352\233\316W\20\2121\264"..., 8192) = 8192
    futex(0x563438b14028, FUTEX_WAKE_PRIVATE, 2147483647) = 2
    futex(0x563438b13fd0, FUTEX_WAKE_PRIVATE, 1) = 1
    read(3, "\234\235\250\352\315\343*\t\215\205o(\231\230\325\f\222\257o\302A3\315z\240}\364'\327\33Wj"..., 8192) = 8192
    ... and so on
    

    Environment:

    • Built with cargo install b3sum (as of 2020-02-03)
    • Stable cargo 1.41.0 (626f0f40e 2019-12-03)
    • ZFS on Linux
    • Ubuntu 20.04
    • Kernel 5.4.0-9-generic
    opened by terorie 18
  • OID Value

    OID Value

    Currently I use this fake OID value (next available one after blake2 family) but we probably need an official one to be defined

    1 3 6 1 4 1 1722 12 2 3 8 : BLAKE3 : blake3

    Reference: http://www.oid-info.com/cgi-bin/display?oid=1+3+6+1+4+1+1722+12+2&action=display

    opened by dlegaultbbry 15
  • BLAKE3 collision resistance

    BLAKE3 collision resistance

    How collision resistant are BLAKE3 hashes of different length comparing to the following table from Wikipedia?

    Hash function | Security claim | Best attack | Publish date | Comment -- | -- | -- | -- | -- SHA256 | 2128 | 31 of 64 rounds (265.5) | 2013-05-28 | Two-block collision.[3] SHA512 | 2256 | 24 of 80 rounds (232.5) | 2008-11-25 | Paper.[4] SHA-3 | Up to 2512 | 6 of 24 rounds (250) | 2017 | Paper.[5] BLAKE2s | 2128 | 2.5 of 10 rounds (2112) | 2009-05-26 | Paper.[6] BLAKE2b | 2256 | 2.5 of 12 rounds (2224) | 2009-05-26 | Paper.[6]

    I am especially interested to compare BLAKE3 128 to BLAKE2s 128 as it is the digest size (#107) I'd like to use.

    opened by abitrolly 14
  • Fix differing behavior in C implementation on big-endian platforms

    Fix differing behavior in C implementation on big-endian platforms

    Hello,

    J-P did not express much interest for “exotic” platforms, so @jakub-zwolakowski only configured the C implementation to be interpreted on a 32-bit big-endian and on a 64-bit big-endian platform, in addition to x86 and x86-64. The chosen platforms, ppc32 and ppc64, have the advantage of defining char as an unsigned integer type, so they provide one more portability check for the same price (BLAKE3 is not written to rely on the signedness of char, but since the type char appears in the prototypes of standard functions, it is difficult to be absolutely sure).

    Long story short, there was no Undefined Behavior for these big-endian platforms but they were computing different digests than the little-endian platforms. One possibility is that a bug in TrustInSoft CI's emulation of these big-endian architectures causes the execution to be imperfectly emulated, but that does not seem very likely, because any ordinary bug would already be visible for shorter plaintexts. Another possibility is that although the C implementation of BLAKE3 is written to behave the same on little- and big-endian platforms with functions named load32 and store32, one spot where these functions were necessary was missed.

    The tests that show the wrong digest being computed are: https://dev.ci.trust-in-soft.com/projects/jakub-zwolakowski/BLAKE3/37?page=1

    The same tests, with the change in this PR applied, compute the correct digest in: https://ci.trust-in-soft.com/projects/pascal-cuoq/BLAKE3/2?page=4

    I expect the change to make more sense to you when you see it than it did to me when writing it. If the change does not seem to make sense, then we can investigate further.

    opened by pascal-cuoq 14
  • b3sum incredibly slow

    b3sum incredibly slow

    b3sum runs incredibly slow.

    PC: Windows 10 Pro i7-6700 32GB RAM

    b3sum built using: cargo install b3sum (v0.1.2)

    Test File: 2.88GB

    Results:

    md5sum: 10 seconds
    b2sum: 5 seconds
    b3sum: 4 Minutes 37 Seconds
    
    opened by tERyceNzAchE 13
  • Fake warning on Fedora 34 (in fact newer gcc version 11.2.1 20210728)

    Fake warning on Fedora 34 (in fact newer gcc version 11.2.1 20210728)

    I found "classical" fake warning on newer gcc

    [[email protected] 47]# make
    g++ -Dunix -O3 -march=native zpaqfranz.cpp -o zpaqfranz -pthread -static
    In function ‘size_t compress_parents_parallel(const uint8_t*, size_t, const uint32_t*, uint8_t, uint8_t*)’,
        inlined from ‘void compress_subtree_to_parent_node(const uint8_t*, size_t, const uint32_t*, uint64_t, uint8_t, uint8_t*)’ at zpaqfranz.cpp:9425:34,
        inlined from ‘void blake3_hasher_update.part.0(blake3_hasher*, const void*, size_t)’ at zpaqfranz.cpp:9609:38:
    zpaqfranz.cpp:9313:11: warning: ‘void* memcpy(void*, const void*, size_t)’ writing 32 bytes into a region of size 0 overflows the destination [-Wstringop-overflow=]
     9313 |     memcpy(&out[parents_array_len * BLAKE3_OUT_LEN],
          |     ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     9314 |            &child_chaining_values[2 * parents_array_len * BLAKE3_OUT_LEN],
          |            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     9315 |            BLAKE3_OUT_LEN);
          |            ~~~~~~~~~~~~~~~
    zpaqfranz.cpp: In function ‘void blake3_hasher_update.part.0(blake3_hasher*, const void*, size_t)’:
    zpaqfranz.cpp:9422:11: note: at offset 32 into destination object ‘out_array’ of size 32
     9422 |   uint8_t out_array[MAX_SIMD_DEGREE_OR_2 * BLAKE3_OUT_LEN / 2];
          |           ^~~~~~~~~
    In function ‘void compress_subtree_to_parent_node(const uint8_t*, size_t, const uint32_t*, uint64_t, uint8_t, uint8_t*)’,
        inlined from ‘void blake3_hasher_update.part.0(blake3_hasher*, const void*, size_t)’ at zpaqfranz.cpp:9609:38:
    zpaqfranz.cpp:9426:11: warning: ‘void* memcpy(void*, const void*, size_t)’ reading between 64 and 96 bytes from a region of size 32 [-Wstringop-overread]
     9426 |     memcpy(cv_array, out_array, num_cvs * BLAKE3_OUT_LEN);
          |     ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    zpaqfranz.cpp: In function ‘void blake3_hasher_update.part.0(blake3_hasher*, const void*, size_t)’:
    zpaqfranz.cpp:9422:11: note: source object ‘out_array’ of size 32
     9422 |   uint8_t out_array[MAX_SIMD_DEGREE_OR_2 * BLAKE3_OUT_LEN / 2];
          |           ^~~~~~~~~
    

    may I suggest some code refactoring (just to do not get those spurious warnings)?

    It happens quite frequently that I have to change perfectly functional code so as not to have to "fight" with compilers often full of bugs in detecting "memory" errors.
    It's not that important, just a hint

    PS Newer Linux-based g++ runs BLAKE3 two time faster then "not so old" g++, on AMD.
    Yes, two time faster, not 2%!

    opened by fcorbelli 12
  • fix disabled-optimization -Wall -Werror

    fix disabled-optimization -Wall -Werror

    if you tried to compile c/blake3_dispatch.c with -Wall -Werror -DBLAKE3_NO_SSE2 -DBLAKE3_NO_SSE41 -DBLAKE3_NO_AVX2 -DBLAKE3_NO_AVX512

    something like this would happen:

    [email protected]:~/projects/BLAKE3/c$ gcc -O0 -o example example.c blake3.c blake3_dispatch.c blake3_portable.c     blake3_sse2_x86-64_unix.S blake3_sse41_x86-64_unix.S blake3_avx2_x86-64_unix.S blake3_avx512_x86-64_unix.S \ 
    -DBLAKE3_NO_SSE2 -DBLAKE3_NO_SSE41 -DBLAKE3_NO_AVX2 -DBLAKE3_NO_AVX512 \
    -Wall -Wextra -Wpedantic -Werror
    blake3_dispatch.c: In function ‘blake3_compress_in_place’:
    blake3_dispatch.c:139:26: error: unused variable ‘features’ [-Werror=unused-variable]
      139 |   const enum cpu_feature features = get_cpu_features();
          |                          ^~~~~~~~
    blake3_dispatch.c: In function ‘blake3_compress_xof’:
    blake3_dispatch.c:167:26: error: unused variable ‘features’ [-Werror=unused-variable]
      167 |   const enum cpu_feature features = get_cpu_features();
          |                          ^~~~~~~~
    blake3_dispatch.c: In function ‘blake3_hash_many’:
    blake3_dispatch.c:195:26: error: unused variable ‘features’ [-Werror=unused-variable]
      195 |   const enum cpu_feature features = get_cpu_features();
          |                          ^~~~~~~~
    blake3_dispatch.c: In function ‘blake3_simd_degree’:
    blake3_dispatch.c:244:26: error: unused variable ‘features’ [-Werror=unused-variable]
      244 |   const enum cpu_feature features = get_cpu_features();
          |                          ^~~~~~~~
    cc1: all warnings being treated as errors
    
    opened by divinity76 12
  • Implement BitXorAssign trait for Hash

    Implement BitXorAssign trait for Hash

    There is currently no way to XOR two blake3::Hash types without making extra allocations, because it is not possible to iterate mutably over a blake3::Hash.

    It is possible to work around it but I think it is a common enough use case for it to be implemented in the crate. I could take care of this if you believe it is worthwhile.

    opened by Eitu33 0
  • Support portable build without intrinsics

    Support portable build without intrinsics

    It is impossible to build currently if you want to have a pure-portable build of the C code. When we build rizin, we need to support TinyCC which does not have immintrin.h nor intrin.h. This patch removes some dup declarations and defines the default values for the missing intrinsics.

    opened by wargio 0
  • Implement digest::DynDigest

    Implement digest::DynDigest

    According to https://crates.io/crates/digest/reverse_dependencies this crate uses digest crate.

    Would it be possible to implement digest::DynDigest trait so that this hasher could be used in for example HashMaps which uses HashMap<&str, Box<dyn digest::DynDigest>> where you can iterate through multiple different hashers?

    opened by raspi 0
  • Add vcpkg installation instructions

    Add vcpkg installation instructions

    blake3 available as a port in vcpkg, a C++ library manager that simplifies installation for blake3 and other project dependencies. Documenting the install process here will help users get started by providing a single set of commands to build blake3, ready to be included in their projects.

    We also test whether our library ports build in various configurations (dynamic, static) on various platforms (OSX, Linux, Windows: x86, x64) to keep a wide coverage for users.

    I'm a maintainer for vcpkg, and here is what the port script looks like. We try to keep the library maintained as close as possible to the original library.

    opened by Cheney-W 0
  • Attempts to implement the blake3 hash algorithm using the standard 2020 programming language, c++, have consistently turned out to be incorrect.

    Attempts to implement the blake3 hash algorithm using the standard 2020 programming language, c++, have consistently turned out to be incorrect.

    I am a cryptography-loving c++ technician, and I have always wanted to implement a simple and portable future c++ version of the Blake3 hash algorithm. But I don't know much about the Rust programming language, and other people only want to use this code base, or just do the code adaptation of the programming language binding interface, but few have understood the Blake3 hash algorithm and contributed code to the open source community. I hope that the official or others can point out my mistakes in the implementation of the Blake3 hash algorithm code understanding, I will contribute my own small capacity, and thank everyone who helped me.

    If you are interested in my project please follow my Include code base for Applied Cryptography https://github.com/Twilight-Dream-Of-Magic/TDOM-EncryptOrDecryptFile-Reborn

    @oconnor663 Could you please explain the question I asked

    opened by Twilight-Dream-Of-Magic 6
Releases(1.3.1)
  • 1.3.1(Feb 14, 2022)

  • 1.3.0(Jan 8, 2022)

  • 1.2.0(Nov 5, 2021)

    version 1.2.0

    Changes since 1.1.0:

    • SECURITY FIX: Fixed an instance of undefined behavior in the Windows SSE2 assembly implementations, which affected both the Rust and C libraries in their default build configurations. See https://github.com/BLAKE3-team/BLAKE3/issues/206. The cause was a vector register that wasn't properly saved and restored. This bug has been present since SSE2 support was initially added in v0.3.7. The effects of this bug depend on surrounding code and compiler optimizations; see test_issue_206_windows_sse2 for an example of this bug causing incorrect hash output. Note that even when surrounding code is arranged to trigger this bug, the SSE2 implementation is normally only invoked on CPUs where SSE4.1 (introduced in 2007) isn't supported. One notable exception, however, is if the Rust library is built in no_std mode, with default_features = false or similar. In that case, runtime CPU feature detection is disabled, and since LLVM assumes that all x86-64 targets support SSE2, the SSE2 implementation will be invoked. For that reason, Rust callers who build blake3 in no_std mode for x86-64 Windows targets are the most likely to trigger this bug. We found this bug in internal testing, and we aren't aware of any callers encountering it in practice.
    • Added the Hasher::count() method.
    Source code(tar.gz)
    Source code(zip)
    b3sum_linux_x64_bin(4.58 MB)
    b3sum_macos_x64_bin(1.30 MB)
    b3sum_windows_x64_bin.exe(935.50 KB)
  • 1.1.0(Oct 21, 2021)

    version 1.1.0

    Changes since 1.0.0:

    • The NEON implementation is now enabled by default on AArch64 targets. Previously it was disabled without the "neon" Cargo feature in Rust or the "BLAKE3_USE_NEON=1" preprocessor flag in C. This is still the case on ARM targets other than AArch64, because of the lack of dynamic CPU feature detection on ARM. Contributed by @rsdy.
    • The previous change leads to some build incompatibilities, particularly in C. If you build the C implementation for AArch64 targets, you now need to include blake3_neon.c, or else you'll get a linker error like "undefined reference to `blake3_hash_many_neon'". If you don't want the NEON implementation, you need to explicitly set "BLAKE3_USE_NEON=0". On the Rust side, AArch64 targets now require the C toolchain by default. build.rs includes workarounds for missing or very old C compilers for x86, but it doesn't currently include such workarounds for AArch64. If we hear about build breaks related to this, we can add more workarounds as appropriate.
    • C-specific Git tags ("c-0.3.7" etc.) have been removed, and all the projects in this repo (Rust "blake3", Rust "b3sum", and the C implementation) will continue to be versioned in lockstep for the foreseeable future.
    Source code(tar.gz)
    Source code(zip)
    b3sum_linux_x64_bin(4.58 MB)
    b3sum_macos_x64_bin(1.30 MB)
    b3sum_windows_x64_bin.exe(935.50 KB)
  • 1.0.0(Jul 25, 2021)

    version 1.0.0

    Changes since 0.3.8:

    • Add Hash::from_hex() and implement FromStr for Hash.
    • Implement Display for Hash, equivalent to Hash::to_hex().
    • Implement PartialEq<[u8]> for Hash, using constant_time_eq.
    • Change derive_key() to return a 32-byte array. As with hash() and keyed_hash(), callers who want a non-default output length can use Hasher::finalize_xof().
    • Replace Hasher::update_with_join() with Hasher::update_rayon(). The former was excessively generic, and the Join trait leaked implementation details. As part of this change, the Join trait is no longer public.
    • Upgraded arrayvec to 0.7.0, which uses const generics. This bumps the minimum supported Rust compiler version to 1.51.
    • Gate the digest and crypto-mac trait implementations behind an unstable feature, "traits-preview". As part of this change upgrade crypto-mac to 0.11.0.
    Source code(tar.gz)
    Source code(zip)
    b3sum_linux_x64_bin(4.61 MB)
    b3sum_macos_x64_bin(1.29 MB)
    b3sum_windows_x64_bin.exe(937.00 KB)
  • 0.3.8(May 25, 2021)

  • 0.3.7(Oct 1, 2020)

    version 0.3.7

    Changes since 0.3.6:

    • BUGFIX: The C implementation was incorrect on big endian systems for inputs longer than 1024 bytes. This bug affected all previous versions of the C implementation. Little endian platforms like x86 were unaffected. The Rust implementation was also unaffected. @jakub-zwolakowski and @pascal-cuoq from TrustInSoft reported this bug: https://github.com/BLAKE3-team/BLAKE3/pull/118
    • BUGFIX: The C build on x86-64 was producing binaries with an executable stack. @tristanheaven reported this bug: https://github.com/BLAKE3-team/BLAKE3/issues/109
    • @mkrupcale added optimized implementations for SSE2. This improves performance on older x86 processors that don't support SSE4.1.
    • The C implementation now exposes the blake3_hasher_init_derive_key_raw function, to make it easier to implement language bindings. Added by @k0001.
    Source code(tar.gz)
    Source code(zip)
    b3sum_linux_x64_bin(4.18 MB)
    b3sum_macos_x64_bin(1.16 MB)
    b3sum_windows_x64_bin.exe(984.00 KB)
  • 0.3.6(Jul 29, 2020)

  • 0.3.5(Jul 10, 2020)

  • 0.3.4(May 23, 2020)

    version 0.3.4

    Changes since 0.3.3:

    • b3sum now supports the --check flag. This is intended to be a drop-in replacement for e.g. md5sum --check from Coreutils. The behavior is somewhat stricter than Coreutils with respect to invalid Unicode in filenames. For a complete description of how --check works, see the file b3sum/what_does_check_do.md.
    • To support the --check feature, backslashes and newlines that appear in filenames are now escaped in the output of b3sum. This is done the same way as in Coreutils.
    • To support --check interoperability between Unix and Windows, backslashes in filepaths on Windows are now replaced with forward slashes in the output of b3sum. Note that this is different from Coreutils.
    Source code(tar.gz)
    Source code(zip)
    b3sum_linux_x64_bin(3.87 MB)
    b3sum_macos_x64_bin(1.12 MB)
    b3sum_windows_x64_bin.exe(950.50 KB)
  • 0.3.3(Apr 28, 2020)

Owner
BLAKE3 team
The BLAKE3 cryptographic hash function
BLAKE3 team
Fastmurmur3 - Fast non-cryptographic hash, with the benchmarks to prove it.

Fastmurmur3 Murmur3 is a fast, non-cryptographic hash function. fastmurmur3 is, in my testing, the fastest implementation of Murmur3. Usage let bytes:

Kurt Wolf 14 Sep 22, 2022
Pure-Rust traits and utilities for constant-time cryptographic implementations.

subtle Pure-Rust traits and utilities for constant-time cryptographic implementations. It consists of a Choice type, and a collection of traits using

dalek cryptography 189 Sep 21, 2022
std::simd implementation of BLAKE3

BLAKE3-STD the first blake3 implementation on std::simd ONLY COMPILES WITH NIGHTLY [dependencies] blake3-std = "0.0.1" OFFICIAL DOC BLAKE3 is a crypto

LemonHX 15 Jun 6, 2022
Bessie - an authenticated, chunked cipher based on BLAKE3

Bessie Bessie is an authenticated, chunked cipher based on BLAKE3. Right now it's in the early design stages. See design.md. Although the Bessie ciphe

Jack O'Connor 12 Sep 6, 2022
A rust binding for nodejs to generate md5 hash value

Hasher A rust binding for creating node module to generate md5 hash value This project was bootstrapped by create-neon. Installing hasher Installing h

Md. Al-Amin 0 Nov 7, 2021
Highly modular & configurable hash & crypto library

Octavo Highly modular & configurable hash & crypto library written in pure Rust. Installation [dependencies] octavo = { git = "https://github.com/libO

Octavo Developers 137 Aug 27, 2022
Reference implementation for the Poseidon Snark-friendly Hash algorithm.

Dusk-Poseidon Reference implementation for the Poseidon Hashing algorithm. Reference Starkad and Poseidon: New Hash Functions for Zero Knowledge Proof

Dusk Network 81 Sep 9, 2022
paq files to hash.

paq paq files to hash. Hash a single file or all files in directory recursively. Installation Requires cargo. Run cargo install paq. Usage Run paq [sr

gregory langlais 3 Jul 17, 2022
An implementation of Jakobsson's Fractal Hash Sequence Traversal algorithm

fractal-hash-traversal An implementation of Jakobsson's Fractal Hash Sequence Traversal algorithm. There is at least one hash traversal algorithm that

Dan Cline 1 Jan 12, 2022
computed data's hash by webAssembly

wasm-hasher computed data's hash by webAssembly support md5,sha1,sha2-224,sha2-356,sha2-384,sha2-512,sha3-224,sha3-256,sha3-384,sha3-512,china-sm3 typ

fuyoo 2 Jul 27, 2022
Generates a unique hash/identifier for a system given a set of parameters.

uniqueid ?? Generates a unique hash/identifier for a system given a set of parameters. Example usage use uniqueid; pub fn main() { let data = vec

Checksum 2 Aug 19, 2022
Left To My Own Devices - NT hash tools

ntcrack Left To My Own Devices - NT cracker A full writeup of how it works is available at the SensePost blog Invocation ./ntcrack <input hashlist> <w

SensePost 18 Aug 4, 2022
Easy to use cryptographic framework for data protection: secure messaging with forward secrecy and secure data storage. Has unified APIs across 14 platforms.

Themis provides strong, usable cryptography for busy people General purpose cryptographic library for storage and messaging for iOS (Swift, Obj-C), An

Cossack Labs 1.6k Sep 21, 2022
Dexios-Core is a library used for managing cryptographic functions and headers that adhere to the Dexios format.

What is it? Dexios-Core is a library used for managing cryptographic functions and headers that adhere to the Dexios format. Security Dexios-Core uses

brxken 3 Jul 4, 2022
Pure Rust implementations of the key-committing (and context-committing) AEADs

kc-aeads Pure Rust implementations of the key-committing (and context-committing) AEADs defined in Bellare and Hoang '22. Crash course on the paper: T

Michael Rosenberg 2 Aug 10, 2022
Making composability with the Zeta DEX a breeze, FuZe provides CPI interfaces and sample implementations for on-chain program integration.

Zeta FuZe ?? Zeta FuZe FuZe is Zeta's cross-program integration ecosystem. This repository contains the Zeta Cross Program Invocation (CPI) interface

Zeta 39 Aug 27, 2022
A (mostly) pure-Rust implementation of various cryptographic algorithms.

Rust-Crypto A (mostly) pure-Rust implementation of various common cryptographic algorithms. Rust-Crypto seeks to create practical, auditable, pure-Rus

null 1.2k Sep 22, 2022
Sodium Oxide: Fast cryptographic library for Rust (bindings to libsodium)

sodiumoxide |Crate|Documentation|Gitter| |:---:|:-----------:|:--------:|:-----:|:------:|:----:| |||| NaCl (pronounced "salt") is a new easy-to-use h

sodiumoxide 638 Sep 3, 2022
Secure storage for cryptographic secrets in Rust

secrets secrets is a library to help Rust programmers safely held cryptographic secrets in memory. It is mostly an ergonomic wrapper around the memory

Stephen Touset 151 Sep 2, 2022