Make production Rust binaries auditable

Overview

rust-audit

Know the exact crate versions used to build your Rust executable. Audit binaries for known bugs or security vulnerabilities in production, at scale, with zero bookkeeping.

This works by embedding data about the dependency tree in JSON format into a dedicated linker section of the compiled executable.

The implementation has gotten to the point where it's time to get some real-world experience with it, but the data format is not yet stable. Linux, Windows and Mac OS are currently supported.

The end goal is to get Cargo itself to encode this information in binaries instead of relying on an external crate. RFC for a proper implementation in Cargo, for which this project paves the way: https://github.com/rust-lang/rfcs/pull/2801

Demo

Clone this repository:

git clone https://github.com/Shnatsel/rust-audit.git
cd rust-audit

Compile the tooling and a sample binary with dependency tree embedded:

cargo build --release

Recover the dependency tree we've just embedded.

target/release/rust-audit-info target/release/hello-auditable

You can also audit the recovered dependency tree for known vulnerabilities using cargo audit:

(cd auditable-serde && cargo build --release --features "toml" --example json-to-toml)
cargo install cargo-audit
target/release/rust-audit-info target/release/hello-auditable > dependency-tree.json
target/release/examples/json-to-toml dependency-tree.json | cargo audit -f -

The auditable-extract crate allows your own tools to easily consume this info.

How to make your crate auditable

Add the following to your Cargo.toml:

build = "build.rs"

[dependencies]
auditable = "0.1"

[build-dependencies]
auditable-build = "0.1"

Create a build.rs file next to Cargo.toml with the following contents:

fn main() {
    auditable_build::collect_dependency_list();
}

Add the following to the beginning your main.rs (or any other file):

static COMPRESSED_DEPENDENCY_LIST: &[u8] = auditable::inject_dependency_list!();

Put the following in some reachable location in the code, e.g. in fn main():

    // Actually use the data to work around a bug in rustc:
    // https://github.com/rust-lang/rust/issues/47384
    // On nightly you can use `test::black_box` instead of `println!`
    println!("{}", COMPRESSED_DEPENDENCY_LIST[0]);

Now you can cargo build and the dependency data will be embedded in the final binary automatically. You can verify that the data is actually embedded using the extraction steps from the demo.

See the auditable "Hello, world!" project for an example of how it all fits together.

FAQ

Doesn't this bloat my binary?

Not really. A "Hello World" on x86 Linux compiles into a ~1Mb file in the best case (recent Rust without jemalloc, LTO enabled). Its dependency tree even with a couple of dependencies is < 1Kb, that's under 1/1000 of the size. We also compress it with zlib to drive the size down further. Since the size of dependency tree info grows linearly with the number of dependencies, it will keep being negligible.

What about embedded platforms?

Embedded platforms where you cannot spare a byte should not add anything in the executable. Instead they should record the hash of every executable in a database and associate the hash with its Cargo.lock, compiler and LLVM version, build date, etc. This would make for an excellent Cargo wrapper or plugin. Since that can be done in a 5-line shell script, writing that tool is left as an exercise to the reader.

Does this impact reproducible builds?

The data format is designed not to disrupt reproducible builds. It contains no timestamps, and the generated JSON is sorted to make sure it is identical between compilations. If anything, this helps with reproducible builds, since you know all the versions for a given binary now.

Is there any tooling to consume this data?

It is interoperable with existing tooling that consumes Cargo.lock via the JSON-to-TOML convertor. You can also write your own tooling fairly easily - auditable-extract and auditable-serde crates handle all the data extraction and parsing for you. See the docs to get started.

What is the data format, exactly?

It is not yet stabilized, so we do not have extensive docs or a JSON schema. However, these Rust data structures map to JSON one-to-one and are extensively commented. The JSON is Zlib-compressed and placed in a linker section with a name that varies by platform.

Can I read this data using a tool written in a different language?

Yes. The data format is designed for interoperability with alternative implementations. You can also use pre-existing platform-specific tools or libraries for data extraction. E.g. on Linux:

objcopy -O binary --only-section=.rust-deps-v0 target/release/hello-auditable /dev/stdout | pigz -zd -

However, don't run legacy tools on untrusted files. Use the auditable-extract crate or the rust-audit-info command-line tool if possible - they are written in 100% safe Rust, so they will not have such vulnerabilities.

Does this disclose any sensitive information?

TL;DR: The list of enabled features is the only newly disclosed information.

All URLs and file paths are redacted, but the crate names, feature names and versions are recorded as-is. At present panic messages already disclose all this info and more, except feature names. Also, chances are that you're legally obligated have to disclose use of specific open-source crates anyway, since MIT and many other licenses require it.

What about recording the compiler version?

It's already there. Run strings your_executable | grep 'rustc version' to see it. Don't try this on files you didn't compile yourself - strings is overdue for a rewrite in safe Rust.

In theory we could duplicate it in the JSON for ease of access, but this can be added later in a backwards-compatible fashion.

What about keeping track of versions of statically linked C libraries?

Good question. I don't think they are exposed in any reasonable way right now. Would be a great addition, but not required for the initial launch. We can add it later in a backwards-compatible way later.

What is blocking uplifting this into Cargo?

  1. Getting some real-world experience with this before committing to a stable data format
  2. https://github.com/rust-lang/rust/issues/47384

Help on these points would be greatly appreciated.

Comments
  • to_toml test in auditable-serde writes outside the build directory

    to_toml test in auditable-serde writes outside the build directory

    Hi

    I'm investigating if this could simplify the tracking of relationship between rust binaries and vulnerable versions of rust libraries in Debian, and as part of that investigation I started by seeing how easy it would be to package this with the Debian tooling.

    I noticed that the to_toml test in auditable-serde fails due to that it tries to write a lock file into Debians crate registry directory and became a bit afraid that the project might be dead in the water due to that it needs to write those lock files during normal operations also, and not just for that unit test.

    I realize that you don't control the cargo_metadata crate, but I thought that it might be worth asking here if this is a use case for auditable that you are interested in supporting?

    Complete test output below for context:

    ---- tests::to_toml stdout ----
    thread 'tests::to_toml' panicked at 'called `Result::unwrap()` on an `Err` value: CargoMetadata { stderr: "error: failed to write /usr/share/cargo/registry/auditable-serde-0.5.2/Cargo.lock\n\nCaused by:\n  failed to open: /usr/share/cargo/registry/auditable-serde-0.5.2/Cargo.lock\n\nCaused by:\n  Permission denied (os error 13)\n" }', src/lib.rs:506:20
    stack backtrace:
       0: rust_begin_unwind
                 at /usr/src/rustc-1.62.1/library/std/src/panicking.rs:584:5
       1: core::panicking::panic_fmt
                 at /usr/src/rustc-1.62.1/library/core/src/panicking.rs:142:14
       2: core::result::unwrap_failed
                 at /usr/src/rustc-1.62.1/library/core/src/result.rs:1785:5
       3: core::result::Result<T,E>::unwrap
                 at /usr/src/rustc-1.62.1/library/core/src/result.rs:1078:23
       4: auditable_serde::tests::load_own_metadata
                 at ./src/lib.rs:506:9
       5: auditable_serde::tests::to_toml
                 at ./src/lib.rs:513:24
       6: auditable_serde::tests::to_toml::{{closure}}
                 at ./src/lib.rs:512:5
       7: core::ops::function::FnOnce::call_once
                 at /usr/src/rustc-1.62.1/library/core/src/ops/function.rs:248:5
       8: core::ops::function::FnOnce::call_once
                 at /usr/src/rustc-1.62.1/library/core/src/ops/function.rs:248:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    
    third party 
    opened by alexanderkjall 13
  • Determine the root package

    Determine the root package

    The current format doesn't have information to determine direct or transitive dependence. In the following example, we can know ansi_term depends on bitflags, but the project may also directly depend on bitflags.

    {
      "packages": [
        {
          "name": "ansi_term",
          "version": "0.12.1",
          "source": "crates.io",
          "dependencies": [
            1
          ]
        },
        {
          "name": "bitflags",
          "version": "1.2.1",
          "source": "crates.io",
          "features": [
            "default"
          ]
        }
      }
    }
    

    For example, package-lock.json has "" so that we can know direct dependencies. https://github.com/firebase/functions-samples/blob/3515b7f38a3c598cdb20152a263372e81719ecda/package-lock.json#L7-L17

    opened by knqyf263 12
  • Provide git tags for binary crate releases

    Provide git tags for binary crate releases

    I'm packaging cargo-auditable and rust-audit-info for Void Linux, and for that it'd be very helpful to have git tags available for at least the binary crates. As far as I can tell, the tags already present are for cargo-auditable itself, so this is mostly about providing tags for rust-audit-info as well. An approach that works well for multiple projects with distinct versioning is using some tag prefix containing the crate name, like this for example: https://gitlab.com/openpgp-card/openpgp-card/-/issues/32#note_958397205

    opened by jcgruenhage 10
  • Signed binaries

    Signed binaries

    Would it be feasible to extend cargo-auditable to signed binaries ?

    I would love to sign all the information and then use auditable also to check that the auditable information was signed correctly.

    https://rust-lang.zulipchat.com/#narrow/stream/146229-wg-secure-code/topic/Signed.20binaries/near/293060631

    opened by pinkforest 10
  • Update dependencies and README

    Update dependencies and README

    This PR upgrades dependencies to the latest versions, and adds a mention to the README about go-rustaudit, and syft which will have experimental support for Rust-audit binaries (in 0.53.0, due imminently).

    opened by tofay 10
  • Externally inject auditable data

    Externally inject auditable data

    I was recently thinking about what it would take to integrate something like this into a cargo install process. The biggest issue I see is that it requires modifying the binary sources to have the data added. I think a potentially more useful approach is a way to inject this data into an arbitrary binary build; maybe via something like a cargo wrapper cargo auditable build.

    This would also avoid issues #9, #11 and #13 (but probably introduce others 😀).

    Is there a reason to prefer the current approach where each binary needs to be configured to include the data?

    opened by Nemo157 10
  • Provide a documented way to use `cargo auditable` as a drop-in replacement for `cargo`

    Provide a documented way to use `cargo auditable` as a drop-in replacement for `cargo`

    This came up in the pull request about enabling cargo auditable in Nix by defaut.

    There are two hurdles here:

    1. cargo auditable calls cargo internally quite a lot. We need to make sure it doesn't recurse.
    2. Some targets are currently not supported, notably WebAssembly, and cargo auditable will error out in this case. This is not an option for a drop-in replacement for cargo - we need to print a warning and keep going.
    opened by Shnatsel 9
  • High level wrapper for parsing binaries

    High level wrapper for parsing binaries

    Performs the parsing in a single function call instead of requiring assembling the pieces yourself and also doing your own error handling.

    Resolves #50

    opened by Shnatsel 8
  • Attestation

    Attestation

    Would it be feasible to support attestation instead of embedding all auditable data ?

    So instead of embedding the whole thing into the binary it could simply include URL and / or a hash of some sort

    Then that URL / hash can be checked via attestation optionally.

    Not sure if worthwhile but it might be good for wasm binaries at least ?

    opened by pinkforest 7
  • Fails when a rustc wrapper like sccache is set

    Fails when a rustc wrapper like sccache is set

    Having a rustc wrapper defined (like build caching with RUSTC_WRAPPER=sccache) makes build fail:

    $ export RUSTC_WRAPPER=sccache
    $ cargo auditable build --release
    error: failed to run `rustc` to learn about target-specific information
    
    Caused by:
      process didn't exit successfully: `/home/amousset/.cargo/bin/sccache /home/amousset/.cargo/bin/cargo-auditable rustc - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit status: 2)
      --- stderr
      sccache: error: failed to execute compile
      sccache: caused by: Compiler not supported: "Unrecognized command: \"-E\"\n"
    
    bug third party 
    opened by amousset 7
  • Dependency data not embedded unless `inject_dependency_list!()` return value is used

    Dependency data not embedded unless `inject_dependency_list!()` return value is used

    Dependency information will not be present in the final binary, unless the data returned by auditable::inject_dependency_list!() is actually used somewhere in the binary (printed, put into test::black_box, etc).

    This is actually a bug in rustc: https://github.com/rust-lang/rust/issues/47384

    bug third party 
    opened by Shnatsel 7
  • Auditability / Cheating

    Auditability / Cheating

    Just a small thought after seeing this project for the first time: The name of this crate implies that the embedded information allows ensuring that no vulnerable dependencies were used. However, if I'm not mistaken, it would be very simple to overwrite this section of the binary, and simply lie about the actual dependencies used, right? (For example, to change versions, or to hide dependencies with "unsuitable licenses".) So while this section makes it easier to find vulnerabilities in your own binaries, it does not make your binary that much more auditable towards third parties that don't trust you.

    Maybe a FAQ section in the README ("Can I be sure that the dependency information embedded in the binary is correct?") would make sense. Or maybe "cargo auditable" isn't the best name? (But you've probably thought more about this than I have. Related to #1.)

    opened by dbrgn 4
  • Incompatibility with sccache on long builds

    Incompatibility with sccache on long builds

    When using cargo auditable with sccache, cargo auditable sometimes panics. We're seeing that panic on long (5> mins) builds:

    sccache: error: failed to execute compile
    sccache: caused by: Compiler not supported: "thread \'main\' panicked at \'called `Result::unwrap()` on an `Err` value: MissingOption(Keys([\"--crate-name\", \"\"]))\', cargo-auditable/src/rustc_wrapper.rs:13:50\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\n"
    

    Running under strace shows that sccache is running /path/to/cargo-auditable rustc -vV when compiling a workspace member. Normally sccache runs this when compiling a dependency, and caches the information. On larger builds that cache expires causing sccache to run that command when compiling a workspace member, causing the panic

    The panic can trivially by reproduced with

    CARGO_PRIMARY_PACKAGE=foo ~/.cargo/bin/cargo-auditable rustc -vV
    thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: MissingOption(Keys(["--crate-name", ""]))', cargo-auditable/src/rustc_wrapper.rs:13:50
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    

    or by creating a new project without any dependencies and building with sccache and cargo auditable:

    tom [ ~ ]$ cargo init new
         Created binary (application) package
    tom [ ~ ]$ cd new
    tom [ ~/new ]$ RUSTC_WRAPPER=/home/tom/.cargo/bin/sccache cargo auditable build
       Compiling new v0.1.0 (/home/tom/new)
    sccache: error: failed to execute compile
    sccache: caused by: Compiler not supported: "thread \'main\' panicked at \'called `Result::unwrap()` on an `Err` value: MissingOption(Keys([\"--crate-name\", \"\"]))\', cargo-auditable/src/rustc_wrapper.rs:13:50\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\n"
    error: could not compile `new`
    
    opened by tofay 5
  • "Publish auditable binary" Github Action

    As suggested by @pinkforest, it would be great to have a "publish auditable binary" Github action that builds the binaries with cargo auditable.

    Ideally people should be able to use as a drop-in replacement for an existing Publish action.

    enhancement help wanted 
    opened by Shnatsel 1
  • WebAssembly support

    WebAssembly support

    It is technically possible to support WebAssembly, since they do allow custom sections: https://webassembly.github.io/spec/core/appendix/custom.html

    This may be useful since the overhead of the audit info is just a few kilobytes and WebAssembly is being applied not just for the web.

    enhancement help wanted 
    opened by Shnatsel 0
  • Cargo Resolver V2 (different feature sets for build and runtime dependencies) is not supported

    Cargo Resolver V2 (different feature sets for build and runtime dependencies) is not supported

    Cargo has made it possible to depend on the same version of a given crate with different feature sets, provided that one version is a runtime dependency and another is a build dependency.

    The dependency resolution in rust-audit was written prior to that change, and it's possible that auditable-serde collates these two packages.

    The deduplication is done on the package ID from cargo-metadata, and we'll need to double-check that this is in fact correct even in the presence of the new Cargo feature resolver:

    https://github.com/Shnatsel/rust-audit/blob/d7fa6fff1861799adab41638267e0457b7ba4698/auditable-serde/src/lib.rs#L219

    bug third party 
    opened by Shnatsel 1
Owner
Sergey "Shnatsel" Davidoff
Feel free to contact me about Rust jobs.
Sergey
Template for jumpstarting production-ready Farcaster frames quickly

Rust Farcaster Frames Template Template for jumpstarting production-ready Farcaster frames quickly. — no fuss, all fun! Features Seamless Integration:

jpgonzalezra 13 Feb 21, 2024
Using the powers of Rust, Go and Dragonfly to make a vanilla-like world generation.

df-rs-gen — Dragonfly Rust Generator Using the powers of Rust, Go and Dragonfly to make a vanilla-like world generation. How to use Clone the repo. gi

Sculas 1 Apr 13, 2022
a simple program to make i3 open windows in alternating orientations

i3-alternating 1 use i3ipc::{reply::NodeLayout, I3Connection, I3EventListener, Subscription, event::{Event, WindowEventInfo}, event::inner::Windo

null 5 Apr 26, 2022
Extreme Bevy is what you end up with by following my tutorial series on how to make a low-latency p2p web game.

Extreme Bevy Extreme Bevy is what you end up with by following my tutorial series on how to make a low-latency p2p web game. There game can be played

Johan Klokkhammer Helsing 39 Jan 5, 2023
Rust-raytracer - 🔭 A simple ray tracer in Rust 🦀

rust-raytracer An implementation of a very simple raytracer based on Ray Tracing in One Weekend by Peter Shirley in Rust. I used this project to learn

David Singleton 159 Nov 28, 2022
Rust-and-opengl-lessons - Collection of example code for learning OpenGL in Rust

rust-and-opengl-lessons Project requires Rust 1.31 Collection of example code for learning OpenGL in Rust 00 - Setup 01 - Window 02 - OpenGL Context 0

Nerijus Arlauskas 348 Dec 11, 2022
Simple retro game made using Rust bracket-lib by following "Herbert Wolverson's Hands on Rust" book.

Flappy Dragon Code from This program is a result of a tutorial i followed from Herbert Wolverson's Hands-on Rust Effective Learning through 2D Game De

Praneeth Chinthaka Ranasinghe 1 Feb 7, 2022
A rust chess implementation using a neural network scoring function built on huggingface/candle + rust + wasm

Rusty Chess What is it? Rusty Chess aims to be a high quality embeddable chess engine that runs entirely locally in the browser (no backend required).

Gareth 3 Nov 3, 2023
A Rust wrapper and bindings of Allegro 5 game programming library

RustAllegro A thin Rust wrapper of Allegro 5. Game loop example extern crate allegro; extern crate allegro_font; use allegro::*; use allegro_font::*;

null 80 Dec 31, 2022
High performance Rust ECS library

Legion aims to be a feature rich high performance Entity component system (ECS) library for Rust game projects with minimal boilerplate. Getting Start

Amethyst Engine 1.4k Jan 5, 2023
A refreshingly simple data-driven game engine built in Rust

What is Bevy? Bevy is a refreshingly simple data-driven game engine built in Rust. It is free and open-source forever! WARNING Bevy is still in the ve

Bevy Engine 21.1k Jan 4, 2023
Rust library to create a Good Game Easily

ggez What is this? ggez is a Rust library to create a Good Game Easily. The current version is 0.6.0-rc0. This is a RELEASE CANDIDATE version, which m

null 3.6k Jan 7, 2023
RTS game/engine in Rust and WebGPU

What is this? A real time strategy game/engine written with Rust and WebGPU. Eventually it will be able to run in a web browser thanks to WebGPU. This

Thomas SIMON 258 Dec 25, 2022
unrust - A pure rust based (webgl 2.0 / native) game engine

unrust A pure rust based (webgl 2.0 / native) game engine Current Version : 0.1.1 This project is under heavily development, all api are very unstable

null 368 Jan 3, 2023
Rust bindings for GDNative

GDNative bindings for Rust Rust bindings to the Godot game engine. Website | User Guide | API Documentation Stability The bindings cover most of the e

null 3.2k Jan 9, 2023
SDL bindings for Rust

Rust-SDL Bindings for SDL in Rust Overview Rust-SDL is a library for talking to SDL from Rust. Low-level C components are wrapped in Rust code to make

Brian Anderson 174 Nov 22, 2022
SDL2 bindings for Rust

Rust-SDL2 Bindings for SDL2 in Rust Changelog for 0.34.2 Overview Rust-SDL2 is a library for talking to the new SDL2.0 libraries from Rust. Low-level

null 2.2k Jan 5, 2023
SFML bindings for Rust

rust-sfml Rust bindings for SFML, the Simple and Fast Multimedia Library. Requirements Linux, Windows, or OS X Rust 1.42 or later SFML 2.5 CSFML 2.5 D

Jeremy Letang 567 Jan 7, 2023
Rust bindings for libtcod 1.6.3 (the Doryen library/roguelike toolkit)

Warning: Not Maintained This project is no longer actively developed or maintained. Please accept our apologies. Open pull requests may still get merg

Tomas Sedovic 226 Nov 17, 2022