Dependency lockfiles for a reproducible build environment 📦🔒

Overview

repro-env

Imagine you had a tool that takes a config like this:

# repro-env.toml
[container]
image = "rust:1-alpine3.18"

and turns it into something like this:

# repro-env.lock
[container]
image = "rust@sha256:22760a18d52be83a74f5df8b190b8e9baa1e6ce7d9bda40630acc8ba5328a2fd"

You commit both into your git repository to document:

  • repro-env.toml: which container image tag you intend to follow (think Cargo.toml)
  • repro-env.lock: which specific image you use for your release build (think Cargo.lock)

The .lock file is auto-generated and can be refreshed with a simple command:

repro-env update

The build is executed in a user-namespace with podman (make sure it's installed), the current directory is mounted to /build/ and a given command is executed inside of that directory:

repro-env build -- cargo build

We want to distribute our binary without having to worry about system libraries, so we ask cargo to create static binaries (also enable release optimizations):

repro-env build -- cargo build --release --target x86_64-unknown-linux-musl

This way we also ensure a different build folder is used (target/x86_64-unknown-linux-musl instead of target/) so our normal development doesn't interfere.

The final executable is available at this location:

./target/x86_64-unknown-linux-musl/release/repro-env --help

Download

With github actions:

- name: Install repro-env
  run: |
    wget 'https://github.com/kpcyrd/repro-env/releases/download/v0.2.0/repro-env'
    echo 'b78727a7b2f635504ab780bfa5dd7e7bbcfb9255607eaf815762e0893aad5663  repro-env' | sha256sum -c -
    sudo install -m755 repro-env -t /usr/bin

Packages: Arch Linux

Arch Linux hosts a comprehensive collection of recent compilers at https://archive.archlinux.org. You can create a [packages] section in your repro-env.toml with system = "archlinux" to install additional packages with pacman.

# repro-env.toml
[container]
image = "docker.io/library/archlinux"

[packages]
system = "archlinux"
dependencies = ["rust-musl", "lua"]

The resolved repro-env.lock is going to contain the sha256 of the resolved container image you use as a base, and a list of [[package]] that should be installed/upgraded inside of the container before starting the build.

# repro-env.lock
[container]
image = "docker.io/library/archlinux@sha256:6568d3f1f278827a4a7d8537f80c2ae36982829a0c6bccff4cec081774025472"

# [...]

[[package]]
name = "rust"
version = "1:1.69.0-3"
system = "archlinux"
url = "https://archive.archlinux.org/packages/r/rust/rust-1%3A1.69.0-3-x86_64.pkg.tar.zst"
sha256 = "b8eb31a2eb80efab27bb68beab80436ed3e1d235a217c3e24ba973936c95839e"
signature = "iIsEABYIADMWIQQGaHodnU+rCLUP2Ss7lKgOUKR3xwUCZExVKBUcaGVmdGlnQGFyY2hsaW51eC5vcmcACgkQO5SoDlCkd8fQkAD6AudRi2qP3WxSn38OOkSRSITciqRevPaVJgrz03JUBEAA/12h9z8dReD07Lqnltx9QTa3Cxppbv7VpJlTCQuavoMG"

[[package]]
name = "rust-musl"
version = "1:1.69.0-3"
system = "archlinux"
url = "https://archive.archlinux.org/packages/r/rust-musl/rust-musl-1%3A1.69.0-3-x86_64.pkg.tar.zst"
sha256 = "5a4854cdac8312dbf72fb87795bcc36bfb34e9218944966e5ac2e62319bbcf22"
signature = "iIsEABYIADMWIQQGaHodnU+rCLUP2Ss7lKgOUKR3xwUCZExVKRUcaGVmdGlnQGFyY2hsaW51eC5vcmcACgkQO5SoDlCkd8cCMQD/W59RkOVPZDXlnmyY27jW61GC86hXOkSLOKa7XMQtpBoBALSugCkG1clSo/EQDbnuS+UY3268HNBvz6mF6i/hhEsB"

Packages: Debian

Debian is a widely accepted choice and hosts an archive of all their packages at https://snapshot.debian.org/. You can create a [packages] section in your repro-env.toml with system = "debian" to install additional packages with apt-get.

# repro-env.toml
[container]
image = "debian:bookworm"

[packages]
system = "debian"
dependencies = ["gcc", "libc6-dev"]

Note this only works with official debian packages (not ubuntu).

The resolved repro-env.lock is going to contain the sha256 of the resolved container image you use as a base, and a list of [[package]] that should be installed/upgraded inside of the container before starting the build.

# repro-env.lock
[container]
image = "debian@sha256:3d868b5eb908155f3784317b3dda2941df87bbbbaa4608f84881de66d9bb297b"

[[package]]
name = "binutils"
version = "2.40-2"
system = "debian"
url = "https://snapshot.debian.org/archive/debian/20230115T211934Z/pool/main/b/binutils/binutils_2.40-2_amd64.deb"
sha256 = "83c3e20b53e1fbd84d764c3ba27d26a0376e361ae5d7fb37120196934dd87424"

[[package]]
name = "binutils-common"
version = "2.40-2"
system = "debian"
url = "https://snapshot.debian.org/archive/debian/20230115T211934Z/pool/main/b/binutils/binutils-common_2.40-2_amd64.deb"
sha256 = "ab314134f43a0891a48f69a9bc33d825da748fa5e0ba2bebb7a5c491b026f1a0"

# [...]

Bootstrapping

There are no inherent bootstrapping challenges, you can use any recent Rust compiler to build a working repro-env binary. This binary can then setup any other build environment (including it's own) and is able to build a bit-for-bit identical copy of the official release binaries hosted on github.

Reproducible Builds

All pre-compiled binaries can be reproduced from source code:

% wget https://github.com/kpcyrd/repro-env/releases/download/v0.1.0/repro-env
[...]
% sha256sum repro-env
5b7e043dea9c2a0afc0180be9263dd5c5b7e69c649749b43c132885e4eca623f  repro-env

Since the build environment is fully documented and tracked in git all we need is checkout the corresponding git tag and run make:

% git clone https://github.com/kpcyrd/repro-env
% cd repro-env
% git checkout v0.1.0
% make
% sha256sum target/x86_64-unknown-linux-musl/release/repro-env
5b7e043dea9c2a0afc0180be9263dd5c5b7e69c649749b43c132885e4eca623f  target/x86_64-unknown-linux-musl/release/repro-env

License

GPL-3.0-or-later

You might also like...
Dione is an anonymize and encrypted messaging system build on top on a peer to peer layer.

Secure and Anonymous Messaging WARNING: Currently Dione is not ready to be used nor does it fulfill its goal of being an anonymous messenger. In order

Example crate using fltk-build

white-frame This is just an example showing the use of fltk-build to create native C/C++ FLTK and cfltk modules for fltk-rs. This repo contains 2 exam

Scrypto Advent Calendar. Learn the new programming langage to build quick and secure DeFi applications.

Scrypto Advent Calendar I am publishing new Christmas related Scrypto examples every day from Dec 1st to Dec 25th. "Watch" this project to get notifie

This is a template to build secret contracts in Rust to run in Secret Network

Secret Contracts Starter Pack This is a template to build secret contracts in Rust to run in Secret Network. To understand the framework better, pleas

Diem’s mission is to build a trusted and innovative financial network that empowers people and businesses around the world.
Diem’s mission is to build a trusted and innovative financial network that empowers people and businesses around the world.

Note to readers: On December 1, 2020, the Libra Association was renamed to Diem Association. The project repos are in the process of being migrated. A

A standalone Aleo prover build upon snarkOS and snarkVM, with multi-threading optimization

Aleo Light Prover Introduction A standalone Aleo prover build upon snarkOS and snarkVM, with multi-threading optimization. It's called "light" because

A template to build smart contracts in Rust to run inside a Cosmos SDK module on all chains that enable it.

CosmWasm Starter Pack This is a template to build smart contracts in Rust to run inside a Cosmos SDK module on all chains that enable it. To understan

Rust library for build smart contracts on Internet Computer, by the Spinner.Cash team.

Spinner Rust library for building smart contracts on the Internet Computer. More specifically it is used by Spinner.Cash, a decentralized layer-2 prot

Bootstrap your MEV bot strategies with a simple boilerplate to build on top of.

MEV Template Designed by DeGatchi. Bootstrap your MEV bot strategies with a simple boilerplate to build on top of. How To Use This Template I wrote an

Releases(v0.2.0)
Owner
I'm good at proving my binaries haven't been tampered with. I volunteer as an Arch Linux package maintainer. I like to do things without computers.
null
Authenticate a tarball through a signed tag in a git repository (with reproducible builds)

auth-tarball-from-git Authenticate a tarball through a signed tag in a git repository (with reproducible builds). The signed git tag contains a hash o

null 14 Aug 16, 2022
A re-write of polkadot staking miner using subxt to avoid hard dependency to each runtime version

Staking Miner v2 WARNING this library is under active development DO NOT USE IN PRODUCTION. The library is a re-write of polkadot staking miner using

Parity Technologies 19 Dec 28, 2022
RSA dependency for rust, with cert verification

About Project End to End encryption (RSA) for multiple languages (cross-platform) with double encryption and double decryption methods Icon Item ?? Up

Zot Cryptography 2 Nov 27, 2022
This is my home environment setup for monitoring temperature and humidity

Home EnViroNment Motivation This is my IoT temperature and humidity monitoring solution for where i live. I found it cheaper to go buy sensors and hoo

Fredrik 1 Jan 5, 2022
Compiler development environment.

compiler-dev 北大编译实践教学用编译器开发环境 (Compiler Development Environment). 该仓库的内容将被打包为 Docker 镜像, 所以不建议直接使用该仓库, 具体使用方法见使用方法一节. 使用方法 WIP. 镜像中包含的内容 必要的工具: git, f

PKU Compiler Course 2 Feb 26, 2022
A fast zero-knowledge proof friendly Move language runtime environment.

zkMove Lite zkMove Lite is a lightweight zero-knowledge proof friendly Move language virtual machine. Move bytecode is automatically "compiled" into c

YoungRocks 43 May 20, 2023
Diem’s mission is to build a trusted and innovative financial network that empowers people and businesses around the world.

Note to readers: On December 1, 2020, the Libra Association was renamed to Diem Association. The project repos are in the process of being migrated. A

Diem 16.7k Jan 8, 2023
Examples and helpers to build NFT contracts on CosmWasm

CosmWasm NFTS This repo is the official repository to work on all NFT standard and examples in the CosmWasm ecosystem. cw721 and cw721-base were moved

CosmWasm 147 Jan 4, 2023
A lightweight and flexible framework to build your tailored blockchain applications.

TRINCI Blockchain Core A lightweight and flexible framework to build your tailored blockchain applications. Requirements The required dependencies to

Affidaty S.p.A. 11 Sep 26, 2022
Dfinity's fungible token standard. Any PRs and comments are welcome,collaborate with us to build this standard

Dfinity's fungible token standard. Any PRs and comments are welcome,collaborate with us to build this standard

Deland Labs 46 Nov 7, 2022