Rust bindings for accessing the Go containers/image stack

Overview

Rust bindings for accessing the Go containers/image stack

This crate contains a Rust API that forks /usr/bin/skopeo and talks to it via a custom API. You can use it to fetch container images in a streaming fashion.

At the time of this writing, you will need a version of skopeo with https://github.com/containers/skopeo/pull/1476

Why?

First, assume one is operating on a codebase that isn't Go, but wants to interact with container images - we can't just include the Go containers/image library.

The primary intended use case of this is for things like ostree-containers where we're using container images to encapsulate host operating system updates, but we don't want to involve the containers/image storage layer.

What we do want from the containers/image library is support for things like signatures and offline mirroring. More on this below.

Forgetting things like ostree exist for a second - imagine that you wanted to encapsulate a set of Debian/RPM/etc packages inside a container image to ship for package-based operating systems. You could use this to stream out the layer containing those packages and extract them directly, rather than serializing everything to disk in the containers/storage disk location, only to copy it out again and delete the first.

Another theoretical use case could be something like krustlet, which fetches WebAssembly blobs inside containers. Here again, we don't want to involve containers/storage.

Desired containers/image features

There are e.g. Rust libraries like dkregistry-rs and oci-distribution and similar for other languages.

However, the containers/image Go library has a lot of additional infrastructure that will impose a maintenance burden to replicate:

  • Signatures (man containers-auth.json)
  • Mirroring/renaming (man containers-registries.conf)
  • Support for ~/.docker/config.json for authentication as well as /run

Status

API is subject to change.

Comments
  • Add certificate_authority to expose --cert-dir

    Add certificate_authority to expose --cert-dir

    Will help allow ostree pull images from internal registries Helps https://github.com/ostreedev/ostree-rs-ext/issues/121

    Also change authfile type from String to PathBuf

    opened by mkenigs 7
  • imageproxy: Add a method to get image configuration

    imageproxy: Add a method to get image configuration

    We need this for a few reasons:

    • It has the layer ordering, which we are currently ignoring but will probably be important for correctness later
    • It is the only thing that has extensible metadata that will survive round trips through docker schema 2
    opened by cgwalters 4
  • Add get_layer_info_json

    Add get_layer_info_json

    This API exposes the "GetLayerInfosForCopy" API in containers/image, which can be used to create a mapping between blob and diff IDs, allowing pulling via diffID.

    Signed-off-by: RishabhSaini [email protected]

    opened by RishabhSaini 2
  • Add support for passing auth creds via fd

    Add support for passing auth creds via fd

    Motivated by https://github.com/ostreedev/ostree-rs-ext/pull/413 which is in turn an adaption of code that lives in rpm-ostree today from https://github.com/coreos/rpm-ostree/commit/d661e8f974f8d2550a865c2866476160e333ec72 which aims to support privilege separation.

    This doesn't lower privilege separation into this project (yet), just aids in passing data across boundaries.

    opened by cgwalters 1
  • change public API to use oci-spec

    change public API to use oci-spec

    While we should offer a way to get the raw bytes, I think we should depend on https://docs.rs/oci-spec/latest/oci_spec/ by default, e.g. get_manifest() -> oci_spec::image::ImageManifest.

    opened by cgwalters 1
  • Add `auth_anonymous` option

    Add `auth_anonymous` option

    This is just exposing skopeo's --no-creds. I would like to flip the ostree-as-system case to use this by default, because I don't think the host OS upgrades should read from /root by default.

    Instead, I plan to change the ostree stack to support something like /etc/ostree/container-auth.json so it can be distinct from what the "root user"'s config is.

    Further, the config needs to be in /etc because /root is on /var on ostree systems, and we want to support transactional updates to the pull config.

    (Anyways, that's all mostly ostree stuff, this is just exposing the option)


    opened by cgwalters 1
  • Add an option to provide externally configured `Command`

    Add an option to provide externally configured `Command`

    This will allow higher level tooling to inject their own sandboxing.

    I would like to add opinionated usage of e.g. systemd-run but this is a useful start.

    opened by cgwalters 1
  • Expose `GetConfig`

    Expose `GetConfig`

    https://github.com/containers/skopeo/pull/1495 landed, but we need to bind it.

    This actually introduces a new problem, which is detecting at runtime support for API methods based on semver.

    opened by cgwalters 1
  • Support insecure registry

    Support insecure registry

    We need to support the equivalent of skopeo inspect --tls-verify=false. This may require an skopeo experimental-image-proxy API addition too - like properties on OpenImage?

    opened by cgwalters 1
  • Handle case where skopeo exits unexpectedly

    Handle case where skopeo exits unexpectedly

    Confusingly, today running skopeo some-unknown-command just outputs usage information to stdout, and exits successfully.

    I'll probably fix that to exit with an error, but either way we need to handle the case where skopeo exits before we get a reply from the socketpair.

    opened by cgwalters 1
  • Fails to pull authenticated repos: with

    Fails to pull authenticated repos: with "Conflicting authentication options"

    The latest release fails to pull images from authenticated repositories. This only happens on rpm-ostree: Version: '2022.18' Which pulls containers-image-proxy-rs latest release.

    rpm-ostree rebase --experimental ostree-unverified-registry:ghcr.io/user/private-repo:latest
    
    Pulling manifest: ostree-unverified-image:docker://ghcr.io/user/private-repo:latest
    error: Conflicting authentication options
    
    opened by jmarrero 0
  • Change API to make all methods work on an opened image

    Change API to make all methods work on an opened image

    This relates to https://github.com/containers/containers-image-proxy-rs/pull/31 - basically it would make way more sense for us to expose methods on an opened image.

    This would also help fix https://github.com/coreos/rpm-ostree/issues/4213 because then we could impl Drop on it so that the image is automatically closed.

    opened by cgwalters 1
  • [DNM] Test that CI successfully fails

    [DNM] Test that CI successfully fails

    I think a good pattern is to keep an open (draft) pull request that should fail CI, and verify that it actually does.

    It's surprisingly easily to e.g. have CI not actually test the code. In this case we have our "reverse dependency testing" which patches the crate to use the local code, and that could easily not work.

    opened by cgwalters 2
Releases(v0.2.0)
  • v0.2.0(Nov 5, 2021)

    What's Changed

    • Remove unused tokio-{stream,util} deps by @cgwalters in https://github.com/containers/containers-image-proxy-rs/pull/2
    • Fixes for child process monitoring by @cgwalters in https://github.com/containers/containers-image-proxy-rs/pull/3
    • Add a lot more tracing usage by @cgwalters in https://github.com/containers/containers-image-proxy-rs/pull/4

    Full Changelog: https://github.com/containers/containers-image-proxy-rs/compare/v0.1.1...v0.2.0

    Source code(tar.gz)
    Source code(zip)
Owner
Containers
Open Repository for Container Tools
Containers
🐱‍👤 Cross-language static library for accessing the Lua state in Garry's Mod server plugins

gmserverplugin This is a utility library for making Server Plugins that access the Lua state in Garry's Mod. Currently, accessing the Lua state from a

William 5 Feb 7, 2022
Stack unwinding library in Rust

Unwinding library in Rust and for Rust This library serves two purposes: Provide a pure Rust alternative to libgcc_eh or libunwind. Provide easier unw

Gary Guo 51 Nov 4, 2022
A dynamically typed, interpreted, stack-based language.

Stacc A dynamically typed, interpreted, stack-based language. How does it work? Each call-frame/scope has its own variables and stack, so you can get/

null 8 Nov 12, 2021
Dc improved: Feature-added rewrite of a 50+ year old RPN calculator/stack machine/programming language

dcim [WIP] dc improved: Feature-added rewrite of a 50+ year old RPN calculator/stack machine/programming language This readme is currently incomplete.

null 3 Jun 18, 2022
A simple image uploader and caching/optimizing proxy

yeet (╯°□°)╯︵ ┻━┻ yeet is an image caching and optimizing proxy, with support for uploading. I built it for ellie.wtf Upload images to S3 with curl Ca

Ellie Huxtable 10 Sep 26, 2023
Rust based WASM/JS bindings for ur-rust

ur-wasm-js WASM/JS bindings for the ur-rust rust library Getting started Installation Either build the library yourself with wasm-pack or install for

Lightning Digital Entertainment 5 Feb 28, 2024
A project for generating C bindings from Rust code

cbindgen   Read the full user docs here! cbindgen creates C/C++11 headers for Rust libraries which expose a public C API. While you could do this by h

Ryan Hunt 1.7k Jan 3, 2023
Automatically generates Rust FFI bindings to C (and some C++) libraries.

bindgen bindgen automatically generates Rust FFI bindings to C (and some C++) libraries. For example, given the C header doggo.h: typedef struct Doggo

The Rust Programming Language 3.2k Jan 4, 2023
Rust-JDBC bindings

jdbc A Rust library that allows you to use JDBC and JDBC drivers. Usage First, add the following to your Cargo.toml: [dependencies] jdbc = "0.1" Next,

Aurora 18 Feb 9, 2022
Lua 5.3 bindings for Rust

rust-lua53 Aims to be complete Rust bindings for Lua 5.3 and beyond. Currently, master is tracking Lua 5.3.3. Requires a Unix-like environment. On Win

J.C. Moyer 150 Dec 14, 2022
Safe Rust bindings to Lua 5.1

rust-lua Copyright 2014 Lily Ballard Description This is a set of Rust bindings to Lua 5.1. The goal is to provide a (relatively) safe interface to Lu

Lily Ballard 124 Jan 5, 2023
mruby safe bindings for Rust

mrusty. mruby safe bindings for Rust mrusty lets you: run Ruby 1.9 files with a very restricted API (without having to install Ruby) reflect Rust stru

Anima 200 Oct 12, 2022
Rust bindings for writing safe and fast native Node.js modules.

Rust bindings for writing safe and fast native Node.js modules. Getting started Once you have the platform dependencies installed, getting started is

The Neon Project 7k Jan 4, 2023
Objective-C Runtime bindings and wrapper for Rust.

Objective-C Runtime bindings and wrapper for Rust. Documentation: http://ssheldon.github.io/rust-objc/objc/ Crate: https://crates.io/crates/objc Messa

Steven Sheldon 336 Jan 2, 2023
High-level Rust bindings to Perl XS API

Perl XS for Rust High-level Rust bindings to Perl XS API. Example xs! { package Array::Sum; sub sum_array(ctx, array: AV) { array.iter().map(|

Vickenty Fesunov 59 Oct 6, 2022
Rust <-> Python bindings

rust-cpython Rust bindings for the python interpreter. Documentation Cargo package: cpython Copyright (c) 2015-2020 Daniel Grunwald. Rust-cpython is l

Daniel Grunwald 1.7k Dec 29, 2022
Rust bindings for the Python interpreter

PyO3 Rust bindings for Python. This includes running and interacting with Python code from a Rust binary, as well as writing native Python modules. Us

PyO3 7.2k Jan 4, 2023
Safe Rust <---> GraalVM Polyglot bindings using procedural macros

The class macro is the primary way to generate bindings to Java types; it will generate a struct (with generics if specified) that implements Pass and Receive and has all the methods you give stubs for. The methods generated can be used like normal rust methods, however mutability is not enforced. The fully-qualified type name should precede a block containing method and constructor stubs. Java primitives like char, int, and byte are aliased to corresponding Rust types.

Alec Petridis 33 Dec 28, 2022
Implementation of Monte Carlo PI approximation algorithm in Rust Python bindings

rusty_pi Implementation of Monte Carlo PI approximation algorithm in Rust Python bindings. Time of 100M iterations approximation on Core i7 10th gen:

Aleksey Popov 1 Jul 6, 2022