vault client using jwt authentication that define environment variables from vault secrets before executing into something else

Overview

envlt

envlt, like env, allows you to define environment variables and then execute into something else, but instead of static values, it uses using simple expressions to fetch secret from a vault server using a JWT Token as authentication.

It is useful in CI/CD environment, like Gitlab to securely access secrets inside your jobs. It allows you to centralize secrets management by using the short lived JWT token CI_JOB_JWT and get rid of all static variables you normally define under Gitlab for that purpose. For more complex cases involving services configuration, secret renewals, and restart of services, you will probably find that rconfd is a better fit for the task.

Also gitlab premium user can define vault secrets directly in the project ci definition, there is no such integrated mechanism for the community edition. You should in that case use the vault command, add some boilerplate to login to the jwt service, get secrets one by one, then export them to environment variables. I didn't fancy embedding the full vault executable only for that purpose and wanted CI/CD jobs definitions to be as straightforward as possible. envlt integrate all theses steps in one binary and never expose the secrets values in the command arguments contrary to a script.

Usage

envlt 0.5.6

Usage: envlt 
   
     [
    
     ] [-u 
     
      ] [-l 
      
       ] [-c 
       
        ] [-T 
        
         ] [-t 
         
          ] [-V 
          
           ] [-v] [-i] [-I] Get vault secrets from path expressions, define environment variables, then execute into args and command Options: -u, --url the vault url ($VAULT_URL or https://localhost:8200/v1) -l, --login-path the login path (/auth/jwt/login) -c, --cacert path of vault CA certificate (/var/run/secrets/kubernetes.io/serviceaccount/ca.crt) -T, --token the JWT token taken from the given variable name or from the given string if it fails (takes precedence over -t) -t, --token-path path of the JWT token (/var/run/secrets/kubernetes.io/serviceaccount/token) -V, --vars an expression NAME[=VALUE] for defining one or several variables. When no VALUE given, an environment variable with the same name is imported, when VALUE doesn't match an expression with a backend, a new variable is defined with the provided VALUE, otherwise the expression is expanded in one or several variables and NAME is used as a prefix. -v, --verbose verbose mode -i, --import import all environment variables before executing into cmd -I, --import-vault import environment variables whose values matches a vault_path a whose expansion is successful --help display usage information 
          
         
        
       
      
     
    
   

By default, envlt starts with an empty context, meaning that no variables are exposed to cmd. There is 3 options to alter this behavior you can mix together:

  • -i import all accessible variables "as is"
  • -I import only the variables that match an expression with a backend
  • -V (re)define variables (takes precedence over -i and -I) or import existing ones

Variable expression

A variable expression following the -V flag has 3 form:

  • NAME: import an environment variable with the same name
  • NAME=VALUE: define a new environment variable with a static value
  • PREFIX=backend:args:path: define one or several variables by fetching their value from a backend. When the returned value is structured (vault backend and const backend with js value), envlt recursively define one variable name for each leaf of the json tree by joining the prefix and path components with _. Path components are keys for dictionaries and indexes (starting at 0) for arrays.

Backends

There are currently 2 supported back-ends.

Vault

vault:role[,GET|PUT|POST|LIST][,key=val]*:path[#json_pointer]
  • role is the role name used for vault authentication,
  • an optional http method that defaults to GET,
  • optional keywords arguments that are sent as json dictionary in the body of the request,
  • a path corresponding to the vault api point (without /v1/),
  • an optional json pointer to define variables from and that defaults to the root of the tree.

The vault secrets are cached by path (pointer excluded) and fetched only once. It is not really for performance reason but because some api points generate different secret each time they are called like pki. You can define that way several variables with different names (none is a prefix of the other) but tied to the same secret.

Const

const:str|js:value

the value is parsed as json if js or kept as is if str

The main use of the const:str:value expression was to be able to differentiate a standard (not imported) variable from one to be imported when using the -I flag, although you can achieve the same result in a more verbose way by explicitly import a regular variable (whose value is not an expression) with -V NAME.

With const:js you can expand several static environment variables sharing a common prefix with one expression. If you use sccache with cargo you can use for example:

-V 'SCCACHE=const:js:{"bucket": "sccache", "endpoint": "minio:443", "s3_use_ssl": true}'

to define 3 variables

  • SCCACHE_BUCKET=sccache
  • SCCACHE_ENDPOINT=minio:443
  • SCCACHE_S3_USE_SSL=true

to speedup compilation by using an S3 or compatible (minio) objects storage as build cache. You would also have to provide access keys AWS_* which could come from a kv2 secret.

Example

If you have a pki backend mounted at /pki, and a kv2 secret defined at kv/abuild with the following content

crt: xxxx
key: xxxx
keyid: xxxx

calling envlt with the following arguments

envlt -V 'FOO=const:js:{"bar": 0, "baz": 1}'
      -V BAR=3
      -V CERT=vault:role,POST,common_name=example.com:pki/issue/example.com \
      -V PACKAGER=vault:myrole:kv/data/secret#/data
      -- command args

will add the following environment variables added to command context.

  • FOO_BAR=0
  • FOO_BAZ=1
  • BAR=3
  • CERT_CERTIFICATE=...
  • CERT_ISSUING_CA=...
  • CERT_CA_CHAIN_0=...
  • ...
  • CERT_CA_CHAIN_n=...
  • CERT_PRIVATE_KEY=...
  • CERT_PRIVATE_KEY_TYPE=...
  • CERT_SERIAL_NUMBER=...
  • PACKAGER_CRT=...
  • PACKAGER_KEY=...
  • PACKAGER_KEYID=...

By default, envlt use a jwt token available in every kubernetes containers at /var/run/secrets/kubernetes.io/serviceaccount/token. This token has claims about the kubernetes container execution context you can use in vault to restrict the access to secrets.

You can also export the variables instead of defining them with -V and use -I option. This is useful in CI/CD where you can define variables in the upper level, and hiding the details to keep the pipeline as simple as possible

export \
  'FOO=const:js:{"bar": 0, "baz": 1}' \
  BAR=const:str:3 \
  CERT=vault:role,POST,common_name=example.com:pki/issue/example.com \
  PACKAGER=vault:myrole:kv/data/secret#/data
envlt -I -V PATH -V HOME -- command args

If you choose not to import all the environment variables (you don't use -i flag along with -I) you can control exactly which subset of variables are exported (the ones matching a backend expression) and add manually other important regular variables like PATH or HOME like in the example above.

Using envlt with Gitlab CI/CD

Configuring vault

Activate vault jwt authentication

vault write auth/jwt/config jwks_url="https://gitlab.com/-/jwks" bound_issuer="gitlab.com"

Create a policy for accessing the secrets

vault policy write mypolicy - <<EOF
path "kv/data/secrets/*" {
  capabilities = [ "read" ]
}
EOF

Create a role. Here You can only login with role, and only a build on a project inside gitlab group mygroup and for a protected tag (release) will have access to the secret.

vault write auth/jwt/role/myrole - <<EOF
{
  "role_type": "jwt",
  "policies": ["mypolicy"],
  "token_explicit_max_ttl": 60,
  "user_claim": "user_email",
  "bound_claims": {
    "group_path": "mygroup",
    "ref_protected": "true",
    "ref_type": "tag"
  }
}

Configuring Gitlab CI/CD

You should make a build image (mybuilder) containing the envlt executable. Then you just have to call envlt in your pipelines script using the JWT token from the environment variable CI_JOB_JWT (note that we use a variable name here instead of a substitution to not expose the token on command line arguments)

You must define a VAULT_URL=const:str:https://localhost:8200 and a SECRET=vault:myrole:kv/data/secrets#/data variables and a good place for that is in the project or group settings.

Here is an example .gitlab-ci.yml

image: mybuilder

build:
  stage: build
  script:
  # The Makefile use files containing secrets generated by rconfd
  - envlt -I -V PATH -V HOME -T CI_TOKEN_JWT -- make
You might also like...
🔥 Firebase authentication for Rust 🦀
🔥 Firebase authentication for Rust 🦀

Fire Auth Rust wrapper for Firebase Authentication REST API Installation Add the following to Cargo.toml: fireauth = "0.1.5" How to use First you need

Xbox live authentication flow for Minecraft with Rust.

MC Auth Xbox live authentication flow for Minecraft in Rust. Why? In order to create tools for Minecraft based on rust that implement the user profile

A HTTP Filter checking for OIDC Authentication, made for Envoy Plugins, written in Rust

WASM OIDC Plugin A plugin for Envoy written in Rust. It is a HTTP Filter, that implements the OIDC Authorization Code Flow. Requests sent to the filte

An oauth2 client implementation providing the Device, Installed and Service Account flows.
An oauth2 client implementation providing the Device, Installed and Service Account flows.

yup-oauth2 is a utility library which implements several OAuth 2.0 flows. It's mainly used by google-apis-rs, to authenticate against Google services.

Extensible, strongly-typed Rust OAuth2 client library

OAuth2 An extensible, strongly-typed implementation of OAuth2 (RFC 6749). Documentation is available on docs.rs. Release notes are available on GitHub

Authorization Server with Rust using Tonic

authorization-server Authorization Server with Rust using Tonic. Function implemented User registration and profile store Change password Login Token

Ointers is a library for representing pointers where some bits have been stolen so that they may be used by the programmer for something else

Ointers is a library for representing pointers where some bits have been stolen so that they may be used by the programmer for something else. In effect, it's a small amount of free storage

Loads environment variables into your structs in one shot.
Loads environment variables into your structs in one shot.

econf Loads environment variables into your structs in one shot. econf allows to override struct fields with environment variables easily. This is use

Deserialize (potentially nested) environment variables into your custom structs
Deserialize (potentially nested) environment variables into your custom structs

envious allows you to deserialize your serde enabled structs from environment variables. See it in action: use serde::{Deserialize, Serialize}; #[der

Something something B language.

badc A terrible, dirty, no-good, rotten B compiler. Written by one really great human being, and one obscenely terrible sheep. Contributing Don't. Ref

⚡️(cd with env) Is a configurable cd wrapper that lets you define your environment per directory.
⚡️(cd with env) Is a configurable cd wrapper that lets you define your environment per directory.

⚡️cdwe (cd with env) A simple configurable cd wrapper that provides powerful utilities for customizing your envionment per directory. (For ZSH / BASH

Distributed Vault For Your Secrets
Distributed Vault For Your Secrets

https://meta-secret.github.io Application Design Activity Diagram graph TD User -- |split password| MSS{MetaSecret} MSS -- |split| Hash1

JWT Authentication in Rust using Axum Framework

Are you interested in building a secure authentication system for your Rust web application? Look no further than the Axum framework and JSON Web Tokens (JWTs)! Axum is a fast and scalable Rust web framework that provides a reliable and efficient platform for developing microservices and APIs.

A Rust Boilerplate server with GraphQL API, Diesel, PostgreSQL, session authentication and JWT
A Rust Boilerplate server with GraphQL API, Diesel, PostgreSQL, session authentication and JWT

Canduma rust Graphql A Rust authentication server with GraphQL API, Diesel, PostgreSQL session authentication and JWT This repository contains a Graph

Rust - JWT Authentication with Actix Web

In this article, we will delve into the implementation of JWT authentication in Rust, covering all crucial steps from generating and verifying JWT tokens with the HS256 algorithm, to registering users, signing them in, logging them out, and safeguarding private routes.

Axum + JWT authentication Middleware that allows you to start building your application fast

axum_jwt_ware Integration Guide Simple Axum + JWT authentication middleware with implemented Login and refresh token. Goal I aim to simplify the proce

Build a config structure from environment variables in Rust without boilerplate

Yasec Yet another stupid environment config (YASEC) creates settings from environment variables. (Envconig-rs fork) Features Nested configuration stru

A Rust-based Garry's Mod module for fetching environment variables.

gm_environ Using Environment Variables in Garry's Mod. Installation Download a copy of the module from the releases (or compile from source) Move the

Envful is a CLI tool that verifies the presence of environment variables
Envful is a CLI tool that verifies the presence of environment variables

Envful is a CLI tool that verifies the presence of environment variables. It looks inside your .env file and the host system. You can use it to run any process while ensuring all the variables are set.

Owner
Eric Burghard
Eric Burghard
Highly flexible library to manage and orchestrate JWT workflow

JWT Vault Highly flexible library to manage and orchestrate JWT workflow Examples | Website | Chat TODO Add more examples Improve coverage Features Ma

Saurav Gupta 65 Nov 8, 2022
A minimal jwt implementation for OIDC

Compact JWT Json Web Tokens (JWT) are a popular method for creating signed transparent tokens that can be verified by clients and servers. They are en

Kanidm 4 Dec 29, 2021
Simple backend app with Actix-web, JWT and MongoDB

Actix Web JWT Example Simple backend app with Actix-web, JWT and MongoDB (JWT Token, Protect Route, Login & Register) While developing the web service

Emre 124 Dec 31, 2022
WebCipher - JWT encryption/decryption algorithms + a JWK Store implementation

webcipher provides JWT authentication utilities and storage mechanism for caching keys and optimizing decryption/encryption processes.

Wavy 1 May 1, 2022
SD-JWT Rust Reference Implementation

SD-JWT Rust Reference Implementation This is the reference implementation of the IETF SD-JWT specification written in Rust. Supported version: 6. Note

OpenWallet Foundation Labs 4 Dec 19, 2023
Example application using a Vue frontend with Rust backend that has authentication + authorization.

This project contains a Rust server that serves a single page application and has authentication + JWT-based authorization.

null 43 Dec 9, 2022
Authenticate to Minecraft using the Microsoft Authentication Scheme from Rust.

Authenticating to Minecraft with the Microsoft Authentication Scheme from Rust This program showcases an implementation of the microsoft authenticatio

ALinuxPerson 17 Dec 22, 2022
An implementation for an authentication API for Rocket applications.

rocket_auth rocket_auth provides a ready-to-use backend agnostic API for authentication management. For more information visit the documentation at ht

null 62 Dec 19, 2022
Rust library for HTTP authentication. Parses challenge lists, responds to Basic and Digest challenges. Likely to be extended with server support and additional auth schemes.

Rust library for HTTP authentication. Parses challenge lists, responds to Basic and Digest challenges. Likely to be extended with server support and a

Scott Lamb 3 Jun 10, 2022
Fast, simple and REST compliant file-server with public/private key authentication written in Rust

stormi Stormi is a fast and simple file-server with public/private key authentication How does it work? Stormi accepts multipart/form-data form with m

Polygon 2 Dec 8, 2022