A Rust library for processing application configuration easily

Overview

Configure me

A Rust library for processing application configuration easily

About

This crate aims to help with reading configuration of application from files, environment variables and command line arguments, merging it together and validating. It auto-generates most of the parsing and deserializing code for you based on specification file. It creates a struct for you, which you can use to read configuration into. It will contain all the parsed and validated fields, so you can access the information quickly easily and idiomatically.

The generated code is formatted to be easy to read and understand.

Wait a second, why this crate doesn't use derive?

I'd love to use derive. Unfortunately it doesn't compose well with man page generation and other tooling.

For a longer version, see docs/why_not_derive.md

Example

Let's say, your application needs these parametrs to run:

  • Port - this is mandatory
  • IP address to bind to - defaults to 0.0.0.0
  • Path to TLS certificate - optional, the server will be unsecure if not given

First create config_spec.toml configuration file specifying all the parameters:

[[param]]
name = "port"
type = "u16"
optional = false
# This text will be used in the documentation (help etc.)
# It's not mandatory, but your progam will be ugly without it.
doc = "Port to listen on."

[[param]]
name = "bind_addr"
type = "::std::net::Ipv4Addr" # Yes, this works and you can use your own types as well! (impl. Deserialize and ParseArg)
default = "::std::net::Ipv4Addr::new(0, 0, 0, 0)" # Rust expression that creates the value.
doc = "IP address to bind to."

[[param]]
name = "tls_cert"
type = "String"
doc = "Path to the TLS certificate. The connections will be unsecure if it isn't provided."
# optional = true is the default, no need to add it here.

Then, create a simple build.rs script like:

extern crate configure_me_codegen;

fn main() -> Result<(), configure_me_codegen::Error> {
    configure_me_codegen::build_script_auto()
}

Tip: use cfg_me to generate a man page for your program.

Add dependencies to Cargo.toml:

[package]
# ...
build = "build.rs"

# This tells auto build script and other tools where to look for your specification.
[package.metadata.configure_me]
spec = "config_spec.toml"

[dependencies]
configure_me = "0.4.0"

[build-dependencies]
configure_me_codegen = "0.4.0"

And finally add appropriate incantations into src/main.rs:

#[macro_use]
extern crate configure_me;

include_config!();

fn main() {
    // Don't worry, unwrap_or_exit() prints a nice message instead of ugly panic.
    let (server_config, _remaining_args) = Config::including_optional_config_files(&["/etc/my_awesome_server/server.conf"]).unwrap_or_exit();

    // Your code here, for example:
    let listener = std::net::TcpListener::bind((server_config.bind_addr, server_config.port)).expect("Failed to bind socket");
}

If you need to generate different files for multiple binaries, create a separate file for each binary and then define them separately in Cargo.toml:

# config for binary foo
[package.metadata.configure_me.bin]
foo = "foo_config_spec.toml"

# config for binary bar
[package.metadata.configure_me.bin]
bar = "bar_config_spec.toml"

And include the file in foo like this:

include_config!("foo");

This needs to be specific because there's no way to detect binary name.

Manual page generation

The crate exports an interface for generating manual pages, but I recommend you to not worry about it. There's a tool for generating extra files (currently only man page) from your specification. You can install it using cargo.

After installing it, you can type cfg_me man to see the generated man page. Run cfg_me -o program_name.1 man to save it to a file.

Debconf generation

This crate also contains experimental debconf support behind debconf feature. It generates templates, configure and postinst files for you. If you plan to package your application, you can use it. Note that this isn't integrated with cargo-deb yet.

In order to use this feature, you must enable the flag in Cargo.toml:

configure_me_codegen = { version = "0.4.0", features = ["debconf"] }

Then add debconf options to your configuration specification:

[debconf]
# Sets the name of the package.
# Enables debconf support.
package_name = "my-awesome-app"

[[param]]
name = "port"
type = "u16"
optional = false
# Documentation IS mandatory for debconf!
doc = "Port to listen on."
# Priority used for debconf questions "high" is recommended for non-default,
# mandatory questions. In case of missing priority, the option is skipped!
debconf_priority = "high"

[[param]]
name = "bind_addr"
type = "::std::net::Ipv4Addr"
# Rust expression that creates the default value.
default = "::std::net::Ipv4Addr::new(0, 0, 0, 0)"
doc = "IP address to bind to."
debconf_priority = "low"
# The default set by debconf.While it might seem redundant, this way the user
# sees the default value when editing.
debconf_default = "0.0.0.0"

[[param]]
name = "tls_cert"
type = "String"
doc = "Path to the TLS certificate. The connections will be unsecure if it isn't provided."
debconf_priority = "medium"

Finally build your application with DEBCONF_OUT environment variable set to existing directory where configure_me should generate the files.

Planned features

This crate is unfinished and there are features I definitelly want:

  • Support for documenting your configuration very well - done
  • Support environment variables - done
  • Generate bash completion
  • Some advanced features

Comparison with clap

clap is a great crate that works well. Unfortunately, it doesn't support reading from config files. It also has stringly-typed API, which adds boilerplate and (arguably small, but non-zero) runtime overhead.

On the other hand, it's much more mature and supports some features this crate doesn't (bash completion and native subcommands).

clap may be more suitable for programs that should be easy to work with from command line, configure_me may be better for long-running processes with a lot of configuration options.

License

MITNFA

Comments
  • add some extra detail to docs

    add some extra detail to docs

    closes #44 This doesn't cover all functionality, but it covers a bunch of common use cases I had to figure out to use this crate for another project.

    Question, if I use the conf_file_param I find that options set in that config file overwrite options set in environment variables. Is that intentional?

    opened by Shamazo 4
  • Consider support out_file setting

    Consider support out_file setting

    set out_dir like pb builder's style:Builder::out_dir.

    
    fn path_in_out_dir<P: AsRef<Path>>(file_name: P) -> Result<PathBuf, Error> {
        //TODO  pass the out_dir config
        let mut out: PathBuf = std::env::var_os("OUT_DIR").ok_or(ErrorData::MissingOutDir)?.into();
        out.push(file_name);
    
        Ok(out)
    }
    
    
    opened by mylinyuzhi 2
  • Change sequence to map

    Change sequence to map

    Consider changing sequence to map in specification Toml input. That means:

    [[param]]
    name = "foo"
    type = "String"
    

    would become:

    [param.foo]
    type = "String"
    

    Alternative that makes Toml input map more directly to Rust code, catching duplicates much more early:

    [field.foo]
    cli = "param" # other possibilities: switch, disabled
    type = "String"
    

    So far I think I prefer the less verbose option, duplicate detection can be written differently.

    @dr-orlovsky @romanz would appreciate your thoughts.

    enhancement v1.0 refactoring API breaking 
    opened by Kixunil 2
  • Multiple configurations for multiple binaries

    Multiple configurations for multiple binaries

    How is it possible to use per-binary configuration and code generation (when there are multiple bins in src/bin section and each one needs a separate config)?

    enhancement good first issue 
    opened by dr-orlovsky 1
  • Skipping default config files

    Skipping default config files

    It'd be great to add an option to enable skipping default config files. It'd resolve https://github.com/romanz/electrs/issues/217 as well as provide some robustness for other interesting cases. Programs that don't have such options are sometimes painful to work with (pkg-config comes to mind).

    Proposed solution: make it possible to add skip_default_configs to a boolean flag. After processing arguments, this is checked and configs are not loaded if it's set to true.

    Pros:

    • Looks very easy to implement
    • Implicitly prevents people from making two different options of the same name (because of the field name in the struct).
    • Makes it possible to access this value - e.g. for logging purposes
    • No need to write a separate documentation generating code

    Cons:

    • One might accidentally make more than one such option - this would work, but it'd be weird - maybe issue a warning?
    • It leaks the value, cementing the implementation detail immediately, removing the ability to change it in the future
    • It forces everyone to document this option, while it might be entirely fine to provide doc string automatically. Maybe we could set it as the default, but I'm not sure if it'd be easy enough.

    Alternatives:

    • Silently ignore bunch of other errors (e.g. EPERM) - lot of hair pulling in cases where the user forgot didn't want this behavior.
    • Print warning if a file is skipped because of any other reason - annoying warning clutter for projects that'd need this feature
    • Make this option always present in configure_me - would remove configuration handling, but be inflexible. There's a precedent with --help, but that is extremely widespread - basically every program understands -h and --help. This isn't true for skipping default config.
    bug enhancement 
    opened by Kixunil 1
  • Man page generation

    Man page generation

    • [x] Wait for merging of description section
    • [x] Wait for release of man with description section
    • [x] Implement description
    • [x] Verify that the resulting man page conforms to standards
    • [x] Add option to override binary name
    • [x] Add ability to write to custom destination as well as default one
    opened by Kixunil 1
  • Beautiful error messages with `codespan-reporting`

    Beautiful error messages with `codespan-reporting`

    One of the challenges of configure_me is its usage being less common. Good error messages, among other things, can make development easier and may help people understand configure_me better.

    This makes several significant improvements of error messages:

    • Uses codespan-reporting to report the exact places of root causes of errors - the messages look very similar to those of rustc!
    • Doesn't stop after first validation error - if multiple errors can be detected and reported they are.
    • Moves identifier validation out of deserialization to allow detecting other errors and print very detailed messages with hints.
    • Treats --help/-h as reserved and reports it as such.
    • Adds detection of duplicate short options - this was previously missing.
    • Spanned messages don't just report the locations but also explain possibly less-obvious details - such as setting default = true in switch makes it inverted.
    • Improves wording of some messages.

    New API and new style of build script

    Error has new methods report and report_and_exit that report the error with colors to stderr. The latter one is intended for use in build script inside unwrap_or_else:

    fn main() {
        configure_me_codegen::build_script_auto()
            .unwrap_or_else(|error| error.report_and_exit());
    }
    

    The addition of unwrap_or_else is offset by removal of -> Result<...>. Note that the existing build scripts still work, they are just colorless.

    opened by Kixunil 0
  • Screwed up merging

    Screwed up merging

    Merging of conf files is illogical, error-prone, confusing and possibly buggy. Tests need to be written and merge_in modified to override &mut self.

    bug refactoring 
    opened by Kixunil 0
  • debconf support

    debconf support

    It'd be neat if template files and scripts could be generated from the configuration.

    When file list is implemented in build script, then it could auto-generate the config file too. That'd help a lot.

    opened by Kixunil 0
  • Convert into parameter

    Convert into parameter

    It might be nice to be able to convert parameter type to a different type. This would be useful for scenarios when newtype is used to implement ParseArg.

    opened by Kixunil 0
  • Store the locations of loaded config files for debugging

    Store the locations of loaded config files for debugging

    It may be useful to be able to debug which config files were loaded. An example where this might've been useful: https://github.com/rootzoll/raspiblitz/issues/3447 I've seen some services doing such logging (Tor, I think) and it was handy.

    The proposed API is to have a Metadata struct returned together with the configuration. It will also contain program path and spans. And potentially more things in the future. We can simply use Vec<PathBuf>. It will slow down startup a bit but hopefully not much. If this ever becomes a problem we may offer a feature/cfg option to not fill it (empty is zero cost).

    enhancement 
    opened by Kixunil 0
  • Consider moving to xtask

    Consider moving to xtask

    xtask pattern could improve compilation times for libraries. However it's not entirely suited for this kind of optimization. So instead we can put codegen into cfg_me and if it's installed just run it, if it's not run xtask which will do the same thing.

    And make sure cfg_me is versioned similarly to cargo +version ... (but maybe scan the project file for version instead?) so that we can make sure codegen matches.

    opened by Kixunil 0
  • Support conditional compilation

    Support conditional compilation

    It'd be nice to be able to condition switches/params on features being enabled or based on platform etc.

    Suggested design:

    • When enabled, everything works like now.
    • When disabled, the parameters are still parsed but generate a nice error.
    • Help page does not mention the parameter but it could mention that the app was built with specific features.
    opened by Kixunil 0
  • Subcommands

    Subcommands

    Support subcommands like e.g. commit is a subcommand of git.

    Example input:

    [general]
    # global options
    
    # This is a global param - given to program in the form of cmd --foo subcmd
    # We could later support cmd subcmd --foo
    [[param]]
    name = "foo"
    type = "String"
    # ...
    
    # same as above an be done with switch
    
    # We use subcmd instead of subcommand to keep it reasonably short because it will be repeated
    [subcmd.subcmdname.general]
    # Only support these two for now, other things available above are mostly non-sensical here
    summary = "This command does something"
    doc = "Long explanation of what this command is all about..."
    # LATER we could also support include = "file-relative-to-current"
    # included files should probably only support params and switches
    
    # works exactly the same as in top-level
    [[subcmd.subcmdname.param]]
    name = "bar"
    type = "String"
    
    # switch is supported here as well
    # nested subcommand is also supported
    

    The generated code will look something like this:

    enum Subcommand {
        Subcmdname(subcmdname::Config),
    }
    
    // recursively generates almost the same thing as top-level
    mod subcmdname {
        struct Config {
            // no _program_name here, it's pointless
            bar: String,
        }
    
        // also has raw module like the top-level
        mod raw {
        }
    }
    

    Specification parsing code will get subcmd: HashMap<Ident, Subcommand> field.

    struct Subcommand {
        global: SubcommandGlobal,
        param: Vec<Param>,
        switch: Vec<Switch>,
    }
    

    Difficult stuff:

    • How to parse and process config files for subcommands? Suggestion: don't support in initial impl
    • How to process env vars for config files? Suggestion: don't support in initial impl
    • Should subcommand be inside Config, or outside? The advantage of being outside is it can get global config without self-reference but maybe that's not useful as immutable references will most likely be used?
    • Should we support more straightforward design as clap does? If there are multiple nested subcommands and mid-commmands don't have any options, then avoiding the struct would lead to easier-to-write code, although harder-to-refactor. Maybe instead have Subcmdname { config: subcmdname::Config, subcmd: subcmdname::Subcommand }, then matching code can just ignore empty struct. This obviously implies "outside" for the question about.
    enhancement help wanted v1.0 
    opened by Kixunil 0
  • Support for multiple binaries mis-documented

    Support for multiple binaries mis-documented

    I am trying to use configure_me for a crate with two executables each with their own config files. This section in the documentation seems wrong:

    # config for binary foo
    [package.metadata.configure_me.bin]
    foo = "foo_config_spec.toml"
    
    # config for binary bar
    [package.metadata.configure_me.bin]
    bar = "bar_config_spec.toml"
    

    I believe it should be

    # config for binary foo
    [package.metadata.configure_me.bin]
    foo = "foo_config_spec.toml"
    bar = "bar_config_spec.toml"
    

    I can open a PR for this later.

    opened by Shamazo 3
  • Case for LNP Node and other LNP/BP microservices

    Case for LNP Node and other LNP/BP microservices

    Just would like to check that I am using this lib efficiently with cases we have in different LNP/BP nodes (LNP Node, BP Node, RGB Node).

    Each node is represented by

    • a set of microservices, each of which can be loaded either in form of separate process (daemon) or as a thread withing a single process, depending on the configuration and environment (mobile app or server + docker).
    • a main service (it can also be just a thread withing mobile app) which launches and manages the rest.

    The main service should be configured with either:

    • config file + command-line args matching config file + env variables (in case it is a process), or
    • with rust structure passed to it's entry point (in case it is started as a thread).

    The rest of services are never intended to be launched by human from command-line (they are started as threads or processes by the main service, or automatically by something like Kubernetes), but they also should be configured with

    • some service-specific command line options/env vars, which should not map to the configuration file (their value is unique per service instance) +
    • read parts of the shared config from the main service - or get it's configuration structure for multi-threaded case.

    What I plan is:

    • to have a single configure_me-based configuration for the main service, generating command-line args for it + support for env variables and man pages
    • to re-use that config file by the rest of services, but using direct TOML reads, not configure_me-based routines
    • to add other per-service command-line configuration options (which do not map to the config file and do not need man page) using Clap
    opened by dr-orlovsky 0
Owner
Martin Habovštiak
Beside programming, I'm interested in security, cryptography, Bitcoin and freedom.
Martin Habovštiak
Kusion Configuration Language (KCL) is an open source configuration language mainly used in Kusion Stack

Kusion Configuration Language (KCL) is an open source configuration language mainly used in Kusion Stack. KCL is a statically typed language for configuration and policy scenarios, based on concepts such as declarative and Object-Oriented Programming (OOP) paradigms.

KusionStack 264 Dec 30, 2022
Just-config is a configuration library for rust

Config Library for Rust Just-config is a configuration library for rust. It strives for the old Unix mantra "Do one thing and to it well".

FlashSystems 7 Apr 15, 2022
cfg-rs: A Configuration Library for Rust Applications

cfg-rs: A Configuration Library for Rust Applications Major Features One method to get all config objects, see get. Automatic derive config object, se

Daniel YU 20 Dec 16, 2022
⚙️ Layered configuration system for Rust applications (with strong support for 12-factor applications).

config-rs Layered configuration system for Rust applications (with strong support for 12-factor applications). Set defaults Set explicit values (to pr

Ryan Leckey 1.8k Jan 9, 2023
Zap - A simple cross-platform configuration management and orchestration tool

Zap - A simple cross-platform orchestration and configuration management tool. The main goal for Zap is to a simple mechanism for managing groups of com

R. Tyler Croy 50 Oct 29, 2022
Uclicious is a flexible reduced boilerplate configuration framework.

Uclicious What is Uclicious Usage Raw API Derive-driven Validators Type Mapping Supported attributes (#[ucl(..)]) Structure level Field level Addition

Andrey Cherkashin 14 Aug 12, 2022
🌽 A simple and pain-free configuration language.

?? Corn A simple and pain-free configuration language. Corn has been designed using inspiration from JSON and Nix to produce a language that's easy an

Jake Stanger 3 Nov 28, 2022
hosts file parsing, modification library, and some derivatives.

hosts-rs hosts: Hosts file parsing, modification library resolve-github: Use Cloudflare DoH to resolve GitHub domains and generate hosts files github-

zu1k 33 Jul 4, 2022
This repository is an experimental WebAssembly build of the [ymfm] Yamaha FM sound cores library.

This repository is an experimental WebAssembly build of the [ymfm] Yamaha FM sound cores library.

hiromasa 36 Dec 25, 2022
Build a config structure from environment variables in Rust without boilerplate

Yasec Yet another stupid environment config (YASEC) creates settings from environment variables. (Envconig-rs fork) Features Nested configuration stru

null 4 Dec 28, 2021
Your next config manager, written in rust

confy Your next config manager, written in rust Based on uncomfyhalomacro/hmph but written for .ini files instead of json :) Getting started Take a lo

Krishna Ramasimha 2 Nov 3, 2021
An easy to configure wrapper for Rust's clippy

An easy to configure wrapper for Rust's clippy

Eric Seppanen 46 Dec 19, 2022
Easy c̵̰͠r̵̛̠ö̴̪s̶̩̒s̵̭̀-t̶̲͝h̶̯̚r̵̺͐e̷̖̽ḁ̴̍d̶̖̔ ȓ̵͙ė̶͎ḟ̴͙e̸̖͛r̶̖͗ë̶̱́ṉ̵̒ĉ̷̥e̷͚̍ s̷̹͌h̷̲̉a̵̭͋r̷̫̊ḭ̵̊n̷̬͂g̵̦̃ f̶̻̊ơ̵̜ṟ̸̈́ R̵̞̋ù̵̺s̷̖̅ţ̸͗!̸̼͋

Rust S̵̓i̸̓n̵̉ I̴n̴f̶e̸r̵n̷a̴l mutability! Howdy, friendly Rust developer! Ever had a value get m̵̯̅ð̶͊v̴̮̾ê̴̼͘d away right under your nose just when

null 294 Dec 23, 2022
A systemd-boot configuration and boot entry configuration parser library

A systemd-boot configuration and boot entry configuration parser library

Kaiyang Wu 2 May 22, 2022
A rust layered configuration loader with zero-boilerplate configuration management.

salak A layered configuration loader with zero-boilerplate configuration management. About Features Placeholder Key Convension Cargo Features Default

Daniel YU 28 Sep 20, 2022
Kusion Configuration Language (KCL) is an open source configuration language mainly used in Kusion Stack

Kusion Configuration Language (KCL) is an open source configuration language mainly used in Kusion Stack. KCL is a statically typed language for configuration and policy scenarios, based on concepts such as declarative and Object-Oriented Programming (OOP) paradigms.

KusionStack 264 Dec 30, 2022
wireguard tool to manage / generate configuration. Maintain one yaml configuration file to quickly build wireguard network.

wgx wireguard tool to manage / generate configuration. Maintain one yaml configuration file to quickly build wireguard network. Usage wgx --h USAGE:

yinheli 6 Nov 3, 2022
A CLI to easily switch between multiple Neovim configuration environments, written in Rust

Neovim Configuration Switcher Neovim Configuration Switcher (short nvims) is a CLI to easily switch between multiple Neovim configuration environments

Nhan Pham 3 Mar 30, 2024
A command line tool for easily generating multiple versions of a configuration file from a single template

MultiConf A command line tool for easily generating multiple versions of a configuration file from a single template. Why? I'm a big fan of the i3 win

Ian Clarke 4 Dec 10, 2022
Neovim Configuration Manager (Swap/Backup/Try Configurations Easily)

ncm-rs Neovim Configuration Manager (Swap/Backup/Try Configurations Easily) I created this package because I wanted to try out Lazyvim (which is why i

instance.id 4 Mar 5, 2023