rustic_core - library for fast, encrypted, deduplicated backups that powers rustic-rs

Overview

Library for fast, encrypted, and deduplicated backups

About

This library is powering rustic-rs. A backup tool that provides fast, encrypted, deduplicated backups. It reads and writes the restic repository format, which is described in their design document.

Note: rustic_core is in an early development stage and its API is subject to change in the next releases. If you want to give feedback on that, please open a thread in our discussions.

Contact

You can ask questions in the Discussions or have a look at the FAQ.

Contact Where?
Issue Tracker GitHub Issues
Discord Discord
Discussions GitHub Discussions

Usage

Add this to your Cargo.toml:

[dependencies]
rustic_core = "0.1"

Crate features

This crate exposes a few features for controlling dependency usage:

  • cli - Enables support for CLI features by enabling merge and clap features. This feature is disabled by default.
  • merge - Enables support for merging multiple values into one, which enables the merge dependency. This is needed for parsing commandline arguments and merging them into one (e.g. config). This feature is disabled by default.
  • clap - Enables a dependency on the clap crate and enables parsing from the commandline. This feature is disabled by default.

Examples

Example: Initializing a new repository

use rustic_core::{ConfigOptions, KeyOptions, Repository, RepositoryOptions};
use simplelog::{Config, LevelFilter, SimpleLogger};
use std::error::Error;

fn main() -> Result<(), Box<dyn Error>> {
    // Display info logs
    let _ = SimpleLogger::init(LevelFilter::Info, Config::default());

    // Init repository
    let repo_opts = RepositoryOptions::default()
        .repository("/tmp/repo")
        .password("test");
    let key_opts = KeyOptions::default();
    let config_opts = ConfigOptions::default();
    let _repo = Repository::new(&repo_opts)?.init(&key_opts, &config_opts)?;

    // -> use _repo for any operation on an open repository
    Ok(())
}

Example: Creating a new snapshot

use rustic_core::{BackupOptions, PathList, Repository, RepositoryOptions, SnapshotOptions};
use simplelog::{Config, LevelFilter, SimpleLogger};
use std::error::Error;

fn main() -> Result<(), Box<dyn Error>> {
    // Display info logs
    let _ = SimpleLogger::init(LevelFilter::Info, Config::default());

    // Open repository
    let repo_opts = RepositoryOptions::default()
        .repository("/tmp/repo")
        .password("test");
    let repo = Repository::new(&repo_opts)?.open()?.to_indexed_ids()?;

    let backup_opts = BackupOptions::default();
    let source = PathList::from_string(".")?.sanitize()?;
    let snap = SnapshotOptions::default()
        .add_tags("tag1,tag2")?
        .to_snapshot()?;

    // Create snapshot
    let snap = repo.backup(&backup_opts, source, snap)?;

    println!("successfully created snapshot:\n{snap:#?}");
    Ok(())
}

Example: Restoring a snapshot

use rustic_core::{LocalDestination, LsOptions, Repository, RepositoryOptions, RestoreOptions};
use simplelog::{Config, LevelFilter, SimpleLogger};
use std::error::Error;

fn main() -> Result<(), Box<dyn Error>> {
    // Display info logs
    let _ = SimpleLogger::init(LevelFilter::Info, Config::default());

    // Open repository
    let repo_opts = RepositoryOptions::default()
        .repository("/tmp/repo")
        .password("test");
    let repo = Repository::new(&repo_opts)?.open()?.to_indexed()?;

    // use latest snapshot without filtering snapshots
    let node = repo.node_from_snapshot_path("latest", |_| true)?;

    // use list of the snapshot contents using no additional filtering
    let streamer_opts = LsOptions::default();
    let ls = repo.ls(&node, &streamer_opts)?;

    let destination = "./restore/"; // restore to this destination dir
    let create = true; // create destination dir, if it doesn't exist
    let dest = LocalDestination::new(destination, create, !node.is_dir())?;

    let opts = RestoreOptions::default();
    let dry_run = false;
    // create restore infos. Note: this also already creates needed dirs in the destination
    let restore_infos = repo.prepare_restore(&opts, ls.clone(), &dest, dry_run)?;

    repo.restore(restore_infos, &opts, ls, &dest)?;
    Ok(())
}

Example: Checking a repository

use rustic_core::{CheckOptions, Repository, RepositoryOptions};
use simplelog::{Config, LevelFilter, SimpleLogger};
use std::error::Error;

fn main() -> Result<(), Box<dyn Error>> {
    // Display info logs
    let _ = SimpleLogger::init(LevelFilter::Info, Config::default());

    // Open repository
    let repo_opts = RepositoryOptions::default()
        .repository("/tmp/repo")
        .password("test");
    let repo = Repository::new(&repo_opts)?.open()?;

    // Check repository with standard options but omitting cache checks
    let opts = CheckOptions::default().trust_cache(true);
    repo.check(opts)?;
    Ok(())
}

Contributing

Found a bug? Open an issue!

Got an idea for an improvement? Don't keep it to yourself!

Please make sure, that you read the contribution guide.

Minimum Rust version policy

This crate's minimum supported rustc version is 1.68.2.

The current policy is that the minimum Rust version required to use this crate can be increased in minor version updates. For example, if crate 1.0 requires Rust 1.20.0, then crate 1.0.z for all values of z will also require Rust 1.20.0 or newer. However, crate 1.y for y > 0 may require a newer minimum version of Rust.

In general, this crate will be conservative with respect to the minimum supported version of Rust.

License

Licensed under either of:

Comments
  • fix(deps): update rust crate rayon to 1.8.0

    fix(deps): update rust crate rayon to 1.8.0

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | rayon | dependencies | minor | 1.7.0 -> 1.8.0 |


    Release Notes

    rayon-rs/rayon (rayon)

    v1.8.0

    Compare Source

    • The minimum supported rustc is now 1.63.
    • Added ThreadPoolBuilder::use_current_thread to use the builder thread as part of the new thread pool. That thread does not run the pool's main loop, but it may participate in work-stealing if it yields to rayon in some way.
    • Implemented FromParallelIterator<T> for Box<[T]>, Rc<[T]>, and Arc<[T]>, as well as FromParallelIterator<Box<str>> and ParallelExtend<Box<str>> for String.
    • ThreadPoolBuilder::build_scoped now uses std::thread::scope.
    • The default number of threads is now determined using std::thread::available_parallelism instead of the num_cpus crate.
    • The internal logging facility has been removed, reducing bloat for all users.
    • Many smaller performance tweaks and documentation updates.

    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

    A-dependencies 
    opened by renovate[bot] 0
  • fix(deps): update rust crate aho-corasick to 1.1.1

    fix(deps): update rust crate aho-corasick to 1.1.1

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | aho-corasick | dependencies | patch | 1.1.0 -> 1.1.1 |


    Release Notes

    BurntSushi/aho-corasick (aho-corasick)

    v1.1.1

    Compare Source


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

    A-dependencies 
    opened by renovate[bot] 0
  • chore(deps): update taiki-e/install-action digest to 2afb713

    chore(deps): update taiki-e/install-action digest to 2afb713

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | taiki-e/install-action | action | digest | 2358ab6 -> 2afb713 |


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

    A-dependencies 
    opened by renovate[bot] 0
  • fix(deps): update rust crate binrw to 0.12.0

    fix(deps): update rust crate binrw to 0.12.0

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | binrw (source) | dependencies | minor | 0.11.2 -> 0.12.0 |


    Release Notes

    jam1garner/binrw (binrw)

    v0.12.0: Version 0.12.0

    Compare Source

    Breaking changes

    • The default behaviour of FilePtr has been changed to always immediately seek to and read the pointed-to value. Helper functions have been added to the file_ptr module to support the common case of reading from an offset table. Additional helper functions and types will be added in future releases to improve support for file indirection.
    • The BinRead::after_parse API has been removed since it was rarely used, usually broken by manual BinRead implementations, and made it impossible to use borrowed values in arguments in some cases. For custom two-pass parsing, one alternative is to create an object that will store data that need to be used during the second pass, pass a mutable reference to that object as an argument to read_options, add data to that object during the first pass, then use the recorded data from the object to perform the required action for the second pass.
    • deref_now, offset_after, and postprocess_now have been removed as they were designed to control the after_parse API which no longer exists. Any field with a deref_now or postprocess_now directive can simply remove the directive to achieve equivalent functionality. Any struct that used offset_after should reorder the fields so the dependent field is after the offset location.

    For more detail on these changes, see #​210.

    • Selected helper functions are no longer re-exported to the root of the crate. Access these helper functions, plus all the other helper functions that were never re-exported to the root, from the helpers module.
    • Using the undocumented internal variable this from an assert directive is no longer supported. Replace this with the supported self keyword instead.

    New features

    • Helper functions for more efficient reading of offset tables have been added to the file_ptr module. These helpers can be combined with the seek_before directive to support both relative and absolute positioning of the pointed-to data. See the documentation for usage information and examples.
    • assert directives on structs, non-unit enums, and data variants can now access the constructed object using the self keyword. (Thanks, @​octylFractal!) (#​219)

    Enhancements

    • Clone is no longer a required trait for arguments passed to parse_with functions. (Thanks, @​octylFractal!)
    • Clone is no longer a required trait for arguments passed to BinReaderExt methods.
    • BinRead is no longer a required trait for dereferencing a value from FilePtr. (#​218)
    • map and try_map functions can now mutably borrow values.
    • dbg now also prints information about any padding and alignment directives on a field.
    • Various documentation improvements.

    Bug fixes

    • The count directive no longer attempts useless conversions. (Thanks, @​poliorcetics!) (#​206)
    • dbg now returns the correct position of a field when using pad_before or align_before. (#​188)
    • Parser errors are no longer discarded if a stream rewind fails. (#​215)
    • Implementation details of binrw will no longer cause borrow checker failures when passing borrowed arguments to child objects.

    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

    A-dependencies 
    opened by renovate[bot] 0
  • Miri fails with `memory allocation of 5368709120 bytes failed`  - exit code: 0xc0000409, STATUS_STACK_BUFFER_OVERRUN

    Miri fails with `memory allocation of 5368709120 bytes failed` - exit code: 0xc0000409, STATUS_STACK_BUFFER_OVERRUN

    Culprit, most likely in this test and surroundings:

        #[test]
        fn chunk_zeros() {
            let mut reader = repeat(0u8);
    
            let poly = random_poly().unwrap();
            let rabin = Rabin64::new_with_polynom(6, poly);
            let mut chunker = ChunkIter::new(&mut reader, usize::MAX, rabin);
    
            let chunk = chunker.next().unwrap().unwrap();
            assert_eq!(constants::MIN_SIZE, chunk.len());
        }
    

    https://github.com/rustic-rs/rustic_core/blob/5675dd5ebea841c6b55c61901a33a42f8504b8e4/src/chunker.rs#L319

    Action-Run: https://github.com/rustic-rs/rustic_core/actions/runs/6240530669/job/16940825841#step:9:481

    memory allocation of 5368709120 bytes failed
    error: test failed, to rerun pass `--lib`
    Error: test failed, to rerun pass `--lib`
    Caused by:
      process didn't exit successfully: `C:\Users\runneradmin\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\bin\cargo-miri.exe runner D:\a\rustic_core\rustic_core\target\miri\x86_64-pc-windows-msvc\debug\deps\rustic_core-a5df6ff461b5f381.exe` (exit code: 0xc0000409, STATUS_STACK_BUFFER_OVERRUN)
    note: test exited abnormally; to see the full output pass --nocapture to the harness.
    Error: The process 'C:\Users\runneradmin\.cargo\bin\cargo.exe' failed with exit code 3221226505
    
    A-testing C-bug C-question O-windows P-high 
    opened by simonsan 0
  • Dependency Dashboard

    Dependency Dashboard

    This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

    Rate-Limited

    These updates are currently rate-limited. Click on a checkbox below to force their creation now.

    • [ ] chore(deps): update actions/checkout digest to 8ade135
    • [ ] chore(deps): update taiki-e/install-action digest to 1c06275
    • [ ] fix(deps): update rust crate clap to 4.4.6
    • [ ] fix(deps): update rust crate reqwest to 0.11.22
    • [ ] fix(deps): update rust crate thiserror to 1.0.49
    • [ ] fix(deps): update rust crate cached to 0.46.0
    • [ ] 🔐 Create all rate-limited PRs at once 🔐

    Open

    These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

    Detected dependencies

    cargo
    Cargo.toml
    • displaydoc 0.2.4
    • thiserror 1.0.48
    • derivative 2.2.0
    • derive_more 0.99.17
    • derive_setters 0.1.6
    • log 0.4.20
    • crossbeam-channel 0.5.8
    • pariter 0.5.1
    • rayon 1.8.0
    • aes256ctr_poly1305aes 0.1.1
    • rand 0.8.5
    • scrypt 0.11.0
    • integer-sqrt 0.1.5
    • binrw 0.11.2
    • hex 0.4.3
    • serde 1.0.188
    • serde-aux 4.2.0
    • serde_derive 1.0.188
    • serde_json 1.0.107
    • serde_with 3.3.0
    • bytes 1.5.0
    • chrono 0.4.31
    • enum-map 2.6.3
    • enum-map-derive 0.14.0
    • zstd 0.12.4
    • aho-corasick 1.1.1
    • cached 0.45.1
    • filetime 0.2.22
    • ignore 0.4.20
    • nix 0.27.1
    • walkdir 2.4.0
    • backoff 0.4.0
    • reqwest 0.11.20
    • url 2.4.1
    • cachedir 0.3.0
    • dirs 5.0.1
    • clap 4.4.4
    • bytesize 1.3.0
    • directories 5.0.1
    • dunce 1.0.4
    • gethostname 0.4.3
    • humantime 2.1.0
    • itertools 0.11.0
    • merge 0.1.0
    • path-dedot 3.1.1
    • shell-words 1.1.0
    • expect-test 1.4.1
    • pretty_assertions 1.4.0
    • public-api 0.32.0
    • quickcheck 1.0.3
    • quickcheck_macros 1.0.0
    • rstest 0.18.2
    • rustdoc-json 0.8.7
    • rustup-toolchain 0.1.5
    • simplelog 0.12.1
    • tempfile 3.8.0
    • sha2 0.10
    • sha2 0.10
    • sha2 0.10
    • xattr 1
    github-actions
    .github/workflows/audit.yml
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • actions-rs/audit-check v1@35b7b53b1e25b55642157ac01b4adceb5b9ebef3
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • EmbarkStudios/cargo-deny-action v1@a50c7d5f86370e02fae8472c398f15a36e517bb8
    .github/workflows/careful.yml
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • taiki-e/install-action v2@2afb713f1c297f70eecb9cb904b36612eba3c25e
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505
    .github/workflows/ci.yml
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • actions-rs/clippy-check v1@b5b5f21f4797c02da247df37026fcd0a5024aa4d
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • taiki-e/install-action v2@2afb713f1c297f70eecb9cb904b36612eba3c25e
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505
    .github/workflows/cross-ci.yml
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    .github/workflows/lint-docs.yml
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • dprint/check v2.2@2f1cf31537886c3bfb05591c031f7744e48ba8a1
    .github/workflows/msrv.yml
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • taiki-e/install-action v2@2afb713f1c297f70eecb9cb904b36612eba3c25e
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505
    .github/workflows/release-ci.yml
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • obi1kenobi/cargo-semver-checks-action v2@e275dda72e250d4df5b564e969e1348d67fefa52
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505
    .github/workflows/release-pr.yml
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • taiki-e/install-action v2@2afb713f1c297f70eecb9cb904b36612eba3c25e
    • cargo-bins/release-pr v2@deeacca5a38bacc74a3f444b798f4b9bba40f6ad
    .github/workflows/triage.yml
    .github/workflows/update-public-api.yml
    • actions/checkout v4@3df4ab11eba7bda6032a0b82a6bb43b11571feac
    • actions-rs/toolchain v1@16499b5e05bf2e26879000db0c1d13f7e13fa3af
    • Swatinem/rust-cache v2@a95ba195448af2da9b00fb742d14ffaaf3c21f43
    • actions-rs/cargo v1@844f36862e911db73fe0815f00a4a2602c279505

    • [ ] Check this box to trigger a request for Renovate to run again on this repository
    A-dependencies 
    opened by renovate[bot] 0
  • Rclone seems to keep running after backup ends

    Rclone seems to keep running after backup ends

    I've noticed since I started using Rustic, I have a lot of leftover rcloneorig processes. It seems like every time a backup runs, the rclone process doesn't get closed afterwards.

    19wolf      5733  3.5  0.1 1284416 49796 ?       Sl   20:22   0:00 rcloneorig --config ~/.rclone.conf serve restic --stdio --b2-hard-delete GoogleDrive:restic
    19wolf      8356  0.0  0.1 1286080 61220 ?       Sl   Sep13   0:11 rcloneorig --config ~/.rclone.conf serve restic GoogleDrive:restic --addr localhost:0
    19wolf      9513  0.0  0.2 1353844 100768 ?      Sl   02:18   1:02 rcloneorig --config ~/.rclone.conf serve restic GoogleDrive:restic --addr localhost:0
    19wolf     16279  0.0  0.1 1285184 52564 ?       Sl   Sep13   0:07 rcloneorig --config ~/.rclone.conf serve restic GoogleDrive:restic --addr localhost:0
    19wolf     20556  0.0  0.1 1284928 54792 ?       Sl   Sep13   0:07 rcloneorig --config ~/.rclone.conf serve restic GoogleDrive:restic --addr localhost:0
    19wolf     22057  0.0  0.1 1285440 57584 ?       Sl   Sep13   0:12 rcloneorig --config ~/.rclone.conf serve restic GoogleDrive:restic --addr localhost:0
    
    C-bug A-backends 
    opened by 19wolf 2
  • Easy mode `rustic_core` API

    Easy mode `rustic_core` API

    Recently, I found two related libraries with (mostly) the same underlying principles as rustic (see Related libraries).

    acid store has the same use case with uploading encrypted, deduplicated data to a backend. elfshaker is mostly useful for directories, where most things don't change that often.

    I find the user-facing CLI-API fascinating, though:

    • elfshaker store <snapshot> – capture the state of the current working directory into a named snapshot <snapshot>.
    • elfshaker pack <pack name> – capture all 'loose' snapshots into a single pack file (this is what gets you the compression win).
    • elfshaker extract <snapshot> – restore the state of a previous snapshot into the current working directory.

    As a side-project, I'm developing a file organization software and a personal tool for backup and drive management, where it would be beneficial to easily backup directories and easily restore certain files after an action. I thought of using rustic_core for that.

    Currently, the general API is relatively verbose for smaller use cases. IMHO, it would need another layer of abstraction for certain easy mode use cases, as in short-term backup -> diff -> restore.

    Like really just taking a directory snapshot before changing things in it. Maybe restoring only the files with differences or easily jumping between the two states: last_snapshot / current state.

    I think it would make sense to adapt such an rustic_core::easy_mode::* API design or really implement that on top of rustic_core in an rustic_easy crate — what would fit best.

    I think essentials would be (ideas):

    • store – capture the state of the current working directory into a named snapshot
    • diff – compare two snapshots/paths (in most cases diff the content of the current working directory against the latest snapshot) and mount a temporary snapshot of the differences to be able to extract data
    • merge – combine all available snapshots into a newly named snapshot
    • restore – restore the state of a previous snapshot into the current working directory
    • extract – just access one file/directory of the snapshot as it would be an archive

    As you can see, there is definitely an intersection with the current API approach (#787). Though this easy mode (as I would call it for now) would leave a user with a less customizable version of parts of the core API – but everything needed to have a minimal repository interaction.

    Implementation details

    • we could store this on-the-fly created repository either in /tmp/, /usr/tmp/ or in ~/.rustic/tmp/, depending maybe on a persistent = true flag, that a user can set if they don't want the OS to clean up the repository after a restart
    • we should be able to automatically generate a temporary profile (as in config.toml) for it or store the metadata (e.g. backup source) in a local file in ~/.rustic/temporary_repos.toml or something like that
    • tbc

    Related libraries

    https://github.com/elfshaker/elfshaker https://github.com/lostatc/acid-store

    C-enhancement 
    opened by simonsan 0
  • `generate`: Bundle a `config` template

    `generate`: Bundle a `config` template

    I think it would be beneficial to bundle a fully annotated config template in the compiled binary with:

    pub static CONFIG_TEMPLATE_TOML: &str = include_str!("config/config_template.toml");
    

    and let users generate a template config with rustic generate config > <path_to_future_config> or rustic generate config -o <path_to_future_config>, if -o/--output is given by the user.

    Besides, I think it would be beneficial, as well, to package and ship rustic with the /config/ folder and maybe unpack it on installation to ~/.config/rustic/examples.

    A-packaging C-enhancement 
    opened by simonsan 0
Releases(0.1.1)
  • 0.1.1(Sep 18, 2023)

    [0.1.1] - 2023-09-18

    Bug Fixes

    • Correct glob-matching for relative paths (#783)

    Documentation

    • Update Readme layout, move docs to separate repository, add rustic_core Readme.md (#820)
    • Add rustic_core logo
    • Set subtitle for rustic_core readme
    • Fix item links in documentation for rustic_core
    • Pass "--document-private-items" to rustdoc via metadata in manifest

    Features

    • Option to disable requiring git repository for git-ignore rules
    • Wait for password-command to exit
    • Add --json option to forget command (#806)

    Miscellaneous Tasks

    • Lint markdown with dprint, run initial dprint fmt (#830)
    • Lint has been removed
    • Add cliff.toml and generate rustic_core changelog
    • Add documentation field to rustic_core manifest
    • Relink to new image location

    Refactor

    • Replace nom with shellwords to split strings (#752)
    • Add metadata to crate manifests (#822)

    Build

    • Bump public-api from 0.29.1 to 0.31.2 (#695)
    • Bump public-api from 0.31.2 to 0.31.3 (#796)
    • Bump rustdoc-json from 0.8.6 to 0.8.7 (#794)

    Prune

    • Add example using rustic_core
    • Don't abort if time is unset for pack-to-delete

    Repoinfo

    • Add options --json, --only-files, --only-index

    Rest/rclone

    • Make # of retries cusomizable and use sensible default

    Restore

    • Download multiple contiguous blobs in one request

    Rustic_core

    • Add NoProgress and NoProgressBars (e.g. for examples)
    Source code(tar.gz)
    Source code(zip)
Owner
rustic
fast, encrypted, and deduplicated backups
rustic
Bijou is a tiny yet fast encrypted file system.

Bijou ✨??✨ Bijou (['bi:ʒu], French for "jewel") is a tiny yet fast encrypted filesystem, built upon RocksDB. Bijou provides a FUSE interface, as well

Mivik 5 Sep 27, 2023
a handy utility to work with encrypted DMGs

edmgutil edmgutil is a simple wrapper utility to hdiutil to help you work with disposable, encrypted DMGs. It can decompress an encrypted ZIP into a n

Sentry 9 Nov 29, 2022
Encrypted memories

Diary - Encrypted memories Diary is a TUI program written in Rust for GNU/Linux / *BSD / Android (It probably works on other platforms too, but who ca

Arun Sojan Parolikkal 44 Dec 23, 2022
An application for creating encrypted vaults for the GNOME desktop.

Vaults An application for creating encrypted vaults for the GNOME desktop. It currently uses gocryptfs and CryFS for encryption. Please always keep a

Martin Pobaschnig 51 Dec 17, 2022
age-encrypted secrets for NixOS; drop-in replacement for agenix

ragenix ragenix provides age-encrypted secrets for NixOS systems which live in the Nix store and are decrypted on system activation. Using ragenix to

YAXI 91 Jan 8, 2023
An encrypted multi client messaging system written in pure Rust

?? Preamble This is a pure Rust multi-client encrypted messaging system, also known as Edode's Secured Messaging System. It is an end-to-end(s) commun

Edode 3 Sep 16, 2022
Dione is an anonymize and encrypted messaging system build on top on a peer to peer layer.

Secure and Anonymous Messaging WARNING: Currently Dione is not ready to be used nor does it fulfill its goal of being an anonymous messenger. In order

Dione 41 Jan 5, 2023
NymDrive is a complete, end-to-end encrypted file syncing daemon that runs over the Nym network.

NymDrive NymDrive is a complete, end-to-end encrypted file syncing daemon that runs over the Nym network. Features Active file monitoring of changes i

Hans Bricks 16 Jul 12, 2022
An open source desktop wallet for nano and banano with end-to-end encrypted, on chain messaging using the dagchat protocol.

An open source wallet with end-to-end encrypted, on chain messaging for nano and banano using the dagchat protocol.

derfarctor 22 Nov 6, 2022
Smarter brute-force password searching for PKZIP encrypted files

Zip Blitz Motivation This program was created for a very specfic problem I had. I had a large encrypted zip file that I lost/forgot the password for.

Michael 4 Jul 29, 2022
An HTTP proxy for assets (mainly images) to route requests through an always-encrypted connection.

camo-rs camo-rs is a frontend-compatible Rust-re-implementation of the now archived NodeJS-based atmos/camo - an HTTP proxy for assets (mainly images)

Dennis Schubert 7 Dec 8, 2022
Program to determine the password of an encrypted ZIP file via dictionary attack.

zip-dict-attack Program to determine the password of an encrypted ZIP file via dictionary attack. Inspired by this article. Usage Cargo is used to bui

null 2 Oct 8, 2022
A simple key-value store with a log-structured, append-only storage architecture where data is encrypted with AES GCM.

akvdb A simple key-value store with a log-structured, append-only storage architecture where data is encrypted with AES GCM. Modified from the actionk

Olle W 3 Oct 10, 2022
Koofr Vault is an open-source, client-side encrypted folder for your Koofr cloud storage offering an extra layer of security for your most sensitive files.

Koofr Vault https://vault.koofr.net Koofr Vault is an open-source, client-side encrypted folder for your Koofr cloud storage offering an extra layer o

Koofr 12 Dec 30, 2022
Sodium Oxide: Fast cryptographic library for Rust (bindings to libsodium)

sodiumoxide |Crate|Documentation|Gitter| |:---:|:-----------:|:--------:|:-----:|:------:|:----:| |||| NaCl (pronounced "salt") is a new easy-to-use h

sodiumoxide 642 Dec 17, 2022
miners is a fast Rust library for the Maximal Information-based Nonparametric Exploration (MIC and MINE family)

miners miners is a fast Rust library for the Maximal Information-based Nonparametric Exploration (MIC and MINE family). miners using rayon and vectori

CuteSocks 7 Nov 2, 2023
High-level networking library that extends the bevy_replicon library to allow snapshot interpolation and client-side prediction

bevy_replicon_snap A Snapshot Interpolation plugin for the networking solution bevy_replicon in the Bevy game engine. This library is a very rough pro

Ben 3 Oct 15, 2023
The fast, light, and robust client for the Ethereum mainnet.

OpenEthereum Fast and feature-rich multi-network Ethereum client. » Download the latest release « Table of Contents Description Technical Overview Bui

OpenEthereum 1.6k Dec 28, 2022
Safe, fast, small crypto using Rust

THE SOFTWARE IS PROVIDED "AS IS" AND BRIAN SMITH AND THE AUTHORS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES

Brian Smith 3k Jan 2, 2023