Fast, deduplicated content and database seeding for WordPress

Related tags

Command-line rust wordpress content s3 rclone seeding restic opendal rustic-rs
Overview

Sprout

Fast, deduplicated content and database seeding for WordPress.

Documentation | Install | Releases

GitHub Release Documentation Build Test Status

  • Store your uploads and database in a secure, central location and easily let your team get up and running with the correct version of content with a one-liner - sprout seed.

  • Sprout lets you commit a small sprout.yaml file as part of your WordPress project. It identifies the exact version of database and wp-uploads content required in order to seed a new environment.

  • You can store your database and content in a Sprout repo - this can be a local path, SFTP, S3 bucket, or any number of supported backends via OpenDAL or rclone. Have complete control over your data.

  • Your data is encrypted and deduplicated, and stored in the proven and trusted Restic format. So much more efficient than chucking ZIPs around.

  • Sprout is FAST. Written in Rust, and provided as a single static binary, it will run anywhere.

  • Sprout will locally stash your current database and wp-uploads during destructive operations - a nice safety net to have!

⚠️ Although I think you'll love it, Sprout is pre-release. You should always back up your WordPress content and database before using Sprout.

asciicast

Installation

Binaries are avalailable for Mac and Linux on both x86_64 and arm64. You can get them via the releases page.

Mac

# x86_64 / Intel Macs
curl -L "https://github.com/talss89/sprout/releases/latest/download/sprout-macos-x86_64.tar.gz" | tar zxf - && sudo install -c -m 0755 sprout /usr/local/bin

# aarch64 / M1 Macs
curl -L "https://github.com/talss89/sprout/releases/latest/download/sprout-macos-aarch64.tar.gz" | tar zxf - && sudo install -c -m 0755 sprout /usr/local/bin

Linux

# x86_64
curl -L "https://github.com/talss89/sprout/releases/latest/download/sprout-linux-x86_64.tar.gz" | tar zxf - && sudo install -c -m 0755 sprout /usr/local/bin

# aarch64 / amd64
curl -L "https://github.com/talss89/sprout/releases/latest/download/sprout-linux-aarch64.tar.gz" | tar zxf - && sudo install -c -m 0755 sprout /usr/local/bin

Windows support is untested, and will require compilation with cargo.

What's going on behind the scenes?

Sprout stores your data in the Restic Repo Format, and uses rustic-rs/rustic_core internally. We dump the database via WP CLI, and replace the site URL with a placeholder value before storing it alongside an encrypted and de-duplicated archive of your wp-uploads folder.

Sprout rewrites the Restic hostname and path properties of each snapshot so your team can snap and seed seamlessly. Content branches are implemented as a virtual directory inside a snapshot.

Comments
  • Feedback for rustic_core (DX)

    Feedback for rustic_core (DX)

    Hey there, as you are an early user of rustic_core, we would like to understand better how the API of rustic_core fares you. Do you have any feedback, do you miss anything? :-)

    opened by simonsan 1
  • bug: `sprout snap <branch>` does not correctly push to new branch

    bug: `sprout snap ` does not correctly push to new branch

    sprout snap my-branch appears to push to the previously set branch, not the new one.

    • on main
    • sprout snap my-branch creates snapshot for main
    • sprout snap my-other-branch creates snapshot for my-branch
    • sprout snap main creates snapshot for my-other-branch
    bug 
    opened by talss89 0
  • test: integration test suite

    test: integration test suite

    This, on the face of it, seems complex due to the need to interact with WP-CLI, WP Database, and Git.

    • [x] Declare a Fact trait which we can use as an interface between a project and WP-CLI + Git. This could solve the problem of setting up a full WP environment.
    • [x] Build a module to build separate test contexts to include a SPROUT_HOME and local repo
    • [x] Provide fake WordPress uploads content to snapshot
    enhancement 
    opened by talss89 0
  • docs: installation, guides, reference docs

    docs: installation, guides, reference docs

    • [x] Clap inline docs - review
    • [x] docs: installation instructions
    • [ ] readme: point to proper docs
    • [ ] guide: S3
    • [ ] guide: Setting up a repo
    • [ ] guide: Snapshotting a project
    • [ ] guide: Using branches
    • [ ] guide: Seeding from a snapshot
    • [ ] guide: Environment vars in repo definition
    • [ ] docs: how to use / what is it
    documentation 
    opened by talss89 0
  • feature: `sprout init` should attempt to find the correct wp-uploads path

    feature: `sprout init` should attempt to find the correct wp-uploads path

    To reduce friction when initialising a new project, we could attempt to determine the correct wp-uploads path instead of a dumb default.

    All of the logic is present and written, but cold on main.

    If wp core is-installed fails, we just warn and revert to the default path.

    enhancement 
    opened by talss89 0
  • feature: label and describe snapshots

    feature: label and describe snapshots

    • [x] Optionally label a snapshot with text.
    • [x] Optionally save a description.
    • [x] Show label in ls table
    • [ ] Create a new command to show snapshot information sprout describe <id>, including description
    enhancement 
    opened by talss89 0
  • docs: planning and improvements

    docs: planning and improvements

    The docs are lacking at the moment! This issue tracks all of the things we'd like to include.

    • [ ] reference: internals (sprt_ tagging, uniqueness digest, branch parents)
    • [ ] guide: repo management / maintenance
    • [ ] guide: stashes
    • [ ] guide: branching
    • [ ] reference: sprout.yaml format
    • [ ] guide: using onedrive
    • [ ] guide: using dropbox
    • [ ] guide: using google drive
    • [ ] guide: using restic http server
    • [ ] guide: sftp
    documentation 
    opened by talss89 0
  • build: reliance on upstream patch for `zstd`

    build: reliance on upstream patch for `zstd`

    To fix the linux-aarch64-musl build, a patch was applied to zstd-sys.

    PR for fix is open. https://github.com/gyscos/zstd-rs/pull/262

    Revert the cargo.toml patch for zstd-sys once this is released.

    upstream 
    opened by talss89 0
  • roadmap: path to 1.0.0

    roadmap: path to 1.0.0

    This issue serves as a planning scratchpad to bring Sprout up to a stable 1.0.0 release.

    <0.1.1: For next pre-release

    ???

    <1.0.0: For stable

    • [x] #19
    • [ ] build: windows
    • [ ] feature: installer for windows

    Backlog / Ideas(?)

    • feature?: repo access token management
    • feature?: share repo definitions
    • feature?: punch-in for user passwords
    • ~feature?: DB anonymisation~ - out of scope

    Wider discussion

    Although I made this for WordPress, there's no reason why this approach wouldn't work for other software projects that depend on large content storage.

    There are some specialisations for WordPress (DB search replace on site URL for example) that would need to be abstracted away. Other types of project would likely have similar specifics.

    Is it worth planning to turn Sprout into a general content seeding and storage tool?

    And if so, at what point do we do that?

    opened by talss89 0
Releases(v0.1.2)
Owner
Tom Lawton
Tom Lawton
GitHub
Rust SDK for the core C2PA (Coalition for Content Provenance and Authenticity) specification

C2PA Rust SDK The Coalition for Content Provenance and Authenticity (C2PA) addresses the prevalence of misleading information online through the devel

Content Authenticity Initiative 46 Jun 30, 2023
βš‘πŸš€ Content Delivery Network written in Rustlang, optimized for speed and latency.

Supported Formats HTML Javscript Css Image PNG JPG JPEG GIF SVG Video MP4 WEBM FLV Audio OGG ACC MP3 Archives ZIP RAR Feeds & Data JSON YAML XML Docum

Noname 3 Apr 9, 2024
:large_orange_diamond: Build beautiful terminal tables with automatic content wrapping

Comfy-table Comfy-table tries to provide utility for building beautiful tables, while being easy to use. Features: Dynamic arrangement of content to a

Arne Beer 525 Jan 8, 2023
A simple CLI tool for converting CSV file content to JSON.

fast-csv-to-json A simple CLI tool for converting CSV file content to JSON. ζˆ‘θŠ±δΊ†δΈ€ε€‹ε°ζ™‚ζ“ε‡ΊδΎ†οΌŒζŽ₯θ‘—ε„ͺεŒ–δΊ†ε…©ε€©ηš„εΏ«ι€Ÿ CSV 轉 JSON CLI 小ε·₯ε…· Installation Install Rust with ru

Ming Chang 3 Apr 5, 2023
Tiny CLI tool that helps to visualize iCal file content in the terminal.

Calio Calio is a tiny CLI tool that helps to visualize iCal file in the terminal. Installation You can either install it via cargo or download the bin

Oscar Cortez 5 Jun 12, 2023
This is an example Nostr rust project to enable '402 Payment Required' responses for requests to paid content.

Nostr Paywall Example This is an example Nostr rust project to enable 402 Payment Required responses for requests to paid content. To prove payment, a

Blake Jakopovic 6 May 6, 2023
Shared k-mer content between two genomes

skc skc is a simple tool for finding shared k-mer content between two genomes. Installation Prebuilt binary curl -sSL skc.mbh.sh | sh # or with wget w

Michael Hall 16 Jun 26, 2023
A Content Discovery Tool insipired from Feroxbuster. Work In Progress

monologue A Content Discovery Tool written in Rust, insipired from Feroxbuster. Installation Dependencies OpenSSL (If You are on linux). Rust programm

Voyage 3 Jul 20, 2023
Show HTML content "inside" your egui rendered application

hframe Show HTML content "inside" your egui rendered application. "hframe" stands for "HTML Frame". Note: hframe only works when the application is co

Franco Profeti 3 Feb 26, 2024
A general-purpose, transactional, relational database that uses Datalog and focuses on graph data and algorithms

cozo A general-purpose, transactional, relational database that uses Datalog for query and focuses on graph data and algorithms. Features Relational d

null 1.9k Jan 9, 2023
🐒 Atuin replaces your existing shell history with a SQLite database, and records additional context for your commands

Atuin replaces your existing shell history with a SQLite database, and records additional context for your commands. Additionally, it provides optional and fully encrypted synchronisation of your history between machines, via an Atuin server.

Ellie Huxtable 4.6k Jan 1, 2023
PyO3 bindings and Python interface to skani, a method for fast fast genomic identity calculation using sparse chaining.

?? ⛓️ ?? Pyskani PyO3 bindings and Python interface to skani, a method for fast fast genomic identity calculation using sparse chaining. ??️ Overview

Martin Larralde 13 Mar 21, 2023
Download your Stripe account to a SQLite database.

tdog tdog is a CLI to download your Stripe account to a SQLite database. table.dog Sponsor development Install # macOS: curl -L --output /usr/local/bi

Table Dog 16 Oct 10, 2022
Integrate Mcfly with fzf to combine a solid command history database with a widely-loved fuzzy search UI

McFly fzf integration Integrate McFly with fzf to combine a solid command history database with a widely-loved fuzzy search UI Features: Advanced hist

null 11 Jan 25, 2023
A utility for exporting administrative/moderation statistics from your Lemmy instance's PostgreSQL database to InfluxDB!

Lemmy (Stats) Data Exporter About This Project This project aims to act as a bridge between Lemmy's PostgreSQL database and InfluxDB, primarily to tra

Russell 3 Jul 5, 2023
A lightweight, embedded key-value database for mobile clients (i.e., iOS, Android), written in Rust.

A lightweight, embedded key-value database for mobile clients (i.e., iOS, Android), written in Rust. ⚠️ Still in testing, not yet ready for production

Tangent 6 Sep 25, 2023
A tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK.

FileQL - File Query Language FileQL is a tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK. Sampl

Amr Hesham 39 Mar 12, 2024
A blazingly fast rust-based bionic reader for blazingly fast reading within a terminal console πŸ¦€

This Rust-based CLI tool reads text and returns it back in bionic reading format for blazingly fast loading and even faster reading! Bionic reading is

Ismet Handzic 5 Aug 5, 2023
Warp is a blazingly fast, Rust-based terminal that makes you and your team more productive at running, debugging, and deploying code and infrastructure.

Warp is a blazingly fast, Rust-based terminal that makes you and your team more productive at running, debugging, and deploying code and infrastructure.

Warp 10.4k Jan 4, 2023