Fast, deduplicated content and database seeding for WordPress

Tom Lawton

Last update: Feb 22, 2024

Related tags

Command-line rust wordpress content s3 rclone seeding restic opendal rustic-rs

Overview

Sprout

Fast, deduplicated content and database seeding for WordPress.

Documentation | Install | Releases

Store your uploads and database in a secure, central location and easily let your team get up and running with the correct version of content with a one-liner - sprout seed.
Sprout lets you commit a small sprout.yaml file as part of your WordPress project. It identifies the exact version of database and wp-uploads content required in order to seed a new environment.
You can store your database and content in a Sprout repo - this can be a local path, SFTP, S3 bucket, or any number of supported backends via OpenDAL or rclone. Have complete control over your data.
Your data is encrypted and deduplicated, and stored in the proven and trusted Restic format. So much more efficient than chucking ZIPs around.
Sprout is FAST. Written in Rust, and provided as a single static binary, it will run anywhere.
Sprout will locally stash your current database and wp-uploads during destructive operations - a nice safety net to have!

⚠️ Although I think you'll love it, Sprout is pre-release. You should always back up your WordPress content and database before using Sprout.

Installation

Binaries are avalailable for Mac and Linux on both x86_64 and arm64. You can get them via the releases page.

Mac

# x86_64 / Intel Macs
curl -L "https://github.com/talss89/sprout/releases/latest/download/sprout-macos-x86_64.tar.gz" | tar zxf - && sudo install -c -m 0755 sprout /usr/local/bin

# aarch64 / M1 Macs
curl -L "https://github.com/talss89/sprout/releases/latest/download/sprout-macos-aarch64.tar.gz" | tar zxf - && sudo install -c -m 0755 sprout /usr/local/bin

Linux

# x86_64
curl -L "https://github.com/talss89/sprout/releases/latest/download/sprout-linux-x86_64.tar.gz" | tar zxf - && sudo install -c -m 0755 sprout /usr/local/bin

# aarch64 / amd64
curl -L "https://github.com/talss89/sprout/releases/latest/download/sprout-linux-aarch64.tar.gz" | tar zxf - && sudo install -c -m 0755 sprout /usr/local/bin

Windows support is untested, and will require compilation with cargo.

What's going on behind the scenes?

Sprout stores your data in the Restic Repo Format, and uses rustic-rs/rustic_core internally. We dump the database via WP CLI, and replace the site URL with a placeholder value before storing it alongside an encrypted and de-duplicated archive of your wp-uploads folder.

Sprout rewrites the Restic hostname and path properties of each snapshot so your team can snap and seed seamlessly. Content branches are implemented as a virtual directory inside a snapshot.

Comments

Feedback for rustic_core (DX)

Hey there, as you are an early user of rustic_core, we would like to understand better how the API of rustic_core fares you. Do you have any feedback, do you miss anything? :-)

opened by simonsan 1
bug: `sprout snap ` does not correctly push to new branch
sprout snap my-branch appears to push to the previously set branch, not the new one.

on main

sprout snap my-branch creates snapshot for main

sprout snap my-other-branch creates snapshot for my-branch

sprout snap main creates snapshot for my-other-branch

bug
opened by talss89 0
test: integration test suite
This, on the face of it, seems complex due to the need to interact with WP-CLI, WP Database, and Git.

[x] Declare a Fact trait which we can use as an interface between a project and WP-CLI + Git. This could solve the problem of setting up a full WP environment.

[x] Build a module to build separate test contexts to include a SPROUT_HOME and local repo

[x] Provide fake WordPress uploads content to snapshot

enhancement
opened by talss89 0
docs: installation, guides, reference docs
[x] Clap inline docs - review

[x] docs: installation instructions

[ ] readme: point to proper docs

[ ] guide: S3

[ ] guide: Setting up a repo

[ ] guide: Snapshotting a project

[ ] guide: Using branches

[ ] guide: Seeding from a snapshot

[ ] guide: Environment vars in repo definition

[ ] docs: how to use / what is it

documentation
opened by talss89 0
feature: `sprout init` should attempt to find the correct wp-uploads path

To reduce friction when initialising a new project, we could attempt to determine the correct wp-uploads path instead of a dumb default.

All of the logic is present and written, but cold on main.

If wp core is-installed fails, we just warn and revert to the default path.
enhancement

opened by talss89 0
feature: label and describe snapshots
[x] Optionally label a snapshot with text.

[x] Optionally save a description.

[x] Show label in ls table

[ ] Create a new command to show snapshot information sprout describe <id>, including description

enhancement
opened by talss89 0
docs: planning and improvements
The docs are lacking at the moment! This issue tracks all of the things we'd like to include.

[ ] reference: internals (sprt_ tagging, uniqueness digest, branch parents)

[ ] guide: repo management / maintenance

[ ] guide: stashes

[ ] guide: branching

[ ] reference: sprout.yaml format

[ ] guide: using onedrive

[ ] guide: using dropbox

[ ] guide: using google drive

[ ] guide: using restic http server

[ ] guide: sftp

documentation
opened by talss89 0
build: reliance on upstream patch for `zstd`

To fix the linux-aarch64-musl build, a patch was applied to zstd-sys.

PR for fix is open. https://github.com/gyscos/zstd-rs/pull/262

Revert the cargo.toml patch for zstd-sys once this is released.
upstream

opened by talss89 0
roadmap: path to 1.0.0
This issue serves as a planning scratchpad to bring Sprout up to a stable 1.0.0 release.

<0.1.1: For next pre-release

???

<1.0.0: For stable

[x] #19

[ ] build: windows

[ ] feature: installer for windows

Backlog / Ideas(?)

feature?: repo access token management

feature?: share repo definitions

feature?: punch-in for user passwords

~feature?: DB anonymisation~ - out of scope

Wider discussion

Although I made this for WordPress, there's no reason why this approach wouldn't work for other software projects that depend on large content storage.

There are some specialisations for WordPress (DB search replace on site URL for example) that would need to be abstracted away. Other types of project would likely have similar specifics.

Is it worth planning to turn Sprout into a general content seeding and storage tool?

And if so, at what point do we do that?
opened by talss89 0

Releases(v0.1.2)

v0.1.2(Feb 23, 2024)
📦 Self-update via sprout update

✨ UI enhancements (show active snapshot ID, general tidy)

🛒 Checkout a specific snapshot ID via sprout seed <id>

What's Changed

minor: ls table tidy by @talss89 in https://github.com/talss89/sprout/pull/30

feat: self update sprout update by @talss89 in https://github.com/talss89/sprout/pull/33

Full Changelog: https://github.com/talss89/sprout/compare/v0.1.1...v0.1.2
Source code(tar.gz)
Source code(zip)
sprout-linux-aarch64.tar.gz(7.49 MB)
sprout-linux-x86_64.tar.gz(7.86 MB)
sprout-macos-aarch64.tar.gz(5.11 MB)
sprout-macos-x86_64.tar.gz(5.69 MB)
v0.1.1(Feb 19, 2024)
🩹 Bugfix for branch push / restore

What's Changed

fix: sprout snap branch does not correctly push to new branch by @talss89 in https://github.com/talss89/sprout/pull/29

Full Changelog: https://github.com/talss89/sprout/compare/v0.1.0...v0.1.1
Source code(tar.gz)
Source code(zip)
sprout-linux-aarch64.tar.gz(9.67 MB)
sprout-linux-x86_64.tar.gz(7.49 MB)
sprout-macos-aarch64.tar.gz(6.57 MB)
sprout-macos-x86_64.tar.gz(7.01 MB)
v0.1.0(Feb 19, 2024)
✨ Improved snapshot fault tolerance and UI details and presentation

⚠️ Breaking Changes

v0.0.3 and older local and remote snapshots will not be found due to a new bundling format

Your repo definition access_key is now repo_key

What's Changed

refactor(test): integration test suite by @talss89 in https://github.com/talss89/sprout/pull/20

test: improve integration coverage by @talss89 in https://github.com/talss89/sprout/pull/21

fix: #23 improve error messages when in invalid project by @talss89 in https://github.com/talss89/sprout/pull/24

fix!: rename access key to repo key to avoid confusion between s3 service credentials by @talss89 in https://github.com/talss89/sprout/pull/25

feat!: merge snapshots into single by @talss89 in https://github.com/talss89/sprout/pull/26

minor: cli styling by @talss89 in https://github.com/talss89/sprout/pull/27

Full Changelog: https://github.com/talss89/sprout/compare/v0.0.3...v0.1.0
Source code(tar.gz)
Source code(zip)
sprout-linux-aarch64.tar.gz(5.41 MB)
sprout-linux-x86_64.tar.gz(6.00 MB)
sprout-macos-aarch64.tar.gz(4.98 MB)
sprout-macos-x86_64.tar.gz(5.55 MB)
v0.0.3(Feb 18, 2024)
✨ We have docs! Check them out here.

🗣️ Feedback is welcome

✨ List and drop your stashes with sprout stash list and sprout stash drop

✨ List your snapshots with sprout ls

💻 Environment variables for your repo secrets (S3 etc.)

💄 General UI improvements, more information

New 0.0.3 demo:

What's Changed

refactor: split progressbar into module by @talss89 in https://github.com/talss89/sprout/pull/6

refactor: major restructure of concerns by @talss89 in https://github.com/talss89/sprout/pull/7

feature: stash list and drop commands by @talss89 in https://github.com/talss89/sprout/pull/8

docs: starlight, auto-generate from clap by @talss89 in https://github.com/talss89/sprout/pull/9

feature: #11 env var substitution in repo definitions by @talss89 in https://github.com/talss89/sprout/pull/12

feature: show sizes and list snapshots (sprout ls) by @talss89 in https://github.com/talss89/sprout/pull/14

docs: first pass by @talss89 in https://github.com/talss89/sprout/pull/17

Full Changelog: https://github.com/talss89/sprout/compare/v0.0.2...v0.0.3
Source code(tar.gz)
Source code(zip)
sprout-linux-aarch64.tar.gz(5.40 MB)
sprout-linux-x86_64.tar.gz(5.98 MB)
sprout-macos-aarch64.tar.gz(4.97 MB)
sprout-macos-x86_64.tar.gz(5.55 MB)
v0.0.2(Feb 8, 2024)
Many improvements in workflow and usability. Now in use on some personal projects.

What's Changed

feat: sprout repo commands by @talss89 in https://github.com/talss89/sprout/pull/4

feat: sensible defaults when initialising by @talss89 in https://github.com/talss89/sprout/pull/5

feat: work with S3 and other remote backends

feat: locally linear parent snapshot histories

feat: automatic access_key generation and management

perf: optimised and reduced binary size

New Contributors

@talss89 made their first contribution in https://github.com/talss89/sprout/pull/4

Full Changelog: https://github.com/talss89/sprout/compare/v0.0.1...v0.0.2
Source code(tar.gz)
Source code(zip)
sprout-linux-aarch64.tar.gz(5.27 MB)
sprout-linux-x86_64.tar.gz(5.83 MB)
sprout-macos-aarch64.tar.gz(4.84 MB)
sprout-macos-x86_64.tar.gz(5.42 MB)
v0.0.1(Feb 5, 2024)

Full Changelog: https://github.com/talss89/sprout/commits/v0.0.1
Source code(tar.gz)
Source code(zip)
sprout-linux-aarch64.tar.gz(9.49 MB)
sprout-linux-x86_64.tar.gz(7.25 MB)
sprout-macos-aarch64.tar.gz(6.36 MB)
sprout-macos-x86_64.tar.gz(6.81 MB)