s3d is an S3 daemon for the Edge written in Rust
The goal of
s3d is to provide a daemon for edge platforms (e.g. factory servers
Since edge applications operate in subideal environments (high latency network, disconnections, hardware failures, eavesdropping, ...),
s3d aims to make the applications run smoothly while it handles the flows of data, security, networking, local capacity, metadata caching, write queueing, etc.
Worth mentioning that the choice of the Rust language is a natural fit for the edge systems, as it is a modern language with a focus on functionality, safety and performance.
s3d builds with the rust toolchain into a single binary that loads a yaml config file, which makes it easy to set up and configure in standalone linux or containerized environments like Kubernetes.
This project is still
This means it's a great time to affect its direction!
If you find it interesting or want to contribute, please feel free to communicate using these options:
- Discord Chat - join to discussion channels
- Github Issues - for bugs/questions/disccussions/suggestions
- Github PR's - add features/fixes/improvements (prefer to have an issue open first)
- Github Projects - track roadmap/progress
- License - Apache-2.0
To start using
s3d you need to install it, run it, and configure your data workflows.
The following steps illustrate a basic flow of s3d usage:
# Install the s3d binary using Rust toolchain cargo install s3d # Sets up the configuration, hub connection, and local storage s3d init # Run the daemon to serve S3 clients can connect to read and write objects # Start and stop will spawn/kill the daemon in the background s3d run s3d start s3d stop # Fetch reads metadata from the hub and stores locally # This includes the list of objects and their metadata, but excludes objects contents s3d fetch [bucket/prefix] # Pull is like fetch but includes objects contents. s3d pull [bucket/prefix] # Simple access to objects from the CLI s3d get bucket/key > file s3d put bucket/key < file s3d ls [bucket/prefix] # Show local changes not pushed to the hub s3d diff [bucket/prefix] # Show bucket status and local store stats (objects, sizes, etc). s3d status [bucket/prefix] # Push bucket changes (merge by last modified time). s3d push [bucket/prefix] # Remove objects from local store based on age s3d prune [bucket/prefix]