Rust Distributed-storage Resources

Paxakos is a pure Rust implementation of a distributed consensus algorithm based on Leslie Lamport's Paxos. It enables distributed systems to consistently modify shared state across their network, even in the presence of failures.

elfshaker is a low-footprint, high-performance version control system fine-tuned for binaries.

PERSIA (Parallel rEcommendation tRaining System with hybrId Acceleration) is developed by AI platform@Kuaishou Technology, collaborating with ETH. It

ROAPI ROAPI automatically spins up read-only APIs for static datasets without requiring you to write a single line of code. It builds on top of Apache

storage A sessioned Merkle key/value store The crate was designed to be the blockchain state database. It provides persistent storage layer for key-va

WRaft: Raft in WebAssembly What is this? A toy implementation of the Raft Consensus Algorithm for WebAssembly, written in Rust. Basically, it synchron

Rust Web Local Storage API A Rust implementation of the Web LocalStorage API, for use in non-browser contexts About the Web Local Storage API MDN docs

Library to call Filen.io API from Rust Filen.io is a cloud storage provider with an open-source desktop client. My goal is to write a library which ca

gcs-rsync Lightweight and efficient Rust gcs rsync for Google Cloud Storage. gcs-sync is faster than gsutil rsync when files change a lot while perfor

MadRaft The labs of Raft consensus algorithm based on MadSim. Some codes are derived from MIT 6.824 and PingCAP Talent Plan: Raft Lab. Thanks for thei

Zenith substitutes PostgreSQL storage layer and redistributes data across a cluster of nodes

Hold My Backup It is a backup tool that creates backups and stores them on an object storage. By default it uses minio but you can use AWS: S3 as well

Forage Forage is for Storage Remote storage: Open storage channels to a remote storage provider over Tor Lightweight: Platform-optimized using Blake3-

Little Raft The lightest distributed consensus library. Run your own replicated state machine! ❤️ Installing Simply import the crate. In your Cargo.to

FS-DKR: One Round Distributed Key Rotation Intro In this note we aim to re-purpose the Fouque-Stern Distributed Key Generation (DKG) to support a secu

Engula is a storage engine that empowers engineers to build reliable and cost-effective databases with less effort and more confidence. Engula is in t

MadSim Magical Automatic Deterministic Simulator for distributed systems. Deterministic simulation MadSim is a Rust async runtime similar to tokio, bu

Skytable is an effort to provide the best of key/value stores, document stores and columnar databases, that is, simplicity, flexibility and queryability at scale. The name 'Skytable' exemplifies our vision to create a database that has limitless possibilities. Skytable was previously known as TerrabaseDB (and then Skybase) and is also nicknamed "STable", "Sky" and "SDB" by the community.

A lightweight and distributed task scheduling platform written in rust.

Dorea DB 🛰 Dorea is a key-value data storage system. It is based on the Bitcask storage model Documentation | Crates.io | API Doucment 简体中文 | English

ForestDB is a key-value storage engine developed by Couchbase Caching and Storage Team, and its main index structure is built from Hierarchic

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values. Authors: Sanjay Ghem

secrets secrets is a library to help Rust programmers safely held cryptographic secrets in memory. It is mostly an ergonomic wrapper around the memory

xaynet Xaynet: Train on the Edge with Federated Learning Want a framework that supports federated learning on the edge, in desktop browsers, integrate

Datafuse Modern Real-Time Data Processing & Analytics DBMS with Cloud-Native Architecture Datafuse is a Real-Time Data Processing & Analytics DBMS wit

A model checker for implementing distributed systems.

Client-side-validation Foundation Libraries This is an implementation defining standard of client-side-validation representing a set of its Foundation

Rustpad is an efficient and minimal open-source collaborative text editor based on the operational transformation algorithm

Hitbox is an asynchronous caching framework supporting multiple backends and suitable for distributed and for single-machine applications.

RiteRaft - A raft framework, for regular people This is an attempt to create a layer on top of tikv/raft-rs, that is easier to use and implement. This

Quickwit This repository will host Quickwit, the big data search engine developed by Quickwit Inc. We will progressively polish and opensource our cod

Holochain License: This repository contains the core Holochain libraries and binaries. This is the most recent and well maintained version of Holochai

Fluvio is a high-performance distributed streaming platform that's written in Rust, built to make it easy to develop real-time applications.

Hecate OpenStreetMap Inspired Data Storage Backend Focused on Performance and GeoJSON Interchange Hecate Feature Comparison Feature Hecate ESRI MapSer

xaynet Xaynet: Train on the Edge with Federated Learning Want a framework that supports federated learning on the edge, in desktop browsers, integrate

Ballista: Distributed Compute Platform Overview Ballista is a distributed compute platform primarily implemented in Rust, powered by Apache Arrow. It

FeoBlog FeoBlog is a distributed blogging platform. It takes a lot of its inspiration from Mastodon and Scuttlebutt. It aims to solve a couple of prob

rusty-workers A cloud-native distributed serverless workers platform. Features JavaScript and WebAssembly engine powered by V8 Fetch API Highly scalab

storage-poc aims at exploring the usage of custom Storages, rather than custom Allocators. Goals This is a Proof-of-Concept aiming at: Demonstrating t

hashfs A file storage service How to use it? Start up the service at terminal # You can specify the storage root path and access domain when you start

Aggregatable DKG and VUF WARNING: this code should not be used in production! Implementation of Aggregatable Distributed Key Generation, a distributed

clepsydra Overview This is a work-in-progress implementation of a core protocol for a minimalist distributed database. It strives to be as small and s

ZboxFS ZboxFS is a zero-details, privacy-focused in-app file system. Its goal is to help application store files securely, privately and reliably. By

Themis provides strong, usable cryptography for busy people General purpose cryptographic library for storage and messaging for iOS (Swift, Obj-C), An

sccache - Shared Compilation Cache sccache is a ccache-like compiler caching tool. It is used as a compiler wrapper and avoids compilation when possib

Concurr: Distributed and Concurrent Command Execution, in Rust This project is dual licensed under MIT and Apache 2.0. Originally inspired by the GNU

Lethe A secure, free, cross-platform and open-source drive wiping utility. Should work with any HDD, SSD (read limitations) and flash drives. The usua

Website | Documentation | Community Chat TiKV is an open-source, distributed, and transactional key-value database. Unlike other traditional NoSQL sys

Skybase The next-generation NoSQL database What is Skybase? Skybase (or SkybaseDB/SDB) is an effort to provide the best of key/value stores, document

PumpkinDB Build status (Linux) Build status (Windows) Project status Usable, between alpha and beta Production-readiness Depends on your risk toleranc

About Lucid KV High performance and distributed KV store w/ REST API. 🦀 Introduction Lucid is an high performance, secure and distributed key-value s

Materialize is a streaming database for real-time applications. Get started Check out our getting started guide. About Materialize lets you ask questi

Holochain License: This repository contains the core Holochain libraries and binaries. This is the most recent and well maintained version of Holochai

Rudr 🚨 NOTE: Rudr is deprecated in favor of the upcoming open application platform project as its successor. There are no plans to produce future rel

Ballista: Distributed Compute Platform Overview Ballista is a distributed compute platform primarily implemented in Rust, powered by Apache Arrow. It