Rust Distributed-storage Resources

VelarixDB is an LSM-based storage engine designed to significantly reduce IO amplification, resulting in better performance and durability for storage

Solidity-Gas-Optimizoor An high performance automated CLI tool that optimizes gas usage in Solidity smart contracts, focusing on storage and function

Table of Contents Overview Usage TODO MVCC in entangleDB SQL Query Execution in entangleDB entangleDB Raft Consensus Engine What I am trying to build

spider-rs The spider project ported to Node.js Getting Started npm i @spider-rs/spider-rs --save import { Website, pageTitle } from "@spider-rs/spider

UnlimCloud provides unlimited cloud storage for your files, utilizing Telegram as the storage solution. Simply log in using your Telegram ID, and you are good to go.

Tartarus - Rust Locker Overview The Hyperswitch Card Vault (Tartarus) is a highly performant and a secure vault to save sensitive data such as payment

reconcile-rs Docs This crate provides a key-data map structure HRTree that can be used together with the reconciliation Service. Different instances c

Rust TUI HTTP Client with API Key Management This project is still in active development and although it is useable, there may still be bugs and signi

Quick-KV A reliable key-value storage for modern software Features Binary Based Data-Store Serde Supported Data Types Thread Safe Links Documentation

RocksQ An inproc RocksDB-based queue with Python bindings. It is implemented in Rust. Features: max capacity limit in number of elements; size calcula

NodeCraft Crafting seamless node operations for distributed systems, which provides foundational traits for node identification and address resolution

मनस् | Manas Solid is a web native protocol to enable interoperable, read-write, collaborative, and decentralized web, truer to web's original vision.

usbsas is a free and open source (GPLv3) tool and framework for securely reading untrusted USB mass storage devices. Description Following the concept

DawnSearch DawnSearch is an open source distributed web search engine that searches by meaning. It uses semantic search (searching on meaning), using

y-sweet: a Yjs server with persistence and auth y-sweet is an open-source server for building realtime applications on top of the Yjs CRDT library. Fe

Postgres Message Queue (PGMQ) A lightweight distributed message queue. Like AWS SQS and RSMQ but on Postgres. Features Lightweight - Built with Rust a

PhotonDB A high-performance storage engine for modern hardware and platforms. PhotonDB is designed from scratch to leverage the power of modern multi-

mvSQLite Distributed, MVCC SQLite that runs on top of FoundationDB. Documentation mvSQLite Features Releases Quick reference Try it Contributing Featu

OmniPaxos Demo This is a small demo of how to transform a simple single-server RocksDB service into a distributed version using OmniPaxos. Related res

TinyBase TinyBase is an in-memory database built with Rust, based on the sled embedded key-value store. It supports indexing and constraints, allowing

DKG This library is an implementation of the distributed key generator required for blind DKG. Overview dkg-core: supports both std and no-std. When b

Version Control An experiment to write a version control system from scratch in Rust. CLI Usage Usage: revtool COMMAND Commands: init initia

The fly.io distributed systems challenges solved in Rust. Live-streamed in https://youtu.be/gboGyccRVXI License Licensed under either of Apache Licens

GlueSQL Discord Storage After discussing how CI testing will be managed, we plan to move it upstream. Precautions for use discord ToS https://discord.

OFFFLIX A simple application to auto manage series on your local storage. Features Resume watching Play next episode Auto increment season Play random

⚠️ TAPLE is in early development and should not be used in production ⚠️ TAPLE Client TAPLE (pronounced T+ 🍎 ['tapəl]) stands for Tracking (Autonomou

I was working on this instead of my finals, hope you appreciate it. I'll add all relevant executables when I can Infinite-Storage-Glitch AKA ISG (writ

rsync-sjtug WIP: This project is still under development, and is not ready for production use. rsync-sjtug is an open-source project designed to provi

A simple to use rust package to generate or parse Twitter snowflake IDs,generate time sortable 64 bits unique ids for distributed systems (inspired from twitter snowflake)

Discord Drive This repo hold an MVP implementation of an concept to store data on discord by dividing them into chunks of 7MB each, which are then upl

fólkvangr A distributed network for cloud-native services About Fólkvangr started out as a way to build social networks that were secure and decentral

Koofr Vault https://vault.koofr.net Koofr Vault is an open-source, client-side encrypted folder for your Koofr cloud storage offering an extra layer o

LSM in a Week Build a simple key-value storage engine in a week! Tutorial The tutorial is available at https://skyzh.github.io/mini-lsm. You can use t

akvdb A simple key-value store with a log-structured, append-only storage architecture where data is encrypted with AES GCM. Modified from the actionk

Carbonado An apocalypse-resistant data storage format for the truly paranoid. Designed to keep encrypted, durable, compressed, provably replicated con

RDeeBee Follow this blog series for more details on this project. This system is inspired by Martin Kleppman's arguments that Event Sourcing system an

Rust API Client for ImageKit.io a file storage and image processing service Usage You must retrieve your Public and Private Keys from the ImageKit Dev

https://meta-secret.github.io Application Design Activity Diagram graph TD User -- |split password| MSS{MetaSecret} MSS -- |split| Hash1

w3s-cli A simple command line program to upload file or directory to web3.storage with optional encryption and compression. Features Uploads single fi

Project Masterpass (working title) Attention! This project is still under heavy development, and SHOULD NOT be used in practice, as the algorithms cou

RSink Simple utility to backup/sync data between devices to the cloud Features 🚀 Blazingly Fast ⛽ Lightweight 🛟 Cross-platform, runs everywhere 📁 S

dbz A library (dbz-lib) and CLI tool (dbz-cli) for working with Databento Binary Encoding (DBZ) files. Python bindings for dbz-lib are provided in the

This demo shows how to connect Phala's Fat Contract to external storage services, both centralized (Amazon s3) and decentralized (Arweave/Filecoin thr

Live Demo | Website | API Workspace on Postman Parseable is an open source, cloud native, log storage and management platform. Parseable helps you ing

Fractal Docker Plugin This plugin handles the creation and deletion of docker volumes backed by copy-on-write filesystems (BTRFS currently), regular s

Storage API This projects hosts the storage metadata API, a backend component which hosts metadata for storage snapshots, the actual data is housed in

blobnet A low-latency file server that responds to requests for chunks of file data. This acts as a non-volatile, over-the-network content cache. Inte

A mini kv database demo that using simplified bitcask storage model with rust implementation.

The rust client for CeresDB. CeresDB is a high-performance, distributed, schema-less, cloud native time-series database that can handle both time-series and analytics workloads.

OBKV Table Client is Rust Library that can be used to access table data from OceanBase storage layer. Its access method is different from JDBC, it skips the SQL parsing layer, so it has significant performance advantage.

CeresDB is a high-performance, distributed, schema-less, cloud native time-series database that can handle both time-series and analytics workloads.

Habitat is open source software that creates platform-independent build artifacts and provides built-in deployment and management capabilities. The go

Bioyino The StatsD server written in Rust Description Bioyino is a distributed statsd-protocol server with carbon backend. Features all basic metric t

hashvec A HashVec is a hash map / dictionary whose key-value pairs are stored (and can be iterated over) in a fixed order, by default the order in whi

DarkBird is a Document oriented, high concurrency in-memory Storage, also persist data to disk to avoid loss any data The darkbird provides the follow

quad-storage This is the crate to save data in persistent local storage in miniquad/macroquad environment. In WASM the data persists even if tab or br

Spacedrive A file explorer from the future. spacedrive.com » Download for macOS · Windows · Linux · iOS · watchOS · Android ~ Links will be added once

RisingWave is a cloud-native streaming database that uses SQL as the interface language. It is designed to reduce the complexity and cost of building real-time applications. RisingWave consumes streaming data, performs continuous queries, and updates results dynamically. As a database system, RisingWave maintains results inside its own storage and allows users to access data efficiently.

Cask A universal, distributed package manager. Installation | Usage | How to publish package? | Design | Contributing | Cask.toml If you are tired of:

Sorock is an experimental "so rocking" scale-out distributed object storage

Alchemy Attribute-Level Caching in Heterogeneous In-Memory DBMS Alchemy is a DRAM-PM hybrid database engine built from scratch to achieve high perform

Build & Release Notes rustup Toolchain We normally want to build with the rustc Debian package. To do that you can set the following rustup configurat

OpenDAL Open Data Access Layer that connect the whole world together. Status OpenDAL is in alpha stage and has been early adopted by databend. Welcome

An elastic and reliable Serverless Data Warehouse, offers Blazing Fast Query and combines Elasticity, Simplicity, Low cost of the Cloud, built to make the Data Cloud easy

klstore Appendable and iterable key/list storage, backed by S3. General Overview Per key, a single writer appends to underlying storage, enabling many

This is an experiment in designing a distributed connected garden experience. It started as an implementation of a blockchain, but has moved on to be a bit more than that. There is no proof of work like a cryptocurrency, but plays with the idea of a distributed consensus-building system.

A lightweight key-value cache system developed for experimental purposes. It can also include distributed systems setup if I can.

wasm-sqlite SQLite compiled to WASM with pluggable data storage. Useful to save SQLite in e.g. Cloudflare Durable Objects (example: https://github.com

uf2conv Adds a uf2 header Microsofts HID Flashing Format (UF2) for copying over to UF2 bootloader mass storage devices. UF2 is factory programmed exte

Cap'n Proto for Rust documentation blog Introduction Cap'n Proto is a type system for distributed systems. With Cap'n Proto, you describe your data an

SFA (Rust) Single File Assets is a file storage format for images. The packed images are not guaranteed to be of same format because the format while

Damavand is a code that simulates quantum circuits. In order to learn more about damavand, refer to the documentation. Development status Core feature

EDED Embedded Distributed Encrypted Database. Research projects to support ESSE. WIP Distributed design features Adapt to personal distributed usecase

photofinish - a little, handy tool to replay events This tiny CLI tool aims to fulfill the need to replay some events and get fixtures. Photofinish re

Stacks 2.0 Reference implementation of the Stacks blockchain in Rust. Stacks 2.0 is a layer-1 blockchain that connects to Bitcoin for security and ena

Raft Problem and Importance When building a distributed system one principal goal is often to build in fault-tolerance. That is, if one particular nod

Chamomile Build a robust stable connection on p2p network features Support build a robust stable connection between two peers on the p2p network. Supp

ri2p Rust implementation of the i2p client/server/router protocols Status Common Commands cargo build: Builds the ri2p binary cargo run: Runs the ri2p

SDB - SignatureDB Distributed, version controlled, SQL database with cryptographically verifiable storage, queries and results. Think git for postgres

PLEASE NOTE THAT THIS REPOSITORY HAS NOW BEEN ARCHIVED All routing development is now via the safe_network repository sn_routing sn_routing - a specia

brassfibre Provides multiple-dtype columner storage, known as DataFrame in pandas/R. Series Single-dtype 1-dimentional vector with label (index). Crea

Rain Rain is an open-source distributed computational framework for processing of large-scale task-based pipelines. Rain aims to lower the entry barri

Welcome to learn Talent Plan Courses! Talent Plan is an open source training program initiated by PingCAP. It aims to create or combine some open sour

toyDB Distributed SQL database in Rust, written as a learning project. Most components are built from scratch, including: Raft-based distributed conse

crdts: family of thoroughly tested hybrid crdt's. A family of CRDT's supporting both State and Op based replication. what is a CRDT? CRDT's are the so

LibreAuth is a collection of tools for user authentication. Features Password / passphrase authentication no character-set limitation reason

Canary Canary is a distributed systems and communications framework, focusing on minimalism, ease of use and performance. Development of Canary utiliz

Damavand is a quantum circuit simulator. It can run on laptops or High Performance Computing architectures, such CPU distributed architectures or multi GPU distributed architectures.

Nouzdb An embedded key-value storage for learning purpose, which is based on the idea of SSTable / LSM-tree. Plan Implement a memtable. Implement the

is the ultimate cloud database for tomorrow's applications Develop easier. Build faster. Scale quicker. What is SurrealDB? SurrealDB is an end-to-end

ChiselStore ChiselStore is an embeddable, distributed SQLite for Rust, powered by Little Raft. SQLite is a fast and compact relational database manage

Zero-Trust Decentralized Package Network Current Development Phase 📢 We are looking for your feedback! This project is currently in the "sandbox" 🏖️

memcrsd memcached server implementation in Rust memcrsd is a key value store implementation in Rust. It is compatible with binary protocol of memcache

secrets secrets is a library to help Rust programmers safely held cryptographic secrets in memory. It is mostly an ergonomic wrapper around the memory

NymDrive NymDrive is a complete, end-to-end encrypted file syncing daemon that runs over the Nym network. Features Active file monitoring of changes i

Rust I18n Rust I18n is use Rust codegen for load YAML file storage translations on compile time, and give you a t! macro for simply get translation te

nbd-rs Disclaimer DO NEVER USE THIS FOR PRODUCTION Do not use this for any data that you cannot afford to lose any moment. Expect data loss, corruptio

Merge together and efficiently time-sort compressed .pcap files stored in AWS S3 object storage (or locally) to stdout for pipelined processing. High performance and parallel implementation for 10 Gbps playback throughput with large numbers of files (~4k).

Garage [ Website and documentation | Binary releases | Git repository | Matrix channel ] Garage is a lightweight S3-compatible distributed object stor

✨ Hands-on Introduction: Build end-to-end encrypted, mutually-authenticated, secure messaging in Rust ✨ Rust and Elixir libraries for end-to-end encry