114 Repositories
Rust object-storage Libraries
An LSM storage engine designed to significantly reduce I/O amplification written in safe rust (Under active development)
VelarixDB is an LSM-based storage engine designed to significantly reduce IO amplification, resulting in better performance and durability for storage
Wrapper over MMTk to simplify integration with runtimes: thread management system, object header, root scanning
vmkit A library which provides bunch of building blocks to make a VM in Rust. Feautures MMTK integration out of the box Thread management provided by
An automated CLI tool that optimizes gas usage in Solidity smart contracts, focusing on storage and function call efficiency.
Solidity-Gas-Optimizoor An high performance automated CLI tool that optimizes gas usage in Solidity smart contracts, focusing on storage and function
A box with vtable, erasing the type of the contained object
VBox VBox is a type erased Box of trait object that stores the vtable pointer. VBox is just like a Boxdyn Trait but erases type Trait so that to use
UnlimCloud provides unlimited cloud storage for your files, utilizing Telegram as the storage solution
UnlimCloud provides unlimited cloud storage for your files, utilizing Telegram as the storage solution. Simply log in using your Telegram ID, and you are good to go.
Hyperswitch Card Vault is an open-source sensitive information storage system built on Rust.
Tartarus - Rust Locker Overview The Hyperswitch Card Vault (Tartarus) is a highly performant and a secure vault to save sensitive data such as payment
Just set of functions to utilize YOLO v3, v4, v7 and v8 version with OpenCV's DNN module
Object detection utilities in Rust programming language for YOLO-based neural networks in OpenCV ecosystem This crate provides some basic structures a
Hash trait that is object-safe
Hash trait that is object-safe This crate provides a DynHash trait that can be used in trait objects. Types that implement the standard library's std:
A reconciliation service to sync a key-value map over multiple instances.
reconcile-rs Docs This crate provides a key-data map structure HRTree that can be used together with the reconciliation Service. Different instances c
HTTP client/libcurl TUI front end in Rust, with request + key storage
Rust TUI HTTP Client with API Key Management This project is still in active development and although it is useable, there may still be bugs and signi
A reliable key-value storage for modern software
Quick-KV A reliable key-value storage for modern software Features Binary Based Data-Store Serde Supported Data Types Thread Safe Links Documentation
RocksDB-based queue with python bindings
RocksQ An inproc RocksDB-based queue with Python bindings. It is implemented in Rust. Features: max capacity limit in number of elements; size calcula
Manas project aims to create a modular framework and ecosystem to create robust storage servers adhering to Solid protocol in rust.
मनस् | Manas Solid is a web native protocol to enable interoperable, read-write, collaborative, and decentralized web, truer to web's original vision.
Tool and framework for securely reading untrusted USB mass storage devices.
usbsas is a free and open source (GPLv3) tool and framework for securely reading untrusted USB mass storage devices. Description Following the concept
A standalone yjs server with persistence to blob storage.
y-sweet: a Yjs server with persistence and auth y-sweet is an open-source server for building realtime applications on top of the Yjs CRDT library. Fe
A high-performance storage engine for modern hardware and platforms.
PhotonDB A high-performance storage engine for modern hardware and platforms. PhotonDB is designed from scratch to leverage the power of modern multi-
CYFS:Next Generation Protocol Family to Build Web3
CYFS is the next-generation technology to build real Web3 by upgrading the basic protocol of Web (TCP/IP+DNS+HTTP). It has a subversive architectural design that everyone brings their own OOD (Owner Online Device) to form a truly decentralized network.
Coffee is a loader for ELF (Executable and Linkable Format) object files written in Rust
Coffee is a loader for ELF (Executable and Linkable Format) object files written in Rust. It provides a mechanism to load and parse ELF files similar to COFFLoader, but specifically designed for ELF files used in Unix-like systems.
High-performance, lock-free local and concurrent object memory pool with automated allocation, cleanup, and verification.
Opool: Fast lock-free concurrent and local object pool Opool is a high-performance Rust library that offers a concurrent and local object pool impleme
A tiny embedded database built in Rust.
TinyBase TinyBase is an in-memory database built with Rust, based on the sled embedded key-value store. It supports indexing and constraints, allowing
Just a simple object renderer, written in under 500 lines using Rust.
All cargoes that the project runs are: bitflags: a crate for defining bitflag types cfg-if: a small macro crate for defining cfg-based - conditional c
A version control system implemented from scratch in Rust.
Version Control An experiment to write a version control system from scratch in Rust. CLI Usage Usage: revtool COMMAND Commands: init initia
SQL database to read and write "discord"
GlueSQL Discord Storage After discussing how CI testing will be managed, we plan to move it upstream. Precautions for use discord ToS https://discord.
A simple local storage media library manager.
OFFFLIX A simple application to auto manage series on your local storage. Features Resume watching Play next episode Auto increment season Play random
ISG lets you use YouTube as cloud storage for ANY files, not just video
I was working on this instead of my finals, hope you appreciate it. I'll add all relevant executables when I can Infinite-Storage-Glitch AKA ISG (writ
Mirroring remote repositories to s3 storage, with atomic updates and periodic garbage collection.
rsync-sjtug WIP: This project is still under development, and is not ready for production use. rsync-sjtug is an open-source project designed to provi
Rust implementation of Surging Object DiGraph (SODG)
This Rust library implements a Surging Object DiGraph (SODG) for reo virtual machine for EO programs. Here is how you can create a di-graph: use sodg:
store virtually unlimited, large files on discord by splitting them into small chunks
Discord Drive This repo hold an MVP implementation of an concept to store data on discord by dividing them into chunks of 7MB each, which are then upl
Process Injection via Component Object Model (COM) IRundown::DoCallback().
COM PROCESS INJECTION for RUST Process Injection via Component Object Model (COM) IRundown::DoCallback(). 该技术由 @modexpblog 挖掘发现,在我对该技术进行深入研究过程中,将原项目 m
Koofr Vault is an open-source, client-side encrypted folder for your Koofr cloud storage offering an extra layer of security for your most sensitive files.
Koofr Vault https://vault.koofr.net Koofr Vault is an open-source, client-side encrypted folder for your Koofr cloud storage offering an extra layer o
A tutorial of building an LSM-Tree storage engine in a week! (WIP)
LSM in a Week Build a simple key-value storage engine in a week! Tutorial The tutorial is available at https://skyzh.github.io/mini-lsm. You can use t
Omi - Object Mapping Intelligently
Omi - Object Mapping Intelligently A library that maps object structures to relational databases and provides interfaces for common queries and update
A simple key-value store with a log-structured, append-only storage architecture where data is encrypted with AES GCM.
akvdb A simple key-value store with a log-structured, append-only storage architecture where data is encrypted with AES GCM. Modified from the actionk
An apocalypse-resistant data storage format for the truly paranoid.
Carbonado An apocalypse-resistant data storage format for the truly paranoid. Designed to keep encrypted, durable, compressed, provably replicated con
Rust API Client for ImageKit.io a file storage and image processing service
Rust API Client for ImageKit.io a file storage and image processing service Usage You must retrieve your Public and Private Keys from the ImageKit Dev
Pure Rust implementation of Javascript Object Signing and Encryption (JOSE)
RustCrypto: JOSE Pure Rust implementation of Javascript Object Signing and Encryption (JOSE) License All crates licensed under either of Apache Licens
A simple command line program to upload file or directory to web3.storage with optional encryption and compression
w3s-cli A simple command line program to upload file or directory to web3.storage with optional encryption and compression. Features Uploads single fi
Project Masterpass is a deterministic databaseless key management algorithm, aimed to help those who cannot protect their encryption keys in storage
Project Masterpass (working title) Attention! This project is still under heavy development, and SHOULD NOT be used in practice, as the algorithms cou
Simple utility to backup/sync data between devices to the cloud
RSink Simple utility to backup/sync data between devices to the cloud Features 🚀 Blazingly Fast ⛽ Lightweight 🛟 Cross-platform, runs everywhere 📁 S
Databento Binary Encoding (DBZ) - Fast message encoding and storage format for market data
dbz A library (dbz-lib) and CLI tool (dbz-cli) for working with Databento Binary Encoding (DBZ) files. Python bindings for dbz-lib are provided in the
Demo: Connect Phala's Fat Contract to external storage services, both centralized (Amazon s3) and decentralized .
This demo shows how to connect Phala's Fat Contract to external storage services, both centralized (Amazon s3) and decentralized (Arweave/Filecoin thr
Cloud native log storage and management for Kubernetes, containerised workloads
Live Demo | Website | API Workspace on Postman Parseable is an open source, cloud native, log storage and management platform. Parseable helps you ing
Object Notation Programming Language?
You may have questions ONLang - Object Notation Language (jsON) VSCode extension - OnLang 1. God, what the f**** is this ONLang is an experimental, es
Replicated storage docker plugin.
Fractal Docker Plugin This plugin handles the creation and deletion of docker volumes backed by copy-on-write filesystems (BTRFS currently), regular s
Storage metadata service.
Storage API This projects hosts the storage metadata API, a backend component which hosts metadata for storage snapshots, the actual data is housed in
Non-volatile, distributed file cache backed by content-addressed storage
blobnet A low-latency file server that responds to requests for chunks of file data. This acts as a non-volatile, over-the-network content cache. Inte
Merge multiple Juniper object definitions into a single object type.
juniper-compose Merge multiple Juniper object definitions into a single object type. crates.io | docs | github Motivation You are building a GraphQL s
Machine learning framework for building object trackers and similarity search engines
Similari Similari is a framework that helps build sophisticated tracking systems. The most frequently met operations that can be efficiently implement
A mini kv database demo that using simplified bitcask storage model with rust implementation
A mini kv database demo that using simplified bitcask storage model with rust implementation.
Ecstasy - a subjective experience of total involvement of the subject, with an object of their awareness.
Ecstasy (from Ancient Greek ἔκστασις ékstasis, meaning 'outside of oneself') is a subjective experience of total involvement of the subject, with an object of their awareness. In classical Greek literature it refers to removal of the mind or body "from its normal place of function."
OBKV Table Client is Rust Library that can be used to access table data from OceanBase storage layer.
OBKV Table Client is Rust Library that can be used to access table data from OceanBase storage layer. Its access method is different from JDBC, it skips the SQL parsing layer, so it has significant performance advantage.
garbage-collecting on-disk object store, supporting higher level KV stores and databases.
marble Garbage-collecting disk-based object-store. See examples/kv.rs for a minimal key-value store built on top of this. Supports 4 methods: read: de
A HashMap/Vector hybrid: efficient, ordered key-value data storage in Rust.
hashvec A HashVec is a hash map / dictionary whose key-value pairs are stored (and can be iterated over) in a fixed order, by default the order in whi
High concurrency, RealTime, In-memory storage inspired by erlang mnesia
DarkBird is a Document oriented, high concurrency in-memory Storage, also persist data to disk to avoid loss any data The darkbird provides the follow
Plugin for macro-, mini-quad (quads) to save data in simple local storage using Web Storage API in WASM and local file on a native platforms.
quad-storage This is the crate to save data in persistent local storage in miniquad/macroquad environment. In WASM the data persists even if tab or br
Sorock is an experimental "so rocking" scale-out distributed object storage
Sorock is an experimental "so rocking" scale-out distributed object storage
Attribute-Level Caching in Heterogeneous In-Memory DBMS
Alchemy Attribute-Level Caching in Heterogeneous In-Memory DBMS Alchemy is a DRAM-PM hybrid database engine built from scratch to achieve high perform
Proxmox Backup Server and Client
Build & Release Notes rustup Toolchain We normally want to build with the rustc Debian package. To do that you can set the following rustup configurat
Open Data Access Layer that connect the whole world together
OpenDAL Open Data Access Layer that connect the whole world together. Status OpenDAL is in alpha stage and has been early adopted by databend. Welcome
Databend aimed to be an open source elastic and reliable serverless data warehouse,
An elastic and reliable Serverless Data Warehouse, offers Blazing Fast Query and combines Elasticity, Simplicity, Low cost of the Cloud, built to make the Data Cloud easy
Appendable and iterable key/list storage, backed by S3, written in rust
klstore Appendable and iterable key/list storage, backed by S3. General Overview Per key, a single writer appends to underlying storage, enabling many
A lightweight key-value cache system developed for experimental purposes
A lightweight key-value cache system developed for experimental purposes. It can also include distributed systems setup if I can.
SQLite compiled to WASM with pluggable data storage
wasm-sqlite SQLite compiled to WASM with pluggable data storage. Useful to save SQLite in e.g. Cloudflare Durable Objects (example: https://github.com
Adds a uf2 header Microsofts HID Flashing Format (UF2) for copying over to UF2 bootloader mass storage devices
uf2conv Adds a uf2 header Microsofts HID Flashing Format (UF2) for copying over to UF2 bootloader mass storage devices. UF2 is factory programmed exte
Parses .off (Object File Format) files. This implementation follows this spec from the Princeton Shape Benchmark.
off-rs - A simple .off file parser Parses .off (Object File Format) files. This implementation follows this spec from the Princeton Shape Benchmark. S
Single File Assets is a file storage format for images
SFA (Rust) Single File Assets is a file storage format for images. The packed images are not guaranteed to be of same format because the format while
An event replay tool for the Trento storage backend.
photofinish - a little, handy tool to replay events This tiny CLI tool aims to fulfill the need to replay some events and get fixtures. Photofinish re
The Stacks 2.0 blockchain implementation
Stacks 2.0 Reference implementation of the Stacks blockchain in Rust. Stacks 2.0 is a layer-1 blockchain that connects to Bitcoin for security and ena
A stack for rust trait objects that minimizes allocations
dynstack A stack for trait objects that minimizes allocations COMPATIBILITY NOTE: dynstack relies on an underspecified fat pointer representation. Tho
Distributed, version controlled, SQL database with cryptographically verifiable storage, queries and results. Think git for postgres.
SDB - SignatureDB Distributed, version controlled, SQL database with cryptographically verifiable storage, queries and results. Think git for postgres
Parser for Object files define the geometry and other properties for objects in Wavefront's Advanced Visualizer.
format of the Rust library load locad blender obj file to Rust NDArray. cargo run test\t10k-images.idx3-ubyte A png file will be generated for the fi
Routing - specialised storage DHT
PLEASE NOTE THAT THIS REPOSITORY HAS NOW BEEN ARCHIVED All routing development is now via the safe_network repository sn_routing sn_routing - a specia
Provides multiple-dtype columner storage, known as DataFrame in pandas/R
brassfibre Provides multiple-dtype columner storage, known as DataFrame in pandas/R. Series Single-dtype 1-dimentional vector with label (index). Crea
LibreAuth is a collection of tools for user authentication.
LibreAuth is a collection of tools for user authentication. Features Password / passphrase authentication no character-set limitation reason
Qoo - Query Object Oriented. Pronunciation is Kuu.
qoo Query Object Oriented. Pronunciation is Kuu. demo use qoo::base::*; use qoo::select::*; use qoo::insert::*; use qoo::update::*; use qoo::delete::*
Eon-rs - A reference parser for EON (Extensible Object Notation)
eon-rs eon-rs is a Rust library for parsing EON. Installation Add eon-rs = "1.0.0" to your Cargo.toml file. Usage use eon_rs; // Read EON from file l
An embedded key-value storage for learning purpose, which is based on the idea of SSTable / LSM-tree.
Nouzdb An embedded key-value storage for learning purpose, which is based on the idea of SSTable / LSM-tree. Plan Implement a memtable. Implement the
My solutions for the 2021 edition of the Advent of Code, using Rust and SOM (Simple Object Machine)
Advent of Code 2021 These are my solutions for the 2021 edition of the Advent of Code. The solutions are all implemented using both Rust and SOM (Simp
DWARF packaging utility, written in Rust, supporting GNU extension and DWARF 5 package formats.
thorin thorin is an DWARF packaging utility for creating DWARF packages (*.dwp files) out of input DWARF objects (*.dwo files; or *.o files with .dwo
CBOR: Concise Binary Object Representation
CBOR 0x(4+4)9 0x49 “The Concise Binary Object Representation (CBOR) is a data format whose design goals include the possibility of extremely small cod
Secure storage for cryptographic secrets in Rust
secrets secrets is a library to help Rust programmers safely held cryptographic secrets in memory. It is mostly an ergonomic wrapper around the memory
NymDrive is a complete, end-to-end encrypted file syncing daemon that runs over the Nym network.
NymDrive NymDrive is a complete, end-to-end encrypted file syncing daemon that runs over the Nym network. Features Active file monitoring of changes i
DWARF packaging utility, written in Rust, supporting GNU extension and DWARF 5 package formats.
thorin thorin is an DWARF packaging utility for creating DWARF packages (*.dwp files) out of input DWARF objects (*.dwo files; or *.o files with .dwo
Rust I18n is use Rust codegen for load YAML file storage translations on compile time, and give you a t! macro for simply get translation texts.
Rust I18n Rust I18n is use Rust codegen for load YAML file storage translations on compile time, and give you a t! macro for simply get translation te
Network Block Storage server, written in Rust. Supports pluggable and chainable underlying storage
nbd-rs Disclaimer DO NEVER USE THIS FOR PRODUCTION Do not use this for any data that you cannot afford to lose any moment. Expect data loss, corruptio
Merge together and efficiently time-sort compressed .pcap files stored in AWS S3 object storage (or locally) to stdout for pipelined processing.
Merge together and efficiently time-sort compressed .pcap files stored in AWS S3 object storage (or locally) to stdout for pipelined processing. High performance and parallel implementation for 10 Gbps playback throughput with large numbers of files (~4k).
Garage is a lightweight S3-compatible distributed object store
Garage [ Website and documentation | Binary releases | Git repository | Matrix channel ] Garage is a lightweight S3-compatible distributed object stor
elfshaker is a low-footprint, high-performance version control system fine-tuned for binaries.
elfshaker is a low-footprint, high-performance version control system fine-tuned for binaries.
Create full-fledged APIs for static datasets without writing a single line of code.
ROAPI ROAPI automatically spins up read-only APIs for static datasets without requiring you to write a single line of code. It builds on top of Apache
A sessioned Merkle key/value store
storage A sessioned Merkle key/value store The crate was designed to be the blockchain state database. It provides persistent storage layer for key-va
A pure Rust implementation of the Web Local Storage API, for use in non-browser contexts
Rust Web Local Storage API A Rust implementation of the Web LocalStorage API, for use in non-browser contexts About the Web Local Storage API MDN docs
Filen.io is a cloud storage provider with an open-source desktop client.
Library to call Filen.io API from Rust Filen.io is a cloud storage provider with an open-source desktop client. My goal is to write a library which ca
Lightweight Google Cloud Storage sync Rust Client with better performance than gsutil rsync
gcs-rsync Lightweight and efficient Rust gcs rsync for Google Cloud Storage. gcs-sync is faster than gsutil rsync when files change a lot while perfor
An object-relational in-memory cache, supports queries with an SQL-like query language.
qlcache An object-relational in-memory cache, supports queries with an SQL-like query language. Warning This is a rather low-level library, and only p
Zenith substitutes PostgreSQL storage layer and redistributes data across a cluster of nodes
Zenith substitutes PostgreSQL storage layer and redistributes data across a cluster of nodes
It is a backup tool that creates backups and stores them on an object storage
Hold My Backup It is a backup tool that creates backups and stores them on an object storage. By default it uses minio but you can use AWS: S3 as well
Educational Rust implemenation of Auction Sniper from Growing Object-Oriented Software, Guided By Tests
Auction Sniper Educational Rust not-OOP implemenation of Auction Sniper from "Growing Object-Oriented Software, Guided By Tests" book More about it in
Forage is for Storage
Forage Forage is for Storage Remote storage: Open storage channels to a remote storage provider over Tor Lightweight: Platform-optimized using Blake3-
Engula empowers engineers to build reliable and cost-effective databases.
Engula is a storage engine that empowers engineers to build reliable and cost-effective databases with less effort and more confidence. Engula is in t
A Key-Value data storage system. - dorea db
Dorea DB 🛰 Dorea is a key-value data storage system. It is based on the Bitcask storage model Documentation | Crates.io | API Doucment 简体中文 | English