Rust Data-management Resources

RustForDataPipelines Testing out if Rust can be used for a normal Data Engineering Pipeline. Check out the full blog post here. https://www.confession

Arenta A terminal-based daily task management tool with minimal overhead. Demo Features Complete CRUD support of daily tasks with intuitive syntax Vis

RedMaple offers an oppinionated yet extremely flexible data modeling system based on events for back-end applications.

Extensions supported ( .xls, .xlsx, .xlsm, .xlsb, .xla, .xlam, .ods ) Speed Test Image shows the results of 5000defs synced from 2 workbooks and 5 she

Http Drogue Http Drogue is a tiny service that downloads files over HTTP from links you provide. It can restart and resume interrupted downloads. Http

🚀 10x easier, 🚀 10x cheaper, 🚀 petabyte scale - Elasticsearch/Splunk/Datadog alternative for 🚀 (logs, metrics, traces). ZincObserve ZincObserve is

fiberplane This repository is a monorepo for Rust code that is used throughout Fiberplane's product. Overview base64uuid - A utility for working with

About A fully-featured set of tools for managing abilities in Bevy. This crate is meant to be used with Leafwing Input Manager, which converts inputs

Few Word Do Trick (fwdt) Few Word Do Trick (fwdt) is a cross-platform general purpose fast logger for humans that supports incomplete csvs for a bette

tx-decoder A quick way to decode a contract's transaction data with only the contract address and abi. E.g, let tx_data = "0xe70dd2fc00000000000000000

Barter-Data A high-performance WebSocket integration library for streaming public market data from leading cryptocurrency exchanges - batteries includ

DriftDB DriftDB is a real-time data backend for browser-based applications. For more information, see driftdb.com. Structure of this repo docs/: main

Tickets.rs - A Ticket Management Tool Tickets.rs is a Desktop application made with egui, that is used for managing and modifying Tickets (or Issues,

RusQTTbom RusQTTbom collects weather data from the Bureau of Meterology (BOM) then publishes said data locally via MQTT messages. BOM weather data is

Shrike Shrike is a set of tools built for the purpose of Neo blockchain data analysis. The infrastructure comprises three components: Indexer - Synchr

healthkit-to-sqlite Command-line tool to convert Apple HealthKit data to a SQLite database. Getting Started Open the Health app on your iOS device. Cl

Compact Generational Arena This crate provides ArenaT, a contiguous growable container which assigns and returns IDs to values when they are added t

ferrilab Redefining the Rust fundamental data model bitvec funty radium Introduction The ferrilab project is a collection of crates that provide more

Allocative: memory profiler for Rust This crate implements a lightweight memory profiler which allows object traversal and memory size introspection.

YClass A program that allows you to inspect and recreate data structures of other processes. Installation To compile YClass you will need Rust. git cl

This repository contains the code examples, data structures, and links from Rust Atomics and Locks. The examples from chapters 1, 2, 3, and 8 can be f

Stdto stdto provides a set of functional traits for conversion between various data representations. | Examples | Docs | Latest Note | stdto = "0.13.0

ipzone Ipzone provides a simple and powerful IP architecture to Rust. | Examples | Docs | Latest Note | ipzone = "0.4.0" Examples use ipzone::prelude:

Probabilistic Principal Component Analysis (PPCA) model This project implements a PPCA model implemented in Rust for Python using pyO3 and maturin. In

DNG-rs   A pure rust library for reading / writing DNG files providing access to the raw data in a zero-copy friendly way. Also containing code for re

🎄 advent-of-code-2022 Overview This repository contains my solutions to Advent of Code 2022. I decided to take this year as an opportunity to learn R

ruscode A visual studio code's workspaces organizer written in Rust Supports Linux, and Windows. What is a workspaces organizer? After months of or ev

Soroban Fiddle https://leighmcculloch.github.io/soroban-fiddle Web frontend-only application that displays data on the Soroban Futurenet network. Feat

akvdb A simple key-value store with a log-structured, append-only storage architecture where data is encrypted with AES GCM. Modified from the actionk

xvc A Fast and Robust MLOps Swiss-Army Knife in Rust ⌛ When to use xvc? Machine Learning Engineers: When you manage large quantities of unstructured d

flowing flowing is a flow graph processing library written in Rust. It shall serve as a general-purpose building block for all kinds of dataflow progr

Carbonado An apocalypse-resistant data storage format for the truly paranoid. Designed to keep encrypted, durable, compressed, provably replicated con

Shell Of A New Machine soanm is a dead-simple tool for easily configuring new UNIX machines, with almost zero prerequisites on the target machine. All

🚀 ghr Yet another repository management with auto-attaching profiles. 🔥 Motivation ghq is the most famous solution to resolve stress of our reposito

rust_aads - Rust Algorithms And Data Structures rust_aads is an open repository with algorithms and data structures, used in computer science and comp

cozo A general-purpose, transactional, relational database that uses Datalog for query and focuses on graph data and algorithms. Features Relational d

Eightfold A library for spatial partitioning of 3D data. Built with nalgebra. Not yet fit for actual use; wait until 1.0.0. Feature Flags spatial :: [

OpenEMC OpenEMC is an open-source firmware implementing an embedded management controller (EMC) on an STM32F1 microcontroller. It consists of a bootlo

ktool - a tool for Kafka ktool is a dead-simple tool for working with data in Kafka: Copy partitions / topics to disk Replay messages Inspect message

UniSBOM is a tool to build a software bill of materials on any platform with a unified data format. Work in progress Support MacOS Uses system_profile

Stuart A Blazingly-Fast Static Site Generator. Download Now » Stuart is a very fast and flexible static site generator, with build times as low as 0.1

PyVer Python PEP-440 Version Parser This package allows for parsing Python PEP-440 version numbers and for comparisons between PEP-440 version numbers

Project Masterpass (working title) Attention! This project is still under heavy development, and SHOULD NOT be used in practice, as the algorithms cou

RSink Simple utility to backup/sync data between devices to the cloud Features 🚀 Blazingly Fast ⛽ Lightweight 🛟 Cross-platform, runs everywhere 📁 S

Coverage Prepare Convert coverage data to HTML reports, LCOV files or terminal tables. coverage-prepare --help: Convert "profraw" coverage data to: *

dbz A library (dbz-lib) and CLI tool (dbz-cli) for working with Databento Binary Encoding (DBZ) files. Python bindings for dbz-lib are provided in the

Live Demo | Website | API Workspace on Postman Parseable is an open source, cloud native, log storage and management platform. Parseable helps you ing

yb (Yocto Buddy) yb is designed to make it easy to setup and (perhaps more importantly) keep Yocto environments up-to-date and in-sync with your team.

wgx wireguard tool to manage / generate configuration. Maintain one yaml configuration file to quickly build wireguard network. Usage wgx --h USAGE:

Firestore for Rust Library provides a simple API for Google Firestore: Create or update documents using Rust structures and Serde; Support for queryin

riscv-opcodes-parser Parse RISC-V opcodes to provide more detailed structured data. License Licensed under either of Apache License, Version 2.0 (LICE

TokioSky Build concurrent and multi-stage data ingestion and data processing pipelines with Rust+Tokio. TokioSky allows developers to consume data eff

Shurly Shurly, this is a URL shortener with API management Features Management of destinations through a REST'ish API Permanent/temporary redirects; p

Segment Segment is a simple & fast in-memory database with a key-value data model written in Rust. Features Dynamic keyspaces Keyspace level control o

Mars: The powerful analysis platform to explore and visualize data from Web3 Features Blazing Fast Create from scratch with Rust. Pipeline Processor E

OptiVorbis A library and application for lossless, format-preserving, two-pass optimization and repair of Vorbis data, reducing its size without alter

Plow - Ontology package manager Plow is package management solution for OWL ontologies, with support for specifying dependencies between packages via

qsv is a command line program for indexing, slicing, analyzing, splitting, enriching, validating & joining CSV files. Commands are simple, fast & composable

Data Compat MC Data tool for the Falcon MC project. Description DataCompatMC is a cli tool designed to help deal with the many different versions of M

trashctl A command line interface for trash Features Add file to trash List files Permanently delete a file Restore file Empty the trash Documentation

Data Compat MC Data tool for the Falcon MC project. Description DataCompatMC is a cli tool designed to help deal with the many different versions of M

ReadySet is a SQL caching engine designed to help developers enhance the performance and scalability of their existing database-backed applications. W

OBKV Table Client is Rust Library that can be used to access table data from OceanBase storage layer. Its access method is different from JDBC, it skips the SQL parsing layer, so it has significant performance advantage.

gpt-disk-rs no_std libraries related to GPT (GUID Partition Table) disk data. There are three Rust packages in this repository: uguid The uguid packag

python-daachorse daachorse is a fast implementation of the Aho-Corasick algorithm using the compact double-array data structure. This is a Python wrap

PRQL Pipelined Relational Query Language, pronounced "Prequel". PRQL is a modern language for transforming data — a simpler and more powerful SQL. Lik

Seed Your Development Database With Real Data ⚡️ Replibyte is a blazingly fast tool to seed your databases with your production data while keeping sen

jsonptr - JSON Pointers for Rust Data structures and logic for resolving, assigning, and deleting by JSON Pointers (RFC 6901). Usage Resolve JSON Poin

kv-par-merge-sort Key-Value Parallel Merge Sort Sort Pod (key, value) data sets that don't fit in memory. This crate provides the kv_par_merge_sort li

TUI for crate management like lazydocker and lazynpm. Shouldve named it as lazycargo but lazycrates sounded good at that time.

Written in Rust, features a simple cli and a client-side webapp. This tool hides data within the least significant bit of an image. Obfuscation techniques are utilized to make the

Habitat is open source software that creates platform-independent build artifacts and provides built-in deployment and management capabilities. The go

Rust Bitcoin Library with support for de/serialization, parsing and executing on data-structures and network messages related to Bitcoin. Heads up for

Noria: data-flow for high-performance web applications Noria is a new streaming data-flow system designed to act as a fast storage backend for read-he

Quickstart • Docs • Guides • Integrations • Chat • Download What is Vector? Vector is a high-performance, end-to-end (agent & aggregator) observabilit

The Declarative Data Generator Synth is a tool for generating realistic data using a declarative data model. Synth is database agnostic and can scale

CipherCore If you have any questions, or, more generally, would like to discuss CipherCore, please join the Slack community. See a vastly extended ver

Deadliner Watch the YouTube video What's Deadliner? Deadliner is a cross-platform desktop application for setting deadline for a project and keeping t

egui_cable A generic and extensible data-oriented widget for connecting ports by cables. I create this for the visual programming editor of Hihaheho/D

NSE Check out the sister projects NsePython and SaveKiteEnctoken which are Python & Javascript libraries to use the NSE and Zerodha APIs respectively

hashvec A HashVec is a hash map / dictionary whose key-value pairs are stored (and can be iterated over) in a fixed order, by default the order in whi

serial-to-tcp An app which reads data from a serial port and serves it on a TCP port. How to use Clone this repo and build the app as outlined below (

WasmBox WasmBox turns running Rust code into a serializable data structure. It does this by compiling it to WebAssembly and running it in a sandbox. T

fxprof-perf-convert A converter from the Linux perf perf.data format into the Firefox Profiler format, specifically into the processed profile format.

ironworks Modular FFXIV data toolkit written in rust. ironworks is pre-1.0, and as such its API should be considered unstable. Breaking API changes wi

Simple Chunk Allocator A simple no_std allocator written in Rust that manages memory in fixed-size chunks/blocks. Useful for basic no_std binaries whe

quad-storage This is the crate to save data in persistent local storage in miniquad/macroquad environment. In WASM the data persists even if tab or br

uniqueid 🔍 Generates a unique hash/identifier for a system given a set of parameters. Example usage use uniqueid; pub fn main() { let data = vec

SCTE-35 lib and parser for Rust Work in progress! This library provide access to parse and encoding of data using the SCTE-35 standard. This standard

cloud-scanner Collect aws cloud usage data, so that it can be combined with impact data of Boavizta API. ⚠ Very early Work in progress ! At the moment

A SOAP client for Brazilian Central Bank's Time Series Management System

imessage-export This crate provides both a library to interact with iMessage data as well as a binary that can perform some useful read-only operation

Introduction Recent Intel processors feature the "Intel Processor Trace" feature, which can be used to capture the full execution trace of a program.

Dioxus Table Make data-driven table rendering easy with Dioxus Installation Until the next release of Dioxus this requires Dioxus nightly from git. Th

A simple cross-platform remote file management tool to upload and download files over HTTP/S

Quickwit is the next-gen search & analytics engine built for logs. It is a highly reliable & cost-efficient alternative to Elasticsearch.

yewv A lightning fast state management module for Yew built with performance and simplicity as a first priority. Who is this for? If you wish to use a

linux-perf-data This repo contains a parser for the perf.data format which is output by the Linux perf tool. It also contains a main.rs which acts sim

Help project managers and project owners with easy-to-understand views of github issue dependencies.

RadixTrie A prefix tree (trie) is a data structure that allows you to store an associative array whose keys are strings. It is a root tree, each edge