377 Repositories
Rust data-modeling Libraries
Next generation compute platform for the post-modern data stack
Tower CLI The Tower CLI is one of the main ways to interact with the Tower environment. You can do basically everything you need inside the Tower CLI,
This crate provides data-types defined by the VDA5050 standard.
VDA5050 Types This crate provides data-types for the rust programming language defined by the VDA5050 standard. VDA5050 is an open standard for commun
Built on the Substrate-based runtime, this repository develops the Paralink Network, a cross-chain Oracle platform for secure and scalable data integration. Our goal is to create a robust infrastructure that ensures efficient and reliable data transmission across blockchain networks.
Getting Started This repository contains Substrate based runtime for Paralink Network. Makefile This project uses a Makefile to document helpful comma
A lightweight and high-performance order-book designed to process level 2 and trades data. Available in Rust and Python
ninjabook A lightweight and high-performance order-book implemented in Rust, designed to process level 2 and trades data. Available in Python and Rust
A fast, lightweight and extensible implementation of a graph data structure.
fast-graph A fast, lightweight and extensible implementation of a graph data structure. Note โ ๏ธ There will be some breaking changes in the coming 1-2
Infer a JSON schema from example data, produce nonsense synthetic data (drivel) according to the schema
drivel drivel is a command-line tool written in Rust for inferring a schema from an example JSON (or JSON lines) file, and generating synthetic data (
Rust Vector for large amounts of data, that does not copy when growing, by using full `mmap`'d pages.
Large Vector Rust Vector for large amounts of data, that does not copy when growing, by using full mmap'd pages. Maturity I made ths to learn about mm
Avail-light is a data availability light client
avail-light Light client for the Avail blockchain Introduction avail-light is a data availability light client with the following functionalities: Lis
A mono-repo for the Engineering Practice Domains of Development, Data, Infrastructure, Testing, and Platforms
Engineering Practice Domains Introduction Welcome to the Engineering Practice Domains at Fearless! This repository serves as a monorepo for our collab
A template to create Rust-based Pure Data externals
Generate a Rust-based external for Pure Data This is a cargo generate template to create a Pure Data external written entirely in Rust. It uses pureda
Rust crate to retrieve public IP address and geolocation data. ๐ฆ
๐ Public IP Address Lookup and Geolocation Information public-ip-address is a simple Rust library for performing public IP lookups from over a dozen
Prefix tree (ordered map and set) data structure using 100% safe Rust
PFX: A 100% safe, blob-oriented prefix tree This crate provides a prefix tree map and set data structure, implemented purely in safe Rust. The API is
Thread-safe cell based on atomic pointers to externally stored data
Simple thread-safe cell PtrCell is an atomic cell type that allows safe, concurrent access to shared data. No std, no data races, no nasal demons (UB)
Automated Solana tool for quick arbitrage, customizable, with real-time data and wallet integration. Trade responsibly.
Solana Arbitrage Trading Tool The Solana Arbitrage Trading Tool is an automated solution crafted to spot and capitalize on arbitrage opportunities wit
Python package for topological data analysis written in Rust. Not limited to just H0 and H1.
Topological Data Analysis (TDA) Contents Installation Compiling from source Roadmap TDA is a python package for topological data analysis written in R
a simple, non-self-describing data-interchange format.
rust-fr 'rust-fr' (aka rust for real) is a simple, non-self-describing data-interchange format. installation You can use either of these methods. Add
A cross-platform tool for embedding GPS data into photographs
nya-exif ไธญๆ | English ไป็ป nya-exif ๆฏไธไธช็จไบๅน้ ็ ง็ GPS ไฟกๆฏ, ๅนถๅๅ ฅๆไปถ EXIF ไฟกๆฏ็ๅทฅๅ ท, ๆฏๆ JPEG ๅ PNG ๅๅๅคง็ธๆบๅๅ็ไธปๆตRAWๆ ผๅผ. ๆฌๅทฅๅ ทๅบไบ Rust ็ผๅ, ๆฏๆๅ จๅนณๅฐไฝฟ็จ Features ๆฏๆ JPEG ๅ PNG ๅๅๅคง
โก๏ธHighly efficient data and string formatting library for Rust.
โก๏ธHighly efficient data and string formatting library for Rust. ๐ Overview Pad and format string slices and generic vectors efficiently with minimal
๐ฆ Evolve your fixed length data files into Apache Arrow tables, fully parallelized!
๐ฆ Evolve your fixed length data files into Apache Arrow tables, fully parallelized! ๐ Overview ... ๐ฆ Installation The easiest way to install evolut
Implements the packet parser for Gran Turismo 7 telemetry data, allowing a developer to retrieve data from a running game.
gran-turismo-query Implements the packet parser for Gran Turismo 7 telemetry data, allowing a developer to retrieve data from a running game. Features
A project management tool for data science and bioinformatics. If you want it, Kerblam it!
Warning kerblam run and kerblam package are complete but still untested. Please do use them, but be careful. Always have a backup of your data and cod
A tool for bulk downloading and exporting the account data inside Solana snapshots.
Memento Memento is a tool used to load and save accounts from old Solana snapshots in Google Cloud Storage. Why should I use Memento? No fighting sola
Tool to retrieve mev-share events data and scan for refunds onchain
MEV-Share Analysis This repository can be used to retrieve historical events sent to mev-share and scan for any refunds that were sent to the users. T
PyO3's PyAny as a serde data format
serde-pyobject PyO3's PyAny as a serde data format Usage Serialize T: Serialize into &'py PyAny: use serde::Serialize; use pyo3::{Python, types::{PyAn
Toolkit for simple calculations related to Data Comunication and Networks (only available in Spanish temporary)
CDR Toolkit Un toolkit creado para la asignatura Comunicaciรณn de Datos y Redes, cursada en la UIB. Es una potente y rรกpida CLI que ayuda a realizar lo
An inline SIMD accelerated hashmap designed for small amount of data.
Small-Map An inline SIMD accelerated hashmap designed for small amount of data. Usage use small_map::SmallMap; // Don't worry about the 16 here. // Wh
A tool to export TiDB database data to files in cases where the TiDB server can't be restored.
tidb-exporter TiDB uses RocksDB as default storage engine(in fact, TiKV uses it). tidb-exporter can export data from pure RocksDB data files even when
Master Rust programming with this comprehensive roadmap
๐ Master Rust programming with this comprehensive roadmap! Explore fundamental and advanced concepts, code examples, and resources.
The ultimate Data Engineering Chadstack. Apache Airflow running Rust. Bring it.
RustOnApacheAirflow The ultimate Data Engineering Chadstack. Apache Airflow running Rust. Bring it. This is part of a larger blog post trying to do so
A Rust crate that implements a range map data structure backed by a Vec.
range_map_vec This crate implements a range map data structure backed by a Vec using binary search. Docs and usage can be found in the corresponding r
Shaping, Processing, and Transforming Data with the Power of Sulfur with Rust
Sulfur WIP https://www.youtube.com/watch?v=PAAvNmoqDq0 "Shaping, Processing, and Transforming Data with the Power of Sulfur" Welcome to the Sulfur pro
Domain modeling. Event sourcing. CQRS.
f(model) - Functional Domain Modeling with Rust Publicly available at crates.io and docs.rs When youโre developing an information system to automate t
RDE1 (Rusty Data Exfiltrator) is client and server tool allowing auditor to extract files from DNS and HTTPS protocols written in Rust. ๐ฆ
Information: RDE1 is an old personal project (end 2022) that I didn't continue development on. It's part of a list of projects that helped me to learn
Manas project aims to create a modular framework and ecosystem to create robust storage servers adhering to Solid protocol in rust.
เคฎเคจเคธเฅ | Manas Solid is a web native protocol to enable interoperable, read-write, collaborative, and decentralized web, truer to web's original vision.
A fast, powerful, flexible and easy to use open source data analysis and manipulation tool written in Rust
fisher-rs fisher-rs is a Rust library that brings powerful data manipulation and analysis capabilities to Rust developers, inspired by the popular pan
Peakrs Dataframe is a library and framework facilitates the extraction, transformation, and loading (ETL) of data.
Peakrs Dataframe Peakrs Dataframe is a library and framework facilitates the extraction, transformation, and loading (ETL) of data. Its first applicat
A fast, powerful, flexible and easy to use open source data analysis and manipulation tool written in Rust
fisher-rs fisher-rs is a Rust library that brings powerful data manipulation and analysis capabilities to Rust developers, inspired by the popular pan
A perfect smoother; A discrete time version of spline smoothing for equally spaced data
Whittaker Smoother Aka Whittaker-Henderson, Whittaker-Eilers Smoother is known as the perfect smoother. Its a discrete-time version of spline smoothin
Postgres Foreign Data Wrapper for Clerk.com API
Pre-requisites Postgres-15 Rust pgrx Getting Started To run the program locally, clone the repository git clone https://github.com/tembo-io/clerk_fdw.
Prototype risk modeling simulation for Portfolio using Arbiter.
proto-sim Prototype simulation using Arbiter as the simulation & agent engine. Build & Run build.sh cargo run Arbiter config The arbiter.toml config
Collection of immutable and persistent data structures written in Rust, inspired by the standard libraries found in Haskell, Closure and OCaml
PRust: (P)ersistent & Immutable Data Structures in (Rust) This library houses a collection of immutable and persistent data structures, inspired by th
Terminal UI for leetcode. Lets you browse questions through different topics. View, solve, run and submit questions from TUI.
Leetcode TUI Use Leetcode in your terminal. Why this TUI: My motivation for creating leetcode-tui stemmed from my preference for tools that are lightw
A Rust library for building modular, fast and compact indexes over genomic data
mazu A Rust library for building modular, fast and compact indexes over genomic data Mazu (ๅชฝ็ฅ)... revered as a tutelary deity of seafarers, including
Transfer data with the LCU in the command-line interface! ๐ฅ
llux llux (short for LCU Lux) is an open-source CLI tool to easily transfer data with the LCU without any script or code setup, hehe! Preview Download
Leptos Query - a robust asynchronous state management library for Leptos,
Leptos Query is a robust asynchronous state management library for Leptos, providing simplified data fetching, integrated reactivity, server-side rendering support, and intelligent cache management.
A naive buffered/sync channel implementation in Rust, using the queue data structure
buffered-queue-rs Introduction This is my attempt at a simple and very naive buffered/synced queue implementation in Rust. The base thread-safe queue
(early experiments toward) a version-control system for structured data
chit: (early experiments toward) a version-control system for structured data please note, very little is actually implemented here. this is not usefu
App to collect ram/cpu usage from OS and show it in pretty graphs
System info collector This is simple app to collect data about system cpu and memory usage over time. After collecting results into csv file, html fil
๐พ๐ฆ ๐บ๐ฎ๐๐๐ฒ๐ฟ๐ถ๐ป๐ด ๐ฟ๐๐๐ ๐๐ต๐ฒ ๐ฟ๐ถ๐ด๐ต๐ ๐๐ฎ๐: ๐๐ต๐ฟ๐ผ๐๐ด๐ต ๐ฐ๐น๐ฎ๐๐๐ถ๐ฐ๐ฎ๐น ๐ฐ๐ผ๐บ๐ฝ๐๐๐ฒ๐ฟ ๐๐ฐ๐ถ๐ฒ๐ป๐ฐ๐ฒ ๐ฝ๐ฟ๐ผ๐ฏ๐น๐ฒ๐บ๐
๐ฆ ๐พ algorithms and data structures in Rust ๐ฃ Rust is a strong-typed (you can't work around the type system, like Python) and static-typed language
reth-indexer reads directly from the reth db and indexes the data into a postgres database all decoded with a simple config file and no extra setup alongside exposing a API ready to query the data.
reth-indexer reth-indexer reads directly from the reth db and indexes the data into a postgres database all decoded with a simple config file and no e
cryo is the easiest way to extract blockchain data to parquet, csv, or json
โ๏ธ ๐ง cryo ๐ง โ๏ธ cryo is the easiest way to extract blockchain data to parquet, csv, or json cryo is also extremely flexible, with many different opti
A vertically scalable stream processing framework focusing on low latency, helping you scale and consume financial data feeds.
DragonflyBot A vertically scalable stream processing framework focusing on low latency, helping you scale and consume financial data feeds. Design The
CYFS๏ผNext Generation Protocol Family to Build Web3
CYFS is the next-generation technology to build real Web3 by upgrading the basic protocol of Web (TCP/IP+DNS+HTTP). It has a subversive architectural design that everyone brings their own OOD (Owner Online Device) to form a truly decentralized network.
Compact, efficient data structures in contiguous byte arrays
Sokoban Compact, efficient data structures in contiguous byte arrays. Benchmarks Based on simple benchmarks, the naive performance of Sokoban data str
Rust library for concurrent data access, using memory-mapped files, zero-copy deserialization, and wait-free synchronization.
mmap-sync mmap-sync is a Rust crate designed to manage high-performance, concurrent data access between a single writer process and multiple reader pr
Rust For Data book
Rust For Data This book is available for free online at https://rustfordata.com You can find the source code for book in ./rust4data-book This book is
A simulator for Street Fighter 6 battle data.
sf6_sim A simulator for Street Fighter 6 battle data. Very early in development. It can currently simulate boxes, cancel lists, and movement. Online v
The most primitive and the fastest implementation of a fixed-size last-in-first-out stack on stack in Rust, for Copy-implementing types
This is the simplest and the fastest (faster than Vec!) implementation of a last-in-first-out stack data structure, on stack, when stack elements are
Read specialized NGS formats as data frames in R, Python, and more.
oxbow Read specialized bioinformatic file formats as data frames in R, Python, and more. File formats create a lot of friction for computational biolo
Extract data from helium-programs via Solana RPC and serves it via HTTP
hnt-explorer This application extracts data from helium-programs via Solana RPC and serves it via HTTP. There are CLI commands meant to run and test t
Platform independent data channels for WebRTC/Rust.
preach Platform independent data channels Preach provides an abstraction for WebRTC data channels that runs on both native and web platforms. Preach m
A Rust program that analyzes your TikTok data.
The TikTok JSON analyzer This is a program to analyze your TikTok data and calculate these statistics : Number of logins (in the last 6 months) and lo
Nodium is an easy-to-use data analysis and automation platform built using Rust, designed to be versatile and modular.
Nodium is an easy-to-use data analysis and automation platform built using Rust, designed to be versatile and modular. Nodium aims to provide a user-friendly visual node-based interface for various tasks.
Reddit - Liberate your Reddit Chats. This tool will export your reddit chats into a plethora of formats
Rexit Rexit - Liberate your Reddit Chats. This tool will export your Reddit chats into a plethora of formats Tool to export Reddit chats into a variet
๐ The fastest map possible in Rust, where keys are integers and the capacity is fixed (faster than Vec!)
It is an alternative on-heap implementation of a map with keys of type usize and a fixed capacity. It works much faster than a standard HashMap becaus
bare-bones "reactive programming" (change propogation) using a central data dependency graph
mini-rx: Tiny reactive programming change propagation a la scala.rx Cargo documentation Example use mini_rx::*; fn example() { // setup let side_ef
Active Directory data collector for BloodHound written in Rust. ๐ฆ
RustHound Summary Limitation Description How to compile it? Using Makefile Using Dockerfile Using Cargo Linux x86_64 static version Windows static ver
UnravelSports repository to display Football/Soccer tracking data in 3D using Rust & Bevy
Introduction This project is a proof-of-concept by UnravelSports to show football/soccer data in 3D using the Rust programming language and the Bevy g
Poisson intensity of limit order execution, calibration of parameters A and k using level 1 tick data
Poisson intensity of limit order execution, calibration of parameters A and k using level 1 tick data Description A limit order placed at a price St ยฑ
A simpler and 5x faster alternative to HashMap in Rust, which doesn't use hashing and doesn't use heap
At least 5x faster alternative of HashMap, for very small maps. It is also faster than FxHashMap, hashbrown, ArrayMap, and nohash-hasher. The smaller
A special rope, designed to work with any data type that is not String
AnyRope AnyRope is an arbitrary data type rope for Rust, designed for similar operations that a rope would do, but targeted at data types that are not
This repository aims to organize codes related to data structures in Rust. ๐ฆ
Rust Data Structure A project with the objective of introducing the main concepts about data structure using Rust! Explore the docs and learn Rust ยป R
Efficient and fast querying and parsing of GTDB's data
xgt xgt is a Rust tool that enables efficient querying and parsing of the GTDB database. xgt consists of a collection of commands mirroring the GTDB A
Type safe multipart/form-data handling for axum.
axum_typed_multipart Designed to seamlessly integrate with Axum, this crate simplifies the process of handling multipart/form-data requests in your we
A fast rendezvous in rust where data can optionally be swapped between the two threads.
rendezvous_swap A rendezvous is an execution barrier between a pair of threads, but this crate also provides the option of swapping data at the synchr
Rosenpass is a formally verified, post-quantum secure VPN that uses WireGuard to transport the actual data.
Rosenpass README This repository contains A description of the Rosenpass protocol The reference implementation of the protocol โ the rosenpass tool A
A CLI and library to convert data to sound, and vice versa (dependency-free)
Data to sound A simple crate to convert data to sound, and sound to data. The sound file format is wave (.wav). You can use it as a library or as a co
Testing out if Rust can be used for a normal Data Engineering Pipeline.
RustForDataPipelines Testing out if Rust can be used for a normal Data Engineering Pipeline. Check out the full blog post here. https://www.confession
RedMaple offers an oppinionated yet extremely flexible data modeling system based on events for back-end applications.
RedMaple offers an oppinionated yet extremely flexible data modeling system based on events for back-end applications.
A 3D modeling and rendering programming language utilizing SDFs.
ForgedThoughts is a modeling and rendering programming language utilizing SDFs and is in early development. For documentation and examples see the Web
Blazingly fast Rust CLI app to sync data from a folder of excel workbooks into generated c# code for unity usage
Extensions supported ( .xls, .xlsx, .xlsm, .xlsb, .xla, .xlam, .ods ) Speed Test Image shows the results of 5000defs synced from 2 workbooks and 5 she
Core Fiberplane data models and methods for transforming them (templates, providers, markdown conversion)
fiberplane This repository is a monorepo for Rust code that is used throughout Fiberplane's product. Overview base64uuid - A utility for working with
๐ฅ ๐ (fwdt) "few word do trick" is a cross platform manual fast logger
Few Word Do Trick (fwdt) Few Word Do Trick (fwdt) is a cross-platform general purpose fast logger for humans that supports incomplete csvs for a bette
A quick way to decode a contract's transaction data with only the contract address and abi.
tx-decoder A quick way to decode a contract's transaction data with only the contract address and abi. E.g, let tx_data = "0xe70dd2fc00000000000000000
A high-performance WebSocket integration library for streaming public market data. Used as a key dependency of the `barter-rs` project.
Barter-Data A high-performance WebSocket integration library for streaming public market data from leading cryptocurrency exchanges - batteries includ
A real-time data backend for browser-based applications.
DriftDB DriftDB is a real-time data backend for browser-based applications. For more information, see driftdb.com. Structure of this repo docs/: main
RusQTTbom takes weather data from the Bureau of Meteorology (BOM) and publishes that data via MQTT messages.
RusQTTbom RusQTTbom collects weather data from the Bureau of Meterology (BOM) then publishes said data locally via MQTT messages. BOM weather data is
Data analysis infrastructure for the Neo N3 blockchain.
Shrike Shrike is a set of tools built for the purpose of Neo blockchain data analysis. The infrastructure comprises three components: Indexer - Synchr
Command-line tool to convert Apple HealthKit data to a SQLite database.
healthkit-to-sqlite Command-line tool to convert Apple HealthKit data to a SQLite database. Getting Started Open the Health app on your iOS device. Cl
A compact generational arena data structure for Rust.
Compact Generational Arena This crate provides ArenaT, a contiguous growable container which assigns and returns IDs to values when they are added t
Experimenting with Rust's fundamental data model
ferrilab Redefining the Rust fundamental data model bitvec funty radium Introduction The ferrilab project is a collection of crates that provide more
Library and proc macro to analyze memory usage of data structures in rust.
Allocative: memory profiler for Rust This crate implements a lightweight memory profiler which allows object traversal and memory size introspection.
Inspect and recreate in-memory data structures
YClass A program that allows you to inspect and recreate data structures of other processes. Installation To compile YClass you will need Rust. git cl
Code examples, data structures, and links from my book, Rust Atomics and Locks.
This repository contains the code examples, data structures, and links from Rust Atomics and Locks. The examples from chapters 1, 2, 3, and 8 can be f
Stdto provides a set of functional traits for conversion between various data representations.
Stdto stdto provides a set of functional traits for conversion between various data representations. | Examples | Docs | Latest Note | stdto = "0.13.0
simple and powerful IP architecture in Rust.
ipzone Ipzone provides a simple and powerful IP architecture to Rust. | Examples | Docs | Latest Note | ipzone = "0.4.0" Examples use ipzone::prelude:
Python+Rust implementation of the Probabilistic Principal Component Analysis model
Probabilistic Principal Component Analysis (PPCA) model This project implements a PPCA model implemented in Rust for Python using pyO3 and maturin. In
Pure rust library for reading / writing DNG files providing access to the raw data in a zero-copy friendly way.
DNG-rs โ A pure rust library for reading / writing DNG files providing access to the raw data in a zero-copy friendly way. Also containing code for re
๐ My solutions to Advent of Code 2022.
๐ advent-of-code-2022 Overview This repository contains my solutions to Advent of Code 2022. I decided to take this year as an opportunity to learn R
Displaying data for the Soroban Futurenet.
Soroban Fiddle https://leighmcculloch.github.io/soroban-fiddle Web frontend-only application that displays data on the Soroban Futurenet network. Feat