Rust Data-engineering Resources

Unix-socket based client/server In order to dig into Sōzu channels, I had to dig into the workings of unix sockets. What this repo contains a small so

Docs This library provides a data view for reading and writing data in a byte array. This library requires feature(generic_const_exprs) to be enabled.

notion-entry NOTE: This project is a work in progress and has a pile of FIXMEs This program lets you input data into Notion from the command line. It

odjitter This crate contains an implementation of the ‘jittering’ technique for pre-processing origin-destination (OD) data. Jittering in a data visua

Shipyard ⚓ Shipyard is an Entity Component System focused on usability and speed. If you have any question or want to follow the development more clos

Rust Astro Notation A library for transcoding between hexadecimal strings in Astro Notation Format and Native Rust data types. Usage In your Cargo.tom

OnTimeDB Scalable and fast data store optimised for time series data such as financial data, events, metrics for real time analysis OnTimeDB is a time

This repo provide some useful data-structures, such as: Array, HashMap, HashSet, BTreeMap, BTreeSet, etc... That lives on the stack. It's a good choic

covid-dc Parses COVID-19 testing data from DC government ArcGIS APIs Example debug output from cargo run RapidSite { attributes: RapidSiteAttribut

This Week in Rust Content for this-week-in-rust.org. Made available under CC-BY-SA. All code Copyright 2014 Ember Arlynx, made available under the MIT

A simple and convenient way to bundle owned data with a borrowing type. The Problem One of the main selling points of Rust is its borrow checker, whic

versioning-solana This repo demonstrates ONE rudimentary way to upgrade/migrate account data changes with solana program changes. What is data version

showata A library of to show data (in browser, evcxr_jupyter) as table, chart.... The crate provides display for: image vector and slice (as table) nd

Deduplicating Training Data Makes Language Models Better This repository contains code to deduplicate language model datasets as descrbed in the paper

kddbscan-rs Rust implementation of the kddbscan clustering algorithm. From the authors of kDDBSCAN algorithm. Due to the adoption of global parameters

som Self Organizing Map Pre-requisites Setup rust To download Rustup and install Rust, run the following in your terminal, then follow the on-screen i

Apache Arrow Powering In-Memory Analytics Apache Arrow is a development platform for in-memory analytics. It contains a set of technologies that enabl

Create, share, fetch and model Atomic Data! This repo consists of three components: A library, a server and a CLI. atomic-server Status: Beta. Breakin

The Algorithms - Rust All algorithms implemented in Rust (for educational purposes) These are for demonstration purposes only. RESTART BUILD Sort Algo

Contest Algorithms in Rust A collection of classic data structures and algorithms, emphasizing usability, beauty and clarity over full generality. As

This library provides implementations of many algorithms and data structures that are useful for bioinformatics. All provided implementations are rigorously tested via continuous integration.

Overview This repository contains the Rust source code for the algorithms in the textbook Algorithms, 4th Edition by Robert Sedgewick and Kevin Wayne.

My Leetcode Solution in Rust Run cargo run {id} to initialize the template submission file of "question #id". Run cargo test test_{id} to test the sol

Rust Algorithm Club 🚧 🚧 This repo is under construction. Most materials are written in Chinese. Check it out here if you are able to read Chinese. W

Coding Challenge LeetCode/Hackerrank e.t.c Using this as an opportunity to improve my knowledge of rust lang If you found this repo useful to you, add

Oinky Oinky is a static site generator, written in Rust, that compiles Handlebars templates and Markdown files into a static site. Oinky also enables

valq   valq provides a macro for querying and extracting value from structured data in very concise manner, like the JavaScript syntax. Look & Feel: u

UNIC: Unicode and Internationalization Crates for Rust https://github.com/open-i18n/rust-unic UNIC is a project to develop components for the Rust pro

Ronin Hello there! Ronin is a ultra-speed Stacks API server. It's super lightweight, but scales easily. Why are we making this? Because we don't like

Clnooms Assembly-level implementation of various interesting data types on Rust Table of Contents Data Types Half-precision floating point (f16) Exten

RedisJSON RedisJSON is a Redis module that implements ECMA-404 The JSON Data Interchange Standard as a native data type. It allows storing, updating a

⚡ Installation Linux curl -s https://raw.githubusercontent.com/xornet-cloud/Reporter/main/scripts/install.sh | sudo bash Windows Invoke-Command -Scrip

grant-rs An open-source project that aims to manage Postgres/Redshift database roles and privileges in GitOps style, written in Rust. Home | Documenta

dts A simple tool to deserialize data from an input encoding, transform it and serialize it back into an output encoding. Requires rust = 1.56.0. Ins

RustCrypto: Authenticated Encryption with Associated Data (AEAD) Algorithms Collection of Authenticated Encryption with Associated Data (AEAD) algorit

📝 Web-based, reactive Datalog notebooks for data analysis and visualization

Mozilla Data Documentation This documentation was written to help Mozillians analyze and interpret data collected by our products, such as Firefox and

BugHunt, Rust This project is aiming to provide "stateful" QuickCheck models for Rust's standard library. That is, we build up a random list of operat

Welcome to Falcon Falcon is a formal binary analysis framework in Rust. Expression-based IL with strong influences from RREIL and Binary Ninja's LLIL.

libgoblin Documentation https://docs.rs/goblin/ changelog Usage Goblin requires rustc 1.40.0. Add to your Cargo.toml [dependencies] goblin = "0.4" Fea

arch-audit pkg-audit-like utility for Arch Linux. Based on data from security.archlinux.org collected by the awesome Arch Security Team. Installation

Redcar A real-time event-oriented data-hub, inspired by the data hub. It is: Universal: the front end uses gRPC to provide services. Fast: benchmarked

This crate named as ABtree but this not means it is a novel data sturcture. It’s just AVL tree and Btree. For the Btree, what makes it different from

skog Swedish for "forest". A pure rust implementation of Adobe's stlab::forest data structure. Introduction A "forest" is a tree-like data structure.

facelink_rs A VtubeStudio plugin that allows iFacialMocap to stream data to the app, enabling full apple ARkit facial tracking to be used for 2D Vtube

algorithm-rs algorithm-rs is common data structures and algorithms for competitive programming in Rust. Contents TBA How To Contribute Contributions a

Percival is a declarative data query and visualization language. It provides a reactive, web-based notebook environment for exploring complex datasets, producing interactive graphics, and sharing results.

grusterylist: makes grocery lists, written in Rust grusterylist uses and can add to local libraries of user-added recipes and grocery items to put tog

data-exporter A prometheus exporter that scrapes remote data or local files and converts them to prometheus metrics. It is similar to json_exporter, b

sucds: Succinct data structures in Rust sucds contains some succinct data structures written in Rust. Data structures So far, the following data struc

ris-live-rs Provides parsing functions for RIS-Live real-time BGP message stream JSON data. The main parsing function, parse_ris_live_message converts

Docchi is a diff-based data management language to implement unlimited undo, auto-save for games, and cloud-apps which needs to save very often. User'

cds - Collection of Data Structures !!! UNDER CONSTRUCTION !!! The version v0.0.1 is a crates.io placeholder. License Licensed under either of Apache

Flight Tracking ERAU SE300 Description Software that allows for weather and plane tracking to facilitate the user in looking at plane paths. Many peop

vmemory Rust library for reading/writing memory in other processes for Windows, macOS, Linux, and in the future potentially, BSD variants. Rationale A

ftcp A tcp proxy server/client which exchange the data in temp files 通过在临时文件中交换数据来进行TCP代理的一个服务端/客户端 学校内网中有针对教学楼的防火墙导致教室电脑难以上网（ 但学校内建有公共ftp服务器，因此就有了这个借

Twitter stream user info extractor This project lets you parse JSON data from the Twitter API or other sources to extract some basic user information,

grant.rs Manage Redshift database roles and privileges in GitOps style. Usage Install binary from crates.io cargo install grant Using grant tool: $ gr

DAB 👀 Overview An Internet Computer open internet service for data. All the data an IC app needs to make a seamless experience, accessible directly o

graph A library that can be used as a building block for high-performant graph algorithms. Graph provides implementations for directed and undirected

An example repository on how to start building graph applications on streaming data. Just clone and start building 💻 💪

World's first decentralized real-time data warehouse, on your laptop Docs | Demo | Tutorials | Examples | FAQ | Chat Get Started Watch this introducto

Eclipse Zenoh-Flow Zenoh-Flow provides a zenoh-based dataflow programming framework for computations that span from the cloud to the device. ⚠️ This s

AppFlowy is an open-source alternative to Notion. You are in charge of your data and customizations. Built with Flutter and Rust.

lf-queue A lock-free multi-producer multi-consumer unbounded queue. Examples [dependencies] lf-queue = "0.1" Single Producer - Single Consumer: use lf

This crate aims to simplify generating gui from a data structure.

Shotover: L7 data-layer proxy

Download mongodb FTDC data Command line tool and lib crate to download FTDC data from mongodb clusters to investigate with e.g. keyhole. Crate For the

The comparable crate defines the trait [Comparable], along with a derive macro for auto-generating instances of this trait for most data types. Primar

Delta: Structural differencing in Rust The delta crate defines the trait Delta, along with a derive macro for auto-generating instances of this trait

rust-data-structures Library containing various Data Structures implemented using Rust. Running You can test the library by running cargo test, an exa

ogma Welcome to the ogma project! ogma is a scripting language focused on ergonomically and efficiently processing tabular data, with batteries includ

flaco Perhaps the fastest and most memory efficient way to pull data from PostgreSQL into pandas and numpy. 🚀 Have a gander at the initial benchmarks

Writing A document-code sync tools for document engineering. Writing 是一个自动 “文档-代码” 同步工具。解析 Markdown 中的代码定义，读取目标代码，并嵌入到新的文档中。 Language parse support by

BGPKIT Parser BGPKIT Parser aims to provides the most ergonomic MRT/BGP message parsing Rust API. BGPKIT Parser has the following features: performant

BGPKIT Broker BGPKIT Broker is a online data API service that allows users to search for publicly available BGP archive files by time, collector, proj

enum_pipline Provides a way to use enums to describe and execute ordered data pipelines. 🦀 🐾 I needed a succinct way to describe 2d pixel map operat

📊 Cube.js — Open-Source Analytics API for Building Data Apps

Hyaline-SMR This crate provides garbage collection using hyaline algorithm for building concurrent data structures. When a thread removes an object fr

Chainlink Terra Developing Requirements This demo requires the following components: Rust: rustup with cargo 1.44.1+. rustc and cargo 1.44.1+. Install

Cyberdeck A library for easily creating WebRTC data channel connections in Rust.

Zenith substitutes PostgreSQL storage layer and redistributes data across a cluster of nodes

prometheus-absent-data-alert-rule-generator prometheus-absent-data-alert-rule-generator is a tool to generate alerts for missing data in time-series t

Double-Linked List This crate provides a doubly-linked list with owned nodes, implemented as a cyclic list. Usage First, add dependency in your Cargo.

tuinance Tuinance is a performant TUI program to display financial data, written completely in Rust. All data is gathered through the Yahoo Finance AP

Canadian Federal Elections election is a small Rust program for processing vote data from Canadian Federal Elections. After building, see election --h

crypto-cli-tools A collection of CLI tools to process cryptocurrency data. Install: RUSTFLAGS="-C target-cpu=native" cargo install crypto-cli-tools cr

#Zei: Findora's Cryptographic Library Zei is a library that provide tools to create and verify public transaction with confidential data. Support: Bas

a graphical tool to visualize binary data

DataBrush DataBrush is a simple library for displaying structured data in the command line. In supports dividing data into chunks and highlighting cer

NeoTwitch Arch Network broadcaster Chat (message buffer) If the message buffer is full then shut down Channel point events If the message buffer is fu

Planter Blazingly fast and simple data generation & seeding for MongoDB Installation Use the package manager cargo to install planter. Add the followi

blendtool Command line tool to extract various data from Blender .blend files. Currently supports dumping Eevee irradiance volumes to .dds, new featur

📺(tv) Tidy Viewer is a cross-platform CLI csv pretty printer that uses column styling to maximize viewer enjoyment.

A time traveling resource monitor for modern Linux systems

ruborute Are you 暴龍天 ?. The ruborute is an interactive command-line tool to get asphyxia@sdvx gaming data. asphyxia-core/plugins: https://github.com/a

🚖 TestDrive What does it do? TestDrive automatically scrapes input/output data from BOJ(Baekjoon Online Judge) and runs tests for your executable bin

arrow-odbc Fill Apache Arrow arrays from ODBC data sources. This crate is build on top of the arrow and odbc-api crate and enables you to read the dat

TensorBase is a new big data warehousing with modern efforts.

A toolkit designed to be a foundation for applications to monitor their performance.