Proof-of-concept for a memory-efficient data structure for zooming billion-event traces

Overview

Proof-of-concept for a gigabyte-scale trace viewer

This repo includes:

  • A memory-efficient representation for event traces
  • An unusually simple and memory-efficient range aggregation index data structure (IForestIndex) for zooming traces of billions of events at 60fps
  • A proof-of-concept Druid UI to demo efficient trace zooming, that isn't remotely useable as a real trace viewer.
You might also like...
๐Ÿ‹: A General Lock following paper
๐Ÿ‹: A General Lock following paper "Optimistic Lock Coupling: A Scalable and Efficient General-Purpose Synchronization Method"

Optimistic Lock Coupling from paper "Optimistic Lock Coupling: A Scalable and Efficient General-Purpose Synchronization Method" In actual projects, th

Safe, efficient, and ergonomic bindings to Wolfram LibraryLink and the Wolfram Language

wolfram-library-link Bindings to the Wolfram LibraryLink interface, making it possible to call Rust code from the Wolfram Language. This library is us

An efficient runtime for asynchronous applications in Rust.

PhotonIO PhotonIO is an efficient runtime for asynchronous applications in Rust. Features Asynchronous filesystem and networking I/O for Linux based o

A pure-rust(with zero dependencies) fenwick tree, for the efficient computation of dynamic prefix sums.

indexset A pure-rust(with zero dependencies, no-std) fenwick tree, for the efficient computation of dynamic prefix sums. Background Did you ever have

The Fast Vector Similarity Library is designed to provide efficient computation of various similarity measures between vectors.
The Fast Vector Similarity Library is designed to provide efficient computation of various similarity measures between vectors.

Fast Vector Similarity Library Introduction The Fast Vector Similarity Library is designed to provide efficient computation of various similarity meas

This crate allows to generate a flat binary with the memory representation of an ELF.

flatelf Library This crate allows to generate a flat binary with the memory representation of an ELF. It also allows to generate a FLATELF with the fo

High concurrency, RealTime, In-memory storage inspired by erlang mnesia
High concurrency, RealTime, In-memory storage inspired by erlang mnesia

DarkBird is a Document oriented, high concurrency in-memory Storage, also persist data to disk to avoid loss any data The darkbird provides the follow

Support SIMD low-memory overhead and high-performance adaptive radix tree.

Artful Artful is an adaptive radix tree library for Rust. At a high-level, it's like a BTreeMap. It is based on the implementation of paper, see The A

A additional Rust compiler pass to detect memory safe bugs of Rust programs.

SafeDrop A additional Rust compiler pass to detect memory safe bugs of Rust programs. SafeDrop performs path-sensitive and field-sensitive inter-proce

Owner
Tristan Hume
Open source enthusiast. Projects in Rust, Ruby, C++, JS, Objective-C and more.
Tristan Hume
A Rust crate that implements a range map data structure backed by a Vec.

range_map_vec This crate implements a range map data structure backed by a Vec using binary search. Docs and usage can be found in the corresponding r

Microsoft 9 Sep 8, 2023
A HashMap/Vector hybrid: efficient, ordered key-value data storage in Rust.

hashvec A HashVec is a hash map / dictionary whose key-value pairs are stored (and can be iterated over) in a fixed order, by default the order in whi

Skye Terran 2 May 16, 2022
Rust library to facilitate event-driven programming.

Squeak Squeak is a zero-dependency Rust library to facilitate event-driven programming. Examples use squeak::{Delegate, Response}; let on_damage_rece

Antoine Gersant 58 Dec 31, 2022
Domain modeling. Event sourcing. CQRS.

f(model) - Functional Domain Modeling with Rust Publicly available at crates.io and docs.rs When youโ€™re developing an information system to automate t

Fraktalio 12 Oct 7, 2023
Harvest Moon: (More) Friends of Mineral Town event script compiler

mary This is a script compiler for Harvest Moon: Friends of Mineral Town and Harvest Moon: More Friends of Mineral Town for the GBA. The end goal is f

Nat (Stan) 5 Oct 23, 2023
Simple Event-Driven Microservice Architecture in Rust

Simple Event-Driven Microservice Architecture in Rust is an open-source project showcasing the simplicity and efficiency of building microservices in Rust. This minimalistic project demonstrates an e-commerce backend system, featuring just two microservices: Catalog and Order.

James Mallon 4 Dec 19, 2023
Library and proc macro to analyze memory usage of data structures in rust.

Allocative: memory profiler for Rust This crate implements a lightweight memory profiler which allows object traversal and memory size introspection.

Meta Experimental 19 Jan 6, 2023
Rust library for concurrent data access, using memory-mapped files, zero-copy deserialization, and wait-free synchronization.

mmap-sync mmap-sync is a Rust crate designed to manage high-performance, concurrent data access between a single writer process and multiple reader pr

Cloudflare 97 Jun 26, 2023
Generate SUMMARY.md files based on your book's file structure

mdbook-autosummary Generate a SUMMARY.md for your mdBook based on your folder structure! Warning The implementation is hacky and has several limitatio

Hyper 3 Sep 30, 2023
An expression based data notation, aimed at transpiling itself to any cascaded data notation.

Lala An expression oriented data notation, aimed at transpiling itself to any cascaded data notation. Lala is separated into three components: Nana, L

null 37 Mar 9, 2022