Lightweight & Fast LWW CRDT Table

Overview

Lightweight & Fast LWW CRDT Table

  • Supports millions of operations per second
  • Suitable for real-time collaboration
  • Supports delta updates
  • It is a CRDT, which means it possesses strong eventual consistency and can be easily used in distributed environments. It allows for syncing tabular data via peer-to-peer connections, supports end-to-end encryption, and facilitates the development of local-first applications.

Currently, it functions solely as an in-memory table with a unique persistence format and is not a comprehensive database solution. It is not suitable for sparse tables yet.

Usage

Update DB

use lww_table::LwwDb;
use std::collections::HashSet;

pub fn main() {
    let mut db = LwwDb::new();
    db.set("my_table", "row1", "col1", 1);
    db.set("my_table", "row1", "col2", 2);
    db.set("my_table", "row2", "col1", 3);
    db.set("my_table", "row2", "col2", 4);
    assert_eq!(db.get_cell("my_table", "row1", "col1").unwrap(), &1.into());
    assert_eq!(
        db.iter_row("my_table", "row1").collect::<HashSet<_>>(),
        HashSet::from([("col1", &1.into()), ("col2", &2.into())])
    );
    println!("{}", db);
    db.delete_row("my_table", "row1");
    println!("{}", db);
}

Output:

LwwDb {
  # my_table
  +--------+------+------+
  | row_id | col2 | col1 |
  +--------+------+------+
  | row1   | 2    | 1    |
  +--------+------+------+
  | row2   | 4    | 3    |
  +--------+------+------+
}

LwwDb {
  # my_table
  +--------+------+------+
  | row_id | col2 | col1 |
  +--------+------+------+
  | row1   | null | null |
  +--------+------+------+
  | row2   | 4    | 3    |
  +--------+------+------+
}

Sync DB

use lww_table::{LwwDb, VectorClock};

pub fn main() {
    let mut db = LwwDb::new();
    db.set("my_table", "row1", "col1", 1);
    db.set("my_table", "row1", "col2", 2);
    db.set("my_table", "row2", "col1", 3);
    db.set("my_table", "row2", "col2", 4);
    let mut db2 = LwwDb::new();
    db2.set("my_table", "row3", "col1", 3);
    db2.set("my_table", "row3", "col2", 5);

    let version: Vec<u8> = db2.version().encode();
    // you can send version via network, save to disk, etc.
    let bytes: Vec<u8> = db.export_updates(VectorClock::decode(&version));
    // you can send bytes via network, save to disk, etc.
    db2.import_updates(&bytes);

    // sync in the other direction
    db.import_updates(&db2.export_updates(db.version().clone()));

    // now two databases are in sync
    assert!(db.check_eq(&mut db2));
    println!("{}", db);
    println!("{}", db2);
}

Output:

LwwDb {
  # my_table
  +--------+------+------+
  | row_id | col2 | col1 |
  +--------+------+------+
  | row1   | 2    | 1    |
  +--------+------+------+
  | row2   | 4    | 3    |
  +--------+------+------+
  | row3   | 5    | 3    |
  +--------+------+------+
}

LwwDb {
  # my_table
  +--------+------+------+
  | row_id | col2 | col1 |
  +--------+------+------+
  | row1   | 2    | 1    |
  +--------+------+------+
  | row2   | 4    | 3    |
  +--------+------+------+
  | row3   | 5    | 3    |
  +--------+------+------+
}

Performance

For a table created by the following code:

let mut db = lww_table::LwwDb::new();
for i in 0..100_000 {
    for j in 0..10 {
        db.set("table", &i.to_string(), j.to_string().as_str(), i + j);
    }
}

The benchmark is conducted on MacBook Pro (13-inch, M1, 2020).

Set 344.884ms
Export updates 272.93475ms
Updates size 2552394 bytes
Import updates 329.477459ms
Export snapshot 26.450417ms
Snapshot size 323420 bytes
Import snapshot 143.268833ms
You might also like...
A blazingly fast rust-based bionic reader for blazingly fast reading within a terminal console 🦀
A blazingly fast rust-based bionic reader for blazingly fast reading within a terminal console 🦀

This Rust-based CLI tool reads text and returns it back in bionic reading format for blazingly fast loading and even faster reading! Bionic reading is

Manage lightweight VMs created from OCI images
Manage lightweight VMs created from OCI images

krunvm krunvm is a CLI-based utility for managing lightweight VMs created from OCI images, using libkrun and buildah. Features Minimal footprint Fast

A lightweight terminal menu for Rust
A lightweight terminal menu for Rust

youchoose A simple, easy to use command line menu for Rust. Usage There are two methods you need to be familiar with to get started: Menu::new which t

a Rust library implementing safe, lightweight context switches, without relying on kernel services

libfringe libfringe is a library implementing safe, lightweight context switches, without relying on kernel services. It can be used in hosted environ

Lemurs - A lightweight TUI display/login manager written in Rust 🐒
Lemurs - A lightweight TUI display/login manager written in Rust 🐒

Lemurs 🐒 A TUI Display/Login Manager written in Rust WIP: Whilst the project is working and installable, there are still a lot of bugs and limitation

A lightweight ShareX-compatible image uploader server

Imitari Imitari is a project to provide an easy and lightweight server for images. It is compatible with software like ShareX. The projects goals are

A lightweight but incredibly powerful and feature-rich BitTorrent tracker. Supports UDP + HTTP(S) and a private tracker mode.

Torrust Tracker Project Description Torrust Tracker is a lightweight but incredibly powerful and feature-rich BitTorrent tracker made using Rust. Feat

A lightweight focus CLI tool built with Rust

A lightweight pomodoro focus tool with cross-platform desktop notifications on Linux, MacOS and Windows.

82 fun and easy to use, lightweight, spinners for Rust, with minimal overhead.
82 fun and easy to use, lightweight, spinners for Rust, with minimal overhead.

Spinners for Rust 82 fun and easy to use, lightweight, spinners for Rust, with minimal overhead, all the way from simple dots, to fun emoji based "spi

Owner
Zixuan Chen
Zixuan Chen
Make data-driven table rendering easy with Dioxus

Dioxus Table Make data-driven table rendering easy with Dioxus Installation Until the next release of Dioxus this requires Dioxus nightly from git. Th

null 9 Oct 9, 2022
A more compact and intuitive ASCII table in your terminal: an alternative to "man 7 ascii" and "ascii"

asciit A more compact and intuitive ASCII table in your terminal: an alternative to man 7 ascii and ascii. Colored numbers and letters are much more e

Qichen Liu 刘启辰 5 Nov 16, 2023
A lightweight and super fast cli todo program written in rust under 200 sloc

todo A lightweight and super fast cli todo program written in rust under 200 sloc installation AUR package: todo-bin use cargo build --release to comp

sioodmy 243 Dec 24, 2022
A safe, fast, lightweight embeddable scripting language written in Rust.

Bud (budlang) A safe, fast, lightweight embeddable scripting language written in Rust. WARNING: This crate is not anywhere near being ready to publish

Khonsu Labs 13 Dec 27, 2022
A fast, simple and lightweight Bloom filter library for Python, fully implemented in Rust.

rBloom A fast, simple and lightweight Bloom filter library for Python, fully implemented in Rust. It's designed to be as pythonic as possible, mimicki

Kenan Hanke 91 Feb 4, 2023
A blazingly fast & lightweight Obsidian CLI ⚡️

obs - the Obsidian CLI ⚡️ Connecting your second brain to the termainl - blazing fast ⚡️ Note ?? obs is under active development and currently only su

null 5 Mar 4, 2023
A fast and lightweight HTTP server implementation in Rust.

server_nano A tiny, fast, and friendly web server written in rust and inspired by express. It uses may to coroutines Usage First, add this to your Car

Jonny Borges 5 May 2, 2023
An easy, fast and lightweight tool to create notes in terminal.

An easy and lightweight tool to create notes in terminal. Features Make notes faster. Manage your notes directly in terminal. Lightweight and fast. Ea

Konstantin Zhigaylo 5 May 24, 2023
A self-contained, unopinionated, fast and lightweight executable launcher.

Kickoff ?? A self-contained, unopinionated, fast and lightweight executable launcher. Supported Platforms Platform Host Target aarch64-apple-macos-non

Nimbus 18 Oct 27, 2023
PyO3 bindings and Python interface to skani, a method for fast fast genomic identity calculation using sparse chaining.

?? ⛓️ ?? Pyskani PyO3 bindings and Python interface to skani, a method for fast fast genomic identity calculation using sparse chaining. ??️ Overview

Martin Larralde 13 Mar 21, 2023