crdt-genome
Synopsis
Experiments with Rust CRDTs using Tokio web application framework Axum.
Background
Exploring some ideas of Martin Kleppmann, particularly Local-First software.
See this Podcast
Also this conference talk
Scenario
A group of Axum executeables represents a group of Actors.
Each actor process maintains a genome, represented by a CRDT List.
pub struct Genome {
genes: ListOfGenes,
}
Each process mutates its genome at random intervals and circulates a CmRDT Op using HTTP POST.
The goal is to observe every genome instance converging to a common value.
Caveat
This little system has serious shortcomings. If a process joins late, or drops out and rejoins, or even loses a single POST request, it will never have the full genome.
Execution
see scripts/run-test.sh
to run some actors, first bring the executeable up to date:
cargo build
Then run instances of the executeable
#!/bin/bash
set -euxo pipefail
target/debug/crdt-genome --actor=0 --count=3 --base=8000 2>&1 | tee actor-0.log &
target/debug/crdt-genome --actor=1 --count=3 --base=8000 2>&1 > actor-1.log &
target/debug/crdt-genome --actor=2 --count=3 --base=8000 2>&1 > actor-2.log &
USAGE:
crdt-genome --actor <actor> --base <base> --count <count>
FLAGS:
-h, --help Prints help information
-V, --version Prints version information
OPTIONS:
-a, --actor <actor> the actor id of this server
-b, --base <base> base port number
-c, --count <count> The number of actors
Verification
We want to verify that every genome is converging to the same value.
Since the system is constantly changing, there is no absolute way to do this.
We use HTTP GET to return a string representation of the genome.
$ curl localhost:8000/genome
40fd70816b4f664be2f28766
At a fixed interval, each process polls all the others and compares its genome representation with theirs. For this test, we simply log the result.
$ grep "match" actor-1.log
Oct 24 15:24:05.500 DEBUG crdt_genome: match count = 6
Oct 24 15:24:10.516 DEBUG crdt_genome: match count = 9
Oct 24 15:24:15.528 DEBUG crdt_genome: match count = 9
Oct 24 15:24:20.540 DEBUG crdt_genome: match count = 9
Oct 24 15:24:25.551 DEBUG crdt_genome: match count = 9
Oct 24 15:24:30.561 DEBUG crdt_genome: match count = 9
Oct 24 15:24:35.573 DEBUG crdt_genome: match count = 9
Oct 24 15:24:40.587 DEBUG crdt_genome: match count = 9
Oct 24 15:24:45.601 DEBUG crdt_genome: match count = 9
Oct 24 15:24:50.613 DEBUG crdt_genome: match count = 9
Oct 24 15:24:55.623 DEBUG crdt_genome: match count = 9
Oct 24 15:25:00.635 DEBUG crdt_genome: match count = 9
Oct 24 15:25:05.646 DEBUG crdt_genome: match count = 9
Oct 24 15:25:10.657 DEBUG crdt_genome: match count = 9
10 Mar 19, 2023
20 Jun 16, 2023
16.2k Jan 2, 2023
1.7k Dec 26, 2022
3 Jan 8, 2023
31 Dec 29, 2022
11 Sep 22, 2023
2.2k Jan 7, 2023
5 Aug 8, 2021
45 Sep 6, 2023
39 Oct 16, 2024
12 Sep 25, 2023
115 Dec 29, 2022
46 Jan 5, 2023
10 Mar 4, 2023
7.9k Dec 31, 2022
16 Jun 11, 2023
3 Dec 2, 2023
11 Dec 15, 2022