Raft distributed consensus for WebAssembly in Rust

Overview

WRaft: Raft in WebAssembly

What is this?

A toy implementation of the Raft Consensus Algorithm for WebAssembly, written in Rust. Basically, it synchronizes state across three browser windows in a peer-to-peer fashion.

Why is this?

Because I was curious to see if I could get it to work 😄 . I can't think of any real-world use-cases off the top of my head, but if you can, please open an issue and let me know!

How does it work?

WRaft uses WebRTC data channels to set up communication between the browser windows. Sadly WebRTC isn't purely peer-to-peer, so there's a separate WebSocket-based service (webrtc-introducer) that "introduces" the browser windows to each other before the cluster can start. The browser windows can be on the same computer or different machines on a LAN (or different networks, theoretically, but I haven't tested that yet). Firefox and Chrome (or any combination of the two) seem to work; Safari seems to not work.

Once the browser windows have been introduced to each other, a Raft cluster is started and messages sent to one browser window are reliably replicated to all three in a consistent order (as long as everything's working correctly 😉 ). The messages are persisted to the browser's local storage so they'll survive browser restarts. The cluster will continue functioning normally even if one window stops, and can recover if two windows stop.

The "replicated messages" could be any serializable Rust type. There's a raft::State trait that allows the user to "plug in" any state-machine-like-thing into the Raft server (using Rust generics). For the example apps, I used yew with the "state machine" more or less mapping to the application state. (You could also use a HashMap to get a distributed key/value store à la etcd.)

There's currently no way to expand beyond three browser windows (or any notion of a "client" outside of the cluster servers). I have some ideas for how it might work, though.

Can I try it?

Yes! There are two basic "demo apps" included in the library, publicly hosted at https://wraft0.eevans.co/. The apps are:

To use either app, you'll need to open the app in different browser windows with different hosts (wraft0.eevans.co, wraft1.eevans.co, and wraft2.eevans.co) (they need to be different host names so they have independent Local Storage). The app should then start up and you can try it!

Is it fast?

I don't have much to compare it to, but from some basic testing it seems pretty fast to me! In the best-case scenario (three Chromium browsers on the same machine) I've seen ~2000 writes/second which should be enough for any use case I can think of 😄 . (The bigger issue is that you hit the local storage quota pretty fast; log compaction would have to be implemented to work around that.) Firefox seems to top out at ~800 writes/second.

What parts of Raft are implemented?

For the Raft nerds out there, so far I've only implemented the "basic algorithm" (basically what's in the TLA spec). To make it more useful, you'd probably need:

  • Log compaction/snapshots (the API is designed to make this possible, but it's completely unimplemented).

  • Cluster membership changes (I haven't really looked into this yet).

No promises that either of those will ever happen, but I might try implementing them if I have time.

Should I use it in production?

No! (At least not in its current state.) Documentation, error handling, and testing are basically non-existent, and I haven't implemented some harder parts of Raft like log compaction and cluster membership changes. There are a few bugs I know about and almost certainly many more I don't!

If you have an actual use case for this, open an issue to let me know and I'll think about turning it into a proper Crate and/or NPM package.

You might also like...
Easy-to-use beanstalkd client for Rust (IronMQ compatible)

rust-beanstalkd Easy-to-use beanstalkd client for Rust (IronMQ compatible) Install Add this dependency to your Cargo.toml beanstalkd = "*" Documentati

libhdfs binding and wrapper APIs for Rust

hdfs-rs libhdfs binding library and rust APIs which safely wraps libhdfs binding APIs Current Status Alpha Status (Rust wrapping APIs can be changed)

Raft distributed consensus algorithm implemented in Rust.
Raft distributed consensus algorithm implemented in Rust.

Raft Problem and Importance When building a distributed system one principal goal is often to build in fault-tolerance. That is, if one particular nod

Fork of async-raft, the Tokio-based Rust implementation of the Raft protocol.

Agreed Fork of async-raft, the Tokio-based Rust implementation of the Raft distributed consensus protocol. Agreed is an implementation of the Raft con

The labs of Raft consensus algorithm based on MadSim.

MadRaft The labs of Raft consensus algorithm based on MadSim. Some codes are derived from MIT 6.824 and PingCAP Talent Plan: Raft Lab. Thanks for thei

🚣‍♀️ 1kloc, well-documented Raft consensus algorithm implementation

miniraft A 1kloc, well-documented Raft consensus algorithm implementation This crate is a minimal implementation of the Raft consensus protocol with

Easy c̵̰͠r̵̛̠ö̴̪s̶̩̒s̵̭̀-t̶̲͝h̶̯̚r̵̺͐e̷̖̽ḁ̴̍d̶̖̔ ȓ̵͙ė̶͎ḟ̴͙e̸̖͛r̶̖͗ë̶̱́ṉ̵̒ĉ̷̥e̷͚̍ s̷̹͌h̷̲̉a̵̭͋r̷̫̊ḭ̵̊n̷̬͂g̵̦̃ f̶̻̊ơ̵̜ṟ̸̈́ R̵̞̋ù̵̺s̷̖̅ţ̸͗!̸̼͋

Rust S̵̓i̸̓n̵̉ I̴n̴f̶e̸r̵n̷a̴l mutability! Howdy, friendly Rust developer! Ever had a value get m̵̯̅ð̶͊v̴̮̾ê̴̼͘d away right under your nose just when

ChiselStore is an embeddable, distributed SQLite for Rust, powered by Little Raft.

ChiselStore ChiselStore is an embeddable, distributed SQLite for Rust, powered by Little Raft. SQLite is a fast and compact relational database manage

Paxakos is a pure Rust implementation of a distributed consensus algorithm

Paxakos is a pure Rust implementation of a distributed consensus algorithm based on Leslie Lamport's Paxos. It enables distributed systems to consistently modify shared state across their network, even in the presence of failures.

The lightest distributed consensus library. Run your own replicated state machine! ❤️

Little Raft The lightest distributed consensus library. Run your own replicated state machine! ❤️ Installing Simply import the crate. In your Cargo.to

Damavand is a quantum circuit simulator. It can  run on laptops or High Performance Computing architectures, such CPU distributed architectures or multi GPU distributed architectures.
Damavand is a quantum circuit simulator. It can run on laptops or High Performance Computing architectures, such CPU distributed architectures or multi GPU distributed architectures.

Damavand is a quantum circuit simulator. It can run on laptops or High Performance Computing architectures, such CPU distributed architectures or multi GPU distributed architectures.

open source training courses about distributed database and distributed systemes
open source training courses about distributed database and distributed systemes

Welcome to learn Talent Plan Courses! Talent Plan is an open source training program initiated by PingCAP. It aims to create or combine some open sour

Damavand is a quantum circuit simulator. It can  run on laptops or High Performance Computing architectures, such CPU distributed architectures or multi GPU distributed architectures.
Damavand is a quantum circuit simulator. It can run on laptops or High Performance Computing architectures, such CPU distributed architectures or multi GPU distributed architectures.

Damavand is a code that simulates quantum circuits. In order to learn more about damavand, refer to the documentation. Development status Core feature

Raft implementation in Rust

rsraft Raft implementation in Rust. The aim of this project is implementing the Raft Consensus Algorithm as described in the paper, with the goal of f

The Raft algorithm implement by Rust.

Raft The Raft algorithm implement by Rust. This project refers to Eli Bendersky's website, the link as follows: https://eli.thegreenplace.net/2020/imp

Another minimal Raft implementation in Rust.

raft-rs Not my first time implementing Raft. I wrote about another implementation in Go I did. But you don't learn a concept well until you've impleme

RiteRaft - A raft framework, for regular people

RiteRaft - A raft framework, for regular people This is an attempt to create a layer on top of tikv/raft-rs, that is easier to use and implement. This

A raft framework, for regular people

RmqttRaft - A raft framework, for regular people This is an attempt to create a layer on top of tikv/raft-rs, that is easier to use and implement. Thi

Implementation of Proof of Existence consensus using Substrate Framework, Frame, Pallets, RUST

Substrate Node Template A fresh FRAME-based Substrate node, ready for hacking 🚀 Getting Started Follow the steps below to get started with the Node T

Owner
Emanuel Evans
Emanuel Evans
The lightest distributed consensus library. Run your own replicated state machine! ❤️

Little Raft The lightest distributed consensus library. Run your own replicated state machine! ❤️ Installing Simply import the crate. In your Cargo.to

Ilya Andreev 359 Dec 26, 2022
The Raft algorithm implement by Rust.

Raft The Raft algorithm implement by Rust. This project refers to Eli Bendersky's website, the link as follows: https://eli.thegreenplace.net/2020/imp

Qiang Zhao 1 Oct 23, 2021
Fluvio is a high-performance distributed streaming platform that's written in Rust

Fluvio is a high-performance distributed streaming platform that's written in Rust, built to make it easy to develop real-time applications.

InfinyOn 1.6k Dec 30, 2022
Magical Automatic Deterministic Simulator for distributed systems in Rust.

MadSim Magical Automatic Deterministic Simulator for distributed systems. Deterministic simulation MadSim is a Rust async runtime similar to tokio, bu

MadSys Research Group 249 Dec 28, 2022
A model checker for implementing distributed systems.

A model checker for implementing distributed systems.

Stateright Actor Framework 1.3k Dec 15, 2022
Sorock is an experimental "so rocking" scale-out distributed object storage

Sorock is an experimental "so rocking" scale-out distributed object storage

Akira Hayakawa 6 Jun 13, 2022
A universal, distributed package manager

Cask A universal, distributed package manager. Installation | Usage | How to publish package? | Design | Contributing | Cask.toml If you are tired of:

null 39 Dec 30, 2022
Easy Hadoop Streaming and MapReduce interfaces in Rust

Efflux Efflux is a set of Rust interfaces for MapReduce and Hadoop Streaming. It enables Rust developers to run batch jobs on Hadoop infrastructure wh

Isaac Whitfield 31 Nov 22, 2022
A fully asynchronous, futures-based Kafka client library for Rust based on librdkafka

rust-rdkafka A fully asynchronous, futures-enabled Apache Kafka client library for Rust based on librdkafka. The library rust-rdkafka provides a safe

Federico Giraud 1.1k Jan 8, 2023
Rust client for Apache Kafka

Kafka Rust Client Project Status This project is starting to be maintained by John Ward, the current status is that I am bringing the project up to da

Yousuf Fauzan 902 Jan 2, 2023