Eventually consistent values for Rust

Overview

Eventuals give you the most up-to-date snapshots of some value. They are like Futures that update over time, continually resolving to an eventually consistent state. You can chain eventuals together, inspect their present value, and subscribe for updates. The eventual paradigm is the culmination of many hard lessons in asynchronous programming.

Consider this code using an imaginary set_interval inspired by JavaScript:

set_interval(|| async { let update = fetch().await; set_ui_data(update); }, 2000)

The above is a typical pattern used to update a UI from external data over time. There are a couple of problems with this code. First, the fetch operations may overlap. This overlap can result in stale data being shown in the UI after new data! Furthermore, there is no clear way to cancel - either from the reading or the writing perspective.

Eventuals solve these and other common issues!

let cancel_on_drop = timer(2000).map(|_| fetch).pipe(set_ui_update);

Here, set_ui_update is guaranteed to progress only forward in time. Different pipeline stages may execute concurrently, and some updates may be skipped if newer values are available. Still, the eventual state will always be consistent with the final writes to data.

Subscriptions

Subscriptions are views of the latest update of an eventual from the reader's perspective. A subscription will only observe an update if the value has changed since the last observation and may skip updates between the previous and current observations. Taken together, this ensures that any reader performs the least amount of work necessary to be eventually consistent with the state of the writer.

You can get a subscription by calling eventual.subscribe() and subscription.next().await any time you are ready to process an update. If there has been an update since the previously processed update the Future is ready immediately, otherwise it will resolve as soon as an update is available.

Snapshots

There are two ways to get the current value of an eventual without a subscription. These are eventual.value_immediate() and eventual.value, depending on whether you want to wait for the first update.

Unit tests

One useful application of eventuals is to make components testable without mocks. Instead of making a mock server (which requires traits) you can supply your component with an eventual and feed the component data. It's not dependency injection. It's data injection.

FAQ

What is the difference between an eventual and a Stream?

A Stream is a collection of distinct, ordered values that are made available asynchronously. An eventual, however, is an asynchronous view of a single value changing over time.

The API seems sparse. I see map and join, but where are filter, reduce, and select?

It is natural to want to apply the full range of functional techniques to eventuals, but this is an anti-pattern. The goal of eventuals is to make the final value of any view of any data pipeline deterministically consistent with the latest write. The path to producing a result may not be deterministic, but the final value is consistent if each building block is also consistent. Not every functional block passes this test. To demonstrate, we can compare the behavior of map and an imaginary filter combinator on the same sequence of writes.

Consider eventual.map(|n| async move { n * 2 }) and eventual.filter(|n| async move { n % 2 == 0 }) both consuming the series writes [0, 1, 2, 3, 4, 5] over time. map, in this case, will always eventually resolve to the value 10. It may also produce some subset of intermediate values along the way (like [0, 4, 8, 10] or [2, 10]). Still, it will always progress forward in time and always resolves to a deterministic value consistent with the final write. However, the final value that filter produces would be a function of which intermediate values were observed. If filter observes the subset of writes [0, 2, 5], it will resolve to 2, but if it observes the subset of writes [1, 4, 5], it will resolve to 4. This bug is exactly the kind eventuals are trying to avoid.

You might also like...
A command line tool that resembles a debugger as well as Cheat Engine, to search for values in memory
A command line tool that resembles a debugger as well as Cheat Engine, to search for values in memory

Summary This is a small command-line tool designed to peek around memory of a running Linux process. It also provides filtering mechanisms similar to

Library uses file-based mmap to store key-values

Library uses file-based mmap to store key-values This is a Rust version of MMKV. By default, this lib uses CRC8 to check data integrity. If include fe

Macro to print variable(s) with values nicely (stripped from release builds)

log_macro Macro to print variable(s) with values nicely (stripped from release builds) Install cargo add log_macro Use Add this to top of file: #[mac

Use LLMs to generate strongly-typed values

Magic Instantiate Quickstart use openai_magic_instantiate::*; #[derive(MagicInstantiate)] struct Person { // Descriptions can help the LLM unders

A fast bump allocator that supports allocation scopes / checkpoints. Aka an arena for values of arbitrary types.

bump-scope A fast bump allocator that supports allocation scopes / checkpoints. Aka an arena for values of arbitrary types. What is bump allocation? A

Rust-advent - Learning Rust by solving advent of code challenges (Streaming live on Twitch every Monday)
Rust-advent - Learning Rust by solving advent of code challenges (Streaming live on Twitch every Monday)

Rust advent 🦀 🐚 Learning Rust by implementing solutions for Advent of Code problems. 🎥 HEY, we are live-streaming our attempts to solve the exercis

Rust-clippy - A bunch of lints to catch common mistakes and improve your Rust code

Clippy A collection of lints to catch common mistakes and improve your Rust code. There are over 450 lints included in this crate! Lints are divided i

Rust-battery - Rust crate providing cross-platform information about the notebook batteries.

battery Rust crate providing cross-platform information about the notebook batteries. Table of contents Overview Supported platforms Install Examples

A Rust-based shell script to create a folder structure to use for a single class every semester. Mostly an excuse to use Rust.

A Rust Course Folder Shell Script PROJECT IN PROGRESS (Spring 2022) When completed, script will create a folder structure of the following schema: [ro

Comments
  • error: could not compile `eventuals`

    error: could not compile `eventuals`

    Actual:

    noselectin the root

    Expected:

    Dependency should build

    Notes:

    error[E0432]: unresolved import `tokio::select`
     --> /home/richardb/.cargo/registry/src/github.com-1ecc6299db9ec823/eventuals-0.4.0/src/eventual/eventual.rs:9:5
      |
    9 | use tokio::select;
      |     ^^^^^^^^^^^^^ no `select` in the root
    

    Background

    • crate version 0.4.0
    • cargo 1.52.0 (69767412a 2021-04-21)
    • rustc 1.52.1 (9bc8c42bb 2021-05-09)
    • edition 2018

    Dependencies:

    actix-web = { version = "3.3.2", features = ["rustls"] }
    actix-cors = "0.5"
    chrono = { version = "0.4", features = ["serde"] }
    clap = "2.33.3"
    eventuals = "0.4.0"
    fern = "0.6"
    futures-util = "0.3.15"
    jsonwebtoken = "7.2.0"
    log = "0.4"
    rustls = "0.19.1"
    serde = "1.0.126"
    serde_json = "1.0.64"
    
    opened by richbayliss 2
  • Prototype

    Prototype

    Quick and dirty implementation. Next thing to do is to go lock-free, cleanup, add more tests, and add map.

    The current idea for map is something like...

    fn map(source: EventualReader, f: Fn) -> EventualReader {
        let (writer, readers) = Eventual::new();
        tokio::spawn(async move {
            while let Ok(v) = source.next().await {
                writer.push(f(v).await);
            }
        });
        readers
    }
    

    Need to make a couple adjustments but that is where this would be headed. I feel it's at a good state for a Zoom call at least.

    Something cool from the above is that cancellation is automatic. When the writer for the source is dropped, this task will observe Err(Closed) instead of Ok(v) which will leave the while loop, dropping this Eventual, and cancelling anything downstream as well. Magic.

    opened by That3Percent 1
  • Finalizer

    Finalizer

    The main update here is that subscriptions can always observe the final value even if the eventual is closed. This makes a lot more sense than what was happening before (at the expense of some terribly messy internal code).

    Also added docs, some minor APIs, and a bump to 0.3.0. I think this should be the last update before putting this to use.

    opened by That3Percent 0
  • More apis

    More apis

    PR because this is how far I got for today.

    Some things were made less-boilerplatey (especially allowing ? in spawn). pipe was added. retry was added (sort of... it probably works) handle_errors was added with fixed tests sleep was removed from all tests

    opened by That3Percent 0
Owner
Edge & Node
Edge & Node
Simple, lightweight, markdown-based notes app I might actually finish eventually

Jupiter A simple markdown & git-based notes app for Linux. Features Lightweight with minimal dependencies Git integration for syncing & versioning Sea

Maxim 6 Jan 31, 2023
SysEx editor for Roland SC-55 (GS), Roland SC-7, and eventually maybe Yamaha XG

SoundPalette: MIDI SysEx Generator This is the source code for SoundPalette, a tool for editing MIDI System Exclusive (SysEx) messages for the Roland

hikari_no_yume 9 Dec 8, 2023
Rust TUI library - Clipping region is a set of min/max x/y values applied to the existing region

TinyBit Clipping region is a set of min/max x/y values applied to the existing region A TUI lib This is not yet production ready T O D O TODO: bugs: T

Togglebit 13 May 3, 2022
Generic extensions for tapping values in Rust.

tap Suffix-Position Pipeline Behavior This crate provides extension methods on all types that allow transparent, temporary, inspection/mutation (tappi

Alexander Payne 213 Dec 30, 2022
Microsoft Excel (XLSX) to Unicode Separated Values (USV) Rust crate

xlsx-to-usv Convert Microsoft Excel (XLSX) to Unicode Separated Values (USV). Built with the USV Rust crate. Syntax: stdin | xlsx-to-usv [options] | s

SixArm 3 Mar 31, 2024
Padding/aligning values without heap allocation

zero-copy-pads Padding/aligning values without heap allocation. Cargo Features std (default feature): Disable #![no_std]. Enable features that require

Khải 5 Nov 29, 2021
Quickly save and retrieve values for shell scripts.

Quickly save and retrieve values for shell scripts.

Alex Andrade 2 Dec 15, 2022
a crate to swap values between possibly-overlapping references

omniswap: a crate to swap values between possibly-overlapping references Motivating Example You cannot simply use std::mem::swap to replace values wit

Masaki Hara 21 Nov 30, 2022
🍅 A command-line tool to get and set values in toml files while preserving comments and formatting

tomato Get, set, and delete values in TOML files while preserving comments and formatting. That's it. That's the feature set. I wrote tomato to satisf

C J Silverio 15 Dec 23, 2022
Encode and decode dynamically constructed values of arbitrary shapes to/from SCALE bytes

scale-value · This crate provides a Value type, which is a runtime representation that is compatible with scale_info::TypeDef. It somewhat analogous t

Parity Technologies 15 Jun 24, 2023