I got 99 problems, but a batch ain't one

Overview

Batch ain't one

GitHub Workflow Status GitHub Crates.io docs.rs

I got 99 problems, but a batch ain't one...

Batch up multiple items for processing as a single unit.

Why

Sometimes it is more efficient to process many items at once rather than one at a time. Especially when the processing step has overheads which can be shared between many items.

Example: Inserting multiple rows

For example, each database operation, such as an INSERT, has the overhead of a round trip to the database.

Unbatched example

Multi-row inserts can share this overhead between many items. This also allows us to share a single database connection to insert these three items, potentially reducing contention if the connection pool is highly utilised.

Batched example

Example: With locking and transactions

Insert into database tables can often be done concurrently. In cases where these must be done serially, throughput can suffer. In the example below, six round trips to the database are necessary for each item. All subsequent items must wait until this is finished.

If each round trip takes 1ms, then this results in a minimum of 6ms per item, or 167 items/sec maximum.

Unbatched example

With batching, we can improve the throughput. Acquiring/releasing the lock and beginning/committing the transaction can be shared for the whole batch. With four items per batch, we can increase the theoretical maximum throughput to 667 items/sec. In reality each INSERT will take longer, but the multi-row inserts are likely to share some overheads in a similar way.

Batched example

Example

use std::{marker::Send, sync::Arc};

use async_trait::async_trait;
use batch_aint_one::{Batcher, Processor, BatchingStrategy};

#[derive(Debug, Clone)]
struct SimpleBatchProcessor;

#[async_trait]
impl Processor<String, String, String> for SimpleBatchProcessor {
    async fn process(&self, key: String, inputs: impl Iterator<Item = String> + Send) -> Result<Vec<String>, String> {
        // In this example:
        // - `key`: "A"
        // - `inputs`: ["1", "2"]
        Ok(inputs.map(|s| s + " processed").collect())
    }
}

tokio_test::block_on(async {
    let batcher = Arc::new(Batcher::new(SimpleBatchProcessor, BatchingStrategy::Size(2)));

    // Request handler 1
    let b1 = batcher.clone();
    tokio::spawn(async move {
        let output = b1.add("A".to_string(), "1".to_string()).await.unwrap();

        assert_eq!("1 processed".to_string(), output);

    });

    // Request handler 2
    let b2 = batcher.clone();
    tokio::spawn(async move {
        let output = b2.add("A".to_string(), "2".to_string()).await.unwrap();

        assert_eq!("2 processed".to_string(), output);
    });
});

Roadmap

  • Tests
  • Better error handling
  • Garbage collection for old generation placeholders
  • Docs
    • Why – motivating example
    • Code examples
  • Observability
  • Metrics?

Further reading

You might also like...
Dechrome is a tool written in Rust for batch removing Chromium-based browsers.

Dechrome Dechrome is a tool written in Rust for batch removing Chromium-based browsers and installing Firefox as a replacement. Warning The script is

🎆 CPU-driven, batch-rendered particle system for the Bevy game engine.
🎆 CPU-driven, batch-rendered particle system for the Bevy game engine.

Bevy Firework 🎆 Bevy firework is a particle system plugin where particles are simulated on the CPU and use GPU batching for rendering. This allows ea

A simple menu to keep all your most used one-liners and scripts in one place
A simple menu to keep all your most used one-liners and scripts in one place

Dama Desktop Agnostic Menu Aggregate This program aims to be a hackable, easy to use menu that can be paired to lightweight window managers in order t

Rust Programming Fundamentals - one course to rule them all, one course to find them...

Ultimate Rust Crash Course This is the companion repository for the Ultimate Rust Crash Course published online, presented live at O'Reilly virtual ev

A super simple /sbin/init for Linux which allows running one and only one program

Summary High-performance /sbin/init program for Linux This is designed to do literally nothing but accept binaries over the network and run them as a

A simple but production-ready link shortener. Repository belonging to one of my YouTube videos

What is this? This is a simple link shortener, which is the result of a tutorial I did on YouTube. Important technologies used Rust (as our most belov

A genetic algorithm for bechmark problems, written to learn Rust.

Genetic Algorithm A genetic algorithm in Rust for the following benchmark problems: Ackley Griewangk Rastrigin Rosenbrock Schwefel Sphere Usage: Insta

Rust implementation of real-coded GA for solving optimization problems and training of neural networks
Rust implementation of real-coded GA for solving optimization problems and training of neural networks

revonet Rust implementation of real-coded genetic algorithm for solving optimization problems and training of neural networks. The latter is also know

Radiate is a parallel genetic programming engine capable of evolving solutions to many problems as well as training learning algorithms.

Radiate Coming from Evolutionary Radiation. Evolutionary radiation is a rapid increase in the number of species with a common ancestor, characterized

Linear Programming for Rust, with an user-friendly API. This crate allows modeling LP problems, and let's you solve them with various solvers.

good_lp A Linear Programming modeler that is easy to use, performant with large problems, and well-typed. use good_lp::{variables, variable, coin_cbc,

Rust implementation of real-coded GA for solving optimization problems and training of neural networks
Rust implementation of real-coded GA for solving optimization problems and training of neural networks

revonet Rust implementation of real-coded genetic algorithm for solving optimization problems and training of neural networks. The latter is also know

Lagoon is a thread pool crate that aims to address many of the problems with existing thread pool crates.

Lagoon is a thread pool crate that aims to address many of the problems with existing thread pool crates. Example Lagoon's scoped jobs can be u

Temp repo to document problems with workers-rs

This is a temporarily repo to share some worker-rs code that is giving me problems. See also: https://github.com/cloudflare/workers-rs/issues/94 Versi

A Discord bot focused on addressing the inherent problems with Discord, to allow a more socialist/anarchist organization of servers.

ACABot A Discord bot focused on addressing the inherent problems with Discord, to allow a more socialist/anarchist organization of servers (or "guilds

My solutions to the advent of code 2021 problems.

Advent of Code 2021 My solutions to the AOC 2021 problems in Rust. Solutions Task Status Day 1 ✔️ , ✔️ Day 2 ✔️ , ✔️ Day 3 ✔️ , ✔️ Day 4 ✔️ , ✔️ Day 5

Eslint - Find and fix problems in your JavaScript code.
Eslint - Find and fix problems in your JavaScript code.

ESLint Website | Configuring | Rules | Contributing | Reporting Bugs | Code of Conduct | Twitter | Mailing List | Chat Room ESLint is a tool for ident

Coding-challenge - Algorithms and Data-structures, problems and solutions in Rust language using cargo-workspaces

Coding Challenge LeetCode/Hackerrank e.t.c Using this as an opportunity to improve my knowledge of rust lang If you found this repo useful to you, add

Rust solutions to problems in Advent of Code '22.

Advent of Code '22 This repository contains my solutions for AoC'22 problems, written in the language that will save us from cpp. Running the code The

Cargo-BOJ: test and submit solutions to BOJ problems

Cargo-BOJ Test and submit solutions to BOJ (Baekjoon Online Judge) problems. Defaults are geared towards Rust solutions, but non-Rust usage is support

Owner
Thom Wright
Thom Wright
All in one (aka Aio) database with async support

All in one (aka Aio) database with async support. Based on libsql/Sqlite, bevy_reflect and tokio, includes a dead simple API to be used (no SQL needed just pure Rust). Comes with automigration.

Milen Denev 11 May 4, 2024
batch eval tool for machine code. eval() but for asm.

yaxpeax-eval yaxpeax-eval is the repo providing yaxeval, a tool to execute machine code with preconditions and report state at exit. currently, yaxeva

iximeow 10 Jan 5, 2023
Creating CLI's just got a whole lot better.

Staq Creating CLI's just got a whole lot better. Don't worry about CLI colouring, networking, Size of Executables, Speed ever again Have any doubts? R

null 12 Jun 6, 2021
Ever got frustrated when you realize that the for loop you were writing

for_each_repeat Ever got frustrated when you realize that the for loop you were writing... fn foo(mut iter: impl Iterator<Item=i32>) { for i in it

null 1 Jun 18, 2022
Batch rename utility for developers

nomino Batch rename utility for developers How to install Pre-Compiled You can download a pre-compiled executable for Linux, MacOS and Windows operati

Navid 458 Dec 27, 2022
rbdt is a python library (written in rust) for parsing robots.txt files for large scale batch processing.

rbdt ?? ?? ?? ?? rbdt is a work in progress, currently being extracted out of another (private) project for the purpose of open sourcing and better so

Knuckleheads' Club 0 Nov 9, 2021
Sol Batch Token Transfer CSV

Sol Batch Token Transfer CSV This script can batch transfer any SPL token to multiple addresses listed in a CSV, and then save the tx hash to a new CS

The Z Institute 10 Nov 6, 2022
A simple web-app allowing you to batch archive groups of repositories from a given organization

ice-repos My goal here is to build a simple web-app allowing you to batch archive groups of repositories from a given organization, using Rust+Yew. As

Nic McPhee 6 Nov 5, 2022
A CLI batch downloader for your Bandcamp collection.

bandsnatch A CLI batch downloader for your Bandcamp collection. Bandsnatch is a Rust tool for downloading all of your Bandcamp purchases all at once i

Michael Mitchell 12 Jan 1, 2023
An efficient, robust, and generalized batch submission service for rollup stacks written in pure rust.

archon is an efficient, robust, and generalized batch submission service for rollup stacks written in pure rust. Note Archon is primarily tested again

refcell.eth 75 Apr 2, 2023