QueingSimulator is an application that can be used to build intuitions about behavior of synchronous request/reply systems

Overview

Queueing Simulator

QueingSimulator is an application that can be used to build intuitions about behavior of synchronous request/reply systems (such as a modern web service) under different loads and configuration parameters. More specifically, it examines the impact request queueing has on possibility of congestion collapse.

Most modern web service and application frameworks use a queueing system to enqueue incoming requests when there are no idle workers available to handle them. These queues can be great for minimizing transient failures, as they allow the application to ride through a temporary spike in incoming requests without dropping any of them on the floor. However, for synchronous request/reply system these can pose a big availability risk. This happens when a large enough queue builds up that by the time a request makes it to the front of the queue, the client would’ve timed out and most likely retried with a fresh new request. When this happens, the server is essentially doing throw away work and the system suffers a congestion collapse. There are several mechanisms for reducing this risk, including approaches like load shedding and cooperative clients. This simulator allows one to explore impact of tuning queueing subsystem (including disabling it altogether or flipping from FIFO to LIFO order) on the possiblity of congestion collapse.

Each execution of the simulator runs a single simulator with specified parameters, and then reports the percentage of the requests that failed, either because they were timed out or rejected by the server when its queue was full. The simulator uses a virtual clock, and as such can simulate hours or even days of runtime in milliseconds.

Parameters:

-r --arrival_rate: Mean arrival rate of new requests per clock tick.

-w —num_workers: Number of workers (such as web server threads) processing incoming requests. (Default: 10)

-t --timeout: Request timeout - a time after which a client gives up on the request, and potentially retries. For a meaningful simulation, this value should be smaller than mean request latency (Default: 1000)

--mean_latency: Mean latency it takes a worker to process each request, after picking it up from the queue. (Default: 50)

--simulation_time: Number of clock ticks to run the simulation. (Default: 1000000)

-q --queue_size: The size of request queue. (Default: 1000)

--lifo: Whether to use LIFO, instead of FIFO queue. (Default: false)

--simulate_spike: Whether to simulate a temporary spike in request latency (as can happen if a server had a temporary slow down (Default: false)

--retry_probability. Probability a failed request will be retried. Must be between 0 and 1. (Default: 0.5)

Building and running

This is a Rust application, so you will need the latest Rust toolchain to build and run it. Hhead on down to https://www.rust-lang.org/tools/install to install the Rust toolchain. Once installed, run “cargo build --release" from the repo directory. The compiled application will be in ./target/release directory.

Running the simulator:

queueingsimulator -r 0.1 --simulate_spike --lifo Failure rate: 0.79%

queueingsimulator -r 0.1 --simulate_spike Failure rate: 9.91%

queueingsimulator -r 0.5 Failure rate: 86.74%

You might also like...
Ampseer examines reads in fastq format and identifies which multiplex PCR primer set was used to generate the SARS-CoV-2 sequencing library they are read from.

Ampseer examines reads in fastq format and identifies which multiplex PCR primer set was used to generate the SARS-CoV-2 sequencing library they are read from.

A simple library with just one struct which is used to wrap around pointers

A simple library with just one struct which is used to wrap around pointers. This can be used to create pointers and share them across threads without the hassle of synchronization if you really do not care about that.

Rewrite of the Discord Bot used for Managing the Infinity Bot List Servers.
Rewrite of the Discord Bot used for Managing the Infinity Bot List Servers.

Arcadia Rewrite of the Discord Bot used for Managing the Infinity Bot List Servers. Contributing Always run fmt.sh before making a Pull Request! MacOS

🤖 Discord bot used @okp4 to bridge the Dataverse!

🤖 Discord bot Discord bot used @okp4 to bridge the Dataverse! ✨ Purpose This project is a Discord bot mainly intended to be used for the OKP4 Discord

Rust library that can be reset if you think it's slow

GoodbyeKT Rust library that can be reset if you think it's slow

Uindex is a data store, for data that can be parsed as sentences in some context-free language.
Uindex is a data store, for data that can be parsed as sentences in some context-free language.

Uindex - Universal index Uindex is a data store, for data that can be parsed as sentences in some context-free language.

Damavand is a quantum circuit simulator. It can  run on laptops or High Performance Computing architectures, such CPU distributed architectures or multi GPU distributed architectures.
Damavand is a quantum circuit simulator. It can run on laptops or High Performance Computing architectures, such CPU distributed architectures or multi GPU distributed architectures.

Damavand is a quantum circuit simulator. It can run on laptops or High Performance Computing architectures, such CPU distributed architectures or multi GPU distributed architectures.

Playing with web dev in Rust. This is a sample Rust microservice that can be deployed on Kubernetes.

Playing with web dev in Rust. This is a sample Rust microservice that can be deployed on Kubernetes.

Edgelord is a library for Cloudflare Workers. You can scaffold a basic bot for discord, slack, etc.

Edge Computing + chūnibyō = Edgelord ✨ 👿 Edgelord Edgelord is now working. You can contribute for it. Edgelord is a Rust library for cloudflare worke

Comments
  • Feature: add request deadline/timeout parameter

    Feature: add request deadline/timeout parameter

    Great work here! And as you write in the README, there‘s several patterns to address „wasted work“. I am curious how the simulator can be enhanced using request deadlines/timeouts (or TTL) as often used in distributed systems and deep call graphs.

    opened by embano1 0
Owner
Joe Magerramov
Joe Magerramov
Rustcat is a port listener that can be used for different purposes.

⚙️ Rustcat ⚙️ -- Basic Netcat Alternative -- About Rustcat is a port listener that can be used for different purposes.

Robiot 489 Dec 28, 2022
A small monitoring process that checks if kstars is active, if not it sends a request to notify a user via telegram that it crashed

Astro monitor A small monitoring process that checks if kstars is active, if not it sends a request to notify a user via telegram that it crashed Inst

Mattia Procopio 2 Jan 10, 2022
HTTP microservice using Axum and Reqwest to request the Google Translate TTS endpoint without rate limits

HTTP microservice using Axum and Reqwest to request the Google Translate TTS endpoint without rate limits

Gnome! 5 Oct 5, 2022
Buildomat manages the provisioning of ephemeral UNIX systems on which to run software builds

B U I L D O M A T a software build labour-saving device Buildomat manages the provisioning of ephemeral UNIX systems (e.g., instances in AWS EC2) on w

Oxide Computer Company 33 Dec 4, 2022
Nimbus is a framework for building parachain consensus systems on cumulus-based parachains.

Cumulo -- Nimbus ⛈️ Nimbus is a framework for building parachain consensus systems on cumulus-based parachains. Given the regular six-second pulse-lik

null 36 Dec 14, 2022
Amethyst is a systems language aimed at being simple, small, portable, and safe.

amethyst Amethyst is a systems language aimed at being simple, small, portable, and safe. What is this language? From the r/ProgLangs discord server:

Amethyst Language 34 Dec 18, 2022
Ointers is a library for representing pointers where some bits have been stolen so that they may be used by the programmer for something else

Ointers is a library for representing pointers where some bits have been stolen so that they may be used by the programmer for something else. In effect, it's a small amount of free storage

Irrustible 8 Jun 4, 2022
Simple MHV6 extension used to download custom songs efficiently and effectively.

nong-downloader-extension A simple MegaHack v6 extension that helps you download NONG songs directly to GD. Compiling. Why would you want to compile??

Alpha 6 Nov 1, 2022
Common processing blocks used with your Runes.

Common Processing Blocks (API Docs) Processing blocks built by Hammer of the Gods that you can use with your Runes. License This project is licensed u

Hammer of the Gods 9 Jul 21, 2022
Smart Contracts to be used by terra-bot

CosmWasm Starter Pack This is a template to build smart contracts in Rust to run inside a Cosmos SDK module on all chains that enable it. To understan

João Silvestre 2 Sep 25, 2022