Docker containers on a synthetic network. Run applications in a context that lets you manipulate their network conditions.


Synthetic Network

Docker containers on a synthetic network. Run applications in a context that lets you manipulate their network conditions.



$ make
CONTAINER_NAME_INTERACTIVE ?= syntheticnet-interactive
CONTAINER_NAME_CHROME ?= syntheticnet-chrome
(add /etc/hosts entry to container) help: # Print this help message image: # Build Docker image: syntheticnet image-vnc: # Build Docker image: syntheticnet:vnc image-chrome: image-vnc # Build Docker image: syntheticnet:chrome run-interactive: image synthetic-network # Debug syntheticnet container. Prereq: create-synthetic-network run-chrome: image-chrome synthetic-network # Run syntheticnet:chrome. Prereq: create-synthetic-network synthetic-network: # Specify SYNTHETIC_NETWORK (this rule is documentation) create-synthetic-network: synthetic-network # Create Docker network: synthetic-network

Run Chrome using Synthetic Network in VNC

  1. ensure Docker for Mac/Windows/Linux is running

$ make create-synthetic-network # You only need to do this once
$ make run-chrome
πŸŽ› Synthetic network GUI will listen on http://localhost:3000

πŸ“Ί Point your VNC client at localhost:5901
  1. open TigerVNC and navigate to

Resolving test domains within the container

$ make run-chrome

Build Container Image

Build syntheticnet image

$ make image

with VNC:

$ make image-vnc

Scripting the Synthetic Network

const SyntheticNetwork = require('synthetic-network/frontend')

const synthnet = new SyntheticNetwork({hostname: "localhost", port: 3000})

await synthnet.get() // Get current configuration

// Double ingress rate
var current_ingress_rate = synthnet.default_link.ingress.rate()

await synthnet.commit() // Apply new configuration

// Add a flow
synthnet.addFlow('udp', {protocol: 'udp'})
await synthnet.commit()

// Print ingress traffic statistics
const ingress_profile = await synthnet.profiles.ingress.get()
for (var flow in ingress_profile.flows)
  console.log(flow, ingress_profile.flows[flow].packets)

// ...

See also: frontend/udp_rate_sine_demo.js

Further reading

Check out the reports under doc/ for details.

The packet processing framework we use to do network conditioning can be found under rush/. Its README points to a screen cast series covering its design and implementation.

  • cargo build --release: cannot find macro `asm` in this scope

    cargo build --release: cannot find macro `asm` in this scope

    Expected behavior

    Hello! When I clone this repo and run make create-synthetic-network and then make run-chrome, I would expect the latter command to build the Docker image and exit successfully per the README.

    Describe the bug (unexpected behavior)

    Instead, when I clone this repo and run make run-chrome the docker build (...) errors while running cargo build --release:

    Click to expand a shell session of the build and error log
    ~/dev $ git clone
    Cloning into 'synthetic-network'...
    remote: Enumerating objects: 1446, done.
    remote: Counting objects: 100% (1446/1446), done.
    remote: Compressing objects: 100% (1187/1187), done.
    remote: Total 1446 (delta 215), reused 1445 (delta 215), pack-reused 0
    Receiving objects: 100% (1446/1446), 4.77 MiB | 6.21 MiB/s, done.
    Resolving deltas: 100% (215/215), done.
    1s ~/dev $ cd synthetic-network/
    0s (main) ~/dev/synthetic-network $ ls Dockerfile         Makefile           aarch64.nix        doc          LICENSE            default.nix        frontend           rush               synth-chrome
    0s (main) ~/dev/synthetic-network $ make create-synthetic-network
    docker network create synthetic-network --subnet=
    7s (main) ~/dev/synthetic-network $ make run-chrome
    docker build -t syntheticnet:vnc --build-arg VNC=true .
    [+] Building 100.4s (14/17)
     => [internal] load build definition from Dockerfile                                                                                                                                                 0.0s
     => => transferring dockerfile: 1.08kB                                                                                                                                                               0.0s
     => [internal] load .dockerignore                                                                                                                                                                    0.0s
     => => transferring context: 2B                                                                                                                                                                      0.0s
     => [internal] load metadata for                                                                                                                                      1.9s
     => [auth] library/node:pull token for                                                                                                                                          0.0s
     => [internal] load build context                                                                                                                                                                    1.0s
     => => transferring context: 12.72MB                                                                                                                                                                 1.0s
     => [ 1/12] FROM                                                                             38.6s
     => => resolve                                                                                0.0s
     => => sha256:0140c4e17aa1f3fd78b939e3c7ac45accdd5274d0194046b484029a5e46b9db6 776B / 776B                                                                                                           0.0s
     => => sha256:2571706a0e9a90a55ea51746239ff8acd489e5c2ce2f119273245ccbc0a94392 2.21kB / 2.21kB                                                                                                       0.0s
     => => sha256:3aab5c2e02a68fcfa5e8c766e1689c6790580f0d51ee647ab67ea57b21fa6dc2 7.83kB / 7.83kB                                                                                                       0.0s
     => => sha256:296c9f0bf5f2cf24e87bc5abd674fc486e8df419d4aa2d362453f64d38900504 43.18MB / 43.18MB                                                                                                    10.1s
     => => sha256:84d546321c8e11789c75187b5b592633207519d75f96e8f03e8378d2da4a72a9 10.18MB / 10.18MB                                                                                                     3.9s
     => => sha256:f91c8ffe219846d18819503633eb252121c21cd20dbff74278b85c4029370992 4.10MB / 4.10MB                                                                                                       1.8s
     => => sha256:df8d698654f0ee6c5be4a19593a2d7a793c4bed9cb10de0c87d1d1c2ce6ba368 47.73MB / 47.73MB                                                                                                    11.2s
     => => sha256:eb8f7931e7aa1d31b83a8db251eb9107d989c9244f8bd7a542788aef3b6d1975 201.71MB / 201.71MB                                                                                                  32.0s
     => => sha256:6516b36f67a70cc6094778e92c2002789a89825b2769c9d5ce9f6427c5c813c2 4.20kB / 4.20kB                                                                                                      10.4s
     => => extracting sha256:296c9f0bf5f2cf24e87bc5abd674fc486e8df419d4aa2d362453f64d38900504                                                                                                            2.3s
     => => sha256:8592970063e59260afa874fb5f28a75d3e4742e7edaaafc1ba690c492e2039d4 23.70MB / 23.70MB                                                                                                    15.0s
     => => sha256:934b7a74b39bb36b47f580e3c33386d8edf20b9f9bc885442b1e78875d253ddf 2.38MB / 2.38MB                                                                                                      11.8s
     => => sha256:68a1166726ff6c90bb9e39a88d2245a71adb142bc16a8731e84ac31b4762ccbc 292B / 292B                                                                                                          12.1s
     => => extracting sha256:84d546321c8e11789c75187b5b592633207519d75f96e8f03e8378d2da4a72a9                                                                                                            0.7s
     => => extracting sha256:f91c8ffe219846d18819503633eb252121c21cd20dbff74278b85c4029370992                                                                                                            0.2s
     => => extracting sha256:df8d698654f0ee6c5be4a19593a2d7a793c4bed9cb10de0c87d1d1c2ce6ba368                                                                                                            3.1s
     => => extracting sha256:eb8f7931e7aa1d31b83a8db251eb9107d989c9244f8bd7a542788aef3b6d1975                                                                                                            5.2s
     => => extracting sha256:6516b36f67a70cc6094778e92c2002789a89825b2769c9d5ce9f6427c5c813c2                                                                                                            0.0s
     => => extracting sha256:8592970063e59260afa874fb5f28a75d3e4742e7edaaafc1ba690c492e2039d4                                                                                                            0.8s
     => => extracting sha256:934b7a74b39bb36b47f580e3c33386d8edf20b9f9bc885442b1e78875d253ddf                                                                                                            0.1s
     => => extracting sha256:68a1166726ff6c90bb9e39a88d2245a71adb142bc16a8731e84ac31b4762ccbc                                                                                                            0.0s
     => [ 2/12] RUN apt-get update && apt-get install -y     iproute2 ethtool iputils-ping iperf3 python3 lsof tcpdump net-tools                                                                         5.7s
     => [ 3/12] ADD /opt/lib/                                                                                                                                                              0.0s
     => [ 4/12] RUN if [ -n "true" ] ; then apt-get install -y tigervnc-standalone-server ratpoison ; else echo "No VNC for you" ; fi                                                                   13.1s
     => [ 5/12] RUN apt-get install -y curl build-essential                                                                                                                                              1.9s
     => [ 6/12] RUN curl --proto '=https' --tlsv1.2 -sSf     | sh -s -- --default-toolchain nightly -y                                                                             26.2s
     => [ 7/12] ADD rush /opt/lib/rush                                                                                                                                                                   0.1s
     => [ 8/12] WORKDIR /opt/lib/rush                                                                                                                                                                    0.0s
     => ERROR [ 9/12] RUN cargo build --release                                                                                                                                                         12.7s
     > [ 9/12] RUN cargo build --release:
    #14 0.695    Compiling libc v0.2.68
    #14 0.695    Compiling proc-macro2 v1.0.24
    #14 0.695    Compiling unicode-xid v0.2.1
    #14 0.695    Compiling syn v1.0.60
    #14 0.695    Compiling getrandom v0.2.2
    #14 0.697    Compiling cfg-if v1.0.0
    #14 0.700    Compiling memchr v2.3.3
    #14 0.845    Compiling serde_derive v1.0.123
    #14 0.873    Compiling ryu v1.0.5
    #14 1.092    Compiling serde v1.0.123
    #14 1.125    Compiling signal-hook v0.3.6
    #14 1.198    Compiling byteorder v1.3.4
    #14 1.206    Compiling ppv-lite86 v0.2.10
    #14 1.207    Compiling serde_json v1.0.62
    #14 1.312    Compiling lazy_static v1.4.0
    #14 1.337    Compiling regex-syntax v0.6.17
    #14 1.403    Compiling itoa v0.4.7
    #14 1.435    Compiling once_cell v1.3.1
    #14 1.687    Compiling thread_local v1.0.1
    #14 1.854    Compiling aho-corasick v0.7.10
    #14 2.162    Compiling quote v1.0.9
    #14 2.342    Compiling signal-hook-registry v1.3.0
    #14 2.470    Compiling rand_core v0.6.2
    #14 2.700    Compiling rand_chacha v0.3.0
    #14 3.014    Compiling rand v0.8.3
    #14 3.564    Compiling regex v1.3.6
    #14 11.58    Compiling rush v0.1.0 (/opt/lib/rush)
    #14 12.45 error: cannot find macro `asm` in this scope
    #14 12.45    --> src/
    #14 12.45     |
    #14 12.45 151 |     asm!("
    #14 12.45     |     ^^^
    #14 12.45     |
    #14 12.45     = note: consider importing one of these items:
    #14 12.45             std::arch::asm
    #14 12.45             core::arch::asm
    #14 12.45
    #14 12.61 error: could not compile `rush` due to previous error
    executor failed running [/bin/sh -c cargo build --release]: exit code: 101
    make: *** [image-vnc] Error 1

    Steps to reproduce

    $ git clone
    $ cd synthetic-network/
    $ make run-chrome


    System information

    I am not familiar with Rust programming. I found this thread that seems surface-similar to the error I received, and so I thought perhaps it was an architecture-specific problem; I am running on arm64 on an M1 Mac. However I see the same error on an x86 Mac.

    OS and arch for both systems:

    # m1 mac, macOS Monterey 12.1
    $ uname -a
    Darwin MacBook-Pro.local 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:41 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T6000 arm64
    $ docker --version
    Docker version 20.10.11, build dea9396
    # x86 mac, macOS Monterey 12.0.1
    $ uname -a
    Darwin Jasons-MacBook-Pro-2.local 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64_x86_64
    $ docker --version
    Docker version 20.10.11, build dea9396
    opened by jasonm 4
  • use chromium instead of google-chrome-stable on arm64

    use chromium instead of google-chrome-stable on arm64

    It appears that google-chrome-stable is not available for arm64 linux. On an M1 macOS machine our Docker VMs are arm64 arch, so this patch replaces google-chrome-stable with Chromium.

    And also symlinks /usr/bin/google-chrome-stable to /usr/bin/chromium. I'm sorry.

    opened by kwindla 2
  • Fix rush build

    Fix rush build


    For reference:

    Namespace the asm! macro:

    Stablization of the asm! macro:

    opened by jasonm 2
  • ARM support

    ARM support

    Feature request

    Perhaps more a question or discussion than feature request, but: do you have any experience running this on ARM platforms e.g. Apple M1 or AWS Graviton?

    Looks like MediaSoup may support ARM ref1 ref2.

    The asm code in rush referenced in #2 / #3 indeed comes with both x86_64 and aarch64 targets.

    Why you need this

    I primarily develop on an M1 Mac. This is very much in the "nice-to-have" territory, but I am curious if others are successfully using this on ARM.

    Alternatives you've considered

    The main issues I encountered with ARM, and workarounds I tried, in case it's useful:

    The chrome image installs google-chrome-stable on Debian, but Google only provides Chrome for Linux amd64. Docker for Mac provides emulation via qemu (support for which should be regarded as "best effort" only). For me, amd64 (x86) Chrome in Docker (via make run-chrome) failed to launch with this error.

    I was not successful in finding an older version of Chrome that successfully launched via apt or deb for Debian 9 (per this repo's Dockerfile FROM node:12.20.2) or Debian 11 (the newest stable version; easy to update by changing this repo's Dockerfile to use FROM node:12-bullseye).

    I also tried various versions of Chromium under arm64, but was not successful.

    I ended up running it successfully on an x86 machine, but thought I'd share notes in case this is useful πŸ˜„

    opened by jasonm 1
  • add new Makefile target

    add new Makefile target "minimal"

    The minimal target will build the smallest synthetic-network image for the rush user proxy to work. This means no frontend is available.

    Since there is no frontend available this requires the use of QOS_SPEC environment variable in order to initialize rush with the right settings.

    (Needs to be rebased on top of #8 once that's merged)

    opened by aconchillo 0
  • BUG: executor failed running [/bin/sh -c cargo build --release]: exit code: 101

    BUG: executor failed running [/bin/sh -c cargo build --release]: exit code: 101

    Expected behavior

    As shown in the, the following should be printed when `make run-chrome' command is run.

    πŸŽ› Synthetic network GUI will listen on http://localhost:3000
    πŸ“Ί Point your VNC client at localhost:5901

    Describe the bug (unexpected behavior)

    However, the following checksum mismatch error is printed.

     => ERROR [ 9/12] RUN cargo build --release                                                                     1.1s
     > [ 9/12] RUN cargo build --release:
    #14 1.030 error: the listed checksum of `/opt/lib/rush/vendor/byteorder/` has changed:
    #14 1.030 expected: 51f0eb3b6139fc1a908d41a7b3cba7d58d684700986b3518f82e5af254c39e8e
    #14 1.030 actual:   3ff0b140f3f49df66dc452b6df345710d2db900d9477d18c045bd48eab5a45c4
    #14 1.030
    #14 1.030 directory sources are not intended to be edited, if modifications are required then it is recommended that `[patch]` is used with a forked copy of the source
    executor failed running [/bin/sh -c cargo build --release]: exit code: 101
    Makefile:18: recipe for target 'image-vnc' failed
    make: *** [image-vnc] Error 1

    Steps to reproduce

    1. Clone the git repo
    2. Run make
    3. Then, make create-synthetic-network
    4. And, make run-chrome

    System information

    • Device: Intel(R) Core(TM) i5-1035G1 CPU @ 1.00GHz 1.19 GHz, 16GB RAM
    • OS, version: Windows 10 Home Single Language, Version: 21H1, OS build: 19043.2006
    • Browser version: Not used yet.
    • Docker version: Docker Desktop 4.12.0, command line: Docker version 20.10.17, build 100c701
    • VNC (if applicable): Not used yet.

    Looking forward to getting started with using synthetic-network.

    opened by qwertynik 0
  • Modeling real-world networks

    Modeling real-world networks

    @steely-glint was wondering on Twitter how well network testing tools such as Synthetic Network simulate real-world network characteristics such as buffering that real congested networks do.

    Seems like a great opportunity to compare notes: what do we do, and what do networks do? What could we do to be more like networks?

    Right now we have these building blocks:

    • RateLimiter: forwards packets up to a bps limit, and drops all packets exceeding the limit (no buffering!). It uses a token bucket algorithm and replenishes tokes every 100us.
    • Loss: drops a fraction of all incoming packets at random and forwards the rest (no buffering!)
    • Latency: buffers packets for X ms before forwarding. (Never drops or reorders packets!)
    • Jitter: like Latency but buffers packets for a random duration up to X ms. Can be configured to reorder packets. (Never drops packets!)

    At the moment they are combined as our QoS path for each direction of each flow like so:


    @steely-glint had some specific use-cases in mind that I think would be interesting to think about how to model in SyntheticNetwork:

    I am trying to mimic 2 common domestic situations.

    1. where a user moves from room to room forcing WiFi to change modulation modes.
    2. where a second family member starts streaming Netflix and halves the available bandwidth.

    I worry about the delay. I suspect that changing the modulation also changes the time to transmit a packet, so the delay would change too. Buffer preserved in both cases ?

    We might be able to model 1) already by scripting RateLimiter and Latency in concert. However we don’t have an explicit way to talk about buffers, It might make sense to decouple simulation of buffering and delay, and make the buffer sizes scriptable as well. Imagine this:


    We could differentiate between wifi latency and geographic network latency, and we could change RateLimiter and Latency to read from Buffers as permitted/needed. If the Buffer size would be scriptable we could possibly simulate arbitrary network configurations?

    As for 2) we might want to add contenders to a flow. For example:


    The Join building block we already have. LoadGen could be interesting to write. It should probably simulate real traffic with pauses/intervals etc, and probably we should make sure that we don’t actually send the generated packets out in the end! ;-)

    Anyways that’s the braindump / food for thought. Looking forward to hear what others think! :-)

    help wanted question 
    opened by eugeneia 0
Video chat API
A tool for defining and running multi-container Docker applications

Ikki Ikki is a tool for defining and running multi-container Docker applications. It is similar to Docker Compose but comes with some differences. Goa

Kirill Vasiltsov 39 Dec 21, 2022
Docker daemon API in Rust

Bollard: an asynchronous rust client library for the docker API Bollard leverages the latest Hyper and Tokio improvements for an asynchronous API cont

Niel Drummond 439 Jan 3, 2023
Implementation of the Docker Registry HTTP API V2 in Rust, that can act as a proxy to other registries

Docker registry server and proxy (I'm bad at creating catchy names, but this one is good enough.) This project aims to implement a Docker Registry HTT

l4p1n (Mathias B.) 2 Dec 30, 2022
Convert your docker-compose into excalidraw

excalidocker-rs Rust-based utility to convert docker-compose.yaml files into excalidraw files. Key features Transform your local docker-compose files

Eugene Tolbakov 26 Jun 15, 2023
It's like "docker stats" but with beautiful, real-time charts into your terminal. πŸ“Š

?? ds - Real-time Stats with Terminal Charts Visualize container stats with beautiful, real-time charts directly in your terminal. Why ds? Missing Cha

Rafael R. Camargo 5 Oct 3, 2023
A simple tool in Rust to split urls in their protocol, host, port, path and query parts.

rexturl A simple tool to split urls in their protocol, host, port, path and query parts. Install cargo install rexturl or clone the source code and r

Volker Schwaberow 3 Oct 22, 2022
Hotwire allows you to study network traffic of a few popular protocols in a simple way

Hotwire Hotwire is a gtk GUI application that leverages the wireshark and tshark infrastructure to capture traffic and explore the contents of tcpdump

null 194 Dec 30, 2022
Small MQTT router. Allows creating multiple inputs/outputs and run action when input triggers.

MQRT Small MQTT router. Allows creating multiple inputs/outputs and run action when input triggers. Features multi-(input/output) multiple actions tie

Nazar Gondaruk 0 Jan 4, 2022
Export MacOS iMessage data + run iMessage Diagnostics

imessage-export This crate provides both a library to interact with iMessage data as well as a binary that can perform some useful read-only operation

Christopher Sardegna 10 Dec 30, 2022
Reverse proxy for HTTP microservices and STDIO. Openfass watchdog which can run webassembly with wasmer-gpu written in rust.

The of-watchdog implements an HTTP server listening on port 8080, and acts as a reverse proxy for running functions and microservices. It can be used independently, or as the entrypoint for a container with OpenFaaS.

yanghaku 7 Sep 15, 2022
A runtime for writing reliable asynchronous applications with Rust. Provides I/O, networking, scheduling, timers, ...

Tokio A runtime for writing reliable, asynchronous, and slim applications with the Rust programming language. It is: Fast: Tokio's zero-cost abstracti

Tokio 18.7k Dec 30, 2022
A Prometheus Aggregation Gateway for FAAS applications

Gravel Gateway Gravel Gateway is a Prometheus Push Gateway for FAAS applications. In particular it allows aggregation to be controlled by the incoming

Colin Douch 85 Nov 23, 2022
A Prometheus Aggregation Gateway for FAAS applications

Gravel Gateway Gravel Gateway is a Prometheus Push Gateway for FAAS applications. In particular it allows aggregation to be controlled by the incoming

Colin Douch 85 Nov 23, 2022
The Graph is a protocol for building decentralized applications (dApps) quickly on Ethereum and IPFS using GraphQL.

Graph Node The Graph is a protocol for building decentralized applications (dApps) quickly on Ethereum and IPFS using GraphQL. Graph Node is an open s 2 Jun 18, 2022
Automatically updates your Cloudflare DNS records for specific zones. Especially useful if you have dynamic IP address

Cloudflare DNS updater What does it do? Cloudflare DNS updater updates specified dns records for specified zones effortlessly and automatically. It wa

Niko Huuskonen 8 Aug 30, 2022
Rust library that helps you change the domain of the link to another domain πŸ¦€πŸ”

Rust library that helps you to change the domain of the link to another domain, the library helps with privacy. It can be used to change the domain of sites that do not care about privacy to another that does.

TheAwiteb 2 Mar 28, 2022
γ€ŒπŸ§±γ€Test a list of payloads and see if you can bypass it

γ€Œ ?? 」About TTWAF TTWAF, or Test This WAF, is a Web Application Firewall (WAF) bypass testing tool. You can test a list of payloads like XSS, LFI, RCE

Amolo Hunters 20 Dec 2, 2022
Fake rest is a fake API generator using a config file to help you develop clients.

About Fake-Rest is a fake API generator using a config file to help you develop clients. It's EASY AS HELL. Usage It's very simple to use. just create

Benyamin Eskandari 9 Feb 1, 2023
Tiny CLI application in rust to scan ports from a given IP and find how many are open. You can also pass the amount of threads for that scan

Port Scanner A simple multi-threaded port scanner written in Rust. Usage Run the port scanner by providing the target IP address and optional flags. $

nicolas lopes 4 Aug 29, 2023