By mirroring traffic to and from your machine, mirrord surrounds your local service with a mirror image of its cloud environment.

Last update: Jun 22, 2022

mirrord logo dark

mirrord logo light

mirrord lets you easily mirror traffic from your Kubernetes cluster to your development environment. It comes as both Visual Studio Code extension and a CLI tool.

Getting Started

mirrord uses your machine's default kubeconfig for access to the Kubernetes API.

Make sure your local process is listening on the same port as the remote pod.


VSCode Extension

Installation

Get the extension here.

How to use

  • Click "Enable mirrord" on the status bar
  • Start debugging your project
  • Choose pod to mirror traffic from
  • The debugged process will start with mirrord, and receive traffic


CLI Tool

Installation

curl -fsSL https://raw.githubusercontent.com/metalbear-co/mirrord/main/scripts/install.sh | bash
  • Windows isn't supported currently (you can use WSL)

How to use

mirrord exec <process command> --pod-name <name of the pod to impersonate>

e.g.

mirrord exec node app.js --pod-name my-pod

How it works

mirrord works by letting you select a pod to mirror traffic from. It launches a privileged pod on the same nodewhich enters the namespace of the selected pod and captures traffic from it.

Contributing

Contributions are welcome via PRs.

Help & Community 🎉 ✉️

Join our Discord Server for questions, support and fun.

Code of Conduct

We take our community seriously and we are dedicated to providing a safe and welcoming environment for everyone. Please take a few minutes to review our Code of Conduct.

License

MIT

GitHub

https://github.com/metalbear-co/mirrord
Comments
  • 1. Resolves #53

    Adds a more graceful exit when the directory does not exist.

    If the desired behavior is any different (for example, printing the absolute path of the nonexistent directory), I can update with the absolute path.

    Reviewed by thedanvail at 2022-06-03 14:53
  • 2. CLI Error

    After installing the mirrord CLI, It didn't turn available through my terminal.

    # Installed the CLI
    $ npm install -g mirrord
    ...
    
    # Checked where it have been installed
    $ which mirrord
    /home/tom/.nvm/versions/node/v17.6.0/bin/mirrord
    
    # Attempted to use it
    $ mirrord -v
    /usr/bin/env: ‘npx ts-node’: No such file or directory
    /usr/bin/env: use -[v]S to pass options in shebang lines
    
    # Checked for ts-node
    $ which ts-node
    /home/tom/.nvm/versions/node/v17.6.0/bin/ts-node
    
    # Confirmed that is available
    $ npx ts-node
    >
    

    If further information is required for your investigation, don't hesitate to ask me for it.

    Reviewed by 7onn at 2022-03-10 15:07
  • 3. V2.0

    Alright my initial PR is adding just a hook for open, but in the process I also ended up refactoring the select! macro usage.

    Before this becomes an even bigger PR, I'm pushing it to get some reviews.

    Reviewed by meowjesty at 2022-05-17 04:28
  • 4. Add E2E tests

    This PR closes #35 adding tests conforming to the suite provided in #50

    Current state of the tests: validation tests:

    • run the node application with mirrord exec in a forked process
    • send GET, POST, PUT, and DELETE requests and keep a counter
    • POST validates data.txt, PUT creates a file with the data received, DELETE reads the file created by PUT, validates the data and removes the file.
    • if all operations succeed, we exit with EXIT_SUCCESS
    • since only a single job and pod is created, we verify that these resources exist and are removed after agent timeout finishes.

    namespace tests [agent]:

    • create a new namespace
    • run a binary with agent namespace assigned the new namespace
    • check if the job and the pod exist in the created namespace within the timeout limit
    • run a binary with agent namespace assigned a non-existent namespace
    • make sure the process crashes
    • verify no resources have been created in either the default or the bad namespace

    namespace tests [pod]:

    • create a new namespace
    • run a binary with pod namespace assigned the new namespace in the form of an env var
    • verify the job exists
    Reviewed by infiniteregrets at 2022-05-16 17:07
  • 5. mirrord extract panics on non-existent directories

    [email protected]:~/mirrord$ target/debug/mirrord extract blah 
    thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', mirrord-cli/src/main.rs:70:52
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    

    It would be nicer to handle the Result type gracefully and return an error or maybe prompt the user whether they want to create the directory (only useful for CLI users)

    Reviewed by infiniteregrets at 2022-05-16 06:09
  • 6. Add detours for dup/dup2/fcntl system calls

    When mirrord exec is called with a simple flask API (https://gist.github.com/infiniteregrets/ba753ffb53ff8b98b2dd8f5051323919), where port 80/any other port is explicitly specified, the application seems to ignore that and instead assign a different port. This could perhaps be due to a call to dup or fcntl with F_DUPFD/F_DUPFD_CLOEXEC flags where mirrord is not aware of the new sockfd and probably getsockname returns a different port to the flask app. So we might need to add detours for the mentioned system calls to bring new sockfds to mirrord book-keeping. More context: Strace of flask API: https://gist.github.com/infiniteregrets/09773cc3f1aee7983f95fc800ff8d969 Agent logs when enforcing port 3689 in the Flask Application: https://gist.github.com/infiniteregrets/7c42aeae359e3a55c756f2ba9c7e1213

    Reviewed by infiniteregrets at 2022-05-16 05:35
  • 7. VSCode extension doesn't list pods outside the default namespace

    Description: I went to test the extension with an app written in Golang which runs outside of my cluster's default namespace. After clicking on Start mirrord in the status bar, only pods placed in the default namespace are listed. Because of this, I wasn't able to proceed with the tests.

    Notes: I'm not sure what useful information I could add here since the debugging mode never got to receive forwarded connections. But feel free to ask anything for I'm willing to help =)

    Reviewed by 7onn at 2022-03-10 15:15
  • 8. mirrord-cli : reqwest dependency missing "blocking" feature

    Bug Description

    mirrord-cli uses the "blocking" feature of reqwest: https://github.com/metalbear-co/mirrord/blob/255fbe55262019f282010951d9d6f8158ec2530d/mirrord-cli/src/main.rs#L193

    but does not enable that feature in its toml file https://github.com/metalbear-co/mirrord/blob/255fbe55262019f282010951d9d6f8158ec2530d/mirrord-cli/Cargo.toml#L26

    should look something like this (docs):

    reqwest = { version = "0.11.10", features = ["blocking"] }
    

    Steps to Reproduce

    1. Clone main branch (as of this commit)
    2. Open project in dev container (I'm using VSCode remote containers extension)
    3. run cargo check or cd mirrord-cli && cargo check
    4. 💥 (see backtrace)

    Backtrace

    if let Ok(client) = reqwest::blocking::Client::builder().build() {
       |                         ^^^^^^^^ could not find `blocking` in `reqwest`
    

    Relevant Logs

    No response

    Your operating system and version

    local: macOS Monterey version 12.4, container: Debian OS variant buster

    Local process

    N/A

    Local process version

    No response

    Additional Info

    If you have already been working with the source code it may be necessary to rebuild your container to simulate this issue.

    I'm not sure if I would call it "cached", but after I solved this issue once "in passing" i was unable to make it happen again regardless of cargo clean ext.

    Reviewed by IsaacCloos at 2022-06-16 04:30
  • 9. Specify custom names for agent image in E2E tests

    The agent image once built is tagged as test:latest, this results in an Elapsed error if the image is not loaded to minikube by any chance, so we should not be hardcoding this. It would be nice to be able to specify the image as an environment variable and read from it, and do the same in the CI.

    Reviewed by infiniteregrets at 2022-06-03 14:04
  • 10. File pr v3

    openat implementation missing.

    Some of the changes in -agent come from @infiniteregrets branch (might be a bit outdated by now). Will update those as well after getting some review on file + refactors I made.

    Reviewed by meowjesty at 2022-06-01 18:30
  • 11. sockets fd isn't removed on close

    Bug Description

    We currently don't hook close, meaning that if a program runs close on a socket we manage we don't delete it from our managed fds.

    Steps to Reproduce

    N/A

    Backtrace

    No response

    Your operating system and version

    N/A

    Your Node version (node --version)

    N/A

    Additional Info

    No response

    Reviewed by aviramha at 2022-05-22 18:43
  • 12. Cannot test file-ops with mirrord VSCode extension

    Bug Description

    mirrord's VSCode extension fails to debug the local process.

    Steps to Reproduce

    • Add this configuration to your launch.json:
     {
          "type": "lldb",
          "request": "launch",
          "name": "ops.py: Debug file ops",
          "program": "python3",
          "args": ["${workspaceFolder}/tests/python-e2e/ops.py"],
          "cwd": "${workspaceFolder}",
          "sourceMap": {},      
          "env": {
            "DYLD_INSERT_LIBRARIES": "${workspaceFolder}/target/debug/libmirrord_layer.dylib",
            "LD_PRELOAD": "${workspaceFolder}/target/debug/libmirrord_layer.so",
            "RUST_LOG": "warn,mirrord=debug",
            "MIRRORD_AGENT_IMPERSONATED_POD_NAME": "http-echo-deployment-64887f7cdb-4lwrs",
            "MIRRORD_AGENT_RUST_LOG": "warn,mirrord=debug",
            "MIRRORD_FILE_OPS": "true",
            "RUST_BACKTRACE": "short", 
            "MIRRORD_ACCEPT_INVALID_CERTIFICATES": "true",
            "MIRRORD_AGENT_IMAGE": "runtime"   
          }
        }
    
    • Start debugging and choose the impersonated pod

    Backtrace

    2022-06-20T17:26:58.258050Z  INFO mirrord_layer: Initializing mirrord-layer!
    2022-06-20T17:26:58.259302Z  WARN mirrord_layer::pod_api: Accepting invalid certificates
    2022-06-20T17:26:59.597035Z DEBUG mirrord_layer::sockets: getpeername hooked
    2022-06-20T17:26:59.597069Z DEBUG mirrord_layer::sockets: getsockname hooked
    2022-06-20T17:26:59.597090Z DEBUG mirrord_layer::sockets: accept4 hooked
    2022-06-20T17:26:59.597106Z DEBUG mirrord_layer::sockets: dup3 hooked
    2022-06-20T17:26:59.597122Z DEBUG mirrord_layer::sockets: accept hooked
    FFF
    ======================================================================
    FAIL: test_lseek (__main__.FileOpsTest)
    Seeks character by character in a file with Lorem Ipsum text in "/tmp" and verifies the concatenation of the text.
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/mehula/mirrord/tests/python-e2e/ops.py", line 27, in test_lseek
        self.assertFalse(self._check_path_exists_on_host(file_path))
    AssertionError: True is not false
    
    ======================================================================
    FAIL: test_openat (__main__.FileOpsTest)
    Opens a directory i.e. "/tmp", then opens a file in temp using openat give the directory file desciptor for "/tmp".
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/mehula/mirrord/tests/python-e2e/ops.py", line 43, in test_openat
        self.assertFalse(self._check_path_exists_on_host(file_path))
    AssertionError: True is not false
    
    ======================================================================
    FAIL: test_read_write_family (__main__.FileOpsTest)
    Reads data from a file in "/tmp" and verifies the text expected is the same as the text written.
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/mehula/mirrord/tests/python-e2e/ops.py", line 14, in test_read_write_family
        self.assertFalse(self._check_path_exists_on_host(file_path))
    AssertionError: True is not false
    
    ----------------------------------------------------------------------
    Ran 3 tests in 0.001s
    
    FAILED (failures=3)
    [email protected]:~/mirrord$ kubectl get pods
    NAME                                    READY   STATUS    RESTARTS     AGE
    http-echo-deployment-64887f7cdb-4lwrs   1/1     Running   7 (6d ago)   6d4h
    mirrord-agent-kpjldmsafe-2zrr4          1/1     Running   0            3s
    mirrord-agent-ykgeipboyw-2g4tj          1/1     Running   0            4s
    [email protected]:~/mirrord$ kubectl logs mirrord-agent-kpjldmsafe-2zrr4
    2022-06-20T17:27:00.470051Z DEBUG mirrord_agent: main -> Initializing mirrord-agent.
    2022-06-20T17:27:00.470269Z DEBUG mirrord_agent::sniffer: packet_worker -> setting namespace
    2022-06-20T17:27:00.470333Z DEBUG mirrord_agent::file: file_worker -> Setting namespace
    2022-06-20T17:27:00.471149Z DEBUG mirrord_agent::runtime: set_namespace -> fd 12
    2022-06-20T17:27:00.471166Z DEBUG mirrord_agent::sniffer: preparing sniffer
    2022-06-20T17:27:00.471169Z DEBUG mirrord_agent::sniffer: prepare_sniffer -> Preparing interface.
    2022-06-20T17:27:00.594518Z DEBUG mirrord_agent: start -> Connection accepted from 127.0.0.1:51646
    2022-06-20T17:27:01.597675Z DEBUG mirrord_agent: peer_handler -> daemon_stream.next received a message None
    2022-06-20T17:27:01.597730Z DEBUG mirrord_agent: start_agent -> Peer 0 closed
    2022-06-20T17:27:01.597750Z DEBUG mirrord_agent: ClientMessage::Close -> peer id 0 sent close
    2022-06-20T17:27:01.597779Z DEBUG mirrord_agent::sniffer: packet_worker -> setting ports []
    2022-06-20T17:27:01.597783Z DEBUG mirrord_agent::sniffer: packet_worker -> empty ports, setting dummy bpf
    [email protected]:~/mirrord$ kubectl logs mirrord-agent-ykgeipboyw-2g4tj
    2022-06-20T17:26:59.111089Z DEBUG mirrord_agent: main -> Initializing mirrord-agent.
    2022-06-20T17:26:59.111252Z DEBUG mirrord_agent::sniffer: packet_worker -> setting namespace
    2022-06-20T17:26:59.111253Z DEBUG mirrord_agent::file: file_worker -> Setting namespace
    2022-06-20T17:26:59.112627Z DEBUG mirrord_agent::runtime: set_namespace -> fd 12
    2022-06-20T17:26:59.112638Z DEBUG mirrord_agent::sniffer: preparing sniffer
    2022-06-20T17:26:59.112641Z DEBUG mirrord_agent::sniffer: prepare_sniffer -> Preparing interface.
    2022-06-20T17:26:59.593772Z DEBUG mirrord_agent: start -> Connection accepted from 127.0.0.1:51642
    2022-06-20T17:27:01.634394Z DEBUG mirrord_agent: peer_handler -> daemon_stream.next received a message None
    2022-06-20T17:27:01.634527Z DEBUG mirrord_agent: ClientMessage::Close -> peer id 0 sent close
    2022-06-20T17:27:01.634581Z DEBUG mirrord_agent::sniffer: packet_worker -> setting ports []
    2022-06-20T17:27:01.634591Z DEBUG mirrord_agent::sniffer: packet_worker -> empty ports, setting dummy bpf
    2022-06-20T17:27:01.634536Z DEBUG mirrord_agent: start_agent -> Peer 0 closed
    
    
    
    ### Relevant Logs
    
    _No response_
    
    ### Your operating system and version
    
    [email protected]:~/mirrord$ uname -a Linux mehul-machine 5.13.0-1029-azure #34~20.04.1-Ubuntu SMP Thu Jun 9 12:37:07 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
    
    ### Local process
    
    python3 tests/python-e2e/ops.py
    
    ### Local process version
    
    [email protected]:~/mirrord$ python3 -V Python 3.8.10
    
    ### Additional Info
    
    _No response_
    Reviewed by infiniteregrets at 2022-06-20 17:31
  • 13. Change e2e to use a "real app"

    Right now the E2E uses something we found online (http echo) but that means that the TCP/HTTP session behavior can be different leading to odd reactions. I think building our own docker image for the test would be better. It should be something that accepts the same APIs that our tests use, have files that we expect to have, etc.

    Long story: Right now we work on TCP level, and therefore if the application running locally and the application running remotely are different, one can close the TCP connection for each HTTP request and the other can reuse it. By using the same app we eliminate randomness.

    Reviewed by aviramha at 2022-06-15 06:26
  • 14. Add flag for file-worker in agent and re-enable the blackbox test

    The test https://github.com/metalbear-co/mirrord/blob/main/mirrord-agent/tests/blackbox.rs#L23 is currently being ignored, a fix would be to enable/disable file operations through a flag

    Reviewed by infiniteregrets at 2022-06-10 13:49
  • 15. Add E2E tests for file operations

    With 2.2.0 release, we need e2e tests for the following functions:

    • [x] open
    • [ ] fopen
    • [ ] fdopen
    • [x] openat
    • [x] read
    • [ ] fread
    • [ ] fileno
    • [x] lseek
    • [x] write
    Reviewed by infiniteregrets at 2022-06-10 13:44
  • 16. Can't execute

    Bug Description

    mirrord-cli in release (downloaded from the script/compiled binary) gets killed when executing the process on macOS

    Steps to Reproduce

    1. download release build
    2. mirrord exec --pod-name test-pod python
    3. kill

    Backtrace

    no backtrace :(
    

    Relevant Logs

    No response

    Your operating system and version

    21.5.0 Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:37 PDT 2022; root:xnu-8020.121.3~4/RELEASE_ARM64_T6000 arm64

    Local process

    python

    Local process version

    No response

    Additional Info

    No response

    Reviewed by aviramha at 2022-06-08 08:08
Save image from your clipboard 📋 as an image file directly from your command line! 🔥

Clpy ?? Save copied image from clipboard as an image file directly from your command line! Note It works only on windows as of now. I'll be adding sup

Aug 6, 2021
Rbenv - Manage your app's Ruby environment

Seamlessly manage your app’s Ruby environment with rbenv. Use rbenv to pick a Ruby version for your application and guarantee that your development en

Jun 22, 2022
A git command to quickly save your local changes in case of earthquake !

git-eq (aka git earthquake) Earthquakes are part of the daily life in many countries like in Taiwan. git-eq is a simple git command to quickly save yo

May 27, 2022
Black-box integration tests for your REST API using the Rust and its test framework

restest Black-box integration test for REST APIs in Rust. This crate provides the [assert_api] macro that allows to declaratively test, given a certai

May 25, 2022
Check the reproducibility status of your Arch Linux packages (read-only mirror)
Check the reproducibility status of your Arch Linux packages (read-only mirror)

arch-repro-status A CLI tool for querying the reproducibility status of the Arch Linux packages using data from a rebuilderd instance such as reproduc

May 19, 2022
Code and Development environment for adventofcode.com - 2021 edition

aoc-2021 Warning Spoiler Alert! If you want to solve the aoc problems on your own, do not read any further. This repository contains solutions for the

Jun 18, 2022
An interactive shell environment for exploring the p2panda protocol
An interactive shell environment for exploring the p2panda protocol

An interactive shell environment for exploring the p2panda protocol. Uses a mock node and clients to simulate network logic.

Dec 12, 2021
A Rust-based Garry's Mod module for fetching environment variables.

gm_environ Using Environment Variables in Garry's Mod. Installation Download a copy of the module from the releases (or compile from source) Move the

Jan 4, 2022
Envful is a CLI tool that verifies the presence of environment variables
Envful is a CLI tool that verifies the presence of environment variables

Envful is a CLI tool that verifies the presence of environment variables. It looks inside your .env file and the host system. You can use it to run any process while ensuring all the variables are set.

Jun 5, 2022
Set Shell Environment Variables across multiple shells with a single configuration file.

Xshe – Cross-Shell Environment Vars xshe allows for setting Shell Environment Variables across multiple shells with a single TOML configuration file.

Jun 19, 2022
A CI inspired approach for local job automation.

nauman A CI inspired approach for local job automation. Features • Installation • Usage • FAQ • Examples • Job Syntax About nauman is an easy-to-use j

May 8, 2022
Tricking shells into interactive mode when local PTY's are not available
Tricking shells into interactive mode when local PTY's are not available

Remote Pseudoterminals Remote Pseudoterminals or "RPTY" is a Rust library which intercepts calls to the Linux kernel's TTY/PTY-related libc functions

Jun 15, 2022
A library that creates a terminal-like window with feature-packed drawing of text and easy input handling. MIRROR.

BearLibTerminal provides a pseudoterminal window with a grid of character cells and a simple yet powerful API for flexible textual output and uncompli

May 11, 2022
An awesome language and its compiler.

Calocom Project Build Build with cargo-make Just run cargo make in the root directory of the project. Or, if you want to build it in release mode, run

Jun 15, 2022
Mirror of https://gitlab.redox-os.org/redox-os/termion
Mirror of https://gitlab.redox-os.org/redox-os/termion

Documentation Examples Changelog Tutorial Termion is a pure Rust, bindless library for low-level handling, manipulating and reading information about

Jun 25, 2022
tkn-watch - watch a PipelineRuns on its way to success or failures
tkn-watch - watch a PipelineRuns on its way to success or failures

tkn-watch - watch a PipelineRuns on its way to success or failures tkn-watch is a simple extension to the tkn command line tool that watches a Pipelin

May 14, 2022
A diff-based data management language to implement unlimited undo, auto-save for games, and cloud-apps which needs to retain every change.

Docchi is a diff-based data management language to implement unlimited undo, auto-save for games, and cloud-apps which needs to save very often. User'

Jan 19, 2022
Command-line tool to generate Rust code for Google Cloud Spanner

nene nene is a command-line tool to generate Rust code for Google Cloud Spanner. nene uses database schema to generate code by using Information Schem

Dec 7, 2021
Provide CRUD CLI for Moco Activities with Jira Cloud Sync Option for faster time tracking.

Moco CLI Provide CRUD CLI for Moco Activities with Jira Cloud Sync Option for faster time tracking. Available commands Login Jira Must be called befor

Jun 14, 2022