Bug Description

When using asdf as my version manager for a bunch of languages, I cannot mirror any apps into a local k8s cluster.

I get the same message as if I had forgotten to set a local or global version for node. I'm sure this is somehow related to running in a different context than if I were running locally... I'm just confused. Running the same command without mirrord exec does work.

Steps to Reproduce

1. mirrord exec npm run dev --target pod/vs-deployment-<hash>

Backtrace

⠇ mirrord cli starting
  ✓ ready to launch process
    ✓ layer extracted
    ✓ agent pod created
    ✓ pod is ready
No version is set for command npm
Consider adding one of the following versions in your config file at /home/jim/Sites/ww-guest-site/.tool-versions```

Relevant Logs

No response

Your operating system and version

Linux kernel 6.0.9

Local process

npm run dev (which runs astro dev)

Local process version

node v18.8.0

Additional Info

No response

Summary

Extend e2e coverage to include net6.0 runtime and Kestrel web server.

Coverage

I've been running the sanity tests against these environments

• [X] macos arm64 (local)

Tasks

• [X] Make tests
• [X] Get Feedback on work
• [X] Update change log after feedback and code updates
• [ ] Update CI to run tests (In progress)

Criticism welcome 😄

Bug Description

I tried to use a binary executable file for testing.

And the pod crashed:

mirrord exec --target pod/busybox-58d584f97b-pbtv8 -- ./main
✓ layer initialized
✓ agent running
  ✓ agent pod created
  ✓ pod is ready
unexpected response - expected env vars response None
Terminated

Steps to Reproduce

1. Deploy a busybox image;
2. Use mirrord mirrord exec --target pod/busybox-58d584f97b-pbtv8 -- ./main;

Backtrace

No response

Relevant Logs

No response

Your operating system and version

Centos 7.9

Local process

None

Local process version

No response

Additional Info

No response

Bug Description

Go binaries cannot access environment variables.

Steps to Reproduce

// main.go
package main

import (
	"fmt"
	"os"
)

func main() {
	fmt.Println(os.Environ())
}

go build main.go

mirrord exec \
        --pod-namespace some-namespace \
        --pod-name some-pod \
        --override-env-vars-include "*" \
        ./main

Compare with

mirrord exec \
        --pod-namespace some-namespace \
        --pod-name some-pod \
        --override-env-vars-include "*" \
        env

Backtrace

No response

Relevant Logs

No response

Your operating system and version

macOS 12.5.1 Monterey

Local process

main: Mach-O 64-bit executable x86_64 go version go1.19 darwin/amd64

Local process version

No response

Additional Info

No response

Bug Description

Running mirrord exec with kubeconfig set to execute aws command to get token, fails with panic.

Steps to Reproduce

1. Create EKS cluster
2. Run aws eks update-kubeconfig --name test-cluster --alias test-cluster
3. Run mirrord exec ...

Backtrace

thread '<unnamed>' panicked at 'failed to create agent in k8s: mirrord-layer: Kube failed with error `auth error: auth exec command '"aws" "--region" "us-east-1" "eks" "get-token" "--cluster-name" "development-cluster"' failed with status signal: 6 (SIGABRT) (core dumped): Output { status: ExitStatus(unix_wait_status(134)), stdout: "", stderr: "thread '<unnamed>' panicked at 'failed to create agent in k8s: mirrord-layer: Kube failed with error `auth error: auth exec command '\"aws\" \"--region\" \"us-east-1\" \"eks\" \"get-token\" \"--cluster-name\" \"development-cluster\"' failed with status signal: 6 (SIGABRT) (core dumped): Output { status: ExitStatus(unix_wait_status(134)), stdout: \"\", stderr: \"thread '<unnamed>' panicked at 'failed to create agent in k8s: mirrord-layer: Kube failed with error `auth error: auth exec command '\\\"aws\\\" \\\"--region\\\" \\\"us-east-1\\\" \\\"eks\\\" \\\"get-token\\\" \\\"--cluster-name\\\" \\\"development-cluster\\\"' failed with status signal: 11 (SIGSEGV) (core dumped): Output { status: ExitStatus(unix_wait_status(139)), stdout: \\\"\\\", stderr: \\\"\\\\r\\\\nmalloc: unknown:0: assertion botched\\\\r\\\\nmalloc: block on free list clobbered\\\\nAborting...\\\" }`!', mirrord-layer/src/connection.rs:92:18\\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\\nfatal runtime error: failed to initiate panic, error 5\\n\" }`!', mirrord-layer/src/connection.rs:92:18\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\nfatal runtime error: failed to initiate panic, error 5\n" }`!', mirrord-layer/src/connection.rs:92:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5

Relevant Logs

No response

Your operating system and version

Manjaro Linux

Local process

Bourne-Again shell script, ASCII text executable

Local process version

No response

Additional Info

mirrord 3.0.19-alpha

Bug Description

I get the following error when I set the MIRRORD_TCP_OUTGOING to true on macOS an and a JVM-based service:

2022-10-26T18:13:26.498142Z  INFO ThreadId(01) mirrord_layer::go_env: replace -> hooking "runtime.goenvs_unix"
2022-10-26T18:13:26.688625Z  INFO ThreadId(15) mirrord_layer::socket::hooks: 14
2022-10-26T18:13:26.718586Z  INFO ThreadId(15) mirrord_layer::error: libc error (doesn't indicate a problem) >> ResponseError(
    RemoteIO(
        RemoteIOError {
            raw_os_error: Some(
                111,
            ),
            kind: ConnectionRefused,
        },
    ),
)
ERROR: transport error 202: connect failed: Unknown error: 111
ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)
JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [./src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c:735]

Process finished with exit code 2

Here are my env vars which I'm using:

DYLD_INSERT_LIBRARIES=/Users/jamowei/Library/Application Support/JetBrains/IntelliJIdea2022.2/plugins/mirrord/libmirrord_layer.dylib
LD_PRELOAD=/Users/jamowei/Library/Application Support/JetBrains/IntelliJIdea2022.2/plugins/mirrord/libmirrord_layer.so
MIRRORD_AGENT_RUST_LOG=DEBUG
RUST_LOG=DEBUG
MIRRORD_ACCEPT_INVALID_CERTIFICATES=true
MIRRORD_EPHEMERAL_CONTAINER=false
MIRRORD_SKIP_PROCESSES=
MIRRORD_OVERRIDE_ENV_VARS_INCLUDE=*
MIRRORD_IMPERSONATED_TARGET=pod/my-service-xxx
MIRRORD_TARGET_NAMESPACE=mynamespace
MIRRORD_FILE_OPS=false
MIRRORD_AGENT_TCP_STEAL_TRAFFIC=false
MIRRORD_REMOTE_DNS=true
MIRRORD_TCP_OUTGOING=true
MIRRORD_UDP_OUTGOING=false

When I set MIRRORD_TCP_OUTGOING to false mirroring works as expected.

Steps to Reproduce

1. set MIRRORD_TCP_OUTGOINGto true
2. start mirrord exec with the JVM-based server
3. get the error message

Backtrace

No response

Relevant Logs

2022-10-26T18:22:08.185829Z TRACE ThreadId(11) tungstenite::protocol: Frames still in queue: 0    
2022-10-26T18:22:08.185827Z TRACE ThreadId(02) handle_daemon_message: mirrord_layer: enter daemon_message=TcpOutgoing(Connect(Err(RemoteIO(RemoteIOError { raw_os_error: Some(111), kind: ConnectionRefused }))))
2022-10-26T18:22:08.185831Z TRACE ThreadId(11) tokio_tungstenite::compat: /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.17.2/src/compat.rs:149 Read.read    
2022-10-26T18:22:08.185834Z TRACE ThreadId(11) tokio_tungstenite::compat: /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.17.2/src/compat.rs:126 AllowStd.with_context    
2022-10-26T18:22:08.185836Z TRACE ThreadId(11) tokio_tungstenite::compat: /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.17.2/src/compat.rs:152 Read.with_context read -> poll_read    
2022-10-26T18:22:08.185839Z TRACE ThreadId(11) tokio_tungstenite::compat: WouldBlock    
2022-10-26T18:22:08.185842Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message: mirrord_layer::outgoing::tcp: enter daemon_message=TcpOutgoing(Connect(Err(RemoteIO(RemoteIOError { raw_os_error: Some(111), kind: ConnectionRefused })))) response=Connect(Err(RemoteIO(RemoteIOError { raw_os_error: Some(111), kind: ConnectionRefused })))
2022-10-26T18:22:08.185850Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message: mirrord_layer::outgoing::tcp: Connect -> connect Err(
    RemoteIO(
        RemoteIOError {
            raw_os_error: Some(
                111,
            ),
            kind: ConnectionRefused,
        },
    ),
) daemon_message=TcpOutgoing(Connect(Err(RemoteIO(RemoteIOError { raw_os_error: Some(111), kind: ConnectionRefused })))) response=Connect(Err(RemoteIO(RemoteIOError { raw_os_error: Some(111), kind: ConnectionRefused })))
2022-10-26T18:22:08.185866Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message: mirrord_layer::outgoing::tcp: exit daemon_message=TcpOutgoing(Connect(Err(RemoteIO(RemoteIOError { raw_os_error: Some(111), kind: ConnectionRefused })))) response=Connect(Err(RemoteIO(RemoteIOError { raw_os_error: Some(111), kind: ConnectionRefused })))
2022-10-26T18:22:08.185871Z TRACE ThreadId(02) handle_daemon_message: mirrord_layer: exit daemon_message=TcpOutgoing(Connect(Err(RemoteIO(RemoteIOError { raw_os_error: Some(111), kind: ConnectionRefused }))))
2022-10-26T18:22:08.185876Z TRACE ThreadId(02) actix_codec::framed: attempting to decode a frame    
2022-10-26T18:22:08.185906Z TRACE ThreadId(15) connect: mirrord_layer::socket::ops: exit sockfd=14 raw_address=0x70000bc22590 address_length=16
2022-10-26T18:22:08.185912Z  INFO ThreadId(15) mirrord_layer::error: libc error (doesn't indicate a problem) >> ResponseError(
    RemoteIO(
        RemoteIOError {
            raw_os_error: Some(
                111,
            ),
            kind: ConnectionRefused,
        },
    ),
)
ERROR: transport error 202: connect failed: Unknown error: 111
ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)
JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [./src/jdk.jdwp.agent/share/native/libjdwp/debugInit.c:735]

Process finished with exit code 2

Your operating system and version

macOS 12.6

Local process

/usr/bin/java: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64 - Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e - Mach-O 64-bit executable arm64e]

Local process version

openjdk version "11.0.15" 2022-04-19 LTS OpenJDK Runtime Environment Zulu11.56+19-CA (build 11.0.15+10-LTS) OpenJDK 64-Bit Server VM Zulu11.56+19-CA (build 11.0.15+10-LTS, mixed mode)

Additional Info

No response

• same default options as CLI
• improved UI layouts
• fixed not reappearing window after pressing cancel-button
• separate tcp and udp outgoing option
• allow only single selection for namespaces and pods
• refactor UI-creating code to a more Kotlin functional style

This PR allows overriding environment variables inside config files.

[feature.env.override]
FOO = "bar"

The above is equivalent to setting FOO manually before running mirrord and then specifying --exclude "FOO". The pain points of this approach become apparent when there are many environment variables and one has to keep something like a .env file in sync with the excludes passed to mirrord.

Bug Description

Downloaded the plugin via the marketplace. When I try to debug a python script via the IDE's debugger (the mirrord button was already pressed), the procces starts but after couple of seconds I get an error.

Steps to Reproduce

1. Hit the mirrord plugin button
2. Hit the debug button of Pycahrm
3. Wait for the debug session to start

Backtrace

No response

Relevant Logs

No response

Your operating system and version

Darwin roberts-MacBook-Pro.local 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64 x86_64

Local process

hitting the run / debug button in the ide

Local process version

No response

Additional Info

No response

Currently we don't load into sip protected binaries on macOS. We have a way to fix it, need to implement. If you face a use case where mirrord failed to load on macOS, please upvote/comment on this ticket with details so we can prioritize it accordingly.

Bug Description

Hi, I have golang application running in k8s cluster and it mounted volumes from secrets and configmap, but when I run the application, it fails because it cannot find the path to the directory.

Steps to Reproduce

1. Have running application on k8s with mounted volumes from configmap and secrets.

   containers:
    volumeMounts:
      - name: token
        mountPath: /etc/github
        readOnly: true
      - name: config
        mountPath: /etc/config
        readOnly: true
   volumes:
     - name: token
       secret:
         secretName: token
     - name: config
       configMap:
         name: config
   

2. Run mirrord exec ./main --target pod/<pod-name>

Backtrace

No response

Relevant Logs

No response

Your operating system and version

Linux Ubuntu 18.04.6 LTS

Local process

./main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=94de9fb4751f32af5cc6a567323cdf5c32b32478, with debug_info, not stripped

Local process version

No response

Additional Info

No response

Go 1.20 is coming soon (February) We need to check the following (for Linux and macOS):

1. We can still load using our current techniques in most common cases (i.e still dynamically linked)
2. Current hooks work, or what work needs to be done.
3. Open relevant issues with tasks need to be done to support it.

Bug Description

When trying to read a mounted directory it fails with fdopendir /tmp/foo: not a directory

Steps to Reproduce

1. Create a pod with a volume mount (e.g. /tmp/foo/bar)
2. Try to read the directory foo
3. See the error

Backtrace

No response

Relevant Logs

Rust Trace Logs Logs are stripped to the dir reading parts.

Your operating system and version

MacBook Pro M1 Max; macOS 13.1 (22C65)

Local process

/private/var/folders/lb/y5b0sgds5nv7z_vr4y9z68kc0000gn/T/GoLand/___go_build_playground_dir: Mach-O 64-bit executable arm64

Local process version

go version go1.19.4 darwin/arm64

Additional Info

I used the GoLand plugin.

package main

import (
	"fmt"
	"os"

	"C"
)

func main() {
	fmt.Println("==== DIR ====")
	stat, err := os.Stat("/tmp/foo")
	if err != nil {
		fmt.Print(err)
		return
	}

	fmt.Printf("Name: %s\n", stat.Name())
	fmt.Printf("Mode String: %s\n", stat.Mode().String())
	fmt.Printf("Mode IsDir: %v\n", stat.Mode().IsDir())
	fmt.Printf("Size: %v\n", stat.Size())

	dir, err := os.ReadDir("/tmp/foo")
	if err != nil {
		fmt.Printf("Reading dir error: %s\n", err)
	} else {
		fmt.Printf("DirEntries: %v\n", dir)
	}

	fmt.Println("==== FILE ====")
	stat, err = os.Stat("/tmp/foo/bar")
	if err != nil {
		fmt.Println(err)
		return
	}

	fmt.Printf("Name: %s\n", stat.Name())
	fmt.Printf("Mode String: %s\n", stat.Mode().String())
	fmt.Printf("Mode IsDir: %v\n", stat.Mode().IsDir())
	fmt.Printf("Size: %v\n", stat.Size())

	file, err := os.ReadFile("/tmp/foo/bar")
	if err != nil {
		fmt.Printf("Reading file error: %s\n", err)
	} else {
		fmt.Printf("File Content: %s\n", file)
	}
}

==== DIR ====
Name: foo
Mode String: drwxrwxrwx
Mode IsDir: true
Size: 4096
Reading dir error: fdopendir /tmp/foo: not a directory
==== FILE ====
Name: bar
Mode String: -rw-r--r--
Mode IsDir: false
Size: 6
File Content: foobar

I have the same namespace in multiple projects, mirrord only lets me see the namespace, so I have no idea what project I am picking from. Would be great to at least see the context, but optimally also change it directly in the UI.