A minimal ngrok liked reverse proxy implemented in Rust.

Kai

Last update: Jun 21, 2022

Related tags

Network programming rok

Overview

rok

A minimal ngrok implementation in Rust, for educational purpose. This work is largely based on rathole, especially the very first commit. Other honorable references are stated below here.

This is by no mean an idiomatic or correct Rust implementation. I am learning Rust and fairly new to writing networking code with tokio.

Implementation and limitations

Last Updated: 21st June 2022

Currently, it's a half baked implementation. It basically:

Have a server to listen to incoming Init packet from a client and reply it with a domain the client can use.
Upon running the client, it will send an Init packet, to get the domain name.
Then, the client send a DataInit packet to the server, setting up another TCP tunnel listening to request from the server and proxy it to the local server. It's currently hardcoded to forward to localhost:4000
Once the server receive an DataInit packet, it will spin up a TcpListener on the assigned port for the particular domain name, which is hardcoded as well. This will then allow the public internet to send request to the domain, which will be proxy to the client local server through the TCP tunnel we setup.
Once the client tunnal receive and respond with a request, the whole TCP tunnel will be shutdown, and it will resend the DataInit packet again to setup another TCP tunnel.
Same applied to the TcpListener that is exposed to the internet. Hence, it's not an efficient implementation as it will spin up and shutdown 2 TCP connection on every request.

On top of that:

It only support 3 domain name, that means it currently support at most 3 client at a time.
It doesn't work with websocket.
~~The domain name is not recycle. This mean that if a client lost it's connection, the domain assigned do not goes back to the domain name pool.~~
~~It doesn't support logging HTTP request and response time.~~

Flow

Last Updated: 24th May 2022

The following flow will be the initial first attempt to make a minimal reverse proxy. It's subject to changes as we involved our implementation.

First, a client send an Init packet to setup the control channel with the server. Then, the server will assigned a domain and port for the control channel and reply the domain name to the client.

        Init
Client ------> Server
       <------
       Domain

Once the client receive the domain name, it send an Ack packet and the server will proceed to update the state to keep track of the control channel spawned.

         Ack
Client ------> Server (proceed to store the control channel information in the state)

Client will then proceed to setup a data channel with the server and a TcpStream to connect to the local service, waiting for the incoming request from the server, so it can proxy the request to the local service and respond back to the server through the data channel.

                TCP connection            DataInit
Local Service <----------------> Client -----------> Server

When server receive the DataInit packet, it start listening to the particular port assigned to the domain name and start to forward any request to the data channel to the client, which the client will proxy it to the local service and reply the responses to us through the same data channel, which we then return to the incoming request.

            TCP listener            Proxy request
Internet <---------------> Server <---------------> Server

Once a request is served, the data channel, TCP listner on the server and TCP connection to the local service from the client will all be shutdown. Step 3, 4 and 5 will be repeated.

References

ngrok: To understand how to setup tunneling on a high level.
rathole: The first commit of a lightweight Rust ngrok liked implementation. Use TCP all the way.
tunnelto: Another Rust ngrok liked implementation. It use websockets to setup the control channel.
tokio.rs: Tokio tutorials.

Proxy sentry request to a sentry server using a tunnel/proxy endpoint

Sentry Tunnel This is a proxy that forwards tunneled sentry requests to the real sentry server. The implementation is based on the explanation provide

14 Dec 20, 2022

UDP proxy with Proxy Protocol and mmproxy support

udppp UDP proxy with Proxy Protocol and mmproxy support. Features Async Support Proxy Protocol V2 SOCKET preserve client IP addresses in L7 proxies(mm

10 Dec 18, 2022

Lightweight proxy that allows redirect HTTP(S) traffic through a proxy.

Proxyswarm Proxyswarm is a lightweight proxy that allows redirect HTTP(S) traffic through a proxy. WARNING: This app isn't recomended for download lar

4 Apr 16, 2022

Web3-proxy: a fast caching and load balancing proxy for web3 (Ethereum or similar) JsonRPC servers.

web3-proxy Web3-proxy is a fast caching and load balancing proxy for web3 (Ethereum or similar) JsonRPC servers. Signed transactions (eth_sendRawTrans

55 Jan 8, 2023

RedLizard - A Rust TCP Reverse Shell with SSL

RedLizard - A Rust TCP Reverse Shell with SSL RedLizard Rust TCP Reverse Shell Server/Client This is a reverse shell in Rust called RedLizard, basical

105 Dec 24, 2022

Interactive bind/reverse PTY shell with Windows&Linux support implementation by Rust.

Cliws Lightweight interactive bind/reverse PTY shell with Windows&Linux support implementation by Rust. Features WebSocket Full pty support: VIM, SSH,

215 Dec 3, 2021

Utility for working with reverse DNS

RDNS RDNS is a small Rust CLI utility for performing single and bulk reverse DNS (PTR) lookups. Usage RDNS 0.1.0 Joe Banks [email protected] Utilities for

2 Sep 22, 2021

A fast, offline reverse geocoder in Python

Reverse Geocoder A Python library for offline reverse geocoding. It improves on an existing library called reverse_geocode developed by Richard Penman

1.8k Dec 26, 2022

server security proxy write by Rust

server-security-proxy server security proxy write by Rust how to use config toml file

3 May 24, 2021

Comments

Connection Logger
This PR is the result of pairing with @erwanor during our time in the Recurse Center. The PR is mainly about:

Logging each HTTP request/response we receive/respond into the client terminal.

On top of that, we also include the duration taken for the local server response to the client.

Lastly, it's beautify by adding alignments and colors to the log out puts.

Implementation Details

Support logging

Initially, we are forwarding the bytes from one TcpStream to another TcpStream using copy_bidirectional. However, since it encapsulates the bytes copied, there's no way we could peek into the bytes we receive. Initially, we thought of two possible ways to implement the logging:

Write our own copy bidirectional manually. Basically, read on TcpStream and the write it to another manually.

Extend copy_bidirectional to accept an event channel. This mean that we can just pass in a Sender to the copy_bidirectional function, which will call send on every read it received. Then, our code can listen to the Receiver and process the bytes as needed.

This approach also means that we need to copy most of the code from the existing copy_bidirectional implementation.

On the next day of Pairing, Erwan thought of another approach: we can implement our own AsyncRead and AsyncWrite trait to wrap the TcpStream and pass it to copy_bidrectional. This way, we could also peek into the bytes when poll_read is called by copy_bidirectional. We decided to approach with this approach and it works really well. This result in the Logger struct that implement AsyncRead + AsyncWrite, where the inner is a TcpStream in our case.

Tracking state

The next feature we want to support is to track the duration takes. This mean that we need to have an Instant::now() created when a request is received and called elapsed() when a response is received. We tried a couple of different ways and ended up adding a LoggerState struct that will hold the shared state by both Logger. The state is then wrapped in Arc<Mutex> to allow it to be hold by multiple struct.

It works as we tested, but we also learn that in browser, a TCP connection might be reused across refresh and and tabs. So, we can't just assume that our TcpStream will be closed once a request is served. Hence, we need to reset the timestamp state everytime a response is logged. This is achieved by calling timestamp.take() to take out the Instant in our Option and replace it with None.
opened by kw7oe 0