Simple low-level web server to serve file uploads with some shell scripting-friendly features

Overview

http_file_uploader

Simple low-level web server to serve file uploads with some shell scripting-friendly features. A bridge between Web's multipart/form-data file upload world and UNIX-ey files and command lines world. Somewhat close in spirit to CGI, but without the flexibility (hopefully with security instead).

HTTP listening features:

  • Listening for connections on TCP address
  • Listening for connections on UNIX path
  • Using stdin/stdout to serve one connection, inetd-style
  • Accepting connections from a pre-bound TCP or UNIX socket e.g. from SystemD socket activation service
  • Decoding multipart request bodies, selecting specific field from it (or optionally just streaming the whole request body). Or just simply starting programs on each request (even without body).

Sink features:

  • Dumping the file being received to stdout
  • Saving the file to filesystem (and maybe moving it elsewhere if upload is finished successfully)
  • Starting external program to handle the upload. The data would appear on stdin.

See "CLI usage message" section below for the exhausive list of options.

Most HTTP request parameters are ignored - it only concerns about the incoming data. Use nginx/caddy to filter and shape request and responses the way you like instead. You can select the field of the form to process, but other fields are ignored.

Installation

Build it from source code with cargo install --path ., install from crates.io using cargo install http_file_uploader or download a pre-built version from Github Releases.

Examples

$ httpfileuploader -l 127.0.0.1:8080 --stdout |
Incoming connection from 127.0.0.1:32876      | $ curl http://127.0.0.1:8080/ --form aaa=www
www                                           | Upload successful

$ httpfileuploade  -l 127.0.0.1:8080 -B -c --url -- echo
Incoming connection from 127.0.0.1:46750      | $ curl http://127.0.0.1:8080/asd?fgh
                                              | /asd?fgh

$ httpfileuploader  -l 127.0.0.1:8080 -o myupload.txt.tmp --rename-complete myupload.txt --once
Incoming connection from 127.0.0.1:48712      | $ curl http://127.0.0.1:8080 --form [email protected]
                                              | Upload successful
$ cmp myupload.txt Cargo.toml

$ http_file_uploader -l 127.0.0.1:1234 -r -P -I --cmdline -- stdbuf -oL /bin/rev&
$ nc 127.0.0.1 1234
POST / HTTP/1.0
Content-Length: 1000

HTTP/1.0 200 OK
date: Thu, 27 Oct 2022 13:26:14 GMT

Hello, world
dlrow ,olleH
12345
54321
^C

CLI usage message

http-file-uploader
  Special web server to allow shell scripts and other simple UNIX-ey programs to handle multipart/form-data HTTP  file uploads

ARGS:
    <argv>...
      Command line array for --cmdline option

OPTIONS:
    -l, --listen <addr>
      Bind and listen specified TCP socket

    -u, --unix <path>
      Optionally remove and bind this UNIX socket for listening incoming connections

    --inetd
      Read from HTTP request from stdin and write HTTP response to stdout

    --accept-tcp
      Expect file descriptor 0 (or specified) to be pre-bound listening TCP socket e.g. from systemd's socket activation
      You may want to specify `--fd 3` for systemd

    --accept-unix
      Expect file descriptor 0 (or specified) to be pre-bound listening UNIX socket e.g. from systemd's socket activation
      You may want to specify `--fd 3` for systemd

    --fd <fd>
      File descriptor to use for --inetd or --accept-... modes instead of 0.

    --once
      Serve only one successful upload, then exit.
      Failed child process executions are not considered as unsuccessful uploads for `--once` purposes, only invalid HTTP requests.
      E.g. trying to write to /dev/full does exit with --once, but failure to open --output file does not.

    -O, --stdout
      Dump contents of the file being uploaded to stdout.

    -o, --output <path>
      Save the file to specified location and overwrite it for each new upload

    -p, --program <path>
      Execute specified program each time the upload starts, without CLI parameters by default and file content as in stdin
      On UNIX, SIGINT is sent to the process if upload is terminated prematurely

    -c, --cmdline
      Execute command line (after --) each time the upload starts. URL is not propagated. Uploaded file content is in stdin.
      On UNIX, SIGINT is sent to the process if upload is terminated prematurely

    -n, --name <field_name>
      Restrict multipart field to specified name instead of taking first encountred field.

    -r, --require-upload
      Require a file to be uploaded, otherwise failing the request.

    -L, --parallelism
      Allow multiple uploads simultaneously without any limit

    -j, --parallelism-limit <limit>
      Limit number of upload-serving processes running in parallel.
      You may want to also use -Q option

    -Q, --queue <len>
      Number of queued waiting requests before starting failing them with 429. Default is no queue.
      Note that single TCP connection can issue multiple requests in parallel, filling up the queue.

    -B, --buffer-child-stdout
      Buffer child process output to return it to HTTP client as text/plain

    -I, --pipe
      Don't bother calculating Content-Length, instead pipe child process's stdout to HTTP reply chunk by chunk

    --remove-incomplete
      Remove --output file if the upload was interrupted

    --rename-complete <path>
      Move --output's file to new path after the upload is fully completed

    -U, --url
      Append request URL as additional command line parameter

    --url-base64
      Append request URL as additional command line parameter, base64-encoded

    -q, --quiet
      Do not announce new connections

    -P, --allow-nonmultipart
      Allow plain, non-multipart/form-data requests (and stream body chunks instead of form field's chunks)

    --no-multipart
      Don't try to decode multipart/form-data content, just stream request body as is always.

    -M, --method
      Append HTTP request method to the command line parameters (before --url if specified)

    -h, --help
      Prints help information.
You might also like...
Some Steam patches, fixing broken Steam features on Asus Rog Ally
Some Steam patches, fixing broken Steam features on Asus Rog Ally

⚙️ Steam Patch Steam Patch is a tool designed to enhance your Steam experience by applying patches to the Steam client. 📥 Installation To install Ste

Threadless Module Stomping In Rust with some features
Threadless Module Stomping In Rust with some features

NovaLdr is a Threadless Module Stomping written in Rust, designed as a learning project while exploring the world of malware development. It uses advanced techniques like indirect syscalls and string encryption to achieve its functionalities. This project is not intended to be a complete or polished product but rather a journey into the technical aspects of malware, showcasing various techniques and features.

bevy_scriptum is a a plugin for Bevy that allows you to write some of your game logic in a scripting language

bevy_scriptum is a a plugin for Bevy that allows you to write some of your game logic in a scripting language. Currently, only Rhai is supported, but more languages may be added in the future.

Low-level Rust library for implementing terminal command line interface, like in embedded systems.

Terminal CLI Need to build an interactive command prompt, with commands, properties and with full autocomplete? This is for you. Example, output only

A low-level ncurses wrapper for Rust

ncurses-rs This is a very thin wrapper around the ncurses TUI lib. NOTE: The ncurses lib is terribly unsafe and ncurses-rs is only the lightest wrappe

Verified Rust for low-level systems code

See Goals for a brief description of the project's goals. Building the project The main project source is in source. tools contains scripts for settin

Unopinionated low level API bindings focused on soundness, safety, and stronger types over raw FFI.

🔥 firehazard 🔥 Create a fire hazard by locking down your (Microsoft) Windows so nobody can escape (your security sandbox.) Unopinionated low level A

Horus is an open source tool for running forensic and administrative tasks at the kernel level using eBPF, a low-overhead in-kernel virtual machine, and the Rust programming language.
Horus is an open source tool for running forensic and administrative tasks at the kernel level using eBPF, a low-overhead in-kernel virtual machine, and the Rust programming language.

Horus Horus is an open-source tool for running forensic and administrative tasks at the kernel level using eBPF, a low-overhead in-kernel virtual mach

Low level access to processors using the AArch64 execution state.

aarch64-cpu Low level access to processors using the AArch64 execution state. Usage Please note that for using this crate's register definitions (as p

Releases(v0.2.0)
Owner
Vitaly Shukela
Vitaly Shukela
A low-level MVCC file format for storing blobs.

Sediment This repository isn't ready for public consumption. It just reached a stage where I wanted to start sharing ideas with others as well as usin

Khonsu Labs 24 Jan 8, 2023
High-performance, low-level framework for composing flexible web integrations

High-performance, low-level framework for composing flexible web integrations. Used mainly as a dependency of `barter-rs` project

Barter 8 Dec 28, 2022
Estratto is a powerful and user-friendly Rust library designed for extracting rich audio features from digital audio signals.

estratto 〜 An Audio Feature Extraction Library estratto is a powerful and user-friendly Rust library designed for extracting rich audio features from

Amber J Blue 5 Aug 25, 2023
Shell scripting that will knock your socks off

atom Shell scripting that will knock your socks off. NOTE: Click the image above for a video demonstration.

adam mcdaniel 256 Dec 14, 2022
A command line interface meant to bridge the gap between Rust and shell scripting

clawbang A command line interface meant to bridge the gap between Rust and shell scripting. Intended for use with HEREDOCs and shebangs: $ clawbang <<

Chris Dickinson 52 Mar 25, 2022
Mod for pxtone Collage that adds some shiny features

ptcMod Mod for pxtone Collage that adds some shiny features READ THIS FIRST The program is extremely invasive to ptCollage. This project makes extensi

David M. 6 Sep 17, 2022
A run-codes cli front end with some extra features

run-cli Run-cli A run-codes cli front end with some extra features Report Bug · Request Feature Table of Contents About The Project Built With Getting

Matheus Vieira 13 Nov 16, 2022
Lightweight alternative Discord client with a smaller footprint and some fancy extensible features.

Dorion Dorion is an alternative Discord client aimed and lower-spec or storage-sensitive PCs that supports themes, plugins, and more! Table of Content

SpikeHD 20 Jan 2, 2023
Rust port of https://github.com/hunar4321/life_code with some fun features.

Smarticles A Rust port of Brainxyz's Artificial Life simulator with some fun features. A simple program to simulate primitive Artificial Life using si

Chevy Ray Johnston 15 Dec 24, 2022
Some Steam patches, fixing broken Steam features on Asus Rog Ally

Ally Steam Patches Ally Steam Patches is a tool designed to enhance your Steam experience by applying patches to the Steam client. Installation To ins

MIkhail Kozlov 2 Jul 10, 2023