Hi, it would be cool to relicense tiny-http under the dual license terms MIT OR Apache-2.0 (at the choice of the licensees) that are common in the Rust community.

To signify consent, please comment with the following:

I agree with relicensing my contributions to this work to the dual MIT/Apache-2.0 license.

Thanks!

Contributor checkoff:

• [x] @tomaka
• [x] @frewsxcv
• [x] @Eijebong
• [x] @moises-silva
• [x] @ashleysommer
• [ ] @spk
• [ ] @erickt
• [x] @ebfe
• [x] @PENGUINLIONG
• [x] @zaneli
• [x] @tomprogrammer
• [x] @romanz
• [x] @phildawes
• [x] @ordepdev
• [x] @nikolaplejic
• [x] @mb64
• [x] @djc
• [x] @algesten

Fixes #144.

HTTPS requests were previously running into a deadlock when trying to respond to requests in cases:

• To respond to the request, a lock needs to be obtained for the openssl TLS socket.
• However, meanwhile, this code is using that lock to try to read any subsequent requests.
• But no more requests are going to be sent by the browser until it gets a response to the first one!

This modifies it to wait for each request to finish responding until it goes on to reading the next request. It works, but it's not perfect; here are some things it doesn't do:

• It's potentially less parallelized with HTTPS than HTTP (but I'm not sure it's possible to do any better)
• Responding to requests out of order with HTTPS won't work (this could conceivably be fixed with some check-is-ready-to-read to supply more requests if they're available)

As the title says, how would one go about creating mock requests for tests. The fields of tiny_http::Request are private so creating the struct does not work for me.

Fixes #189.

• Create MockRequest so downstream users can build Request-like objects for testing without exposing Request too much
• Implement a builder pattern for MockRequest (i.e. the user can chain methods together to build the request they want)
• Add into_reader() and headers() to Response so the user can inspect test results

Without this fix:

HTTP/1.1 400 Bad Request
Server: tiny-http (Rust)
Date: Sat,  2 Jan 2021 20:23:02 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 163

{"description":"could not read the body from the request, or could not execute the CGI program","cause":{"description":"Error while decoding chunks","cause":null}}

With this fix:

HTTP/1.1 400 Bad Request
Server: tiny-http (Rust)
Date: Sat,  9 Jan 2021 00:23:11 GMT
Content-Length: 0

Fixes #173.

I'm finding it awkward to construct and test Method objects. I've found myself doing clumsy things like:

let get_ = tiny_http::Method::from_str("GET").unwrap();
let put_ = tiny_http::Method::from_str("PUT").unwrap();
if req.get_method().eq(&get_) || req.get_method().eq(&put_) { ... }

is there a simpler way to construct and/or test values of type Method?

It seems to me that modelling type Method as an enum might be simpler, easier and more efficient. Am I missing something?

Still loving tiny-http! _Greg

Please be warned I'm new to rust :)

This allows giving up on receiving a request after a specified timeout duration.

It's working for me.

I wouldn't strictly need this if I knew how to kill/stop the thread pool from another thread and force the recv() to unblock. In C you can kill the thread and if blocked on a syscall you get an error back from the syscall, you can check EINTR and decide whether to quit or issue the syscall again. I'm not sure how one would implement such a thing in rust, so the next best thing was to simply timeout on recv() and I can check out on my app once in a while for a shutdown flag.

I'd like to find a better way, but, this works and is not completely awful, is it? :)

If this is not acceptable I'd like to initiate a discussion on ideas on how to handle shutdown correctly. Currently if tiny-http is used in a loadable module (I am using it inside a project that can load/unload modules from an interactive command line a system administrator controls), when the server goes out of scope the socket lingers because the thread never terminates unless there's a new tcp connection received (accept is a blocking call).

An alternative to this would be to put the socket in non-blocking mode and then somehow poll() or select() on it with a timeout to check every sec for the shutdown flag (close trigger bool).

I may be wrong, but I don't see many downsides to this implementation. The Drop() call could hang for a bit on unusual situations like the server being inaccessible to itself (e.g 127 -> 127 firewall blocking or whatever), but the connection will timeout and move on. Most cases are better off with this as far as I can tell.

Thoughts?

This a minor update of #155 to use the latest version of rustls. I think this is useful because it allows tiny-http to serve TLS on platforms that don't have OpenSSL, like OpenBSD.

Hi Devs,

Thanks for all the work that's gone into this library over the last few years!

I want to check if there are plans to keep maintaining this crate. I've noticed that minor PRs have been left untouched for months, and while @frewsxcv has been triaging issues here and there, and @rawler has been reviewing PRs, there hasn't been much movement in the repo beyond security fixes.

I also saw that @tomaka said they were no longer maintaining the crate at all.

I've been extensively using Rouille and thus tiny-http in production for some time now, so if you do want to put the crate up for adoption, or if you just want someone else to deal with the issues and PRs, I'm ready to help.

I hit the following problem and saw it in the issues too (#146) : how do you gracefully shutdown the server ? There is effectively no way of breaking out of incoming_requests() other than receiving a client request.

Rust favors message passing. We can't kill threads or send them signals Unix style. We can only pass messages between threads via channels to tell them to shut down on their own (or with synchronization primitives such as condvar, or with shared memory under lock). After some research, it turns out the standard library does blocking IO (ie the accept method on TcpListener used by tiny-http). There really is no way to break out of it except with a client request (or on error). This is already dealt with by a previous commit by @moises-silva (cde7981) : on object destruction we briefly connect to the server to unblock the thread stuck on accept. (side note : for async IO look for mio)

But a for loop on incoming_requests it still blocking. It's not stuck on accept but inside the message queue waiting on condvar for new client requests. This commit adds a special request variant that says to unblock the loop and return no client request (since there is none we just want to shutdown). I slightly modified the message_queue API (ie the pop method now returns an Option), which is internal. There are no changes on the public side, ie on server, except the addition of the unblock method that effectively unblocks one thread stuck in incoming_requests (or recv).

Since there is no breaking change to the public API, all existing code work as previously. The addition of unblock() allows to implement graceful server shutdown by calling it from another thread. Hopefully the two tests added make its usage clear. Let me know what you think.

Hello, I think it is required to not handle http headers and header fields of any size the clients might provide. (DOS, OOM)
This PR limits the header body to 8192 bytes and the header lines to 2048 bytes. Popular servers (apache, nginx) have similar restrictions.

When a client is going over the limits the HTTP status 431 Request Header Fields Too Large is returned.

I've seen that it has been already the intention in #56 to implement this.
I hope this PR is welcome.

Hello, this PR aims to tackle one problem I had: streaming the output of a program while it is running over HTTP, similarly to the Github/GitLab CI "live" screens. In fact, the goal is to perform a request to the server with cURL inside a gitlab-ci job, and see the output unfolds as the program is executed:

              web browser
----------    ------------> -------------------
| CLIENT |                  |  GitLab CI job  |
----------   <------------  -------------------
               streams the     |              ∧
                  output       |              |
                               | executes     | prints the
                               |              | output
                               |              | "live"
                               ∨              |
                             -------------------
                             |       cURL      |
                             -------------------
                              |               ∧
                              |               | streaming
                              | HTTP          | (chunked
                              | query         | transfer)
                              |               | of stdin
                              |               | and stdout
                  spawns      ∨               |
-----------  <------------   --------------------
| program |                  | tiny-http server |
-----------   ------------>  --------------------
               sends stdout
               and stdin in
                 a pipe

Without this patch, the program executes in its entirety, and the output is sent all at once once the pipe where the program writes its output is closed.

To achieve that goal, I submit two "sub-features":

• The ability to disable buffering inside Response objects with the Response::with_buffered method. Enabling this will force the transfer encoding to be TransferEncoding::Chunked and will ask the chunks encoder to flush to its underlying writer on every write.
• To get "instantaneous" write, disabling buffering in the chunks encoder is not enough, as the underlying writer returned when calling Server::recv() (ClientConnection.sink) is in fact a BufWriter wrapping the "real" output. The buffered_writer option in ServerConfig, when set to false while instantiating a new server, omits the BufWriter to write to the TcpStream withotu any buffering. The cost of that abstraction is that ClientConnection.sink now boxes the writer to be able to choose between BufWriter<RefinedTcpStream> or RefinedTcpStream dynamically, which means there is now one additional pointer deference. I do however expect the performance impact to be small as this pointer is then stored in an Arc<Mutex<>>, and I think locking/unlocking the mutex should be more costly that deferencing the pointer.

I expect this to decrease performance when sending big files, which is why these two sub-features are disabled by default, and must explicitly be opted-in (by calling the with_buffered method for the first, and by instanciating the server with the buffered_writer setting for the second).

Also note that the current iteration of this work breaks the current ABI, as it updates ServerConfig to add the buffered_writer option, and ServerConfig was already exposed through the Server::new function.

See #172 and so on.

This would also allow more flexibility with TLS, including TLS authentication (client certs? TLS-SRP? TLS-OPAQUE?). But the main use-case is still unix sockets and other transports.

Server:: from_listener creates a util::TaskPool which has no limit on the number of threads. This makes tiny-http trivially vulnerable to thread exhaustion. Combined with #220, this behavior means that a single overload causes total and permanent server failure.

When the server fails to spawn a thread, a mutex gets poisoned and the server stops handling requests. It does not close its socket and Server::run does not panic.

The mutex that gets poisoned is util::TaskPool.sharing.todo. TaskPool::spawn holds it while calling TaskPool::add_thread which can panic.

Let's change it to handle thread spawn failure.

• Wait for an existing thread to become free and then hand the request to it.
• If there are no threads, sleep for 100ms and then try to spawn again.