I've been benchmarking axum's JSON handling today using rewrk with great success! But I'm having issues testing JSON input since you cannot set method, headers, and body, so figured I'd try and add it.

Example usage:

rewrk -d 10s -h http://localhost:3000 -m post -H "content-type: application/json" -b "{\"msg\":\"foobar\"}"

Just let me know if you want me to split things up into separate PRs.

• Reordered dependencies and removed their patch version.
• Much more readable and less complex client code.
• Better implemented timeouts.
• Better error messages.
• Use anyhow on client code.
• Error logging.
• Uses tokio-native-tls instead of tokio-rustls.

This can be a potentially big issue for people and likely miss-leading as the current system does not support messuring the headers aswell, not sure what a possible soloution is if hyper doesnt give a direct interface for this.

Hi, I am currently building a framework benchmark tool using rust, and just switched from using wrk to rewrk. Loving it so far as it doesn't has inconsistencies like wrk.

There are some issues with clap when it comes to parsing through another program, for example rust's own std::process::Command utility. I am also submitting a PR to get rid all these issues.

Problem

Here's a demo code that my benchmark program is using (using absolute path just to debug it further)

The exact same arguments seem to work okay with wrk, and any other programs. However, because of this clap issue, rewrk expects us to either specify arguments using t=2 threads=2 or t2.

Solution

Threads

// It's trying to parse a `usize` when the value is like ` 3` - with a space, which fails
// clap doesn't removes that space
let threads: usize = match args.value_of("threads").unwrap_or("1").parse()

connections

// Same issue as above
let conns: usize = match args.value_of("connections").unwrap_or("1").parse()

rounds

// Same issue as above
let rounds: usize = args  .value_of("rounds") .unwrap_or("1").parse()

host/url

// again the `uri` fails to parse since it has a space ` http://localhost:3000`
let user_input = UserInput::new(bench_type, uri_string).await?;

There's a quick fix - trimming values and that will do it for all the use cases. I'll be submitting a PR that fixes all these.

When 0 requests are processed successfully, various functions would panic when trying to calculate request averages. This avoids calling those functions by bailing early.

It seems that at higher concurrent connections (possibly when the comouter running the bench cannot produce that many) cause the tool to silence any results.

Reproduced at 1024 connections on my 2c 2t latop

Command will download, compile, and install using cargo install

Note: I copied the convention of using markdown bulleted lists inside markdown numbered lists

Fixes #25

Based on the README file, it seems like the only way to install this tool is by cloning the repository, building from source, and then figuring out a symbolic link to put the binary in $PATH. This is not unlike my experience with wrk itself, but it leaves an easier option to be desired.

Specifically, it would be nice if there was a one-line, cargo install based command to download, compile, and install rewrk to some bin directory in $PATH.

~~Publishing on https://crates.io is likely a prerequisite.~~

Errors should be ignored by default until error logging is implemented and there should be a command line argument to return early if an error happens.

When benchmarking ASP.NET API endpoints, rewrk is yielding some bizarre numbers (unexpected low) and almost all requests got error responses. On the other hand, testing Rust API endpoints (Actix, Axum) and Node.js endpoints (Nest, Fastify) seems fine. Any thoughts? Thanks~

I am running Fedora 36.

ASP.NET Core (.net 6 + Minimal API) result:

> rewrk -c 256 -t 12 -d 15s -h http://127.0.0.1:5000      
Beginning round 1...
Benchmarking 256 connections @ http://127.0.0.1:5000 for 15 second(s)
  Latencies:
    Avg      Stdev    Min      Max      
    4.96ms   1.82ms   0.12ms   29.61ms  
  Requests:
    Total: 772075  Req/Sec: 51517.15
  Transfer:
    Total: 88.36 MB Transfer Rate: 5.90 MB/Sec

772112 Errors: connection closed

> wrk -c 256 -t 12 -d 15s http://127.0.0.1:5000           
Running 15s test @ http://127.0.0.1:5000
  12 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   360.27us    0.90ms  43.45ms   96.32%
    Req/Sec    78.35k    11.33k  184.48k    76.26%
  14071720 requests in 15.10s, 1.60GB read
Requests/sec: 932057.72
Transfer/sec:    108.44MB

NestJS (w/ Fastify):

> rewrk -c 256 -t 12 -d 15s -h http://127.0.0.1:3000                 
Beginning round 1...
Benchmarking 256 connections @ http://127.0.0.1:3000 for 15 second(s)
  Latencies:
    Avg      Stdev    Min      Max      
    2.18ms   0.56ms   1.02ms   39.25ms  
  Requests:
    Total: 1756945 Req/Sec: 117196.97
  Transfer:
    Total: 296.57 MB Transfer Rate: 19.78 MB/Sec

> wrk -c 256 -t 12 -d 15s http://127.0.0.1:3000     
Running 15s test @ http://127.0.0.1:3000
  12 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.17ms  202.46us  15.30ms   96.03%
    Req/Sec     9.75k     1.13k   39.03k    99.11%
  1748814 requests in 15.09s, 295.20MB read
Requests/sec: 115855.62
Transfer/sec:     19.56MB

This merges the v2 changes to ReWrk.

As part of v2, the ReWrk project and part of the lnx-cli tool are being merged to make it easier to maintain as well as make the tooling more adaptable and generic giving it a wider range of benchmarking use cases.

This PR is very much still a WIP but plans to add the following:

• Benchmarking warm-up
• Body upload
• Sample data export: CSV, JSON, etc...
• Scriptable setup process ran before each round of benchmarking. Provided by Rhai.
• Benchmarking as a library: ReWrk will become an importable crate and a CLI tool allowing people to extend ReWrk and the benchmarking logic into other tooling.

Should we have app server running to expose the metrics which can be consumed by Prometheus. Also I am thinking of publishing a histogram as a report which can give further metrics details in more granular fashion

I think we will have issues while building musl due to openssl dependency on native-tls.

Please let me know and I can work on replacing it.

I do agree that we will lose support for older cipher suites.