Background

It would be nice if there is a table showing the result of the benchmark between each web framework.

Expectation

It is something like that:

| framework | Latencies.Avg | ... | Requests.Total | ... | Transfer.Total | ... | Maximum Memory Usage | | -- | -- | -- | -- | -- | -- | -- | -- | | actix-web | 0.50ms | ... | 30178057 | ... | 3.65 GB | .. | 13.7 MB | | ... |

Anything else

I am willing to add this feature.

osinfo:

ProductName: macOS ProductVersion: 11.5.2 BuildVersion: 20G95

kern.maxfilesperproc: 49152 kern.maxfiles: 98304

result:

➜ echo "Results for hyper:" && rewrk -t 12 -c 300 -d 10s -h http://localhost:3000/

Results for hyper: Beginning round 1... Benchmarking 300 connections @ http://localhost:3000/ for 10 second(s) failed to run benchmark round due to error: error combining results: connection closed

use wrk it's ok:

➜ wrk -t12 -c400 -d30s --latency http://localhost:3000/ wrk http://localhost:3000/ Running 30s test @ http://localhost:3000/ 12 threads and 400 connections Thread Stats Avg Stdev Max +/- Stdev Latency 808.44us 247.22us 3.95ms 72.97% Req/Sec 23.51k 9.67k 141.20k 79.11% Latency Distribution 50% 819.00us 75% 0.96ms 90% 1.10ms 99% 1.40ms 7020424 requests in 30.10s, 589.18MB read Socket errors: connect 157, read 89, write 0, timeout 0 Requests/sec: 233235.83 Transfer/sec: 19.57MB

It would help newbies if some explanation is added to benchmarks results like lower is better or higher is better. I think this explanation should be printed by rewrk by defualt.

I just wanted to be able to more easily compare the results between frameworks, so I made a pretty printed monospaced table with the benchmark results, so the results align vertically and you can see exactly the difference between frameworks.

Depends on pq and csvkit that were available in mac homebrew for me at least.

benchmark with my MBP(rewrk with -c 100) the results:

$ echo "Results for hyper:" && ./rewrk -t 12 -c 100 -d 10s -h http://localhost:3000/
Results for hyper:
Beginning round 1...
Benchmarking 100 connections @ http://localhost:3000/ for 10 second(s)
  Latencies:
    Avg      Stdev    Min      Max
    1.28ms   0.90ms   0.07ms   9.99ms
  Requests:
    Total: 774594  Req/Sec: 77440.91
  Transfer:
    Total: 65.01 MB Transfer Rate: 6.50 MB/Sec


$ echo "Results for axum:" && ./rewrk -t 12 -c 100 -d 10s -h http://localhost:3000/
Results for axum:
Beginning round 1...
Benchmarking 100 connections @ http://localhost:3000/ for 10 second(s)
  Latencies:
    Avg      Stdev    Min      Max
    1.27ms   0.80ms   0.08ms   8.55ms
  Requests:
    Total: 780470  Req/Sec: 78041.91
  Transfer:
    Total: 66.24 MB Transfer Rate: 6.62 MB/Sec


$ echo "Results for actix-web:" && ./rewrk -t 12 -c 100 -d 10s -h http://localhost:3000/
Results for actix-web:
Beginning round 1...
Benchmarking 100 connections @ http://localhost:3000/ for 10 second(s)
  Latencies:
    Avg      Stdev    Min      Max
    1.55ms   1.82ms   0.07ms   14.54ms
  Requests:
    Total: 642066  Req/Sec: 64173.35
  Transfer:
    Total: 79.60 MB Transfer Rate: 7.96 MB/Sec