Hey, cool tool. :)

I noticed that downloading is pretty slow as compared to wget. For instance, on downloading a random large file, using ht -d I get 7-13MB/s on repeated tests while using wget I get a good 60MB/s (my full connection speed). Any idea?

Sorry I can't be more specific but literally all I do is a multi-run test of a 2-5GB-sized file between wget and ht -d.

I was made aware [1] the name Yahc might be confused with a rust community member who goes by the name Yaahc. Therefore I am planning to rename this project using one of the following:

• ht: This would have been the best pick, but it is already being used by a similar project https://github.com/mark-burnett/ht
• rq: This name is being used by a popular python job queue library. Although it is possible to append -rs at the end to avoid a name clash.
• htyr: Not sure what it means but for some reason I like it.

[1] https://www.reddit.com/r/rust/comments/k0aika/yahc_yet_another_httpie_clone_in_rust/gdgza01?utm_source=share&utm_medium=web2x&context=3

Description

This PR attempts to implement same/similar functionalities of HTTPS of HTTPie:

• [x] --verify To skip the host’s SSL certificate verification.
• [x] --cert/cert-key To use a client side certificate for the SSL communication.

@otaconix have contributed a lot of work on this PR, thank you.

Hi,

I was using xh on a Raspberry Pi 4 which was recently switched to the Raspberry Pi OS arm64 release. After that, the arm-unknown-linux-gnueabihf release doesn't work anymore:

$ uname -a
Linux raspberrypi 5.10.63-v8+ #1488 SMP PREEMPT Thu Nov 18 16:16:16 GMT 2021 aarch64 GNU/Linux
$ cd xh-v0.14.1-arm-unknown-linux-gnueabihf/
$ ls -lh
total 6.1M
drwxr-xr-x 2 pi pi 4.0K Nov 26 20:35 completions
drwxr-xr-x 2 pi pi 4.0K Nov 26 20:35 doc
-rw-r--r-- 1 pi pi 1.1K Nov 26 20:35 LICENSE
-rw-r--r-- 1 pi pi 9.7K Nov 26 20:35 README.md
-rwxr-xr-x 1 pi pi 6.1M Nov 26 20:35 xh
$ ./xh
bash: ./xh: No such file or directory

Would it be possible to also release aarch64/arm64 binaries?

Regards, Tiago.

Previously the .netrc only handled authorization that contains both a username and password.

This is a divergence from the behavior of httpie.

This only activates when --auth-type bearer is specified without the --auth flag, so I believe it should be a compatible change. (I believe previously using it like this would have always reported an error?)

I have a plethora of great statically linked binary installs in Linux (Ubuntu). These include ripgrep, exa, bat, fd etc. None shines as well as chezmoi when it comes to the one-line installer.

I recommend to implement a one-line binary installer to follow that of chezmoi. See this documentation: https://github.com/twpayne/chezmoi/blob/master/docs/INSTALL.md#one-line-binary-install

In effect, what it does, is to download the latest statically linked binary for your operating system and architecture into ./bin This means I can put myself in whatever dir I want and just call the installer and it will be installed into the bin path of my choosing. Be that ~/bin or .local/bin or whatever I prefer.

Hey, I really like this tool and use it quite a bit for debugging. As such, I sometimes need to specifically request a certain version of HTTP from a server. Could you add a mode to force the HTTP version? Curl has --http1.0, --http1.1, --http2, --http3 and I think that makes sense.

Please consider adding those flags.

Hello.

Trying to move httpie example:

http GET <url> param1=value1 param2=value2 'Authorization: Bearer <supersecrettoken>' --verify no
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 122
Content-Type: application/json
Date: Wed, 14 Jul 2021 19:01:23 GMT
Server: nginx/1.12.2
Strict-Transport-Security: max-age=31536000;

{
    "code": 200,
    "result": {
        <some_data>
    },
    "status": "ok"
}

to xh:

xh --bearer <supersecrettoken> --verify no <url> param1=value1 param2=value2
Error: error sending request for url (<url>): error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution

Caused by:
    0: error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution
    1: dns error: failed to lookup address information: Temporary failure in name resolution
    2: failed to lookup address information: Temporary failure in name resolution

What am I doing wrong?

What I expect

Xh copies case of user-specified headers.

What actually happens

Xh treats all headers as case-insensitive and lowercases them.

Why is this a problem?

1. It's a difference from httpie.
2. It has a negative net effect when interacting with non-compliant servers.

To explain 2...

Some services treat headers as case-sensitive (e.g with X-* headers, especially for auth). The HTTP specification does specify that headers are case-insensitive, however, consider the following matrix. If headers are treated as case insensitive, this will break servers that (incorrectly) treat them as case-sensitive, while not having a negative effect on compliant servers. If headers are treated as case sensitive, this will work with the non-compliant servers, while not having a negative effect on compliant servers.

It would also be ok to auto-capitalize headers (e.g foO-baR-baZ could become Foo-Bar-Baz), since that's what standard industry practice anyway, but this is not what HTTPie does.

How to test compliance

xh --print=H localhost My-OdDlY-Capitalized-Header:Here compare to http --print=H localhost My-OdDlY-Capitalized-Header:Here

Dear @ducaale, could you be so kind to tell what is the xh equivalent of the following curl command to upload a file?

$ curl.exe -F [email protected] https://transfer.sh/

The reqwest crate does all the heavy lifting (including using the http_proxy and https_proxy environment variables when the --proxy option isn't present). As noted, it's possible to use different proxies for different protocols.

The "socks" configuration for the reqwest crate was enabled to allow users to use a socks proxy.

xh behaves differently from curl considering PROXY environment variables. On Windows xh reads proxy settings form the registry (that's unexpected from curl).

Need an option to force --NO-PROXY.

Changes

Add a --download-encoding flag to choose an Accept-Encoding header other than "identity" during a download request. During the download, the "decompress" function that reads the response and writes it to the output will passthrough data in whatever Content-Encoding was received if --download-encoding was specified.

Motivation

When downloading a large file, I want to be able to transfer it compressed and leave it compressed. For example, stream download a 500MiB CSV file in the gzip encoding, saving it to a data.csv.gz file.

jwtrueb@jmbp xh % cargo run --release -- get http://localhost:3000/download/csv/asdf -d --download-encoding gzip
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Type: application/csv
Date: Tue, 29 Nov 2022 23:17:12 GMT
Transfer-Encoding: chunked
Vary: accept-encoding

Downloading to "asdf"
Done. 147B in 0.00177s (80.92KiB/s)
jwtrueb@jmbp xh % gzcat asdf
ts,user_id,device_id,source,pressure,temperature,humidity
2022-11-28T20:13:12.704030+00:00,2,5,1,99999.0,25.3,16.4
2022-11-28T20:14:02.446438+00:00,2,5,1,99699.0,25.1,16.2

When a TLS certificate has the wrong hostnames it would be helpful to print which hostnames it does have.

HTTPie happens to do this, but we don't:

$ http https://wrong.host.badssl.com

http: error: SSLError: HTTPSConnectionPool(host='wrong.host.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'wrong.host.badssl.com' doesn't match either of '*.badssl.com', 'badssl.com'"))) while doing a GET request to URL: https://wrong.host.badssl.com/

$ xh https://wrong.host.badssl.com xh: error: error sending request for url (https://wrong.host.badssl.com/): error trying to connect: invalid peer certificate contents: invalid peer certificate: CertNotValidForName

Caused by: 0: error trying to connect: invalid peer certificate contents: invalid peer certificate: CertNotValidForName 1: invalid peer certificate contents: invalid peer certificate: CertNotValidForName

$ xh --native-tls https://wrong.host.badssl.com xh: error: error sending request for url (https://wrong.host.badssl.com/): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1914: (Hostname mismatch)

Caused by: 0: error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1914: (Hostname mismatch) 1: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1914: (Hostname mismatch) 2: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1914:

(The above --native-tls error message is with OpenSSL on Linux, maybe Windows or macOS already includes this information.)

When a certificate is expired it's also useful to see for how long it's been expired, but that's less important. (HTTPie doesn't do this.)

I don't know how to implement this, or if we can implement it at all. It might involve library changes, and it might involve bespoke error handling in main(). It might involve messy extraction of the certificate before we drop the reqwest client. It might be really hard.

cURL has the ability to require and verify OCSP stapled responses with the --cert-status option. Having this feature available in xh would make it great for testing both the HTTPS and TLS setup of a server with a single request.

A similar but far more cumbersome option is the --crlfile option; stapling is a much simpler approach, IMO.

Can you please consider adding support for OAuth 2.0, at least the client_credentials grant?

Example:

xh --auth-type oauth2 --auth <client_id>:<client_secret> --oauth-grant client_credentials --oauth-token-url https://example.org/oauth/token GET example.org/api/v1/hello