Black-box integration tests for your REST API using the Rust and its test framework

Related tags

Command-line restest
Overview

restest

Black-box integration test for REST APIs in Rust.

This crate provides the [assert_api] macro that allows to declaratively test, given a certain request, that the response emitted by the server is correct.

Example

#![feature(assert_matches)]

use serde::{Deserialize, Serialize};

restest::port! { 8080 }

restest::assert_api! {
    POST "/user",
    PostUser {
        year_of_birth: 2000,
    } => User {
        year_of_birth: 2000,
        ..
    }
}

#[derive(Debug, Serialize)]
struct PostUser {
    year_of_birth: usize,
}

#[derive(Debug, Deserialize)]
struct User {
    year_of_birth: usize,
    id: Uuid
}

Writing tests

The [port] macro sets the port at which the request must be run.

The tests are written as normal Rust tests (ie: functions annotated with #[test]). As we're using asynchronous code, we must write async tests, perhaps using #[tokio::test].

More specifically, the [assert_api] macro can be used in order to query the server API and analyze its response.

Running tests

The server must be running in the background when cargo test is run.

Required toolchain

The nightly feature allows the [assert_api] macro to expand to nightly-specific code. This offers the following features:

  • better panic message when the request body does not match the expected pattern (requires assert_matches),
  • ability to reuse variables matched in the response body (requires let_else) (still WIP).

These two features must be added at the crate root using the following two lines:

#![feature(assert_matches)]
#![feature(let_else)]

Code of Conduct

We have a Code of Conduct so as to create a more enjoyable community and work environment. Please see the CODE_OF_CONDUCT file for more details.

License

Licensed under either of

at your option.

Dual MIT/Apache2 is strictly more permissive

Comments
  • Handle strings and objects

    Handle strings and objects

    There was no way to match against strings in the assert_body_matches macro. Similarly, there was no way to match against bare JSON objects. This PR implements both.

    The implementation of object matching is quite simple for now. For instance, it does not allow for { a, b, c } as a destructuring pattern (the same destructuring can be achieved with { a: a as T, b: b as U, c: c as V }.

    It turns out that using Deserialize to convert a json value to a given type was a good idea, since it accidentally allowed us to perform deserialization on binding. In other word, the following code was compiling fine:

    assert_body_matches! {
        serde_json::json! {
            {
                "name": "Grace Hopper",
                "id": Uuid::new_v4(),
            }
        },
        user as User
    };
    
    #[derive(Deserialize)]
    struct User {
        name: String,
        id: Uuid,
    }
    

    As this is a nice-to-have behaviour, this PR added a test for it.

    Also: we now use Rust 2021 🎉🎉.

    opened by scrabsha 1
  • Format multiple-segment path as part of macro invocation

    Format multiple-segment path as part of macro invocation

    Previous implementation of restest relied on the users to create the whole full URL themselves, perhaps by concatenating multiple segments in a call to format!.

    This PR allows the users to gives us the URL segments, and assert_api will create the full URL on its own. The segments must be placed in brackets and separated by commas. Single-segment-URL can still be written without brackets. Each segment can have a different type, but it must implement ToString.

    opened by scrabsha 1
  • feat(request): add context description

    feat(request): add context description

    (This builds on the PR #22 to add more informations on the expect status)

    Here we had the context of the request to the expect_status panic message, allowing us to know which request failed the test.

    Default value will be "Method + url", but it can be overriden with a with_context("Some string") to specify our own context.

    image

    fixes #21

    opened by cbonaudo 0
  • feat(expect_status): add more information to the assert_eq for debug

    feat(expect_status): add more information to the assert_eq for debug

    • This adds a lot of debug informations to understand what went wrong during the request, and to help contextualising the request performed.

    Maybe not the cleanest since we .expect(). But since we are tryng to .json after it would still be panicking anyway currently, so it should not be dirtier than the current solution (about the opening of the response with either .json() or .text() ). Even if this could be improved later on.

    All input is welcome about this PR!

    fixes #20

    opened by cbonaudo 0
  • Being able to contextualise a Restest request

    Being able to contextualise a Restest request

    In a file with a big amount of Restest request, it is pretty difficult to figure out which request triggered the failling result.

    • It could be interesting to have a "context" (a bit like anyhow does it), that we can provide to the request so it can display which one failed.

    (For exemple Jest does something similar with the possible annotation of snapshot so we can name them something descriptive)

    • Another possibility could be to maybe display the Url/Method used, it's still not too precise, but a lot more than the current state of error message we get.
    opened by cbonaudo 0
  • More informations on Response

    More informations on Response "expect_status"

    When we expect_status a Response, if it fails we have very little data on what happened to help the devs figure out what went wrong (or what went well). Having a display of what was sent in the request could be very helpful I guess.

    A bit like what was drafted here image maybe not the exact same implementation, but this brings the body received to our attention.

    opened by cbonaudo 0
  • Make API more streamlined for GET requests (+ requests without body)

    Make API more streamlined for GET requests (+ requests without body)

    Currently, requests without body require calling .with_body on the builder:

    let request = Request::get(path!["users", id]).with_body(());
    

    This could be acceptable for POST requests that are usually expected to have a body, for GET requests this is completely different:

    A payload within a GET request message has no defined semantics; sending a payload body on a GET request might cause some existing implementations to reject the request.

    (RFC 7231, HTTP/1.1 Semantics and Content)

    Since GET requests should not contain a body, the API call above should at the very least be simplified.

    let request = Request::get(path!["users", id]);
    

    for example.

    enhancement 
    opened by zdimension 0
  • Update readme (thanks, cargo-abcr!)

    Update readme (thanks, cargo-abcr!)

    Steps to reproduce:

    • kindly ask to @scrabsha the source code of (yet unreleased) cargo-abcr,
    • remove every broken link.

    There rendered version is available here

    opened by scrabsha 0
  • Make assert_body_matches work on Vec

    Make assert_body_matches work on Vec

    (and other fun things)

    This PR is a near rewrite of the initial assert_body_matches proc macro. The following things have been modified:

    • the macro invocation is now split in three clear steps,
    • slice patterns now allow to match over Vec,
    • some documentation for each step.

    Pattern matching over Vec is possible by changing whole initial match body { pattern => output } expression into a nested match expression. Each match destructures a new Vec, the last one additionally stores the final guards and returns all the values that are brought in scope by the macro call.

    Example:

    The following call of assert_body_matches:

    restest::assert_body_matches!(
        value,
        Foo {
            bar: [
                [a, 42],
                [101, b, c],
            ]
        },
    );
    

    Expands in:

    let (a, b, c,) = match value {
        Foo {
            bar: __restest__array_0,
        } => match __restest__array_0 {
            [__restest__array_0, __restest__array_1] => match __restest__array_0 {
                [a, 42] => match __restest__array_1 {
                    [101, b, c] => (a, b, c,),
                    _ => panic!("Matching failed"),
                },
                _ => panic!("Matching failed"),
            },
            _ => panic!("Matching failed"),
        },
        _ => panic!("Matching failed"),
    };
    
    opened by scrabsha 0
  • Completely and definitively remove the assert_api macro

    Completely and definitively remove the assert_api macro

    This functionnality has been replaced by various OOP-ish patterns combined with the assert_body_matches proc macro.

    Some documentation is not accurate. It has been commented out and will be replaced by fresh examples soon:tm:.

    opened by scrabsha 0
  • Refactor the request creation process

    Refactor the request creation process

    The previous lifecycle was simple and declarative: using restest::assert_api to perform any request and check it on the fly. This was good, but did not allow for more complex stuff such as header declaration, host selection or expected status.

    This PR changes this quite a bit. From now on, the following will be done in order to perform an API call and check:

    • a (global) CONTEXT is created. It stores informations about the host. It can be reused for multiple requests and multiple tests.
    • a Request is created. It stores everything about a specific request: what method to use, which URL and what body,
    • the context runs the request (Context::run),
    • we ensure that the server response has a specific HTTP code and deserialize the body,
    • we ensure that the body has specific properties (with assert_body_matches).

    For instance:

    const CONTEXT: restest::Context = Context::new().with_port(8080);
    
    let request = Request::get(path!["users", id]).with_body(());
    
    let response = CONTEXT
        .run(request)
        .await
        .expect_status(StatusCode::OK)
        .await;
    
    // We can ensure that the returned year of birth is now correct.
    assert_body_matches! {
        response,
        User { year_of_birth: 2000, .. },
    };
    

    The user_server_test.rs has been updated to reflect this style. Crate documentation will be updated in a next PR.

    opened by scrabsha 0
  • feat(ensure_status): display request body in most error cases

    feat(ensure_status): display request body in most error cases

    Tried to use a simpler 4 way path (either body is incorrect, or can't be deserialized, or status is unexpected or Happy path.

    This allows to get the body first hand and to have it consistently to describe what we are receiving to the debugging user that is using restest for his tests.

    opened by cbonaudo 0
  • Proposal: Add additional methods to RequestResult

    Proposal: Add additional methods to RequestResult

    This PR adds the following methods to RequestResult:

    • deserialize_body: Deserializes the body without performing checks on status code or content.
    • ensure_status_ignore_body: Checks the status code while ignoring body.
    • expect_status_ignore_body: Checks the status code while ignoring body, panic on failure.
    • ensure_empty_body: Checks that the body is empty.
    • ensure_status_empty_body: Checks the status code and the emptiness of the body.
    • expect_status_empty_body: Checks the status code and the emptiness of the body, panic on failure.

    The last two items, specifically, are an attempt to fix #25 while both

    • keeping the public-facing API unchanged for the sole existing use case (check status code & body)
    • providing both try (ensure_* -> Result) and panic (expect_* -> Either<(), !>) versions of the corresponding methods

    Any criticism on the API structure in itself would be much appreciated.

    • is preservation of the public API surface important enough at this point in the project to justify opting for ensure_thing1_thing2 rather than a builder pattern (.ensure_thing1().ensure_thing2())? The latter, if chosen, would require breaking the current API since expect_status should then only check for status, not body as currently is the case.
    enhancement 
    opened by zdimension 0
  • Having equivalent ensure! methods

    Having equivalent ensure! methods

    Using assert_body_matches, and expect_status are used to panic during the tests.

    I think that having an equivalent with ensure!-like functionnality could be a nice possibility if that's what we want to use in our tests.

    • ensure_body_matches, that returns a Result if body do not match
    • ensure_status, that returns a Result if the status do not match

    !!! - It could make sense to split the expect_status and the json() of the response before implementing this so "ensure_status" doesn't need to json() the response too.

    opened by cbonaudo 0
  • Handle empty response bodies

    Handle empty response bodies

    Currently, the expect_status method deserializes the body to some type, but this fails for an empty body (where we would expect to be able to use expect_status::<()>).

    bug 
    opened by zdimension 0
  • A better panic message for assert_body_matches

    A better panic message for assert_body_matches

    The following code snippet shows is an obvious incorrect call to assert_body_matches:

    use restest::assert_body_matches;
    
    struct User {
        first_name: String,
        last_name: String,
    }
    
    #[test]
    fn basic_message() {
        assert_body_matches! {
            User {
                first_name: "Grace".to_string(),
                last_name: "Hopper".to_string(),
            },
            User {
                first_name: _,
                last_name: "Lovelace",
            },
        };
    }
    

    As of 24fc843, it generates the following panic message:

    thread 'basic_message' panicked at 'Matching failed', tests/error_message.rs:10:5
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    

    This is... less than ideal.

    What could be done

    What i'd love to see is a way to see is something like this:

    thread 'basic_message' panicked at 'Failed to match the request output with the provided pattern:
                   Value returned:           |           Pattern expected:
                                             |
       |   {                                 |  {
       |       first_name: "Grace",          |      first_name: _,
    !! |       last_name: "Hopper",          |      last_name: "Lovelace",
       |   }                                 |  }
    ', tests/error_message.rs:5:5
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    

    Note that this output JSON-ifies the pattern. We could Rust-ify the value instead:

                   Value returned:           |           Pattern expected:
                                             |
       |   User {                            |  User {
       |       first_name: "Grace",          |      first_name: _,
    !! |       last_name: "Hopper",          |      last_name: "Lovelace",
       |   }                                 |  }
    

    Implementation details

    We can gather a set of "constraints" of the value returned by the server by using a custom Serializer, which could, from the previous example, return us the following constraints:

    .first_name == "Grace"
    .last_name == "Hopper"
    

    We could gather the constraints from the pattern by defining a custom [Visitor], which could extract similar constraints from the pattern:

    .first_name: _
    .last_name: "Lovelace"
    

    We could then visit each constraint from the pattern, get the corresponding constraint in the value-side, compare them (write the !! if required), then pretty-print both on each side.

    opened by scrabsha 0
Owner
IOmentum
IOmentum
ABQ is a universal test runner that runs test suites in parallel. It’s the best tool for splitting test suites into parallel jobs locally or on CI

?? abq.build   ?? @rwx_research   ?? discord   ?? documentation ABQ is a universal test runner that runs test suites in parallel. It’s the best tool f

RWX 13 Apr 7, 2023
Coppers is a custom test harnass for Rust that measures the energy usage of your test suite.

Coppers Coppers is a test harness for Rust that can measure the evolution of power consumptions of a Rust program between different versions with the

Thijs Raymakers 175 Dec 4, 2022
By mirroring traffic to and from your machine, mirrord surrounds your local service with a mirror image of its cloud environment.

mirrord lets you easily mirror traffic from your Kubernetes cluster to your development environment. It comes as both Visual Studio Code extension and

MetalBear 2.1k Jan 3, 2023
Assure that your tests are there, and well written.

cargo-is-tested [ ???? ] El libro contiene instrucciones e información detallada en Español. cargo-is-tested is a way to check which of your items are

Alex ✨ Cosmic Princess ✨ 15 Jan 8, 2023
SKYULL is a command-line interface (CLI) in development that creates REST API project structure templates with the aim of making it easy and fast to start a new project.

SKYULL is a command-line interface (CLI) in development that creates REST API project structure templates with the aim of making it easy and fast to start a new project. With just a few primary configurations, such as project name, you can get started quickly.

Gabriel Michaliszen 4 May 9, 2023
A small CLI tool to query ArcGIS REST API services, implemented in Rust. The server response is returned as pretty JSON.

A small CLI tool to query ArcGIS REST API services, implemented in Rust. The server response is returned as pretty JSON.

Andrew Vitale 2 Apr 25, 2022
Small microservice to render Lottie animation files via an http REST API.

Lottie Renderer Service Small microservice to render Lottie animation files via an http REST API. Run via docker docker run -p 8080:8080 ghcr.io/mikbo

Mikbot 3 Oct 22, 2022
Snapshot testing for a herd of CLI tests

trycmd Snapshot testing for a herd of CLI tests trycmd aims to simplify the process for running a large collection of end-to-end CLI test cases, takin

null 57 Jan 3, 2023
Cucumber testing framework for Rust. Fully native, no external test runners or dependencies.

Cucumber testing framework for Rust An implementation of the Cucumber testing framework for Rust. Fully native, no external test runners or dependenci

Cucumber Rust 393 Dec 31, 2022
create and test the style and formatting of text in your terminal applications

description: create and test the style and formatting of text in your terminal applications docs: https://docs.rs/termstyle termstyle is a library tha

Rett Berg 18 Jul 3, 2021
`boxy` - declarative box-drawing characters

boxy - declarative box-drawing characters Box-drawing characters are used extensively in text user interfaces software for drawing lines, boxes, and o

Miguel Young 7 Dec 30, 2022
Benson Box built on Substrate for a world UNcorporated.

Benson Box Benson Box built on Substrate. For getting started and technical guides, please refer to the Benson Wiki. Contributing All PRs are welcome!

Arthur·Thomas 13 Mar 13, 2022
Boxxy puts bad Linux applications in a box with only their files.

boxxy is a tool for boxing up misbehaving Linux applications and forcing them to put their files and directories in the right place, without symlinks!

amy null 910 Feb 22, 2023
A cat(1) clone with syntax highlighting and Git integration.

A cat(1) clone with syntax highlighting and Git integration. Key Features • How To Use • Installation • Customization • Project goals, alternatives [中

David Peter 38.9k Jan 8, 2023
An awesome language and its compiler.

Calocom Project Build Build with cargo-make Just run cargo make in the root directory of the project. Or, if you want to build it in release mode, run

Aoyang Yu 31 Sep 16, 2022
🔔 CLI utility to send notifications to Slack via integration webhooks

Slack notifier Just a simple CLI tool to send notifications to Slack. Please note that this project is just a playground to start learning Rust, it is

Green.Mod 2 May 21, 2022
A high-performance WebSocket integration library for streaming public market data. Used as a key dependency of the `barter-rs` project.

Barter-Data A high-performance WebSocket integration library for streaming public market data from leading cryptocurrency exchanges - batteries includ

Barter 23 Feb 3, 2023
tkn-watch - watch a PipelineRuns on its way to success or failures

tkn-watch - watch a PipelineRuns on its way to success or failures tkn-watch is a simple extension to the tkn command line tool that watches a Pipelin

Chmouel Boudjnah 11 Dec 15, 2022