Rust testing library

Overview

K9 - Rust Testing Library

Crates.io Docs.rs Rust CI

k9_header

Snapshot testing + better assertions

Available test macros

  • snapshot
  • assert_equal
  • assert_greater_than
  • assert_greater_than_or_equal
  • assert_lesser_than
  • assert_lesser_than_or_equal
  • assert_matches_regex
  • assert_err_matches_regex
  • assert_matches_snapshot
  • assert_matches_inline_snapshot
  • assert_ok
  • assert_err

See https://docs.rs/k9 for API documentation

snapshot!() macro

Snapshot macro provides the functionality to capture the Debug representation of any value and make sure it does not change over time.

If it does change, the test will fail and print the difference between "old" and "new" values.

If the change is expected and valid, running cargo test with K9_UPDATE_SNAPSHOTS=1 env variable set will automatically take the new value and insert it into the test source code file as a second argument, after which all subsequent test runs should start passing again.

inline_snapshot_demo

assert_equal!() macro

Rust already provides a good built-in test runner and a set of assertion macros like assert! and assert_eq!. They work great for for quick unit tests, but once the codebase and test suites grows to a certain point it gets harder and harder to test things and keep tests readable.

For example, when testing that two structs are equal using assert_eq! macro the output does not provide a lot of help in understanding why exactly this test failed.

#[derive(PartialEq, Debug)]
struct Person {
    name: &'static str,
    age: usize,
}

#[test]
fn test_eq() {
    let person1 = Person {name: "Bob", age: 12 };
    let person2 = Person {name: "Alice", age: 20 };
    assert_eq!(person1, person2, "These two must be the same person!");
}

All we get is usually a wall of wite text collapsed into a single line and you have to find the difference between two structs yourself. Which becomes very time consuming when structs are 10+ fields.

---- eq::test_eq stdout ----
thread 'eq::test_eq' panicked at 'assertion failed: `(left == right)`
  left: `Person { name: "Bob", age: 12 }`,
 right: `Person { name: "Alice", age: 20 }`: These two must be the same person!', src/eq.rs:13:5

using k9::assert_equal macro improves this output and prints the difference between two structs:

use k9::assert_equal;
assert_equal!(person1, person2, "These two must be the same person!");

assert_equal_example

Non-equality based assertions

Testing equality is very simple and can definitely work for most of the cases, but one of the disadvantages of only using assert! and assert_eq! is the error messages when something fails. For example, if you're testing that your code produces valid URL

let url = generate_some_url();
assert_eq!(URL_REGEX.is_match(url), true);

What you get is

thread 'eq::test_eq3' panicked at 'assertion failed: `(left == right)`
  left: `false`,
 right: `true`', src/eq.rs:19:5

Which doesn't help much. Especially, if you're new to the code base, seeing things like expected 'true' but got 'false' will make you go and look at the code before you even know what the problem can be, which can be very time consuming.

What we probably want to see is:

assert_matches_regex_example

Which gives us enough context on what the problem is and how to fix it without for us having to go and run/debug the test first.

Comments
  • assert_equal does not handle comparing vectors to arrays

    assert_equal does not handle comparing vectors to arrays

    This is a parity issue with assert_eq!

    // This works:
    assert_eq!(vec![1, 2, 3], [1, 2, 3]);
    
    // This does not:
    assert_equal!(vec![1, 2, 3], [1, 2, 3]);
    
    error[E0308]: mismatched types
       --> src/sampler.rs:429:38
        |
    429 |         assert_equal!(vec![1, 2, 3], [1, 2, 3]);
        |                                      ^^^^^^^^^
        |                                      |
        |                                      expected struct `std::vec::Vec`, found array `[{integer}; 3]`
        |                                      help: try using a conversion method: `[1, 2, 3].to_vec()`
        |
        = note: expected struct `std::vec::Vec<{integer}>`
                    found array `[{integer}; 3]`
    
    opened by gregtatum 5
  • assert_matches_snapshot needs Display impl

    assert_matches_snapshot needs Display impl

    It seems that assert_matches_snapshot!(T) needs T to implement Display. This is a pretty restrictive requirement IMO, and Debug should suffice.

    The current workaround id to format! T :assert_matches_snapshot!(format!("{:?}")), but this should not be necessary.

    What do you think?

    opened by MarinPostma 4
  • Warnings in Rust 1.51

    Warnings in Rust 1.51

    warning: panic message is not a string literal
       --> term/src/test/mod.rs:366:5
        |
    366 | /     k9::snapshot!(
    367 | |         term.get_semantic_zones().unwrap(),
    368 | |         "
    369 | | [
    ...   |
    378 | | "
    379 | |     );
        | |______^
        |
        = note: `#[warn(non_fmt_panic)]` on by default
        = note: this is no longer accepted in Rust 2021
        = note: this warning originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
    

    and similarly for the assert_equals macro, which I've aliased here:

    warning: panic message is not a string literal
       --> wezterm-font/src/shaper/harfbuzz.rs:512:13
        |
    512 | /             assert_eq!(
    513 | |                 info,
    514 | |                 vec![GlyphInfo {
    515 | |                     cluster: 0,
    ...   |
    526 | |                 },]
    527 | |             );
        | |______________^
        |
        = note: this is no longer accepted in Rust 2021
        = note: this warning originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
    
    opened by wez 3
  • assert_panics!

    assert_panics!

    // success if the closure inside panics, panics if the closure doesn't panic
    assert_panics!(|| {
      panic!("sup y'all");
    });
    

    see https://doc.rust-lang.org/std/panic/fn.catch_unwind.html

    opened by aaronabramov 2
  • ☂️ DX Discussion

    ☂️ DX Discussion

    These are my notes and thoughts after trying out k9.

    • wish it showed PASS like Jest
    • show docs on hover in IDE (like other crates)
    • better error message for snapshot failure (when no snapshot present)
    • couldn't get snapshot assertion to work with example from docs (i blame myself)
    • also, could explain the _r assertions better and when you might want to use one

    README

    opened by jsjoeio 2
  • include LICENSE file in published crates

    include LICENSE file in published crates

    The MIT license requires that published / redistributed sources include a copy of the license text. This isn't the case for source code of the k9 crate that is published / redistributed via https://crates.io.

    This PR adds a symlink in the k9 directory that points at the LICENSE file in the repository root, which makes cargo include the file when packaging / publishing the crate.

    Note that if you are publishing this crate from an operating system which doesn't support symbolic links (Windows, I'm looking at you), a full copy of the file might be needed in the k9 directory, instead, to have cargo pick it up correctly.

    opened by decathorpe 1
  • Fix spelling mistakes

    Fix spelling mistakes

    I spotted a mistake in the README, so cloned the repo to open a PR, and VSCode found a few more in the rest of the repo, so have also fixed those whilst I'm here :-)

    opened by edmorley 1
  • Fix error assertions

    Fix error assertions

    This pull requests fixes two problems with Result-related assertions I've come across:

    1. It decouples the type arguments in assert_err and assert_ok, so that constructs like
      let result: Result<usize, &str> = Ok(1);
      assert_err!(result);
      

      should work now

    2. It fixes the argument names in the three-argument variant of assert_err_matches_regex, so that this variant can be used again.
    opened by korrat 1
  • Add qualified `use k9` test

    Add qualified `use k9` test

    Was about to fix the use of k9::assert_matches_inline_snapshot! and after writing the test realized it does work as intented so no fixing necessary, just adding the test to make sure it keeps working.

    opened by EduardoHi 1
  • Force colors to be enabled when running tests with buck

    Force colors to be enabled when running tests with buck

    This PR improves support for colors in output when running tests built with buck. By default the colored module would not produce output since it disables colors when stdout is a tty: https://github.com/mackwic/colored/blob/master/src/control.rs#L105-L114

    We detect if buck is being used and if so, force colored output.

    opened by mhlakhani 1
  • `assert_eq!(left, right, format, args...)` parity

    `assert_eq!(left, right, format, args...)` parity

    The standard assertion macros generally pass the error message and arguments to the panic! macro. k9 just seems to allow for a lone string without formatting, which means that migrating this example from the bottom of the docs:

    https://doc.rust-lang.org/std/macro.assert_eq.html

    assert_eq!(a, b, "we are testing addition with {} and {}", a, b);
    

    to k9 needs to done like this:

    assert_equal!(a, b, format!("we are testing addition with {} and {}", a, b));
    
    opened by wez 1
  • [WIP] Migration to ? operator

    [WIP] Migration to ? operator

    Hi Aaron,

    This PR addresses #83 although it's very much a work in progress and I am working to fix all the bugs and issues caused by introducing the new change.

    Thank you for letting me work on this.

    Yours Sincerely, Hannan

    opened by abdulhannanali 0
  • [WIP] Preliminary prototype for collapsed output

    [WIP] Preliminary prototype for collapsed output

    Usecase: Difference between two structs. Assertion: assert_equal

    Current Output: image

    Collapsed Output: image

    Method: String Iteration

    Implementation:

    1. Computed the index of each occurrence with reference to (- or +). // [4, 5, 16, 17, 20, 21]
    2. I have a cur_index and a peek_index.
    3. If the difference between cur_index and peek_index is <= 5, I'll have those index and the intermediate values in the final collapsed output else I have a .... // [4, 5, 16, 17, 18, 19, 20, 21]
    opened by TheWebDevel 12
  • codeframe in assertion failures

    codeframe in assertion failures

    we have line! and column! macros. we can technically take out a pice of code with some context around it where assertion failed and add it to the error message e.g.

    Assertion Failure!
    
    43 | fn do something() {
    44 |   let stuff = 1
    45 |   assert_equal!(stuff, 2);
    46 |  }
    opened by aaronabramov 1
  • Trailing comma in multiline macros

    Trailing comma in multiline macros

        assert_err_matches_regex!(
            conn.query_raw("SELECT 1").await,
            "time out"
        );
    

    works, but

        assert_err_matches_regex!(
            conn.query_raw("SELECT 1").await,
            "time out",
        );
    

    doesn't

    opened by aaronabramov 3
Owner
Aaron Abramov
Aaron Abramov
insta: a snapshot testing library for Rust

insta: a snapshot testing library for Rust Introduction Snapshots tests (also sometimes called approval tests) are tests that assert values against a

Armin Ronacher 1.4k Jan 1, 2023
Simple assertion library for unit testing in python with a fluent API

Simple assertions library for unit testing in Python with a nice fluent API. Supports both Python 2 and 3.

snakedye 19 Sep 10, 2022
This is a tiny (but delightful!) utility library for exhaustive testing.

Exhaustigen This is a tiny (but delightful!) utility library for exhaustive testing. It is based (directly) on the idea and code in the following blog

Graydon Hoare 34 Dec 14, 2022
A minimalist property-based testing library based on the arbitrary crate.

A minimalist property-based testing library based on the arbitrary crate.

Aleksey Kladov 61 Dec 21, 2022
Testing Framework for Rust

Polish Polish is Test-Driven Development done right Getting Started Installing the Package The crates.io package is kept up-to-date with all the major

Fadi Hanna Al-Kass 49 Dec 18, 2022
ArchTest is a rule based architecture testing tool for rust

ArchTest is a rule based architecture testing tool. It applies static analyses on the specified rust project to extract use relationships.

Tom Dymel 7 Sep 26, 2021
Automated property based testing for Rust (with shrinking).

quickcheck QuickCheck is a way to do property based testing using randomly generated input. This crate comes with the ability to randomly generate and

Andrew Gallant 2k Jan 2, 2023
Hypothesis-like property testing for Rust

Proptest Introduction Proptest is a property testing framework (i.e., the QuickCheck family) inspired by the Hypothesis framework for Python. It allow

Jason Lingle 1.1k Jan 1, 2023
Simple goldenfile testing in Rust.

?? Rust Goldenfile Simple goldenfile testing in Rust. Goldenfile tests generate one or more output files as they run. At the end of the test, the gene

Calder Coalson 24 Nov 26, 2022
Cucumber testing framework for Rust. Fully native, no external test runners or dependencies.

An implementation of the Cucumber testing framework for Rust. Fully native, no external test runners or dependencies.

Brendan Molloy 394 Jan 1, 2023
Loom is a concurrency permutation testing tool for Rust.

Loom is a testing tool for concurrent Rust code

Tokio 1.4k Jan 9, 2023
Drill is an HTTP load testing application written in Rust inspired by Ansible syntax

Drill is an HTTP load testing application written in Rust inspired by Ansible syntax

Ferran Basora 1.5k Jan 1, 2023
assay - A super powered testing macro for Rust

assay - A super powered testing macro for Rust as·say /ˈaˌsā,aˈsā/ noun - the testing of a metal or ore to determine its ingredients and quality. Rust

Michael Gattozzi 105 Dec 4, 2022
Testing Framework for Rust

Polish Polish is Test-Driven Development done right Getting Started Installing the Package The crates.io package is kept up-to-date with all the major

Fadi Hanna Al-Kass 49 Dec 18, 2022
Rnp - A simple cloud-friendly tool for testing network reachability.

Rnp - A simple cloud-friendly tool for testing network reachability. Release Status Crates.io Github release Nuget packages NOTE: This project is in e

Riff 50 Dec 13, 2022
Viceroy provides local testing for developers working with Compute@Edge.

Viceroy provides local testing for developers working with Compute@Edge. It allows you to run services written against the Compute@Edge APIs on your local development machine, and allows you to configure testing backends for your service to communicate with.

Fastly 99 Jan 7, 2023
Declarative Testing Framework

Demonstrate allows tests to be written without as a much repetitive code within the demonstrate! macro, which will generate the corresponding full tests.

Austin Baugh 41 Aug 17, 2022
🧵 Generate self-describing strings of a given length to help aid software testing

rust-counter-strings Counter strings generator written in rust to help aid software testing What is a counterstring? "A counterstring is a graduated s

Thomas Chaplin 23 Jun 24, 2022
🔥 Unit testing framework for Subgraph development on The Graph protocol. ⚙️

?? Welcome to Matchstick - a unit testing framework for The Graph protocol. Try out your mapping logic in a sandboxed environment and ensure your hand

null 157 Dec 20, 2022