Cucumber testing framework for Rust. Fully native, no external test runners or dependencies.

Overview

cucumber-rust

Documentation Actions Status

An implementation of the Cucumber testing framework for Rust. Fully native, no external test runners or dependencies.

Usage

See our example repository.

Supporting crates

The full gamut of Cucumber's Gherkin language is implemented by the gherkin-rust project. Most features of the Gherkin language are parsed already and accessible via the relevant structs.

License

This project is licensed under either of

at your option.

Comments
  • [WIP] Add async tests

    [WIP] Add async tests

    I've begun adding async support to Cucumber.

    Async test running functionality

    • [x] Running async tests
    • [x] Basic async steps
    • [x] Regex async steps
    • [x] Regex typed async steps
    • [x] Ability to configure whether scenarios should run in parallel

    Async ergonomics

    • [x] Add support for async tests
    • [x] Add support for async regex tests
    • [x] Ability to select executor to use
    • [x] Re-implement nice printed output
    • [x] Document usage
    opened by bbqsrc 77
  • Replay failed tests

    Replay failed tests

    I was wondering if it's something that we can add to the framework. We already have a function to repeat failed tests output but we don't have a function to run failed tests again. It's common to have flaky tests that sometimes can be solved by running them again.

    For example :

    #[tokio::main]
    async fn main() {
        AnimalWorld::cucumber()
            .replay_failed(2)
            .run_and_exit("tests/features/book/output/terminal_repeat_failed.feature")
            .await;
    }
    

    Where 2 is the maximum number of times the tests should be run again in case of test failures. Let's say we have the tests A, B, C and D :

    1. During the first run only A passes
    2. Then B, C and D are run again (1 replay left if we have new test failures)
    3. Only B passes, we run again C and D (0 replay left)
    4. D fails again, this one is certainly more than just unstable

    Regarding the output we have two choices :

    • Print all test executions : it's transparent but can be repetitive when the tests fail multiple times (like D in this example)
    • Or just print the result once the last replay is done (which can be the maximum, here 2, or a previous run if all tests are passing)
    enhancement 
    opened by theredfish 23
  • Feature: xunit/junit output

    Feature: xunit/junit output

    I believe that adding j/xunit output support would go a long way to streamlining the tests, as those are the formats supported by IDE's and CI tools, so it would make cucumber-rust fit in like a glove with how devs work right now, rather than relying on reading the text output.

    I will probably give it a whack myself, as it's of high interest to me, but certainly, welcome any pointers and suggestions!

    enhancement 
    opened by Puciek 18
  • "and" keyword / prevent step definition repetition?

    I've a step that's used after both a given and then.

    I couldn't find the and keyword and figured that I've to use given (etc.) respectively. In this case this leads to having to write the step definition twice (with given and then), which isn't ideal.

    question 
    opened by ivanschuetz 15
  • Support pass-through for all arguments that `cargo test` supports

    Support pass-through for all arguments that `cargo test` supports

    Right now if you pass test-thread argument to cargo test, cucumber will fail:

         Running `/home/humb1t/workspace/soramitsu/iroha/target/debug/deps/cucumber-09561164203f68fb --test-threads=1 --skip=cucumber`
    error: Found argument '--test-threads' which wasn't expected, or isn't valid in this context
    

    Command: cargo test --workspace --verbose -- --test-threads=1

    It will be great to support this feature or at list ignore it instead of failure.

    enhancement 
    opened by humb1t 15
  • Command line option to disable printing `World` when there are failed tests or panics

    Command line option to disable printing `World` when there are failed tests or panics

    Hello,

    First of all, thanks for the work on this awesome crate.

    If I understand correctly, currently there is no way to disable printing the World in error messages. Whenever there is a failed test or panic, the whole World struct is printed. In some use cases this object can be quite large, making the errors difficult to navigate.

    Would it be possible to add a command line flag that controls this behaviour?

    enhancement 
    opened by yds12 12
  • Replace structopt by clap (#155)

    Replace structopt by clap (#155)

    This pull request is a draft, waiting for the official release of clap v3. However, if needed, It can already be reviewed as the api shouldn't drastically change.

    The doc comments on top of structs deriving clap::Args or clap::Parser is still an issue with Clap v3. So I let the workaround in place.

    Closes #155

    enhancement 
    opened by theredfish 12
  • Support for example values - or better examples

    Support for example values - or better examples

    I expected to access the values in example table easier, like scenario outline can be execute for each row of the example table and in those scenarios we can access the values easier, like:

    builder.given_example("a number <num>", |mut world, step, vars| {
      let string_num = vars.value("num");
      world.num_param = string_num::parse::<i32>::parse().unwrap();
      world
    })
    ...
    

    Or do we need to collect all examples in the World, but how? The example test is not implement for that.

    enhancement k::documentation 
    opened by jonasrichard 12
  • CLI redesign (#58, #134, #140)

    CLI redesign (#58, #134, #140)

    Resolves #58, #134, #140

    Synopsis

    In v0.10.0 CLI was preserved as is with error messages for unsupported options. But we want to rework it for more flexibility.

    Solution

    Redesign CLI and make it composable.

    Checklist

    • Created PR:
      • [x] In draft mode
      • [x] Name contains Draft: prefix
      • [x] Name contains issue reference
      • [x] Has assignee
    • [x] Documentation is updated (if required)
    • [x] Tests are updated (if required)
    • [x] Changes conform code style
    • [x] CHANGELOG entry is added (if required)
    • [x] FCM (final commit message) is posted
      • [x] and approved
    • [x] Review is completed and changes are approved
    • Before merge:
      • [x] Milestone is set
      • [x] PR's name and description are correct and up-to-date
      • [x] Draft: prefix is removed
      • [x] All temporary labels are removed
    enhancement 
    opened by ilslv 11
  • Architectural redesign and concurrent execution

    Architectural redesign and concurrent execution

    Pre-history and motivation

    We hardly use cucumber_rust in our production projects having ~800 scenarios and recently have started to suffer from its limitations: no concurrent runner, inability to customize its parts without changing upstream code.

    This PR is our long-going effort to make the cucumber_rust extensible, customizable, while ergonomic and rational out-of-the-box for most cases. It represents an almost complete redesign and rewrite of the library with introducing some new features along and fixing many existing issues. It was battle-tested in our code base for quite a while, and showed significant improvements of test execution times. All the documentation, images and the Book are made up-to-date with the changes.

    New architecture

    The new design of the library splits it to the 3 main components in the following pipeline:

    Parser -> Runner -> Writter
    

    where:

    • Parser is responsible for sourcing Scenarios for execution.
    • Runner is responsible for executing the stream of Scenarios.
    • Writer is responsible for outputting execution results.

    All the 3 components may be customized or changed to a custom implementation, which solves any questions about implementing custom output, or custom execution order, or anything else.

    Still the library provides reasonable defaults:

    • parser::Basic - reads from .feature files and supports # language: comment.
    • runner::Basic - executes all the scenarios concurrently by default (max 64 at the same time), and customizable to run fully (max 1 concurrent) or partially sequential (via @serial tag).
    • writer::Basic - simply outputs execution results to STDOUT, while writer::Normalized and writer::Summarized are out-of-the-box wrappers introducing output order normalization and writing summary of execution (for anyone implementing custom Writter to no bother about these problems).

    Output

    We've also stripped from the default output the filenames and line numbers for successful steps, as from our experience they make the output unreasonable bloated and hard-to-read. We still keep them for failing steps, though.

    Codegen

    macros feature is now enabled by default, as provides more ergonomic and understandable way to write step matchers.

    Code style

    rustc lints of the project are somewhat hardened and clippy::pedantic lints are checked on CI now. We believe that tightening code style is beneficial for open source libraries and future contributions, despite it may mandate some strange or verbose situations in code.

    Releasing and CI

    Release process now is fully automated via GitHub actions. Just prepare the repository for a release, commit and push the appropriate version tag.

    ⚠️ ACTION REQUIRED: You should set the CRATESIO_TOKEN to the project's GitHub secrets to release via CI.

    CI now fully checks and tests the project, including the Book.

    Resolved issues

    • #56: writer::Basic has really minimal output for success steps now.

    • #62: All the GIFs are ap-to-date now/

    • #63: We've tried our best to document everything pretty well, and enabled Clippy and rustc lints to keep an eye on that.

    • #64: Possible by using regex now.

    • #71: Fixed and not applicable anymore.

    • #80: Parsing errors are now propagated to the Writer.

    • #86: Just fixed.

    • #88: Fixed and not applicable anymore.

    • #102: Now possible with Cucumber::language().

    • #103: Can be done by Cucumber::filter_run_and_exit().

    • #119: writer::Basic pretty-prints only if terminal was detected.

    • #123: Now possible with writer::Summarised::fail_on_skipped().

    • #126: Now uses # language: comment to deduce language dynamically.

    • (partially) #50 and #127: While not provided out-of-the-box, may be implemented via custom Writer.

    Contribution and maintaining.

    We understand that this PR brings a huge impact to the library and may be undesirable by its author. If the later is true, we're OK to go on with our own fork and maintain it. However, from the Rust ecosystem perspective, it would be better to not introduce any ecosystem splits and disperse contributors efforts. That's why we also ask @bbqsrc to add me (@tyranron) and @ilslv to project maintainers, so we can continue to maintain the project and contribute to it more easily and actively. Ideally, we see the project to migrate to some GitHub org (like cucumber-rs or smthng), where new maintainers may step-in to the project more easily in the future.

    opened by tyranron 11
  • Are “Cucumber Expressions” supported ?

    Are “Cucumber Expressions” supported ?

    I couldn't find any references to them, in documentation, examples or even implementation code.

    For the record, Cucumber Expressions are the usual way to write step definitions in most languages, it's slightly less powerful than Regex but much more readable.

    For example consider:

    I have {float} cucumbers in my belly

    vs

    I have ([+-]?([0-9]+([.][0-9]*)?|[.][0-9]+)) cucumbers in my belly

    (Float regex matcher taken from Stackoverflow)

    enhancement 
    opened by StyMaar 10
  • `tracing` integration

    `tracing` integration

    I believe we can support better logging by integrating support for brilliant tracing crate, as current solution described in #177 is quite unintuitive and requires additional setup.

    We can implement custom tracing::Subscriber and pass Feature, Scenario and Step context inside tracing::Span. So we can retrieve this information and pass logging event as a new kind of cucumber::Events for logging them, when appropriate time comes.

    Unresolved questions, that require additional investigations:

    1. spawning a future, that will log after a Step has been finished may lead to strange output behaviour
    #[given(...)]
    fn delayed_log(w: &mut World) {
        spawn(
            async {
                sleep(Duration::from_secs(1).await;
                tracing::info!("Delayed");
            }
            .instrument(tracing::Span::current())
        )
    }
    

    Moreover, spawned Futures by default don't inherit Span from context they were spawned from. See https://github.com/tokio-rs/tracing/issues/394 for more info.

    1. Possible integrations with existing tracing ecosystem.
    enhancement k::refactor 
    opened by ilslv 3
  • Scenario execution priority attributes

    Scenario execution priority attributes

    Priority attribute tells the executor how early need to add scenario to the execution queue.

    In some cases there is no way to tell the executor to run scenarios in specific order. For example: Scenarios that will be executing for a long time must be added to execution queue before others.

    Design proposal:

    @priority [integer]
    Scenario: my slow scenario
    

    Scenarios with lower priority value will be added to execution queue earlier (or vice versa).

    feature k::api k::design 
    opened by 50U10FCA7 7
  • Cucumber Messages protocol

    Cucumber Messages protocol

    Revealed from https://github.com/cucumber-rs/gherkin/discussions/28#discussion-3257377

    Cucumber Messages is a message protocol for representing results and other information from Cucumber - Replaces json and junit formatters.

    Stability/Performance:

    With Cucumber Messages, several messages containing smaller pieces of information are emitted continuously to a stream, avoiding high memory consumption and enabling real-time processing of results.

    Additional functionality (motivation):

    The protocol aims to decouple various components of the Cucumber platform so that:

    • Each component only needs to know about a subset of messages

    • Gherkin is decoupled from the Cucumber execution component

      • This is part of a strategy to support other formats such as Markdown and Excel

    @ilslv what do you think about this? We have the similar thing in our implementation, but hand-baked. Should we support it as a bare-bone of our implementation, or rather have just yet another Writer for it?

    rfc postponed feature k::api k::UI/UX 
    opened by tyranron 2
Releases(v0.19.1)
Testing Framework for Rust

Polish Polish is Test-Driven Development done right Getting Started Installing the Package The crates.io package is kept up-to-date with all the major

Fadi Hanna Al-Kass 49 Dec 18, 2022
Fixture-based test framework for Rust

Fixture-based test framework for Rust Introduction rstest uses procedural macros to help you on writing fixtures and table-based tests. To use it, add

Michele d'Amico 567 Dec 24, 2022
Declarative Testing Framework

Demonstrate allows tests to be written without as a much repetitive code within the demonstrate! macro, which will generate the corresponding full tests.

Austin Baugh 41 Aug 17, 2022
🔥 Unit testing framework for Subgraph development on The Graph protocol. ⚙️

?? Welcome to Matchstick - a unit testing framework for The Graph protocol. Try out your mapping logic in a sandboxed environment and ensure your hand

null 157 Dec 20, 2022
Travis CI and AppVeyor template to test your Rust crate on 5 architectures and publish binary releases of it for Linux, macOS and Windows

trust Travis CI and AppVeyor template to test your Rust crate on 5 architectures and publish binary releases of it for Linux, macOS and Windows Featur

Jorge Aparicio 1.2k Dec 30, 2022
ArchTest is a rule based architecture testing tool for rust

ArchTest is a rule based architecture testing tool. It applies static analyses on the specified rust project to extract use relationships.

Tom Dymel 7 Sep 26, 2021
Automated property based testing for Rust (with shrinking).

quickcheck QuickCheck is a way to do property based testing using randomly generated input. This crate comes with the ability to randomly generate and

Andrew Gallant 2k Jan 2, 2023
Hypothesis-like property testing for Rust

Proptest Introduction Proptest is a property testing framework (i.e., the QuickCheck family) inspired by the Hypothesis framework for Python. It allow

Jason Lingle 1.1k Jan 1, 2023
Simple goldenfile testing in Rust.

?? Rust Goldenfile Simple goldenfile testing in Rust. Goldenfile tests generate one or more output files as they run. At the end of the test, the gene

Calder Coalson 24 Nov 26, 2022
Loom is a concurrency permutation testing tool for Rust.

Loom is a testing tool for concurrent Rust code

Tokio 1.4k Jan 9, 2023
Fluent test assertions for Rust.

This is a fork the unmaintained crate spectral. Spectral as not changed for five years and yet is still very usable, the goal of this fork is to add n

Paul Delafosse 24 Dec 20, 2022
A series of test cases to evaluate async Rust on the nrf52840 in terms of power usage and ergonomics.

A series of test cases to evaluate async Rust on the nrf52840 in terms of power usage and ergonomics. This is an experiment that uses unstable features only available on nightly rust.

Tweede golf 1 Oct 15, 2021
Test for rust-based plugin system for swc

rust-dylib-test Steps Run cargo build in plugin_a Ensure that plugin_a dynamically links to runtime/common by otool -L plugin_a/target/debug/libplugin

Donny/강동윤 1 Apr 6, 2022
Drill is an HTTP load testing application written in Rust inspired by Ansible syntax

Drill is an HTTP load testing application written in Rust inspired by Ansible syntax

Ferran Basora 1.5k Jan 1, 2023
assay - A super powered testing macro for Rust

assay - A super powered testing macro for Rust as·say /ˈaˌsā,aˈsā/ noun - the testing of a metal or ore to determine its ingredients and quality. Rust

Michael Gattozzi 105 Dec 4, 2022
Rust testing library

K9 - Rust Testing Library Snapshot testing + better assertions Available test macros snapshot assert_equal assert_greater_than assert_greater_than_or_

Aaron Abramov 269 Dec 10, 2022
Rustress - stress testing library in Rust. For fun

rustress Simple network stress testing library. To get familiar with Rust Planned features (Subject to change) Multithreaded client/server Throughput

Hakan Sönmez 7 Sep 22, 2022
Verdun is a HTTP stress-test/benchmark tool written in Rust.

Verdun is a HTTP stress-test/benchmark tool written in Rust. ?? It supports testing a single URL, loading multiples URLs from a file or automatically navigating a website (auto discovery)

Alex Hortopan 2 Feb 23, 2022
Nextest is a next-generation test runner for Rust.

nextest Nextest is a next-generation test runner for Rust. For more, check out the website. This repository contains the source code for: cargo-nextes

null 1.3k Jan 8, 2023