Cucumber testing framework for Rust. Fully native, no external test runners or dependencies.

Overview

Cucumber testing framework for Rust

Crates.io Documentation CI Rust 1.61+ Unsafe Forbidden

An implementation of the Cucumber testing framework for Rust. Fully native, no external test runners or dependencies.

Usage

Describe testing scenarios in .feature files:

Feature: Eating too much cucumbers may not be good for you
    
  Scenario: Eating a few isn't a problem
    Given Alice is hungry
    When she eats 3 cucumbers
    Then she is full

Implement World trait and describe steps:

use std::{convert::Infallible, time::Duration};

use async_trait::async_trait;
use cucumber::{given, then, when, WorldInit};
use tokio::time::sleep;

#[derive(Debug, WorldInit)]
struct World {
    user: Option<String>,
    capacity: usize,
}

#[async_trait(?Send)]
impl cucumber::World for World {
    type Error = Infallible;

    async fn new() -> Result<Self, Self::Error> {
        Ok(Self { user: None, capacity: 0 })
    }
}

#[given(expr = "{word} is hungry")] // Cucumber Expression
async fn someone_is_hungry(w: &mut World, user: String) {
    sleep(Duration::from_secs(2)).await;
    
    w.user = Some(user);
}

#[when(regex = r"^(?:he|she|they) eats? (\d+) cucumbers?$")]
async fn eat_cucumbers(w: &mut World, count: usize) {
    sleep(Duration::from_secs(2)).await;

    w.capacity += count;
    
    assert!(w.capacity < 4, "{} exploded!", w.user.as_ref().unwrap());
}

#[then("she is full")]
async fn is_full(w: &mut World) {
    sleep(Duration::from_secs(2)).await;

    assert_eq!(w.capacity, 3, "{} isn't full!", w.user.as_ref().unwrap());
}

#[tokio::main]
async fn main() {
    World::run("tests/features/readme").await;
}

Add test to Cargo.toml:

[[test]]
name = "readme"
harness = false  # allows Cucumber to print output instead of libtest

For more examples check out the Book (current | edge).

Cargo features

  • macros (default): Enables step attributes and auto-wiring.
  • timestamps: Enables timestamps collecting for all Cucumber events.
  • output-json (implies timestamps): Enables support for outputting in Cucumber JSON format.
  • output-junit (implies timestamps): Enables support for outputting JUnit XML report.

Supporting crates

The full gamut of Cucumber's Gherkin language is implemented by the gherkin crate. Most features of the Gherkin language are parsed already and accessible via the relevant structs.

Known issues

  • Scenario Outline is treated the same as Outline or Example in the parser (gherkin/#19).

License

This project is licensed under either of

at your option.

Comments
  • [WIP] Add async tests

    [WIP] Add async tests

    I've begun adding async support to Cucumber.

    Async test running functionality

    • [x] Running async tests
    • [x] Basic async steps
    • [x] Regex async steps
    • [x] Regex typed async steps
    • [x] Ability to configure whether scenarios should run in parallel

    Async ergonomics

    • [x] Add support for async tests
    • [x] Add support for async regex tests
    • [x] Ability to select executor to use
    • [x] Re-implement nice printed output
    • [x] Document usage
    opened by bbqsrc 77
  • Replay failed tests

    Replay failed tests

    I was wondering if it's something that we can add to the framework. We already have a function to repeat failed tests output but we don't have a function to run failed tests again. It's common to have flaky tests that sometimes can be solved by running them again.

    For example :

    #[tokio::main]
    async fn main() {
        AnimalWorld::cucumber()
            .replay_failed(2)
            .run_and_exit("tests/features/book/output/terminal_repeat_failed.feature")
            .await;
    }
    

    Where 2 is the maximum number of times the tests should be run again in case of test failures. Let's say we have the tests A, B, C and D :

    1. During the first run only A passes
    2. Then B, C and D are run again (1 replay left if we have new test failures)
    3. Only B passes, we run again C and D (0 replay left)
    4. D fails again, this one is certainly more than just unstable

    Regarding the output we have two choices :

    • Print all test executions : it's transparent but can be repetitive when the tests fail multiple times (like D in this example)
    • Or just print the result once the last replay is done (which can be the maximum, here 2, or a previous run if all tests are passing)
    enhancement 
    opened by theredfish 23
  • Feature: xunit/junit output

    Feature: xunit/junit output

    I believe that adding j/xunit output support would go a long way to streamlining the tests, as those are the formats supported by IDE's and CI tools, so it would make cucumber-rust fit in like a glove with how devs work right now, rather than relying on reading the text output.

    I will probably give it a whack myself, as it's of high interest to me, but certainly, welcome any pointers and suggestions!

    enhancement 
    opened by Puciek 18
  • "and" keyword / prevent step definition repetition?

    I've a step that's used after both a given and then.

    I couldn't find the and keyword and figured that I've to use given (etc.) respectively. In this case this leads to having to write the step definition twice (with given and then), which isn't ideal.

    question 
    opened by ivanschuetz 15
  • Support pass-through for all arguments that `cargo test` supports

    Support pass-through for all arguments that `cargo test` supports

    Right now if you pass test-thread argument to cargo test, cucumber will fail:

         Running `/home/humb1t/workspace/soramitsu/iroha/target/debug/deps/cucumber-09561164203f68fb --test-threads=1 --skip=cucumber`
    error: Found argument '--test-threads' which wasn't expected, or isn't valid in this context
    

    Command: cargo test --workspace --verbose -- --test-threads=1

    It will be great to support this feature or at list ignore it instead of failure.

    enhancement 
    opened by humb1t 15
  • Command line option to disable printing `World` when there are failed tests or panics

    Command line option to disable printing `World` when there are failed tests or panics

    Hello,

    First of all, thanks for the work on this awesome crate.

    If I understand correctly, currently there is no way to disable printing the World in error messages. Whenever there is a failed test or panic, the whole World struct is printed. In some use cases this object can be quite large, making the errors difficult to navigate.

    Would it be possible to add a command line flag that controls this behaviour?

    enhancement 
    opened by yds12 12
  • Replace structopt by clap (#155)

    Replace structopt by clap (#155)

    This pull request is a draft, waiting for the official release of clap v3. However, if needed, It can already be reviewed as the api shouldn't drastically change.

    The doc comments on top of structs deriving clap::Args or clap::Parser is still an issue with Clap v3. So I let the workaround in place.

    Closes #155

    enhancement 
    opened by theredfish 12
  • Support for example values - or better examples

    Support for example values - or better examples

    I expected to access the values in example table easier, like scenario outline can be execute for each row of the example table and in those scenarios we can access the values easier, like:

    builder.given_example("a number <num>", |mut world, step, vars| {
      let string_num = vars.value("num");
      world.num_param = string_num::parse::<i32>::parse().unwrap();
      world
    })
    ...
    

    Or do we need to collect all examples in the World, but how? The example test is not implement for that.

    enhancement k::documentation 
    opened by jonasrichard 12
  • CLI redesign (#58, #134, #140)

    CLI redesign (#58, #134, #140)

    Resolves #58, #134, #140

    Synopsis

    In v0.10.0 CLI was preserved as is with error messages for unsupported options. But we want to rework it for more flexibility.

    Solution

    Redesign CLI and make it composable.

    Checklist

    • Created PR:
      • [x] In draft mode
      • [x] Name contains Draft: prefix
      • [x] Name contains issue reference
      • [x] Has assignee
    • [x] Documentation is updated (if required)
    • [x] Tests are updated (if required)
    • [x] Changes conform code style
    • [x] CHANGELOG entry is added (if required)
    • [x] FCM (final commit message) is posted
      • [x] and approved
    • [x] Review is completed and changes are approved
    • Before merge:
      • [x] Milestone is set
      • [x] PR's name and description are correct and up-to-date
      • [x] Draft: prefix is removed
      • [x] All temporary labels are removed
    enhancement 
    opened by ilslv 11
  • Architectural redesign and concurrent execution

    Architectural redesign and concurrent execution

    Pre-history and motivation

    We hardly use cucumber_rust in our production projects having ~800 scenarios and recently have started to suffer from its limitations: no concurrent runner, inability to customize its parts without changing upstream code.

    This PR is our long-going effort to make the cucumber_rust extensible, customizable, while ergonomic and rational out-of-the-box for most cases. It represents an almost complete redesign and rewrite of the library with introducing some new features along and fixing many existing issues. It was battle-tested in our code base for quite a while, and showed significant improvements of test execution times. All the documentation, images and the Book are made up-to-date with the changes.

    New architecture

    The new design of the library splits it to the 3 main components in the following pipeline:

    Parser -> Runner -> Writter
    

    where:

    • Parser is responsible for sourcing Scenarios for execution.
    • Runner is responsible for executing the stream of Scenarios.
    • Writer is responsible for outputting execution results.

    All the 3 components may be customized or changed to a custom implementation, which solves any questions about implementing custom output, or custom execution order, or anything else.

    Still the library provides reasonable defaults:

    • parser::Basic - reads from .feature files and supports # language: comment.
    • runner::Basic - executes all the scenarios concurrently by default (max 64 at the same time), and customizable to run fully (max 1 concurrent) or partially sequential (via @serial tag).
    • writer::Basic - simply outputs execution results to STDOUT, while writer::Normalized and writer::Summarized are out-of-the-box wrappers introducing output order normalization and writing summary of execution (for anyone implementing custom Writter to no bother about these problems).

    Output

    We've also stripped from the default output the filenames and line numbers for successful steps, as from our experience they make the output unreasonable bloated and hard-to-read. We still keep them for failing steps, though.

    Codegen

    macros feature is now enabled by default, as provides more ergonomic and understandable way to write step matchers.

    Code style

    rustc lints of the project are somewhat hardened and clippy::pedantic lints are checked on CI now. We believe that tightening code style is beneficial for open source libraries and future contributions, despite it may mandate some strange or verbose situations in code.

    Releasing and CI

    Release process now is fully automated via GitHub actions. Just prepare the repository for a release, commit and push the appropriate version tag.

    ⚠️ ACTION REQUIRED: You should set the CRATESIO_TOKEN to the project's GitHub secrets to release via CI.

    CI now fully checks and tests the project, including the Book.

    Resolved issues

    • #56: writer::Basic has really minimal output for success steps now.

    • #62: All the GIFs are ap-to-date now/

    • #63: We've tried our best to document everything pretty well, and enabled Clippy and rustc lints to keep an eye on that.

    • #64: Possible by using regex now.

    • #71: Fixed and not applicable anymore.

    • #80: Parsing errors are now propagated to the Writer.

    • #86: Just fixed.

    • #88: Fixed and not applicable anymore.

    • #102: Now possible with Cucumber::language().

    • #103: Can be done by Cucumber::filter_run_and_exit().

    • #119: writer::Basic pretty-prints only if terminal was detected.

    • #123: Now possible with writer::Summarised::fail_on_skipped().

    • #126: Now uses # language: comment to deduce language dynamically.

    • (partially) #50 and #127: While not provided out-of-the-box, may be implemented via custom Writer.

    Contribution and maintaining.

    We understand that this PR brings a huge impact to the library and may be undesirable by its author. If the later is true, we're OK to go on with our own fork and maintain it. However, from the Rust ecosystem perspective, it would be better to not introduce any ecosystem splits and disperse contributors efforts. That's why we also ask @bbqsrc to add me (@tyranron) and @ilslv to project maintainers, so we can continue to maintain the project and contribute to it more easily and actively. Ideally, we see the project to migrate to some GitHub org (like cucumber-rs or smthng), where new maintainers may step-in to the project more easily in the future.

    opened by tyranron 11
  • Are “Cucumber Expressions” supported ?

    Are “Cucumber Expressions” supported ?

    I couldn't find any references to them, in documentation, examples or even implementation code.

    For the record, Cucumber Expressions are the usual way to write step definitions in most languages, it's slightly less powerful than Regex but much more readable.

    For example consider:

    I have {float} cucumbers in my belly

    vs

    I have ([+-]?([0-9]+([.][0-9]*)?|[.][0-9]+)) cucumbers in my belly

    (Float regex matcher taken from Stackoverflow)

    enhancement 
    opened by StyMaar 10
  • `tracing` integration

    `tracing` integration

    I believe we can support better logging by integrating support for brilliant tracing crate, as current solution described in #177 is quite unintuitive and requires additional setup.

    We can implement custom tracing::Subscriber and pass Feature, Scenario and Step context inside tracing::Span. So we can retrieve this information and pass logging event as a new kind of cucumber::Events for logging them, when appropriate time comes.

    Unresolved questions, that require additional investigations:

    1. spawning a future, that will log after a Step has been finished may lead to strange output behaviour
    #[given(...)]
    fn delayed_log(w: &mut World) {
        spawn(
            async {
                sleep(Duration::from_secs(1).await;
                tracing::info!("Delayed");
            }
            .instrument(tracing::Span::current())
        )
    }
    

    Moreover, spawned Futures by default don't inherit Span from context they were spawned from. See https://github.com/tokio-rs/tracing/issues/394 for more info.

    1. Possible integrations with existing tracing ecosystem.
    enhancement k::refactor 
    opened by ilslv 3
  • Scenario execution priority attributes

    Scenario execution priority attributes

    Priority attribute tells the executor how early need to add scenario to the execution queue.

    In some cases there is no way to tell the executor to run scenarios in specific order. For example: Scenarios that will be executing for a long time must be added to execution queue before others.

    Design proposal:

    @priority [integer]
    Scenario: my slow scenario
    

    Scenarios with lower priority value will be added to execution queue earlier (or vice versa).

    feature k::api k::design 
    opened by 50U10FCA7 7
  • Cucumber Messages protocol

    Cucumber Messages protocol

    Revealed from https://github.com/cucumber-rs/gherkin/discussions/28#discussion-3257377

    Cucumber Messages is a message protocol for representing results and other information from Cucumber - Replaces json and junit formatters.

    Stability/Performance:

    With Cucumber Messages, several messages containing smaller pieces of information are emitted continuously to a stream, avoiding high memory consumption and enabling real-time processing of results.

    Additional functionality (motivation):

    The protocol aims to decouple various components of the Cucumber platform so that:

    • Each component only needs to know about a subset of messages

    • Gherkin is decoupled from the Cucumber execution component

      • This is part of a strategy to support other formats such as Markdown and Excel

    @ilslv what do you think about this? We have the similar thing in our implementation, but hand-baked. Should we support it as a bare-bone of our implementation, or rather have just yet another Writer for it?

    rfc postponed feature k::api k::UI/UX 
    opened by tyranron 2
Releases(v0.17.0)
Execution of and interaction with external processes and pipelines

subprocess The subprocess library provides facilities for execution of and interaction with external processes and pipelines, inspired by Python's sub

Hrvoje Nikšić 370 Dec 1, 2022
AUR external package builder

AUR Build Server Goal This project aims to provide an external package making server based on any PKGBUILD based project. Right now it pulls AUR packa

Seïfane Idouchach 2 Sep 11, 2022
TMM is a Linux native game modding tool. it allows to install and depoly mods for Linux native and wine games.

Tux Mod Manager TMM is a Linux native mod manager made with the Tauri toolkit. It can install, load, remove and deploy mods for both Linux native and

Mathiew May 117 Nov 27, 2022
Black-box integration tests for your REST API using the Rust and its test framework

restest Black-box integration test for REST APIs in Rust. This crate provides the [assert_api] macro that allows to declaratively test, given a certai

IOmentum 10 Nov 23, 2022
zigfi is an open-source stocks, commodities and cryptocurrencies price monitoring CLI app, written fully in Rust, where you can organize assets you're watching easily into watchlists for easy access on your terminal.

zigfi zigfi is an open-source stocks, commodities and cryptocurrencies price monitoring CLI app, written fully in Rust, where you can organize assets

Aldrin Zigmund Cortez Velasco 18 Oct 24, 2022
A fully modular window manager, extremely extensibile and easily approachable.

AquariWM is a fully modular window manager, allowing extreme extensibility while remaining easily approachable. Installation AquariWM is currently in

AquariWM Window Manager 8 Nov 14, 2022
Pure-Rust rewrite of the Linux fontconfig library (no system dependencies) - using ttf-parser and allsorts

rust-fontconfig Pure-Rust rewrite of the Linux fontconfig library (no system dependencies) - using allsorts as a font parser in order to parse .woff,

Felix Schütt 28 Oct 29, 2022
A CLI tool that allow you to create a temporary new rust project using cargo with already installed dependencies

cargo-temp A CLI tool that allow you to create a new rust project in a temporary directory with already installed dependencies. Install Requires Rust

Yohan Boogaert 61 Oct 31, 2022
Rust library to convert RGB 24-bit colors into ANSI 256 (8-bit) color codes with zero dependencies and at compile-time.

rgb2ansi256 rgb2ansi256 is a small Rust library to convert RGB 24-bit colors into ANSI 256 (8-bit) color codes with zero dependencies and const fn. Th

Linda_pp 7 Nov 17, 2022
A utility for managing cargo dependencies from the command line.

cargo edit This tool extends Cargo to allow you to add, remove, and upgrade dependencies by modifying your Cargo.toml file from the command line. Curr

Pascal Hertleif 2.7k Nov 23, 2022
This utility traverses through your filesystem looking for open-source dependencies that are seeking donations by parsing README.md and FUNDING.yml files

This utility traverses through your filesystem looking for open-source dependencies that are seeking donations by parsing README.md and FUNDING.yml files

Mufeed VH 38 Nov 22, 2022
A complete imgui-rs example using dependencies only from crates.io.

Dear imgui-rs, hello. This is a fairly basic, but complete and standalone example application for the Rust version of dear imgui (https://github.com/o

null 2 Sep 5, 2022
Utility to inherit dependencies from workspace file if it occurs 'n' or more times throughout the project.

Cargo Workspace Dependency Inheritor Utility that inherits dependencies from the main workspace if they occur n or more times in the workspace. Worksp

Timon 9 Oct 14, 2022
Simple test app based on rust-psp

PSP Test App Simple test app based on rust-psp. Demonstrating the usage of C libs. Build Download and unzip the prebuilt PSPSDK (built from clang-psp)

Yifeng Wang 3 Nov 17, 2021
A tui to test regexes on the rust regex crate

regex-tui Structure src/ ├── app.rs -> holds the states and renders the widgets ├── event.rs -> handles the terminal events (key press, mouse cl

null 1 Oct 21, 2021
Sanctity is an acronym for rust ansi16 color test utility

sanctity ?? sanctity is an acronym for rust ansi16 color test utility. It prints your own text or the default text in all of the 16 terminal colors.

null 1 Apr 6, 2022
create and test the style and formatting of text in your terminal applications

description: create and test the style and formatting of text in your terminal applications docs: https://docs.rs/termstyle termstyle is a library tha

Rett Berg 18 Jul 3, 2021
Terminal-based typing test.

ttyper Ttyper is a terminal-based typing test built with Rust and tui-rs. installation With Cargo: cargo install ttyper usage For usage instructions,

Max Niederman 522 Dec 2, 2022
A common library and set of test cases for transforming OSM tags to lane specifications

osm2lanes See discussion for context. This repo is currently just for starting this experiment. No license chosen yet. Structure data tests.json—tests

A/B Street 29 Nov 16, 2022