Splits test files into multiple groups to run tests in parallel nodes

Related tags

Text processing split-test
Overview

split-test

split-test splits tests into multiple groups based on timing data to run tests in parallel.

Installation

Download binary from GitHub releases page:

https://github.com/mtsmfm/split-test/releases

Usage

split-test command outputs test groups to stdout depends on its executing time.

Let's say we have three test files spec/a_spec.rb, spec/b_spec.rb, spec/c_spec.rb and the following test report report/report.xml.

<testsuites>
<testsuite classname="all" name="all" time="18.0">
<testcase classname="a" name="a" file="./spec/a_spec.rb" time="5.0"></testcase>
<testcase classname="b" name="b" file="./spec/b_spec.rb" time="10.0"></testcase>
<testcase classname="c" name="c" file="./spec/c_spec.rb" time="3.0"></testcase>
</testsuite>
</testsuites>

If you run them in series, it'll take 18 seconds.

You can use split-test to split them into two groups and run in parallel.

For node 0:

$ split-test --junit-xml-report-dir report --node-index 0 --node-total 2 --tests-glob 'spec/**/*_spec.rb'

You'll get the following result on stdout:

/path/to/spec/a_spec.rb
/path/to/spec/c_spec.rb

For node 1:

$ split-test --junit-xml-report-dir . --node-index 1 --node-total 2 --tests-glob 'spec/**/*_spec.rb'

You'll get:

/path/to/spec/b_spec.rb

Please be sure to increment --node-index arg.

You can use --debug option to make sure how it's grouped:

$ split-test --junit-xml-report-dir . --node-index 1 --node-total 2 --tests-glob 'spec/**/*_spec.rb' --debug
[2021-01-09T02:55:04Z DEBUG split_test] {"/path/to/spec/b_spec.rb": 10.0, "/path/to/spec/c_spec.rb": 3.0, "/path/to/spec/a_spec.rb": 5.0}
[2021-01-09T02:55:04Z DEBUG split_test] node 0: recorded_total_time: 8
[2021-01-09T02:55:04Z DEBUG split_test] /path/to/spec/a_spec.rb
[2021-01-09T02:55:04Z DEBUG split_test] /path/to/spec/c_spec.rb
[2021-01-09T02:55:04Z DEBUG split_test] node 1: recorded_total_time: 10
[2021-01-09T02:55:04Z DEBUG split_test] /path/to/spec/b_spec.rb
/path/to/spec/b_spec.rb

Pass the result to test command to run grouped tests:

$ rspec $(split-test --junit-xml-report-dir report --node-index 0 --node-total 2 --tests-glob 'spec/**/*_spec.rb' --debug)

GitHub Actions

on: push

jobs:
  # Download test-report and save as test-report-tmp to use the exactly same test report across parallel jobs.
  download-test-report:
    runs-on: ubuntu-latest
    steps:
      # Use dawidd6/action-download-artifact to download JUnit Format XML test report from another branch
      # https://github.com/actions/download-artifact/issues/3
      - uses: dawidd6/action-download-artifact@v2
        with:
          branch: main
          name: test-report
          workflow: ci.yml
          path: report
        # Use continue-on-error to run tests even if test-report is not uploaded
        continue-on-error: true
      - uses: actions/upload-artifact@v2
        with:
          name: test-report-tmp
          path: report

  test:
    needs: download-test-report
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node_index: [0, 1, 2]
    steps:
      - uses: actions/checkout@v2
      - uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true
          ruby-version: 3.0.0
      - uses: actions/download-artifact@v2
        with:
          name: test-report-tmp
          path: report-tmp
        # Use continue-on-error to run tests even if test-report is not uploaded
        continue-on-error: true
      - run: |
          curl -L --out split-test https://github.com/mtsmfm/split-test/releases/download/v0.3.0/split-test-x86_64-unknown-linux-gnu
          chmod +x split-test
      - run: bin/rspec --format progress --format RspecJunitFormatter --out report/rspec-${{ matrix.node_index }}.xml $(./split-test --junit-xml-report-dir report-tmp --node-index ${{ matrix.node_index }} --node-total 3 --tests-glob 'spec/**/*_spec.rb' --debug)
      - uses: actions/upload-artifact@v2
        with:
          name: test-report
          path: report
          if-no-files-found: error
        # Upload test-report on main branch only to avoid conflicting test report
        if: github.ref == 'refs/heads/main'

You can find working example on https://github.com/mtsmfm/split-test-example

Note

split-test is inspired by circleci tests split command.

split-test assumes test report has additional attribute file or filepath.

<testsuites>
<testsuite>
<testcase classname="spec.0_spec" name="0 is expected to eq 0" file="./spec/0_spec.rb" time="0.000373"></testcase>
</testsuite>
</testsuites>

To be exact it isn't JUnit standard.

You might also like...
A tool to calculate mean and standard deviation from multiple tests using PageSpeed Insights API.

psi-sample PSI Test tool is an open source tool to assist web developers that runs Page Speed Insight test manually! Installing To install the psi-tes

Igor Brasileiro 14 Sep 10, 2022
Black-box integration tests for your REST API using the Rust and its test framework

restest Black-box integration test for REST APIs in Rust. This crate provides the [assert_api] macro that allows to declaratively test, given a certai

IOmentum 10 Nov 23, 2022
Parser and test runner for testing compatable common Ethereum full node tests against Polygon Zero's EVM.

EVM Test Parses and runs compatible common Ethereum tests from ethereum/tests against Polygon Zero's EVM. Note: This repo is currently very early in d

Mir Protocol 3 Nov 4, 2022
Single-reader, multi-writer & single-reader, multi-verifier; broadcasts reads to multiple writeable destinations in parallel

Bus Writer This Rust crate provides a generic single-reader, multi-writer, with support for callbacks for monitoring progress. It also provides a gene

Pop!_OS 26 Feb 7, 2022
Download a file using multiple threads in parallel for faster download speeds.

multidl Download a file using multiple threads in parallel for faster download speeds. Uses 0 external dependencies. Usage Usage: multidl [--help] ADD

Divyanshu Agrawal 2 Sep 12, 2021
Android Device Pool - A tool to run device tests against a pool of devices.

adp (Android Device Pool) What is this? A tool to run device tests against a pool of devices. It will 'checkout' a device to run your tests against an

Evan Tatarka 1 Oct 21, 2021
A simple, opinionated way to run containers for tests in your Rust project.

rustainers rustainers is a simple, opinionated way to run containers for tests. TLDR More information about this crate can be found in the crate docum

wefox 4 Nov 23, 2023
1 library and 2 binary crates to run SSH/SCP commands on a
1 library and 2 binary crates to run SSH/SCP commands on a "mass" of hosts in parallel

massh 1 library and 2 binary crates to run SSH/SCP commands on a "mass" of hosts in parallel. The binary crates are CLI and GUI "frontends" for the li

null 2 Oct 16, 2022
Rust bindings to Linux Control Groups (cgroups)

cgroups-fs Native Rust library for managing Linux Control Groups (cgroups). This crate, curently, only supports the original, V1 hierarchy. You are we

Vlad Frolov 28 Nov 21, 2022
A simple web-app allowing you to batch archive groups of repositories from a given organization

ice-repos My goal here is to build a simple web-app allowing you to batch archive groups of repositories from a given organization, using Rust+Yew. As

Nic McPhee 6 Nov 5, 2022
Small program which groups images based on the GPS position.

gps-cluster This small program will take some pictures in input, and based on the metadata on every image, it will group them by their GPS position, i

Alessio Bandiera 2 Sep 12, 2022
Library for Unix users and groups in Rust.

uzers-rs Adoption and continuation of the unmaintained ogham/rust-users crate. Big shout-out to its creator Benjamin Sago. This is a library for acces

null 8 Aug 25, 2023
Library for Unix users and groups in Rust.

uzers-rs Adoption and continuation of the unmaintained ogham/rust-users crate. Big shout-out to its creator Benjamin Sago. This is a library for acces

null 8 Sep 18, 2023
atttribute macro for running a flaky test multiple times

flaky_test This attribute macro will register and run a test 3 times, erroring only if all three times fail. Useful for situations when a test is flak

Deno Land 23 Mar 23, 2022
Coppers is a custom test harnass for Rust that measures the energy usage of your test suite.
Coppers is a custom test harnass for Rust that measures the energy usage of your test suite.

Coppers Coppers is a test harness for Rust that can measure the evolution of power consumptions of a Rust program between different versions with the

Thijs Raymakers 175 Dec 4, 2022
A simple tool to test sqlx with postgres. It will automatically create a database and drop it after the test.

sqlx-db-tester This a tool to test sqlx with postgres. It only supports tokio runtime at this moment. How to use it You should first create a TestDb d

Tyr Chen 4 Nov 29, 2022
Formats output of Solana's cargo test-bpf/test-sbf command
Formats output of Solana's cargo test-bpf/test-sbf command

solfmt Formats output of Solana's cargo test-bpf/test-sbf command. Installation cargo install solfmt Usage Run the your test command as usual (cargo t

Thorsten Lorenz 4 Dec 22, 2022
OpenAPI-based test coverage analysis tool that helps teams improve integration test coverage in CI/CD pipelines
OpenAPI-based test coverage analysis tool that helps teams improve integration test coverage in CI/CD pipelines

Ready-to-use OpenAPI test coverage analysis tool that helps teams improve integration CoveAPI is an advanced test coverage analysis tool based on the

Yasser Tahiri 18 Aug 3, 2023
Collects accurate files while running in parallel through directories. (Simple, Fast, Powerful)

collectfiles Collects accurate files while running in parallel through directories. (Simple, Fast, Powerful) | Docs | Latest Note | [dependencies] col

Doha Lee 2 Jun 1, 2022
Comments
  • actions returns -out as an ambiguous option which result to exit code 2

    actions returns -out as an ambiguous option which result to exit code 2

    We started to get the following exit status when running split test and caused CI to not run. It is possible that GitHub changed ubuntu-latest library a bit and caused some update to the curl. It would be great if the team can check it out.

    Run curl -L --out split-test https://github.com/mtsmfm/split-test/releases/download/v1.0.0/split-test-x86_64-unknown-linux-gnu
[10](https://github.com/yasslab/railsguides.jp_web/actions/runs/3653925624/jobs/6174061904#step:11:11)
curl: option --out: is ambiguous
[11](https://github.com/yasslab/railsguides.jp_web/actions/runs/3653925624/jobs/6174061904#step:11:12)
curl: try 'curl --help' or 'curl --manual' for more information
[12](https://github.com/yasslab/railsguides.jp_web/actions/runs/3653925624/jobs/6174061904#step:11:13)
Error: Process completed with exit code 2.

    I do not have public repo available to share the fix and where it caused the issue but can share some screenshots. Hope it helps!

    Screen Shot 2022-12-09 at 12 53 19 Screen Shot 2022-12-09 at 12 53 45
    opened by nanophate 1
Owner
Fumiaki MATSUSHIMA
Fumiaki MATSUSHIMA
GitHub
This tool is for those who often want to search for a string deeply into a directory in recursive mode, but not with the great tool: grep, ack, ripgrep .........一个工具最大的价值不是它有多少功能，而是它能够让你以多快的速度达成所愿......

SSS - so stupid search tool <阿Q的哥锐普> English Documentation install install from source code 1.install rust toolchain curl --proto '=https' --tlsv1.2 -

null 136 Dec 11, 2022
bottom encodes UTF-8 text into a sequence comprised of bottom emoji

bottom encodes UTF-8 text into a sequence comprised of bottom emoji (with , sprinkled in for good measure) followed by ????. It can encode any valid UTF-8 - being a bottom transcends language, after all - and decode back into UTF-8.

Bottom Software Foundation 345 Dec 30, 2022
Find and replace text in source files

Ruplacer Find and replace text in source files: $ ruplacer old new src/ Patching src/a_dir/sub/foo.txt -- old is everywhere, old is old ++ new is ever

Tanker 331 Dec 28, 2022
Find files (ff) by name, fast!

Find Files (ff) Find Files (ff) utility recursively searches the files whose names match the specified RegExp pattern in the provided directory (defau

Vishal Telangre 310 Dec 29, 2022
⏮ ⏯ ⏭ A Rust library to easily read forwards, backwards or randomly through the lines of huge files.

EasyReader The main goal of this library is to allow long navigations through the lines of large files, freely moving forwards and backwards or gettin

Michele Federici 81 Dec 6, 2022
Difftastic is an experimental structured diff tool that compares files based on their syntax.

Difftastic is an experimental structured diff tool that compares files based on their syntax.

Wilfred Hughes 13.9k Jan 2, 2023
Subtitles-rs - Use SRT subtitle files to study foreign languages

Rust subtitle utilities Are you looking for substudy? Try here. (substudy has been merged into the subtitles-rs project.) This repository contains a n

Eric Kidd 268 Dec 29, 2022
A command line tool for renaming your ipa files quickly and easily.

ipa_renamer A command line tool for renaming your ipa files quickly and easily. Usage ipa_renamer 0.0.1 A command line tool for renaming your ipa file

Noah Hsu 31 Dec 31, 2022
ABQ is a universal test runner that runs test suites in parallel. It’s the best tool for splitting test suites into parallel jobs locally or on CI

?? abq.build   ?? @rwx_research   ?? discord   ?? documentation ABQ is a universal test runner that runs test suites in parallel. It’s the best tool f

RWX 13 Apr 7, 2023
The main repo for manta relay nodes and manta parachain nodes

Manta Manta is a privacy preserving DeFi stack on Polkadot/Substrate. The code currently hasn't been properly security audited (work in progress), use

Manta Network 99 Dec 14, 2022