ripgrep recursively searches directories for a regex pattern while respecting your gitignore

Overview

ripgrep (rg)

ripgrep is a line-oriented search tool that recursively searches the current directory for a regex pattern. By default, ripgrep will respect gitignore rules and automatically skip hidden files/directories and binary files. ripgrep has first class support on Windows, macOS and Linux, with binary downloads available for every release. ripgrep is similar to other popular search tools like The Silver Searcher, ack and grep.

Build status Crates.io Packaging status

Dual-licensed under MIT or the UNLICENSE.

CHANGELOG

Please see the CHANGELOG for a release history.

Documentation quick links

Screenshot of search results

A screenshot of a sample search with ripgrep

Quick examples comparing tools

This example searches the entire Linux kernel source tree (after running make defconfig && make -j8) for [A-Z]+_SUSPEND, where all matches must be words. Timings were collected on a system with an Intel i7-6900K 3.2 GHz.

Please remember that a single benchmark is never enough! See my blog post on ripgrep for a very detailed comparison with more benchmarks and analysis.

Tool Command Line count Time
ripgrep (Unicode) rg -n -w '[A-Z]+_SUSPEND' 452 0.136s
git grep git grep -P -n -w '[A-Z]+_SUSPEND' 452 0.348s
ugrep (Unicode) ugrep -r --ignore-files --no-hidden -I -w '[A-Z]+_SUSPEND' 452 0.506s
The Silver Searcher ag -w '[A-Z]+_SUSPEND' 452 0.654s
git grep LC_ALL=C git grep -E -n -w '[A-Z]+_SUSPEND' 452 1.150s
ack ack -w '[A-Z]+_SUSPEND' 452 4.054s
git grep (Unicode) LC_ALL=en_US.UTF-8 git grep -E -n -w '[A-Z]+_SUSPEND' 452 4.205s

Here's another benchmark on the same corpus as above that disregards gitignore files and searches with a whitelist instead. The corpus is the same as in the previous benchmark, and the flags passed to each command ensure that they are doing equivalent work:

Tool Command Line count Time
ripgrep rg -uuu -tc -n -w '[A-Z]+_SUSPEND' 388 0.096s
ugrep ugrep -r -n --include='*.c' --include='*.h' -w '[A-Z]+_SUSPEND' 388 0.493s
GNU grep egrep -r -n --include='*.c' --include='*.h' -w '[A-Z]+_SUSPEND' 388 0.806s

And finally, a straight-up comparison between ripgrep, ugrep and GNU grep on a single large file cached in memory (~13GB, OpenSubtitles.raw.en.gz):

Tool Command Line count Time
ripgrep rg -w 'Sherlock [A-Z]\w+' 7882 2.769s
ugrep ugrep -w 'Sherlock [A-Z]\w+' 7882 6.802s
GNU grep LC_ALL=en_US.UTF-8 egrep -w 'Sherlock [A-Z]\w+' 7882 9.027s

In the above benchmark, passing the -n flag (for showing line numbers) increases the times to 3.423s for ripgrep and 13.031s for GNU grep. ugrep times are unaffected by the presence or absence of -n.

Why should I use ripgrep?

  • It can replace many use cases served by other search tools because it contains most of their features and is generally faster. (See the FAQ for more details on whether ripgrep can truly replace grep.)
  • Like other tools specialized to code search, ripgrep defaults to recursive directory search and won't search files ignored by your .gitignore/.ignore/.rgignore files. It also ignores hidden and binary files by default. ripgrep also implements full support for .gitignore, whereas there are many bugs related to that functionality in other code search tools claiming to provide the same functionality.
  • ripgrep can search specific types of files. For example, rg -tpy foo limits your search to Python files and rg -Tjs foo excludes JavaScript files from your search. ripgrep can be taught about new file types with custom matching rules.
  • ripgrep supports many features found in grep, such as showing the context of search results, searching multiple patterns, highlighting matches with color and full Unicode support. Unlike GNU grep, ripgrep stays fast while supporting Unicode (which is always on).
  • ripgrep has optional support for switching its regex engine to use PCRE2. Among other things, this makes it possible to use look-around and backreferences in your patterns, which are not supported in ripgrep's default regex engine. PCRE2 support can be enabled with -P/--pcre2 (use PCRE2 always) or --auto-hybrid-regex (use PCRE2 only if needed). An alternative syntax is provided via the --engine (default|pcre2|auto-hybrid) option.
  • ripgrep supports searching files in text encodings other than UTF-8, such as UTF-16, latin-1, GBK, EUC-JP, Shift_JIS and more. (Some support for automatically detecting UTF-16 is provided. Other text encodings must be specifically specified with the -E/--encoding flag.)
  • ripgrep supports searching files compressed in a common format (brotli, bzip2, gzip, lz4, lzma, xz, or zstandard) with the -z/--search-zip flag.
  • ripgrep supports arbitrary input preprocessing filters which could be PDF text extraction, less supported decompression, decrypting, automatic encoding detection and so on.

In other words, use ripgrep if you like speed, filtering by default, fewer bugs and Unicode support.

Why shouldn't I use ripgrep?

Despite initially not wanting to add every feature under the sun to ripgrep, over time, ripgrep has grown support for most features found in other file searching tools. This includes searching for results spanning across multiple lines, and opt-in support for PCRE2, which provides look-around and backreference support.

At this point, the primary reasons not to use ripgrep probably consist of one or more of the following:

  • You need a portable and ubiquitous tool. While ripgrep works on Windows, macOS and Linux, it is not ubiquitous and it does not conform to any standard such as POSIX. The best tool for this job is good old grep.
  • There still exists some other feature (or bug) not listed in this README that you rely on that's in another tool that isn't in ripgrep.
  • There is a performance edge case where ripgrep doesn't do well where another tool does do well. (Please file a bug report!)
  • ripgrep isn't possible to install on your machine or isn't available for your platform. (Please file a bug report!)

Is it really faster than everything else?

Generally, yes. A large number of benchmarks with detailed analysis for each is available on my blog.

Summarizing, ripgrep is fast because:

  • It is built on top of Rust's regex engine. Rust's regex engine uses finite automata, SIMD and aggressive literal optimizations to make searching very fast. (PCRE2 support can be opted into with the -P/--pcre2 flag.)
  • Rust's regex library maintains performance with full Unicode support by building UTF-8 decoding directly into its deterministic finite automaton engine.
  • It supports searching with either memory maps or by searching incrementally with an intermediate buffer. The former is better for single files and the latter is better for large directories. ripgrep chooses the best searching strategy for you automatically.
  • Applies your ignore patterns in .gitignore files using a RegexSet. That means a single file path can be matched against multiple glob patterns simultaneously.
  • It uses a lock-free parallel recursive directory iterator, courtesy of crossbeam and ignore.

Feature comparison

Andy Lester, author of ack, has published an excellent table comparing the features of ack, ag, git-grep, GNU grep and ripgrep: https://beyondgrep.com/feature-comparison/

Note that ripgrep has grown a few significant new features recently that are not yet present in Andy's table. This includes, but is not limited to, configuration files, passthru, support for searching compressed files, multiline search and opt-in fancy regex support via PCRE2.

Installation

The binary name for ripgrep is rg.

Archives of precompiled binaries for ripgrep are available for Windows, macOS and Linux. Linux and Windows binaries are static executables. Users of platforms not explicitly mentioned below are advised to download one of these archives.

If you're a macOS Homebrew or a Linuxbrew user, then you can install ripgrep from homebrew-core:

$ brew install ripgrep

If you're a MacPorts user, then you can install ripgrep from the official ports:

$ sudo port install ripgrep

If you're a Windows Chocolatey user, then you can install ripgrep from the official repo:

$ choco install ripgrep

If you're a Windows Scoop user, then you can install ripgrep from the official bucket:

$ scoop install ripgrep

If you're an Arch Linux user, then you can install ripgrep from the official repos:

$ pacman -S ripgrep

If you're a Gentoo user, you can install ripgrep from the official repo:

$ emerge sys-apps/ripgrep

If you're a Fedora user, you can install ripgrep from official repositories.

$ sudo dnf install ripgrep

If you're an openSUSE user, ripgrep is included in openSUSE Tumbleweed and openSUSE Leap since 15.1.

$ sudo zypper install ripgrep

If you're a RHEL/CentOS 7/8 user, you can install ripgrep from copr:

$ sudo yum-config-manager --add-repo=https://copr.fedorainfracloud.org/coprs/carlwgeorge/ripgrep/repo/epel-7/carlwgeorge-ripgrep-epel-7.repo
$ sudo yum install ripgrep

If you're a Nix user, you can install ripgrep from nixpkgs:

$ nix-env --install ripgrep
$ # (Or using the attribute name, which is also ripgrep.)

If you're a Debian user (or a user of a Debian derivative like Ubuntu), then ripgrep can be installed using a binary .deb file provided in each ripgrep release.

$ curl -LO https://github.com/BurntSushi/ripgrep/releases/download/12.1.1/ripgrep_12.1.1_amd64.deb
$ sudo dpkg -i ripgrep_12.1.1_amd64.deb

If you run Debian Buster (currently Debian stable) or Debian sid, ripgrep is officially maintained by Debian.

$ sudo apt-get install ripgrep

If you're an Ubuntu Cosmic (18.10) (or newer) user, ripgrep is available using the same packaging as Debian:

$ sudo apt-get install ripgrep

(N.B. Various snaps for ripgrep on Ubuntu are also available, but none of them seem to work right and generate a number of very strange bug reports that I don't know how to fix and don't have the time to fix. Therefore, it is no longer a recommended installation option.)

If you're a FreeBSD user, then you can install ripgrep from the official ports:

# pkg install ripgrep

If you're an OpenBSD user, then you can install ripgrep from the official ports:

$ doas pkg_add ripgrep

If you're a NetBSD user, then you can install ripgrep from pkgsrc:

# pkgin install ripgrep

If you're a Haiku x86_64 user, then you can install ripgrep from the official ports:

$ pkgman install ripgrep

If you're a Haiku x86_gcc2 user, then you can install ripgrep from the same port as Haiku x86_64 using the x86 secondary architecture build:

$ pkgman install ripgrep_x86

If you're a Rust programmer, ripgrep can be installed with cargo.

  • Note that the minimum supported version of Rust for ripgrep is 1.34.0, although ripgrep may work with older versions.
  • Note that the binary may be bigger than expected because it contains debug symbols. This is intentional. To remove debug symbols and therefore reduce the file size, run strip on the binary.
$ cargo install ripgrep

Building

ripgrep is written in Rust, so you'll need to grab a Rust installation in order to compile it. ripgrep compiles with Rust 1.34.0 (stable) or newer. In general, ripgrep tracks the latest stable release of the Rust compiler.

To build ripgrep:

$ git clone https://github.com/BurntSushi/ripgrep
$ cd ripgrep
$ cargo build --release
$ ./target/release/rg --version
0.1.3

If you have a Rust nightly compiler and a recent Intel CPU, then you can enable additional optional SIMD acceleration like so:

RUSTFLAGS="-C target-cpu=native" cargo build --release --features 'simd-accel'

The simd-accel feature enables SIMD support in certain ripgrep dependencies (responsible for transcoding). They are not necessary to get SIMD optimizations for search; those are enabled automatically. Hopefully, some day, the simd-accel feature will similarly become unnecessary. WARNING: Currently, enabling this option can increase compilation times dramatically.

Finally, optional PCRE2 support can be built with ripgrep by enabling the pcre2 feature:

$ cargo build --release --features 'pcre2'

(Tip: use --features 'pcre2 simd-accel' to also include compile time SIMD optimizations, which will only work with a nightly compiler.)

Enabling the PCRE2 feature works with a stable Rust compiler and will attempt to automatically find and link with your system's PCRE2 library via pkg-config. If one doesn't exist, then ripgrep will build PCRE2 from source using your system's C compiler and then statically link it into the final executable. Static linking can be forced even when there is an available PCRE2 system library by either building ripgrep with the MUSL target or by setting PCRE2_SYS_STATIC=1.

ripgrep can be built with the MUSL target on Linux by first installing the MUSL library on your system (consult your friendly neighborhood package manager). Then you just need to add MUSL support to your Rust toolchain and rebuild ripgrep, which yields a fully static executable:

$ rustup target add x86_64-unknown-linux-musl
$ cargo build --release --target x86_64-unknown-linux-musl

Applying the --features flag from above works as expected. If you want to build a static executable with MUSL and with PCRE2, then you will need to have musl-gcc installed, which might be in a separate package from the actual MUSL library, depending on your Linux distribution.

Running tests

ripgrep is relatively well-tested, including both unit tests and integration tests. To run the full test suite, use:

$ cargo test --all

from the repository root.

Vulnerability reporting

For reporting a security vulnerability, please contact Andrew Gallant, which has my email address and PGP public key if you wish to send an encrypted message.

Translations

The following is a list of known translations of ripgrep's documentation. These are unofficially maintained and may not be up to date.

Comments
  • add to package repositories

    add to package repositories

    It'd be nice to at least get it into Ubuntu and homebrew. Sadly, I think either one of those will be quite difficult since Rust isn't packaged in either one of them.

    The lowest hanging fruit is the Archlinux User Repository (AUR).

    Fedora is getting Rust packaged soon, so that may be plausible.

    Sadly, we may need to live with binaries distributed here for the time being.

    enhancement 
    opened by BurntSushi 153
  • support searching across multiple lines

    support searching across multiple lines

    Say for example I'm trying to find instances of click that reside in a listeners block, like so:

    listeners: {
        foo: ...
        click: ....
    }
    

    According to the Rust regex docs, I should be able to do: rg '(?s)listeners.+click', but this doesn't seem to work. Does ripgrep not support multiline regex?

    enhancement libripgrep 
    opened by isobit 103
  • Support persistent configuration (`.rgrc`?)

    Support persistent configuration (`.rgrc`?)

    I'd love to keep different types for instance, and not have to specify --type-add each time.

    My specific case is type zsh which searches zsh config files (.zshrc *.zsh, and some custom .zsh*).

    Maybe it could support more stuff like no-heading, --heading etc as well, which would allow people to customize rg more.

    (I know I can just alias rgzsh="rg --type-add 'zsh:*.zsh' --type-add 'zsh:.zshrc' --type zsh", but meh)

    Loving the tool, btw!

    question 
    opened by SimenB 91
  • Searching filenames (`-g` option in `ag`)

    Searching filenames (`-g` option in `ag`)

    Ag supports using -g to recursively search filenames matching the provided pattern. Ripgrep doesn't have this feature (to the best of my knowledge; I've read the --help output three times looking for it); it's the only ag feature holding me back from fully switching to ripgrep. :)

    Since ripgrep already has something unrelated mapped to -g, the option name should be something else. The incompatibility with ag will be unfortunate, but survivable.

    opened by Valloric 61
  • maximum line length

    maximum line length

    I'd like ripgrep to have the ability to either hide or trim lines that are very long. Some lines take up my entire screen and are borderline useless to look at. It's possible that finding an intelligent way to shorten them would be best, since my guess is that the actual matched text is much smaller than the full line. However, this is harder to implement.

    I don't think this should be enabled by default. It seems a little surprising for ripgrep to hide lines like that. In general, I like the work flow of, "run a search, see huge lines, confirm that I don't care about them and run ripgrep again with an option to hide them." It may however be plausible to enable this limit if results are being dumped to a terminal (we already enable colors, line numbers and file headings).

    enhancement help wanted libripgrep 
    opened by BurntSushi 56
  • Add support for replace in files

    Add support for replace in files

    One of the usecases that many people have is a simple search and replace in multiple files. Usually I achieve this functionality with xargs sed for example with ag -l 'hello' | xargs sed -i "s/hello/bye/g. Since rg contains the replace flag it will be nice to add support for this feature by replacing in the files directly.

    This will be a nice addition after https://github.com/BurntSushi/ripgrep/issues/26

    An example on how to do this with sed can be found in the comments here. https://github.com/BurntSushi/ripgrep/issues/74#issuecomment-376659557

    opened by kfir-drivenets 53
  • Multiple testsuite failures on big-endian targets

    Multiple testsuite failures on big-endian targets

    The testsuite of ripgrep 0.10.0 is failing on all of Debian's big-endian targets:

    failures:
    
    ---- feature::f34_only_matching_line_column stdout ----
    thread 'feature::f34_only_matching_line_column' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    sherlock:1:57:Sherlock
    sherlock:3:49:Sherlock
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    sherlock:1:57:Sherlock
    sherlock:2:49:Sherlock
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/feature.rs:110:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::feature::f34_only_matching_line_column::{{closure}}
                 at tests/feature.rs:110
       8: integration::feature::f34_only_matching_line_column
                 at tests/macros.rs:7
       9: integration::feature::f34_only_matching_line_column::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- feature::f917_trim stdout ----
    thread 'feature::f917_trim' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2-For the Doctor Watsons of this world, as opposed to the Sherlock
    3:Holmeses, success in the province of detective work must always
    4-be, to a very large extent, the result of luck. Sherlock Holmes
    5-can extract a clew from a wisp of straw or a flake of cigar ash;
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2-For the Doctor Watsons of this world, as opposed to the Sherlock
    2:Holmeses, success in the province of detective work must always
    2-be, to a very large extent, the result of luck. Sherlock Holmes
    2-can extract a clew from a wisp of straw or a flake of cigar ash;
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/feature.rs:591:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::feature::f917_trim::{{closure}}
                 at tests/feature.rs:591
       8: integration::feature::f917_trim
                 at tests/macros.rs:7
       9: integration::feature::f917_trim::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- feature::f917_trim_match stdout ----
    thread 'feature::f917_trim_match' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2-For the Doctor Watsons of this world, as opposed to the Sherlock
    3:Holmeses, success in the province of detective work must always
    4-be, to a very large extent, the result of luck. Sherlock Holmes
    5-can extract a clew from a wisp of straw or a flake of cigar ash;
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2-For the Doctor Watsons of this world, as opposed to the Sherlock
    2:Holmeses, success in the province of detective work must always
    2-be, to a very large extent, the result of luck. Sherlock Holmes
    2-can extract a clew from a wisp of straw or a flake of cigar ash;
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/feature.rs:619:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::feature::f917_trim_match::{{closure}}
                 at tests/feature.rs:619
       8: integration::feature::f917_trim_match
                 at tests/macros.rs:7
       9: integration::feature::f917_trim_match::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- json::basic stdout ----
    thread 'json::basic' panicked at 'assertion failed: `(left == right)`
      left: `Context { path: Some(Text { text: "sherlock" }), lines: Text { text: "Holmeses, success in the province of detective work must always\n" }, line_number: Some(1), absolute_offset: 65, submatches: [] }`,
     right: `Context { path: Some(Text { text: "sherlock" }), lines: Text { text: "Holmeses, success in the province of detective work must always\n" }, line_number: Some(2), absolute_offset: 65, submatches: [] }`', tests/json.rs:151:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::json::basic::{{closure}}
                 at tests/json.rs:151
       8: integration::json::basic
                 at tests/macros.rs:7
       9: integration::json::basic::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- misc::after_context_line_numbers stdout ----
    thread 'misc::after_context_line_numbers' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:For the Doctor Watsons of this world, as opposed to the Sherlock
    2-Holmeses, success in the province of detective work must always
    3:be, to a very large extent, the result of luck. Sherlock Holmes
    4-can extract a clew from a wisp of straw or a flake of cigar ash;
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:For the Doctor Watsons of this world, as opposed to the Sherlock
    1-Holmeses, success in the province of detective work must always
    2:be, to a very large extent, the result of luck. Sherlock Holmes
    3-can extract a clew from a wisp of straw or a flake of cigar ash;
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/misc.rs:427:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::misc::after_context_line_numbers::{{closure}}
                 at tests/misc.rs:427
       8: integration::misc::after_context_line_numbers
                 at tests/macros.rs:7
       9: integration::misc::after_context_line_numbers::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- misc::before_context_line_numbers stdout ----
    thread 'misc::before_context_line_numbers' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:For the Doctor Watsons of this world, as opposed to the Sherlock
    2-Holmeses, success in the province of detective work must always
    3:be, to a very large extent, the result of luck. Sherlock Holmes
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:For the Doctor Watsons of this world, as opposed to the Sherlock
    1-Holmeses, success in the province of detective work must always
    2:be, to a very large extent, the result of luck. Sherlock Holmes
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/misc.rs:451:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::misc::before_context_line_numbers::{{closure}}
                 at tests/misc.rs:451
       8: integration::misc::before_context_line_numbers
                 at tests/macros.rs:7
       9: integration::misc::before_context_line_numbers::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- misc::columns stdout ----
    thread 'misc::columns' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:57:For the Doctor Watsons of this world, as opposed to the Sherlock
    3:49:be, to a very large extent, the result of luck. Sherlock Holmes
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:57:For the Doctor Watsons of this world, as opposed to the Sherlock
    2:49:be, to a very large extent, the result of luck. Sherlock Holmes
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/misc.rs:49:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::misc::columns::{{closure}}
                 at tests/misc.rs:49
       8: integration::misc::columns
                 at tests/macros.rs:7
       9: integration::misc::columns::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- misc::context_line_numbers stdout ----
    thread 'misc::context_line_numbers' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:For the Doctor Watsons of this world, as opposed to the Sherlock
    2-Holmeses, success in the province of detective work must always
    --
    5-but Doctor Watson has to have it taken out for him and dusted,
    6:and exhibited clearly, with a label attached.
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:For the Doctor Watsons of this world, as opposed to the Sherlock
    1-Holmeses, success in the province of detective work must always
    --
    3-but Doctor Watson has to have it taken out for him and dusted,
    3:and exhibited clearly, with a label attached.
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/misc.rs:479:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::misc::context_line_numbers::{{closure}}
                 at tests/misc.rs:479
       8: integration::misc::context_line_numbers
                 at tests/macros.rs:7
       9: integration::misc::context_line_numbers::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- misc::inverted_line_numbers stdout ----
    thread 'misc::inverted_line_numbers' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    2:Holmeses, success in the province of detective work must always
    4:can extract a clew from a wisp of straw or a flake of cigar ash;
    5:but Doctor Watson has to have it taken out for him and dusted,
    6:and exhibited clearly, with a label attached.
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:Holmeses, success in the province of detective work must always
    3:can extract a clew from a wisp of straw or a flake of cigar ash;
    3:but Doctor Watson has to have it taken out for him and dusted,
    3:and exhibited clearly, with a label attached.
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/misc.rs:121:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::misc::inverted_line_numbers::{{closure}}
                 at tests/misc.rs:121
       8: integration::misc::inverted_line_numbers
                 at tests/macros.rs:7
       9: integration::misc::inverted_line_numbers::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- misc::line_numbers stdout ----
    thread 'misc::line_numbers' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:For the Doctor Watsons of this world, as opposed to the Sherlock
    3:be, to a very large extent, the result of luck. Sherlock Holmes
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:For the Doctor Watsons of this world, as opposed to the Sherlock
    2:be, to a very large extent, the result of luck. Sherlock Holmes
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/misc.rs:38:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::misc::line_numbers::{{closure}}
                 at tests/misc.rs:38
       8: integration::misc::line_numbers
                 at tests/macros.rs:7
       9: integration::misc::line_numbers::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- misc::vimgrep stdout ----
    thread 'misc::vimgrep' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    sherlock:1:16:For the Doctor Watsons of this world, as opposed to the Sherlock
    sherlock:1:57:For the Doctor Watsons of this world, as opposed to the Sherlock
    sherlock:3:49:be, to a very large extent, the result of luck. Sherlock Holmes
    sherlock:5:12:but Doctor Watson has to have it taken out for him and dusted,
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    sherlock:1:16:For the Doctor Watsons of this world, as opposed to the Sherlock
    sherlock:1:57:For the Doctor Watsons of this world, as opposed to the Sherlock
    sherlock:2:49:be, to a very large extent, the result of luck. Sherlock Holmes
    sherlock:3:12:but Doctor Watson has to have it taken out for him and dusted,
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/misc.rs:775:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::misc::vimgrep::{{closure}}
                 at tests/misc.rs:775
       8: integration::misc::vimgrep
                 at tests/macros.rs:7
       9: integration::misc::vimgrep::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- multiline::context stdout ----
    thread 'multiline::context' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1-For the Doctor Watsons of this world, as opposed to the Sherlock
    2:Holmeses, success in the province of detective work must always
    3:be, to a very large extent, the result of luck. Sherlock Holmes
    4-can extract a clew from a wisp of straw or a flake of cigar ash;
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1-For the Doctor Watsons of this world, as opposed to the Sherlock
    1:Holmeses, success in the province of detective work must always
    2:be, to a very large extent, the result of luck. Sherlock Holmes
    3-can extract a clew from a wisp of straw or a flake of cigar ash;
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/multiline.rs:108:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::multiline::context::{{closure}}
                 at tests/multiline.rs:108
       8: integration::multiline::context
                 at tests/macros.rs:7
       9: integration::multiline::context::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- multiline::only_matching stdout ----
    thread 'multiline::only_matching' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:Watson
    1:Sherlock
    2:Holmes
    3:Sherlock Holmes
    5:Watson
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1:Watson
    1:Sherlock
    2:Holmes
    2:Sherlock Holmes
    3:Watson
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/multiline.rs:59:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::multiline::only_matching::{{closure}}
                 at tests/multiline.rs:59
       8: integration::multiline::only_matching
                 at tests/macros.rs:7
       9: integration::multiline::only_matching::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    ---- multiline::vimgrep stdout ----
    thread 'multiline::vimgrep' panicked at '
    printed outputs differ!
    
    expected:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    sherlock:1:16:For the Doctor Watsons of this world, as opposed to the Sherlock
    sherlock:1:57:For the Doctor Watsons of this world, as opposed to the Sherlock
    sherlock:2:57:Holmeses, success in the province of detective work must always
    sherlock:3:49:be, to a very large extent, the result of luck. Sherlock Holmes
    sherlock:5:12:but Doctor Watson has to have it taken out for him and dusted,
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    got:
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    sherlock:1:16:For the Doctor Watsons of this world, as opposed to the Sherlock
    sherlock:1:57:For the Doctor Watsons of this world, as opposed to the Sherlock
    sherlock:2:57:Holmeses, success in the province of detective work must always
    sherlock:2:49:be, to a very large extent, the result of luck. Sherlock Holmes
    sherlock:3:12:but Doctor Watson has to have it taken out for him and dusted,
    
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ', tests/multiline.rs:77:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::continue_panic_fmt
       6: std::panicking::begin_panic_fmt
       7: integration::multiline::vimgrep::{{closure}}
                 at tests/multiline.rs:77
       8: integration::multiline::vimgrep
                 at tests/macros.rs:7
       9: integration::multiline::vimgrep::{{closure}}
                 at tests/macros.rs:5
      10: core::ops::function::FnOnce::call_once
                 at libcore/ops/function.rs:238
      11: <F as alloc::boxed::FnBox<A>>::call_box
      12: __rust_maybe_catch_panic
    
    
    failures:
        feature::f34_only_matching_line_column
        feature::f917_trim
        feature::f917_trim_match
        json::basic
        misc::after_context_line_numbers
        misc::before_context_line_numbers
        misc::columns
        misc::context_line_numbers
        misc::inverted_line_numbers
        misc::line_numbers
        misc::vimgrep
        multiline::context
        multiline::only_matching
        multiline::vimgrep
    
    test result: FAILED. 173 passed; 14 failed; 0 ignored; 0 measured; 0 filtered out
    

    Full build logs here: https://buildd.debian.org/status/package.php?p=rust-ripgrep&suite=sid

    Click on the "Build-Attempted" fields in the "Status" column for more information.

    For access to a big-endian Linux machine to be able to debug the issue, you can request an account from the gcc compile farm (https://gcc.gnu.org/wiki/CompileFarm) and test on Linux/sparc64, for example.

    question 
    opened by glaubitz 52
  • preserve order of input files in output without sacrificing parallelism

    preserve order of input files in output without sacrificing parallelism

    There should be a way to tell rg to preserve the order of input files specified on the command line in the output.

    E.g., if I'm searching a bunch of files which are in chronological order, I want the output to be in chronological order.

    enhancement 
    opened by jikamens 45
  • ripgrep much slower than (rip)grep

    ripgrep much slower than (rip)grep

    What version of ripgrep are you using?

    ripgrep 11.0.1 -SIMD -AVX (compiled) +SIMD -AVX (runtime)

    How did you install ripgrep?

    pacman -S ripgrep

    What operating system are you using ripgrep on?

    Arch Linux

    Describe your question, feature request, or bug.

    ripgrep is 3-4-5 times slower than GNU grep on simple string pattern.

    If this is a bug, what is the actual behavior?

    grep file\.php -r .  0.19s user 0.10s system 98% cpu 0.298 total
    rg (--no-ignore) file\.php  3.68s user 0.45s system 365% cpu 1.129 total
    rg --debug --hidden --no-ignore file\.php  6.12s user 0.42s system 383% cpu 1.706 total
    DEBUG|grep_regex::literal|grep-regex/src/literal.rs:115: required literal found: "file"
    DEBUG|globset|globset/src/lib.rs:435: built glob set; 0 literals, 0 basenames, 11 extensions, 0 prefixes, 0 suffixes, 0 required extensions, 0 regexes
    DEBUG|globset|globset/src/lib.rs:435: built glob set; 0 literals, 0 basenames, 11 extensions, 0 prefixes, 0 suffixes, 0 required extensions, 0 regexes
    DEBUG|globset|globset/src/lib.rs:435: built glob set; 0 literals, 0 basenames, 11 extensions, 0 prefixes, 0 suffixes, 0 required extensions, 0 regexes
    DEBUG|globset|globset/src/lib.rs:435: built glob set; 0 literals, 0 basenames, 11 extensions, 0 prefixes, 0 suffixes, 0 required extensions, 0 regexes
    DEBUG|globset|globset/src/lib.rs:435: built glob set; 0 literals, 0 basenames, 11 extensions, 0 prefixes, 0 suffixes, 0 required extensions, 0 regexes
    

    As I tried to find one-by-one what top-level file (or directory) causes the slowdown, I came to the conclusion: none. Ripgrep should be faster... So I did this table: (Sorry, for hiding the names, I'm not sure if I'm allowed to show them.)

    #!/bin/sh
    $ echo GREPWHAT=$GREPWHAT RGFLAGS=$RGFLAGS GREPFLAGS=$GREPFLAGS; find -maxdepth 1 | xargs -I{} sh -c 'echo $(echo {} | tr a-z x):$(du -sh {} | cut -f1):$(find {} | wc -l):$( ( time rg "$GREPWHAT" $RGFLAGS {} &>/dev/null ) 2>&1 | tr \\t\\n \  ):$( ( time find -mindepth 1 -maxdepth 1 -not -path {} | xargs rg "$GREPWHAT" $RGFLAGS  &>/dev/null ) 2>&1 | tr \\t\\n \  ):$( ( time grep "$GREPWHAT" $GREPFLAGS -r {} &>/dev/null ) 2>&1 | tr \\t\\n \ ):$( ( time find -mindepth 1 -maxdepth 1 -not -path {} | xargs grep "$GREPWHAT" $GREPFLAGS -r &>/dev/null ) 2>&1 | tr \\t\\n \  )' | column -ts: -Rsize,files,rgtime,rgwothistime,greptime,grepwothistime -Nname,size,files,rgtime,rgwothistime,greptime,grepwothistime
    
    GREPWHAT=xxxxxxxxxxxxxxxxxxx.xxx RGFLAGS=-j1 GREPFLAGS=
    name                  size  files                                      rgtime                                rgwothistime                                    greptime                             grepwothistime
    .                     127M  12961   real 0m0.964s user 0m3.295s sys 0m0.348s    real 0m0.971s user 0m3.285s sys 0m0.388s    real 0m0.146s user 0m0.039s sys 0m0.100s    real 0m0.148s user 0m0.069s sys 0m0.078s
    ./xx                  4.0K      1   real 0m0.009s user 0m0.006s sys 0m0.003s    real 0m0.919s user 0m3.097s sys 0m0.406s    real 0m0.003s user 0m0.000s sys 0m0.003s    real 0m0.146s user 0m0.055s sys 0m0.089s
    ./_xxx_xxxxxx.xxx     428K      1   real 0m0.011s user 0m0.008s sys 0m0.004s    real 0m0.936s user 0m3.123s sys 0m0.388s    real 0m0.004s user 0m0.004s sys 0m0.000s    real 0m0.144s user 0m0.043s sys 0m0.100s
    ./xxxxxx               89M  11741   real 0m0.217s user 0m0.576s sys 0m0.211s    real 0m0.221s user 0m0.688s sys 0m0.109s    real 0m0.106s user 0m0.036s sys 0m0.064s    real 0m0.051s user 0m0.023s sys 0m0.029s
    ./xxxxx                16K      7   real 0m0.010s user 0m0.009s sys 0m0.004s    real 0m0.942s user 0m3.101s sys 0m0.418s    real 0m0.003s user 0m0.003s sys 0m0.000s    real 0m0.147s user 0m0.049s sys 0m0.098s
    ./xxxxxxx              52K     28   real 0m0.014s user 0m0.015s sys 0m0.004s    real 0m0.946s user 0m3.148s sys 0m0.418s    real 0m0.003s user 0m0.003s sys 0m0.000s    real 0m0.152s user 0m0.048s sys 0m0.099s
    ./xxxxxx.xxx          4.0K      1   real 0m0.008s user 0m0.006s sys 0m0.003s    real 0m0.929s user 0m3.113s sys 0m0.400s    real 0m0.003s user 0m0.000s sys 0m0.003s    real 0m0.143s user 0m0.051s sys 0m0.091s
    ./xxxxxx               20K      5   real 0m0.012s user 0m0.015s sys 0m0.000s    real 0m0.926s user 0m3.114s sys 0m0.421s    real 0m0.003s user 0m0.000s sys 0m0.003s    real 0m0.144s user 0m0.038s sys 0m0.105s
    ./xxxxxxxxx           248K     60   real 0m0.013s user 0m0.015s sys 0m0.005s    real 0m0.939s user 0m3.139s sys 0m0.384s    real 0m0.004s user 0m0.000s sys 0m0.004s    real 0m0.151s user 0m0.056s sys 0m0.095s
    ./xxxxxx.xx           4.0K      1   real 0m0.006s user 0m0.000s sys 0m0.006s    real 0m0.937s user 0m3.176s sys 0m0.413s    real 0m0.004s user 0m0.004s sys 0m0.000s    real 0m0.145s user 0m0.055s sys 0m0.089s
    ./xxxxxx               35M    522   real 0m0.068s user 0m0.138s sys 0m0.077s    real 0m0.227s user 0m0.639s sys 0m0.211s    real 0m0.036s user 0m0.025s sys 0m0.011s    real 0m0.114s user 0m0.031s sys 0m0.082s
    ./xxxx.xxx            4.0K      1   real 0m0.006s user 0m0.006s sys 0m0.001s    real 0m0.933s user 0m3.123s sys 0m0.408s    real 0m0.003s user 0m0.000s sys 0m0.003s    real 0m0.150s user 0m0.045s sys 0m0.104s
    ./xxxxxxx.xxx         4.0K      1   real 0m0.007s user 0m0.001s sys 0m0.006s    real 0m0.941s user 0m3.139s sys 0m0.409s    real 0m0.003s user 0m0.003s sys 0m0.000s    real 0m0.145s user 0m0.072s sys 0m0.072s
    ./xxxxx.xxx           4.0K      1   real 0m0.008s user 0m0.004s sys 0m0.004s    real 0m0.957s user 0m3.127s sys 0m0.419s    real 0m0.004s user 0m0.004s sys 0m0.000s    real 0m0.154s user 0m0.048s sys 0m0.101s
    ./xxxxxxxx            276K     52   real 0m0.013s user 0m0.016s sys 0m0.000s    real 0m1.037s user 0m2.956s sys 0m0.385s    real 0m0.004s user 0m0.004s sys 0m0.000s    real 0m0.147s user 0m0.036s sys 0m0.109s
    ./xxxxxx              128K     34   real 0m0.012s user 0m0.010s sys 0m0.005s    real 0m0.934s user 0m3.142s sys 0m0.398s    real 0m0.003s user 0m0.000s sys 0m0.003s    real 0m0.146s user 0m0.062s sys 0m0.083s
    ./xxxxxxxx.xxxx       212K      1   real 0m0.009s user 0m0.005s sys 0m0.004s    real 0m0.924s user 0m3.123s sys 0m0.362s    real 0m0.004s user 0m0.000s sys 0m0.003s    real 0m0.147s user 0m0.063s sys 0m0.083s
    ./xxxxxxxx.xxxx       4.0K      1   real 0m0.008s user 0m0.003s sys 0m0.005s    real 0m0.928s user 0m3.067s sys 0m0.422s    real 0m0.004s user 0m0.004s sys 0m0.000s    real 0m0.151s user 0m0.057s sys 0m0.088s
    ./xxxxxx              1.2M    294   real 0m0.018s user 0m0.027s sys 0m0.008s    real 0m0.852s user 0m2.761s sys 0m0.476s    real 0m0.007s user 0m0.000s sys 0m0.007s    real 0m0.141s user 0m0.033s sys 0m0.108s
    ./xxxxxxxxx            12K      5   real 0m0.012s user 0m0.016s sys 0m0.000s    real 0m0.987s user 0m2.797s sys 0m0.365s    real 0m0.003s user 0m0.000s sys 0m0.003s    real 0m0.148s user 0m0.038s sys 0m0.109s
    ./xxxxxxx             4.0K      1   real 0m0.005s user 0m0.000s sys 0m0.005s    real 0m1.074s user 0m2.989s sys 0m0.386s    real 0m0.005s user 0m0.004s sys 0m0.000s    real 0m0.143s user 0m0.057s sys 0m0.085s
    ./xxx                 764K    198   real 0m0.014s user 0m0.011s sys 0m0.012s    real 0m0.970s user 0m2.792s sys 0m0.481s    real 0m0.005s user 0m0.005s sys 0m0.000s    real 0m0.201s user 0m0.065s sys 0m0.133s
    ./.xxxxxxxx.xxxx.xxx   44K      1   real 0m0.007s user 0m0.003s sys 0m0.003s    real 0m1.393s user 0m2.965s sys 0m0.569s    real 0m0.004s user 0m0.000s sys 0m0.003s    real 0m0.205s user 0m0.079s sys 0m0.119s
    ./.xxxxxxxxx          4.0K      1   real 0m0.007s user 0m0.001s sys 0m0.006s    real 0m1.140s user 0m2.975s sys 0m0.556s    real 0m0.004s user 0m0.000s sys 0m0.004s    real 0m0.156s user 0m0.051s sys 0m0.099s
    ./.xxxxxxxxxxxxx      4.0K      1   real 0m0.006s user 0m0.003s sys 0m0.003s    real 0m0.954s user 0m3.055s sys 0m0.484s    real 0m0.003s user 0m0.003s sys 0m0.000s    real 0m0.149s user 0m0.070s sys 0m0.078s
    ./.xxx.xxxxxxx        4.0K      1   real 0m0.007s user 0m0.004s sys 0m0.003s    real 0m0.956s user 0m3.130s sys 0m0.402s    real 0m0.003s user 0m0.003s sys 0m0.000s    real 0m0.149s user 0m0.068s sys 0m0.076s
    $ # Note that the sum of ripgrep times doesn't match with . time.
    $ # wothis means: grepping in the directory with that file excluded.
    $ time tokei
    -------------------------------------------------------------------------------
     Language            Files        Lines         Code     Comments       Blanks
    -------------------------------------------------------------------------------
    ...
    -------------------------------------------------------------------------------
     Total                9256      1221713       827992       262436       131285
    -------------------------------------------------------------------------------
    
    real    0m1.061s
    user    0m3.384s
    sys     0m0.295s
    $ # Note that tokei has similar running time.
    $ stat -f . | head -3
      File: "."
        ID: 0        Namelen: 255     Type: tmpfs
    Block size: 4096       Fundamental block size: 4096
    $ # Fast as hell tmpfs. Does it count?
    

    Tomorrow I will make a line length statistics if it counts, but I think it should be an issue around directory listing. Anyway, I would like to help, so tell me what to do/share. --debug didn't help too much.

    question 
    opened by zsugabubus 38
  • support searching compressed files using in-process decompression

    support searching compressed files using in-process decompression

    I'd like to use ripgrep for grepping log files, because it's faster then grep. But my logs are gzipped, and if I zcat | rg them I'll loose log filenames in output.

    Also, would be great if bzip2 and xz decompressors will be supported too with automatic archive type detection.

    question icebox 
    opened by danbst 38
  • Color customization

    Color customization

    We should definitely try to support customizing colors in the output similar to how ag does it. Currently they support 3 color customizations, as per this issue:

    • --colors-match
    • --colors-path
    • --colors-line-number

    Implementing the part where we set colors based on an argument is not particularly hard (just a matter of translating user input to term::Attr enums). Parsing input in the bash terminal format like ag does was not difficult to implement. However, this limits Windows usage because we don't have an easy translation between those colors.

    How should we go about supporting these color customizations for both Linux, Mac and Windows?

    enhancement 
    opened by eugene-bulkin 38
  • fix(globset/serde1): allow deserializing owned strings into `Glob`

    fix(globset/serde1): allow deserializing owned strings into `Glob`

    This PR modifies how Glob is deserialized such that it now allows for owned strings (previously, only borrowed) to be deserialized into Glob. This takes inspiration from what is done in the regex_serde crate.

    For more details, see the discussion in #2386.

    The first commit includes tests, which fail without the solution. The second commit include the solution.

    This fixes #2386.

    I'd be happy to hear from your thoughts and I'm open to any review :-)

    opened by jeertmans 0
  • support standard character escape sequences (e.g. \t, \x1D) in the replacement string

    support standard character escape sequences (e.g. \t, \x1D) in the replacement string

    I'd like ripgrep to natively support replacements with common escape sequences (e.g --replace '\t' to replace with a horizontal tab)

    It is quite unwieldy to have to write e.g. rg ... --replace ...$(echo -ne '\x1F')... instead of just being able to do rg ... --replace ...\x1F...

    Additionally, being able to easily add '\x00' would be very nice in certain situations (for example, when delimiting the end of file names that contain enough special characters to make it quite annoying to deal with (since \0 is the only illegal character in filepaths)).

    Perhaps you could simply implement the stuff described in https://docs.rs/regex/latest/regex/#escape-sequences (with perhaps the addition of a few others such as '\e' and '\0' for convenience, especially the ones available with echo -eat thetypical shell)?

    Personally, by far the most used of these for me would be \x1[C-F], \x1b (for highlighting sections of the output in the terminal), \t, \n, and \0, but I frequently wish to make use of this capability, and have to either resort to ugly workarounds like the one previously mentioned, or use a different program(s).

    If you don't wish to modify the base functionality of --replace due to it possibly breaking existing scripts (though I'd think that because of the replacement string being evaluated at startup time, and the user presumably being the one providing the input, and the relevant characters themselves, breakage would likely be small and easily fixable), perhaps you can create --extended-replace <extended replacement string> as an alternative to the current --replace that supports standard character escapes, and allow doing -rx <extended replacement string> (with the shorthand explicitly having -r followed immediately by 'x' and then the replacement string being the next arg (unlike -r currently, which (somewhat confusingly) takes the rest of the current arg, if present as the replacement instead of the next argument) (I presume this could work due to rg's implementation. The use of 'x' is also kinda nice due to it being prominent in '\x' and "extend").

    I figure this all wouldn't be too difficult to implement due to it mostly already being implemented elsewhere, would have a minimal (negative) impact on rg as a whole, and be quite useful in certain situations whilst adding functionality many people might expect to be present already, so is this something that could be added to rg, or would I have to maintain a fork of it if I want this?

    Documentation could be something like

    -rx, --extended-replace REPLACEMENT_TEXT
      Like --replace, but interprets various backslash escapes
      \x([0-9A-Fa-f]{2,}) for the UTF-8 character with hexadecimal value $1
      Additionally, interprets
        \\ as a literal '\'
        \e as \x1b
        \0 as \x00 (the null byte)
        \t as \x09 (horizontal tab)
        \n ...
    
    opened by mwaitzman 1
  • globset: `serde` feature - cannot deserialize `String` into `Glob`

    globset: `serde` feature - cannot deserialize `String` into `Glob`

    Hi, I hope that posting issues about subcrates is ok.

    What version of globset are you using?

    globset = { version = "0.4.9", features = ["serde1"] }.

    Describe your bug.

    Because the implementation of trait Deserialize uses &str, it does not accept deserializing owned String.

    https://github.com/BurntSushi/ripgrep/blob/61101289fabc032fd8e90009c41d0b78e6dfc9a2/crates/globset/src/serde_impl.rs#L15-L22

    What are the steps to reproduce the behavior?

    This problem occurred to me when I tried to parse Globs from a toml::Value object (already obtained from a file). As toml::Value owns strings, it caused a problem.

    If needed, I can include a MWE (but quite large).

    What is the actual behavior?

    When parsing some globs, here is the error obtained:

    Error: TomlDecode(Error { inner: ErrorInner { kind: Custom, line: None, col: 0, at: None, message: "invalid type: string \"src/**.rs\", expected a borrowed string", key: [] } })
    

    What is the expected behavior?

    Not sure if it is interesting performance-wise, but using a custom deserialize_with function, with String instead of &str, fixes the problem (glob.as_str()) has to be used afterward).

    I can provide tests and make a PR if you are ok with this solution. Otherwise, I am also open to discussions :-)

    opened by jeertmans 2
  • The `--pretty` option doesn't override config file

    The `--pretty` option doesn't override config file

    What version of ripgrep are you using?

    ripgrep 13.0.0
    -SIMD -AVX (compiled)
    +SIMD +AVX (runtime)
    

    How did you install ripgrep?

    Installed from Debian bookworm

    What operating system are you using ripgrep on?

    Debian

    Describe your bug.

    I use --no-heading \n --no-line-number in my config file referenced by RIPGREP_CONFIG_PATH. I can explicitly add the --heading etc. options for a one-off commandline execution, but --pretty doesn't work as expected. I'd expect to have it be a true "convenience alias" as documented.

    Instead, the output continues to use the options from the config file, including neither heading nor line numbers.

    bug 
    opened by mrjoel 4
  • Try harder to fix #916

    Try harder to fix #916

    The underlying Rust issue1 was recently re-introduced and re-fixed. Rather than wait for a new Rust release, apply effectively the same fix as a workaround in the ignore crate itself.

    opened by tavianator 0
  • Option for writing results to file

    Option for writing results to file

    Describe your feature request

    An option --output or --outfile to write what ripgrep would print in a terminal to a file instead but byte-exact.

    Usually, you would pipe the stdout to a file like rg "expr" infile.txt > outfile.txt to do this. However, there are circumstances where this doesn't work as expected:

    • Line breaks may not be preserved by the terminal as-is. Both cmd.exe (using conhost.exe) and PowerShell (pwsh using the Terminal app) on Windows turn \r and \n into \r\n. A sequence of \n\r ends up as \r\n\r\n. Furthermore, a trailing newline (\r\n) is appended.

      Git Bash from Git for Windows leaves the line breaks as they are but appends a trailing \n.

      \r makes the terminals on Windows write over the current line, but it's difficult to understand what's going on because redirecting the output to a file does not give you the exact byte stream either.

    • Special terminal characters can mess with the redirection https://github.com/BurntSushi/ripgrep/issues/1309#issuecomment-633927149

    • Spaces after the command might end up in the output (although this appears to be limited to spaces before | and cmd.exe) https://github.com/BurntSushi/ripgrep/issues/1500

    enhancement help wanted 
    opened by Simran-B 2
Releases(13.0.0)
Owner
Andrew Gallant
I love to code.
Andrew Gallant
Bitcoin Push Notification Service (BPNS) allows you to receive notifications of Bitcoin transactions of your non-custodial wallets on a provider of your choice, all while respecting your privacy

Bitcoin Push Notification Service (BPNS) Description Bitcoin Push Notification Service (BPNS) allows you to receive notifications of Bitcoin transacti

BPNS 1 May 2, 2022
A lightweight async Web crawler in Rust, optimized for concurrent scraping while respecting `robots.txt` rules.

??️ crawly A lightweight and efficient web crawler in Rust, optimized for concurrent scraping while respecting robots.txt rules. ?? Features Concurren

CrystalSoft 5 Aug 29, 2023
Easy c̵̰͠r̵̛̠ö̴̪s̶̩̒s̵̭̀-t̶̲͝h̶̯̚r̵̺͐e̷̖̽ḁ̴̍d̶̖̔ ȓ̵͙ė̶͎ḟ̴͙e̸̖͛r̶̖͗ë̶̱́ṉ̵̒ĉ̷̥e̷͚̍ s̷̹͌h̷̲̉a̵̭͋r̷̫̊ḭ̵̊n̷̬͂g̵̦̃ f̶̻̊ơ̵̜ṟ̸̈́ R̵̞̋ù̵̺s̷̖̅ţ̸͗!̸̼͋

Rust S̵̓i̸̓n̵̉ I̴n̴f̶e̸r̵n̷a̴l mutability! Howdy, friendly Rust developer! Ever had a value get m̵̯̅ð̶͊v̴̮̾ê̴̼͘d away right under your nose just when

null 294 Dec 23, 2022
A Rust macro for writing regex pattern matching.

regexm A Rust macro for writing regex pattern matching.

Takayuki Maeda 46 Oct 24, 2022
CLI tool that extracts a regex pattern from a list of urls ( Rust )

rextract CLI tool that extracts a regex pattern from a list of urls. The tool is written in Rust and supports PCRE. Installation Step 1: Visit https:/

null 45 Dec 11, 2022
This crate converts Rust compatible regex-syntax to Vim's NFA engine compatible regex.

This crate converts Rust compatible regex-syntax to Vim's NFA engine compatible regex.

kaiuri 1 Feb 11, 2022
Unified directories for different use cases of an application, providing standard directories for local development, when run as service or when run by a user.

UniDirs Unified directories for different use cases of an application, providing standard directories for local development, when run as service or wh

Dominik Nakamura 3 Sep 30, 2022
Collects accurate files while running in parallel through directories. (Simple, Fast, Powerful)

collectfiles Collects accurate files while running in parallel through directories. (Simple, Fast, Powerful) | Docs | Latest Note | [dependencies] col

Doha Lee 2 Jun 1, 2022
A Rust based simple cli to delete all files included in `.gitignore` file from your repo

clean-local A simple CLI to remove all the files and folders included in the .gitignore file. Written in RUST Supports glob and wildcard paths in .git

Vilva Athiban P B 5 Jul 8, 2023
Create `.gitignore` files using one or more templates from TopTal, GitHub or your own collection

gitnr A cross-platform CLI utility to create .gitignore files using templates. Use templates from the GitHub & TopTal collections Use local files and

reemus.dev 25 Sep 2, 2023
Employ your built-in wetware pattern recognition and signal processing facilities to understand your network traffic

Nethoscope Employ your built-in wetware pattern recognition and signal processing facilities to understand your network traffic. Check video on how it

Vesa Vilhonen 86 Dec 5, 2022
Rust implementation of multi-index hashing for neighbor searches on binary codes in the Hamming space

mih-rs Rust implementation of multi-index hashing (MIH) for neighbor searches on binary codes in the Hamming space, described in the paper Norouzi, Pu

Shunsuke Kanda 8 Sep 23, 2022
⚙️ A cute .gitignore generator with support for custom templates

Gign A Gitignore Generator Table of Contents Examples Install Custom templates Help Examples # This is how you going to use it probably most of the ti

Vadim 6 Dec 7, 2022
A modern-looking, lightening fast, privacy respecting, secure, self-hostable meta search engine.

Websurfx A modern-looking, lightning-fast, privacy-respecting, secure meta search engine (pronounced as websurface or web-surface /wɛbˈsɜːrfəs/.) writ

neon_arch 16 May 8, 2023
🌊 ~ seaward is a crawler which searches for links or a specified word in a website.

?? seaward Installation cargo install seaward On NetBSD a pre-compiled binary is available from the official repositories. To install it, simply run:

null 3 Jul 16, 2023
The open source distributed web search engine that searches by meaning.

DawnSearch DawnSearch is an open source distributed web search engine that searches by meaning. It uses semantic search (searching on meaning), using

DawnSearch 4 Aug 8, 2023
rga: ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz, etc.

rga: ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz, etc. rga is a line-oriented search tool that allows you to look for a r

null 5.2k Jan 2, 2023
Fast line based iteration almost entirely lifted from ripgrep's grep_searcher.

?? ripline This is not the greatest line reader in the world, this is just a tribute. Fast line based iteration almost entirely lifted from ripgrep's

Seth 11 Feb 18, 2022
Node.js bindings to the ripgrep library, for fast file searching in JavaScript without child processes!

ripgrepjs ripgrepjs: Node.js bindings to the ripgrep library, for direct integration with JS programs without spawning an extra subprocess! This proje

Annika 1 May 10, 2022