Find and replace text in source files

Overview

Tanker logo

crates.io image Build Coverage

Ruplacer

Find and replace text in source files:

$ ruplacer old new src/
Patching src/a_dir/sub/foo.txt
-- old is everywhere, old is old
++ new is everywhere, new is new

Patching src/top.txt
-- old is nice
++ new is nice

Would perform 2 replacements on 2 matching files.
Re-run ruplacer with --go to write these changes to disk

Installing with cargo

Install rust and cargo, for example with rustup.

Then run:

cargo install ruplacer

Alternative installation methods

  • Pre-compiled binaries for Linux, macOS, and Windows are available as assets of the latest release.

  • ruplacer can also be installed from homebrew:

$ brew install TankerHQ/homebrew-repo/ruplacer

Basic usage

ruplacer pattern replacement [path]

If the path is not given, it defaults to the current working directory.

Ruplacer will then walk through every file in <path> while honoring .gitignore files found on the way.

Binary files and text files containing non-UTF8 characters will be skipped. Then for every remaining file, it will read the contents, replace all lines matching the pattern by the replacement, and print the difference.

If you are OK with the replacements, re-run ruplacer with the --go option to actually write the changes to disk.

Regex

By default, pattern will be compiled into a Rust regex.

Note that it's slightly different from Perl-style regular expressions. Also, you must use $1, $2 to reference groups captured from pattern inside replacement.

For instance, this replaces 'last, first' by 'first last':

$ ruplacer '(\w+), (\w+)' '$2 $1'

(note the use of single quotes to avoid any processing by the shell)

If you don't want the pattern to be used as a regex, use the --no-regex command line flag.

This makes it possible to look for special characters without escaping them:

# This is a regex that matches the letter a
# or the letter o
$ ruplacer '(a|o)' u
- tata toto
+ tutu tutu
- (a|o)
+ (u|u)

# This is the literal string: '(a|o)'
$ ruplacer --no-regex '(a|o)' u
# or
$ ruplacer '\(a\|o|)' u
- (a|o)
+ u

Subvert mode

Ruplacer has a --subvert option which works across a variety of case styles (lower case, snake case, and so on):

$ ruplacer --subvert foo_bar spam_eggs
Patching src/foo.txt
-- foo_bar, FooBar, and FOO_BAR!
++ spam_eggs, SpamEggs, and SPAM_EGGS!

Filter files by type

Inspired by ripgrep, you can also select or ignore certain "file types":

# Select only C++ files
$ ruplacer old new --type cpp
# Ignore all js files
$ ruplacer old new --type-ignore js

Each "file type" is just a list of glob pattern. For instance: the cpp file type matches *.C, *.H, *.cc, *.cpp and so on ...

You can see the whole list by using ruplacer --file-types.

Comments
  • Better diff algorithm

    Better diff algorithm

    Right now if you replace foo_bar by spam_eggs, the underscore won't get colored when printing the diff.

    I've tried and fail to fix the algorithm, so maybe we should use an other diff library instead.

    Maybe this one: https://docs.rs/diff/0.1.11/diff/index.html ?

    Or we port diff-so-fancy from Perl to Rust :P

    opened by dmerejkowsky 9
  • Rewrite core functionality without using the difference crate

    Rewrite core functionality without using the difference crate

    Instead of just using string.replace or re.replace_all, actually do the work ourselves and compute a list of input and output fragments

    Each fragment is a struct containing some text (either added or removed) and some position

    This allows use to display the exact diff.

    Implementation notes:

    • Replace line_patcher module by just replacer

    • Compute the list of patters and replacements much earlier (and just one) when using --subvert

    • Also, implement Train-Case (also known as Http-Case) handling

    • Display the patch while the source files are read. This means less shuffling of data around

    • Tweak output

    Old:

    Patching foo/bar.js --- old is old +++ new is new

    New:

    foo/bar.js 3: old is old foo/bar.js 3: new is new

    • We no longer allocate n*n bytes when patching a line of size 'n', so the fix for issue #52 is no longer needed

    Fix #15

    opened by dmerejkowsky 7
  • Outdated version in brew

    Outdated version in brew

    While trying to apply "--ignored" option I discovered that my version is 0.4.1 while newest is 0.6.4. I can't install if from assets (macOS warns me about it), but will try to install from source. It would be much easier to me if you will update your brew package version. Thanks.

    opened by Masynchin 5
  • subvert does not replace all lower/upper case matches

    subvert does not replace all lower/upper case matches

    Steps to reproduce:

    echo 'stuffStuff stuffstuff STUFFSTUFF' > f
    ruplacer stuffStuff fooFoo --subvert
    

    Obtained result: only the first "stuffStuff" is replaced Expected result: all three should be replaced

    opened by blastrock 5
  • pattern/replacement starting with dashes

    pattern/replacement starting with dashes

    Hi,

    Is there a way to have a pattern or replacement starting with dashes? Right now ruplacer think of them as invalid options. Would it be possible to add "explicit" options? Something like ruplacer <-p|--pattern> --my-pattern-starting-with-double-dashes <-r|--replacement> -my-replacement-with-only-one-dash-at-the-beginning ?

    opened by s-d-m 4
  • select 'ruplacement`by piping them  though fzf or other selector

    select 'ruplacement`by piping them though fzf or other selector

    Hi ! First, thank for this amazing tool ! I use it almost on a daily basis :) Sometime ruplacer over-deliver replacement candidate, and I wold like to select only a subset of them.
    Would it be possible to pipe the candidate into a selector like fzf to achieve this purpose ?

    If not, could you give me some pointer on how to achieve it ?

    opened by sucrecacao 4
  • Add `-V`, `--version` option

    Add `-V`, `--version` option

    I find it very helpful to have a tool tell me it's version so I can consider upgrading to a newer version or track down bugs.

    This change adds the common --version option/switch (along with -V). Fortunately, Clap makes this very easy by using a magic command attribute: https://github.com/clap-rs/clap/blob/v3.1.15/examples/derive_ref/README.md#command-attributes

    Output of ruplacer --version:

    ruplacer 0.6.4
    

    Additional output of ruplacer --help:

    ruplacer 0.6.4
    
    […]
    
        -V, --version
                Print version information
    
    […]
    
    opened by homeworkprod 3
  • Handle trailing newlines consistently in FilePatcher

    Handle trailing newlines consistently in FilePatcher

    Previously, FilePatcher could not differentiate between files with and without trailing newlines, and always wrote files with trailing newlines. As a result, files without trailing newlines that were otherwise unchanged would be modified.

    This commit introduces a new LineIterator that preserves the trailing delimiter, if present, and ensures that files are written with the same newline structure they had when read.

    opened by LawnGnome 3
  • Add support for glob pattern

    Add support for glob pattern

    Workaround for https://github.com/dmerejkowsky/ruplacer/issues/56. Instead of regex, I think glob pattern would be more suited because the ignore crate already supported it.

    opened by ndtoan96 3
  • Fix typo in main.rs

    Fix typo in main.rs

    Hello, I'm Hayashi-Yudai.

    I use your repository and learn from the codes. This time, I found very tiny typo in help message. I'm grad if you merge my pull request.

    Sincerely.

    opened by Hayashi-Yudai 3
  • Replacement with empty string

    Replacement with empty string

    Thanks for creating ruplacer! It's been great for renaming variables across source files.

    Is there a way to replace matches with the empty string? For example,

    ruplacer "string" ""
    

    fails with

    error: The following required arguments were not provided:
        <replacement>
    

    I've been working around this by just adding an empty capture group () and then replacing with '$1'.

    opened by gpoore 2
  • Tool seems to try to (re)write files that don't contain matches

    Tool seems to try to (re)write files that don't contain matches

    $ echo "hello world" > a.txt
    $ echo "foo bar baz" > b.txt
    $ chmod 400 b.txt
    $ ruplacer 'hello' 'goodbye'
    ./a.txt:1 - hello world
    ./a.txt:1 + goodbye world
    
    Would perform 1 replacement on 1 matching file
    Re-run ruplacer with --go to write these changes to the filesystem
    $ ruplacer 'hello' 'goodbye' --go
    Error: Could not write ./b.txt
    
    Caused by:
        Permission denied (os error 13)
    
    opened by 08d2 0
  • Use capturing group next to literal

    Use capturing group next to literal

    Proposed tag: question.

    I want to make all class {Name}Test to be internal class {Name}Test, so I try to run:

    ruplacer 'class (\w+)Test' 'internal class $1Test'
    

    But it doesn't work as expected. I avoided it with

    ruplacer 'class (\w+)(Test)' 'internal class $1$2'
    

    But how can solve my issue without adding second capturing group?

    opened by Masynchin 2
  • Add multi-threading support

    Add multi-threading support

    This (currently WIP) PR proposes switching to ignore's parallel walker API. It's not complete yet (threads are still only set to 1), but I figured that I should start discussion with y'all here upstream before I got too carried away, since there's actually quite a few design considerations here, and I haven't even validated that this is something y'all want in the first place! :)

    See individual commits for more details on changes. Note that this is based on #93, and commits specifically for this PR start with the commit titled refactor: make `Stats` multithread-friendly.

    opened by ErichDonGubler 0
  • Allow multiple search-and-replace paths

    Allow multiple search-and-replace paths

    Intended to resolve #51, pending discussion (there's some trade-offs!). See individual commits for more details on changes.

    Open questions:

    • [ ] What new tests should be added? Cases that I see:
      • [ ] The case mentioned in the first commit (refactor: skip duplicate canonicalized paths) is a good candidate.
    opened by ErichDonGubler 0
  • `--ignored` seems not to work

    `--ignored` seems not to work

    Problem

    I want to patch code of ignored files (dependencies of my python project). But there is no difference with or without --ignored - Error: nothing found to replace. At the same time ripgrep correctly finds provided pattern.

    To reproduce

    Libs used:

    • python==3.10.2
    • ripgrep==13.0.0 (or rg)
    • ruplacer==0.6.4

    It might be overcomplicated, but it is my real situation. Steps to reproduce:

    mkdir reproduce
    cd reproduce
    
    git init
    echo venv/ > .gitignore
    
    python3 -m venv venv
    source venv/bin/activate
    pip install "Django==1.11.29"
    
    # no any output since all files ignored by gitignore:
    rg 'from collections import Iterable'
    ruplacer 'from collections import Iterable' 'from collections.abc import Iterable'
    
    # shows expected
    rg 'from collections import Iterable' --no-ignore
    
    # doesn't shows expected
    ruplacer 'from collections import Iterable' 'from collections.abc import Iterable' --ignored
    
    opened by Masynchin 3
Releases(v0.8.1)
Owner
Tanker
End-to-end encryption as a service
Tanker
Text Expression Runner – Readable and easy to use text expressions

ter - Text Expression Runner ter is a cli to run text expressions and perform basic text operations such as filtering, ignoring and replacing on the c

Maximilian Schulke 72 Jul 31, 2022
Find files (ff) by name, fast!

Find Files (ff) Find Files (ff) utility recursively searches the files whose names match the specified RegExp pattern in the provided directory (defau

Vishal Telangre 310 Dec 29, 2022
Source text parsing, lexing, and AST related functionality for Deno

Source text parsing, lexing, and AST related functionality for Deno.

Deno Land 90 Jan 1, 2023
Find all your TODO notes with one command!

Todo_r Find all your notes with one command! Todo_r is a simple rust command line utility that keeps track of your todo items in code. It is pronounce

Lavi Blumberg 34 Apr 22, 2022
Text calculator with support for units and conversion

cpc calculation + conversion cpc parses and evaluates strings of math, with support for units and conversion. 128-bit decimal floating points are used

Kasper 82 Jan 4, 2023
An efficient and powerful Rust library for word wrapping text.

Textwrap Textwrap is a library for wrapping and indenting text. It is most often used by command-line programs to format dynamic output nicely so it l

Martin Geisler 322 Dec 26, 2022
👄 The most accurate natural language detection library in the Rust ecosystem, suitable for long and short text alike

Table of Contents What does this library do? Why does this library exist? Which languages are supported? How good is it? Why is it better than other l

Peter M. Stahl 569 Jan 3, 2023
Semantic text segmentation. For sentence boundary detection, compound splitting and more.

NNSplit A tool to split text using a neural network. The main application is sentence boundary detection, but e. g. compound splitting for German is a

Benjamin Minixhofer 273 Dec 29, 2022
A fast, low-resource Natural Language Processing and Text Correction library written in Rust.

nlprule A fast, low-resource Natural Language Processing and Error Correction library written in Rust. nlprule implements a rule- and lookup-based app

Benjamin Minixhofer 496 Jan 8, 2023
Font independent text analysis support for shaping and layout.

lipi Lipi (Sanskrit for 'writing, letters, alphabet') is a pure Rust crate that provides font independent text analysis support for shaping and layout

Chad Brokaw 12 Sep 22, 2022
lingua-rs Python binding. An accurate natural language detection library, suitable for long and short text alike.

lingua-py lingua-rs Python binding. An accurate natural language detection library, suitable for long and short text alike. Installation pip install l

messense 7 Dec 30, 2022
bottom encodes UTF-8 text into a sequence comprised of bottom emoji

bottom encodes UTF-8 text into a sequence comprised of bottom emoji (with , sprinkled in for good measure) followed by ????. It can encode any valid UTF-8 - being a bottom transcends language, after all - and decode back into UTF-8.

Bottom Software Foundation 345 Dec 30, 2022
fastest text uwuifier in the west

uwuify fastest text uwuifier in the west transforms Hey... I think I really love you. Do you want a headpat? into hey... i think i w-weawwy wuv you.

Daniel Liu 1.2k Dec 29, 2022
A crate using DeepSpeech bindings to convert mic audio from speech to text

DS-TRANSCRIBER Need an Offline Speech To Text converter? Records your mic, and returns a String containing what was said. Features Begins transcriptio

null 32 Oct 8, 2022
Sorta Text Format in UTF-8

STFU-8: Sorta Text Format in UTF-8 STFU-8 is a hacky text encoding/decoding protocol for data that might be not quite UTF-8 but is still mostly UTF-8.

Rett Berg 18 Sep 4, 2022
The fastest way to identify any mysterious text or analyze strings from a file, just ask `lemmeknow` !

The fastest way to identify anything lemmeknow ⚡ Identify any mysterious text or analyze strings from a file, just ask lemmeknow. lemmeknow can be use

Swanand Mulay 594 Dec 30, 2022
better tools for text parsing

nom-text Goal: a library that extends nom to provide better tools for text formats (programming languages, configuration files). current needs Recogni

null 5 Oct 18, 2022
Makdown-like text parser.

Makdown-like text parser.

Ryo Nakamura 1 Dec 7, 2021
A Rust wrapper for the Text synthesization service TextSynth API

A Rust wrapper for the Text synthesization service TextSynth API

ALinuxPerson 2 Mar 24, 2022