Text Expression Runner – Readable and easy to use text expressions

Overview

ter - Text Expression Runner

ter is a cli to run text expressions and perform basic text operations such as filtering, ignoring and replacing on the command line. There are many great tools that do this job. But most other tools have one in common: They are hard to memorize if you dont use them regularly. ter tries to solve this issue by providing a super simple cli & expression language which can be easily memorized and is well documented.

Quickstart

$ ter filter 'equals "foobar"' -m word				# matches all occurences `foobar` in the text
$ ter filter 'length 20'							# matches all lines with 20 chars
$ ter ignore 'numeric or special'					# ignores all lines which contain only numbers and special chars
$ ter replace 'numeric and length 5' 12345 -m word	# replaces all 5 digit numbers with `12345`

Common tasks where ter excels grep in readability

Task ter grep
Find all words containing a string ter filter 'contains "substr"' -m word grep -oh "\w*substr\w*"
Find all lines in a file with a specific length ter filter 'length 10' grep -x '.\{10\}'
Ignore all lines containing a string ter ignore 'contains "hide me"' grep -v "hide me"
Replacing all words following a specific pattern ter replace 'numeric and length 5' 12345 -m word grep itself cant replace, you need to use sed for that (which gets even more complicated).
Replacing all email addresses in a file with your email ter replace 'contains "@" and contains ".com"' [email protected] -m word Same as above.

When to use other tools

As said earlier: ter is no direct competitor to grep, awk, etc.! If you find yourself reaching the limits of the text expression language, you probably want to use more advanced tools.

Installing

At the moment ter can be installed only via cargo using:

$ cargo install ter

Documentation

There are the following global options:

  • -m / --mode, sets the operation mode, can be either line or word, defaults to line

And there are the following global flags:

  • -f / --first`, print only the first match if available
  • -l / --last`, print only the last match if available
  • --skip n, skip the first n matches
  • --limit n, show at most n matches
ter filter [FLAGS] [OPTIONS]  [FILE]
ter ignore [FLAGS] [OPTIONS]  [FILE]
ter replace [FLAGS] [OPTIONS]   [FILE]

If no file is provided ter tries to read from stdin.

Examples

$ docker ps | ter filter 'alphanumeric and length 12' -m word # prints all docker container ids

The Text Expression Language

This is a super simple format of writing readable and easy to memorize text processing expressions - there are many great and far more advanced languages and tools to process text on the commandline out there but all of them have one problem in common - they're unreadable and hard to memorize if not used often.

The Text Expression Languages provides only 9 Attributes to query by. These attributes indicate the format of a string which gets tested against it.

Attribute Resolve to true if the tested string
starts starts with the given string
ends ends with the given string
contains contains a substring equal to the given string
equals exactly equals the given string
length has the given length
numeric contains only numeric chars
alpha contains only alphabetic chars
alphanumeric contains only alphanumeric chars
special contains only special chars

Currently there are only two binary logical operations: and and or

Operator Boolean Algebra
and Conjunction
or Disjunction

Attributes can be concattenated by logical operators.

Examples

starts "FOO" and ends "BAR"
contains "@" and contains ".com"
length 5 and length 10
numeric and length 8

Limitations

This Syntax might not cover all use cases. It's not meant to do that. If you find yourself reaching the limits of this language you might want to use more advanced tools (such as awk, grep, sed..)


The code for the language itself lives in a seperate repository.

You might also like...
Semantic text segmentation. For sentence boundary detection, compound splitting and more.

NNSplit A tool to split text using a neural network. The main application is sentence boundary detection, but e. g. compound splitting for German is a

A fast, low-resource Natural Language Processing and Text Correction library written in Rust.

nlprule A fast, low-resource Natural Language Processing and Error Correction library written in Rust. nlprule implements a rule- and lookup-based app

Source text parsing, lexing, and AST related functionality for Deno

Source text parsing, lexing, and AST related functionality for Deno.

Font independent text analysis support for shaping and layout.

lipi Lipi (Sanskrit for 'writing, letters, alphabet') is a pure Rust crate that provides font independent text analysis support for shaping and layout

lingua-rs Python binding. An accurate natural language detection library, suitable for long and short text alike.

lingua-py lingua-rs Python binding. An accurate natural language detection library, suitable for long and short text alike. Installation pip install l

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

rust-bert Rust native Transformer-based models implementation. Port of Hugging Face's Transformers library, using the tch-rs crate and pre-processing

bottom encodes UTF-8 text into a sequence comprised of bottom emoji
bottom encodes UTF-8 text into a sequence comprised of bottom emoji

bottom encodes UTF-8 text into a sequence comprised of bottom emoji (with , sprinkled in for good measure) followed by 👉👈. It can encode any valid UTF-8 - being a bottom transcends language, after all - and decode back into UTF-8.

fastest text uwuifier in the west

uwuify fastest text uwuifier in the west transforms Hey... I think I really love you. Do you want a headpat? into hey... i think i w-weawwy wuv you.

A crate using DeepSpeech bindings to convert mic audio from speech to text

DS-TRANSCRIBER Need an Offline Speech To Text converter? Records your mic, and returns a String containing what was said. Features Begins transcriptio

Owner
Maximilian Schulke
Maximilian Schulke
A command-line tool and library for generating regular expressions from user-provided test cases

Table of Contents What does this tool do? Do I still need to learn to write regexes then? Current features How to install? 4.1 The command-line tool 4

Peter M. Stahl 5.8k Dec 30, 2022
An implementation of regular expressions for Rust. This implementation uses finite automata and guarantees linear time matching on all inputs.

regex A Rust library for parsing, compiling, and executing regular expressions. Its syntax is similar to Perl-style regular expressions, but lacks a f

The Rust Programming Language 2.6k Jan 8, 2023
WriteForAll is a text file style checker, that compares text documents with editorial tips to make text better.

WriteForAll: tips to make text better WriteForAll is a text file style checker, that compares text documents with editorial tips to make text better.

Joel Parker Henderson 2 Dec 27, 2022
Simple expression transformer that is not Coq.

Noq Not Coq. Simple expression transformer that is not Coq. Quick Start $ cargo run ./examples/add.noq Main Idea The Main Idea is being able to define

Tsoding 187 Jan 7, 2023
Fast and easy random number generation.

alea A zero-dependency crate for fast number generation, with a focus on ease of use (no more passing &mut rng everywhere!). The implementation is bas

Jeff Shen 26 Dec 13, 2022
Easy reading and writing of `serde` structs to/from Google Sheets

serde_sheets Read and write structs directly from google sheets using serde and csv Implement serde::Serialize to write and serde::Deserialize to read

null 5 Jul 20, 2022
Text calculator with support for units and conversion

cpc calculation + conversion cpc parses and evaluates strings of math, with support for units and conversion. 128-bit decimal floating points are used

Kasper 82 Jan 4, 2023
Find and replace text in source files

Ruplacer Find and replace text in source files: $ ruplacer old new src/ Patching src/a_dir/sub/foo.txt -- old is everywhere, old is old ++ new is ever

Tanker 331 Dec 28, 2022
An efficient and powerful Rust library for word wrapping text.

Textwrap Textwrap is a library for wrapping and indenting text. It is most often used by command-line programs to format dynamic output nicely so it l

Martin Geisler 322 Dec 26, 2022
👄 The most accurate natural language detection library in the Rust ecosystem, suitable for long and short text alike

Table of Contents What does this library do? Why does this library exist? Which languages are supported? How good is it? Why is it better than other l

Peter M. Stahl 569 Jan 3, 2023