A tool for adding new lines to files, skipping duplicates and write in Rust!

z3r0yu

Last update: Nov 16, 2023

Related tags

Command-line anew

Overview

anew

A tool for adding new lines to files written in Rust.

The tool aids in appending lines from stdin to a file, but only if they don't already appear in the file. Outputs new lines to stdout too, making it a bit like a tee -a that removes duplicates.

Install

Via cargo install

cargo install anew

Manual installation

git clone https://github.com/zer0yu/anew
cd anew
cargo build

or you can download the binary from releases

Usage

❯ anew --help
A tool for adding new lines to files, skipping duplicates

Usage: anew [OPTIONS] <FILEPATH>

Arguments:
  <FILEPATH>  Destination file

Options:
  -q, --quiet-mode  Do not output new lines to stdout
  -s, --sort        Sort lines (natsort)
  -t, --trim        Trim whitespaces
  -r, --rewrite     Rewrite existing destination file to remove duplicates
  -d, --dry-run     Do not write to file, only output what would be written
  -h, --help        Print help
  -V, --version     Print version

Usage Example

Here, a file called things.txt contains a list of numbers. newthings.txt contains a second list of numbers, some of which appear in things.txt and some of which do not. anew is used to append the latter to things.txt.

Usage 1: Add differences to things.txt

❯ cat things.txt
One
Zero
Two
One

❯ cat newthings.txt
Three
One
Five
Two
Four

❯ cat newthings.txt | anew things.txt
Three
Five
Four

❯ cat things.txt
One
Zero
Two
One
Three
Five
Four

Usage 2: Disable terminal output

❯ cat newthings.txt | anew things.txt -q
Three
Five
Four

Usage 3: Sorting the contents of things.txt after adding new differences

❯ cat newthings.txt | ./anew things.txt -q -s

❯ cat things.txt
Five
Four
One
Three
Two
Zero

PS:

anew uses the fastest sorting algorithm, so don't worry about efficiency.
Sort mode automatically de-duplicates, so -s and -r do not need to be used at the same time.

Usage 4: De-duplication of things.txt after adding new differences

❯ cat newthings.txt | ./anew things.txt -q -r

❯ cat things.txt
One
Zero
Two
Three
Five
Four

Efficiency Comparison

We use two files newoutput.txt and output.txt of size 10MB as input to the program to compare the difference in speed between tomnomnom's Go implementation, rwese's Rust implementation, and this project's Rust implementation.

This project

❯ time cat newoutput.txt | ./anew output.txt -q
cat newoutput.txt  0.00s user 0.02s system 1% cpu 1.398 total
./anew output.txt -q  1.46s user 0.22s system 97% cpu 1.717 total

anew implemented by rwese

❯ time cat newoutput.txt | ./anew_rwese output.txt -q
cat newoutput.txt  0.00s user 0.02s system 0% cpu 2:28.38 total
./anew output.txt -q  6.95s user 101.08s system 72% cpu 2:29.49 total

anew implemented by tomnomnom

❯ time cat newoutput.txt | ./anew_go -q output.txt
cat newoutput.txt  0.00s user 0.02s system 0% cpu 4.797 total
./anew_go -q output.txt  2.11s user 3.14s system 108% cpu 4.838 total

As can be seen from the above, the project has been implemented most efficiently!

References

You might also like...

Just a simple object renderer, written in under 500 lines using Rust.

All cargoes that the project runs are: bitflags: a crate for defining bitflag types cfg-if: a small macro crate for defining cfg-based - conditional c

3 May 4, 2023

Scriptable tool to read and write UEFI variables from EFI shell. View, save, edit and restore hidden UEFI (BIOS) Setup settings faster than with the OEM menu forms.

UEFI Variable Tool (UVT) UEFI Variable Tool (UVT) is a command-line application that runs from the UEFI shell. It can be launched in seconds from any

4 Dec 11, 2023

A tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK.

FileQL - File Query Language FileQL is a tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK. Sampl

39 Mar 12, 2024

A CLI tool that allow you to create a temporary new rust project using cargo with already installed dependencies

cargo-temp A CLI tool that allow you to create a new rust project in a temporary directory with already installed dependencies. Install Requires Rust

61 Oct 31, 2022

A rust library + CLI tool that tells you when swas will upload new video through complex calculations

A rust library + CLI tool that tells you when swas will upload new video through complex calculations. It also lets you search and play youtube videos of swas and other channels. Searching about youtube channels is also an option. Basically it's a youtube search cli tool written in rust.

4 Jun 10, 2022

Sets of libraries and tools to write applications and libraries mixing OCaml and Rust

Sets of libraries and tools to write applications and libraries mixing OCaml and Rust. These libraries will help keeping your types and data structures synchronized, and enable seamless exchange between OCaml and Rust

36 Jan 28, 2023

Dash is a CLI tool that rapidly sets up new projects by running a series of pre-defined commands.

Dash Dash is a CLI tool that rapidly sets up new projects by running a series of pre-defined commands. Features Quick Initialization: Initialize the c

4 Nov 7, 2023

Rust File Management CLI is a command-line tool written in Rust that provides essential file management functionalities. Whether you're working with files or directories, this tool simplifies common file operations with ease.

Rust FileOps Rust File Management CLI is a command-line tool written in Rust that provides essential file management functionalities. Whether you're w

5 May 2, 2024

Write Cross-platform application with React-like decralative UI framework and scalable ECS architecture all in Rust.

bevy_dioxus Dioxus Plugin for Bevy Write Cross-platform application with React-like decralative UI framework and scalable ECS architecture all in Rust

269 Dec 29, 2022

A tool for adding new lines to files, skipping duplicates and write in Rust!

Related tags

Overview

anew

A tool for adding new lines to files written in Rust.

Install

Usage

Usage Example

Efficiency Comparison

References

You might also like...

Just a simple object renderer, written in under 500 lines using Rust.

Scriptable tool to read and write UEFI variables from EFI shell. View, save, edit and restore hidden UEFI (BIOS) Setup settings faster than with the OEM menu forms.

A tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK.

A CLI tool that allow you to create a temporary new rust project using cargo with already installed dependencies

A rust library + CLI tool that tells you when swas will upload new video through complex calculations

Sets of libraries and tools to write applications and libraries mixing OCaml and Rust

Dash is a CLI tool that rapidly sets up new projects by running a series of pre-defined commands.

Rust File Management CLI is a command-line tool written in Rust that provides essential file management functionalities. Whether you're working with files or directories, this tool simplifies common file operations with ease.

Write Cross-platform application with React-like decralative UI framework and scalable ECS architecture all in Rust.

Owner

z3r0yu

Command line utility to remove duplicates from the given input.

locdev is a handy CLI tool that simplifies the process of adding, removing, and listing entries in the hosts file.

Turns lines of text into SVG files.

Command-line tool that provides a workflow for extending, editing, diffing, and writing to vim-style grep lines.

Shell Of A New Machine: Quickly configure new environments

A little tool to create region-free openingTitle.arc files for New Super Mario Bros. Wii, or to convert them from one region to another

Neovim plugin for moving lines up and down, written in Rust

🦀 Stupid simple presentation of the number of words, characters and lines on your clipboard.

A fast, resilient, isomorphic hacker news clone in ~1k lines of rust.

a cargo subcommand for counting lines of code in Rust projects