A tool for adding new lines to files, skipping duplicates and write in Rust!

Related tags

Command-line anew
Overview

anew

A tool for adding new lines to files written in Rust.

The tool aids in appending lines from stdin to a file, but only if they don't already appear in the file. Outputs new lines to stdout too, making it a bit like a tee -a that removes duplicates.

Install

Via cargo install

cargo install anew

Manual installation

git clone https://github.com/zer0yu/anew
cd anew
cargo build

or you can download the binary from releases

Usage

❯ anew --help
A tool for adding new lines to files, skipping duplicates

Usage: anew [OPTIONS] <FILEPATH>

Arguments:
  <FILEPATH>  Destination file

Options:
  -q, --quiet-mode  Do not output new lines to stdout
  -s, --sort        Sort lines (natsort)
  -t, --trim        Trim whitespaces
  -r, --rewrite     Rewrite existing destination file to remove duplicates
  -d, --dry-run     Do not write to file, only output what would be written
  -h, --help        Print help
  -V, --version     Print version

Usage Example

Here, a file called things.txt contains a list of numbers. newthings.txt contains a second list of numbers, some of which appear in things.txt and some of which do not. anew is used to append the latter to things.txt.

Usage 1: Add differences to things.txt

❯ cat things.txt
One
Zero
Two
One

❯ cat newthings.txt
Three
One
Five
Two
Four

❯ cat newthings.txt | anew things.txt
Three
Five
Four

❯ cat things.txt
One
Zero
Two
One
Three
Five
Four

Usage 2: Disable terminal output

❯ cat newthings.txt | anew things.txt -q
Three
Five
Four

Usage 3: Sorting the contents of things.txt after adding new differences

❯ cat newthings.txt | ./anew things.txt -q -s

❯ cat things.txt
Five
Four
One
Three
Two
Zero

PS:

  1. anew uses the fastest sorting algorithm, so don't worry about efficiency.
  2. Sort mode automatically de-duplicates, so -s and -r do not need to be used at the same time.

Usage 4: De-duplication of things.txt after adding new differences

❯ cat newthings.txt | ./anew things.txt -q -r

❯ cat things.txt
One
Zero
Two
Three
Five
Four

Efficiency Comparison

We use two files newoutput.txt and output.txt of size 10MB as input to the program to compare the difference in speed between tomnomnom's Go implementation, rwese's Rust implementation, and this project's Rust implementation.

This project

❯ time cat newoutput.txt | ./anew output.txt -q
cat newoutput.txt  0.00s user 0.02s system 1% cpu 1.398 total
./anew output.txt -q  1.46s user 0.22s system 97% cpu 1.717 total

anew implemented by rwese

❯ time cat newoutput.txt | ./anew_rwese output.txt -q
cat newoutput.txt  0.00s user 0.02s system 0% cpu 2:28.38 total
./anew output.txt -q  6.95s user 101.08s system 72% cpu 2:29.49 total

anew implemented by tomnomnom

❯ time cat newoutput.txt | ./anew_go -q output.txt
cat newoutput.txt  0.00s user 0.02s system 0% cpu 4.797 total
./anew_go -q output.txt  2.11s user 3.14s system 108% cpu 4.838 total

As can be seen from the above, the project has been implemented most efficiently!

References

  1. anew@tomnomnom
  2. anew@rwese
You might also like...
Just a simple object renderer, written in under 500 lines using Rust.
Just a simple object renderer, written in under 500 lines using Rust.

All cargoes that the project runs are: bitflags: a crate for defining bitflag types cfg-if: a small macro crate for defining cfg-based - conditional c

null 3 May 4, 2023
Scriptable tool to read and write UEFI variables from EFI shell. View, save, edit and restore hidden UEFI (BIOS) Setup settings faster than with the OEM menu forms.
Scriptable tool to read and write UEFI variables from EFI shell. View, save, edit and restore hidden UEFI (BIOS) Setup settings faster than with the OEM menu forms.

UEFI Variable Tool (UVT) UEFI Variable Tool (UVT) is a command-line application that runs from the UEFI shell. It can be launched in seconds from any

null 4 Dec 11, 2023
A tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK.
A tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK.

FileQL - File Query Language FileQL is a tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK. Sampl

Amr Hesham 39 Mar 12, 2024
A CLI tool that allow you to create a temporary new rust project using cargo with already installed dependencies
A CLI tool that allow you to create a temporary new rust project using cargo with already installed dependencies

cargo-temp A CLI tool that allow you to create a new rust project in a temporary directory with already installed dependencies. Install Requires Rust

Yohan Boogaert 61 Oct 31, 2022
A rust library + CLI tool that tells you when swas will upload new video through complex calculations
A rust library + CLI tool that tells you when swas will upload new video through complex calculations

A rust library + CLI tool that tells you when swas will upload new video through complex calculations. It also lets you search and play youtube videos of swas and other channels. Searching about youtube channels is also an option. Basically it's a youtube search cli tool written in rust.

midnightFirefly 4 Jun 10, 2022
Sets of libraries and tools to write applications and libraries mixing OCaml and Rust

Sets of libraries and tools to write applications and libraries mixing OCaml and Rust. These libraries will help keeping your types and data structures synchronized, and enable seamless exchange between OCaml and Rust

Meta 36 Jan 28, 2023
Dash is a CLI tool that rapidly sets up new projects by running a series of pre-defined commands.
Dash is a CLI tool that rapidly sets up new projects by running a series of pre-defined commands.

Dash Dash is a CLI tool that rapidly sets up new projects by running a series of pre-defined commands. Features Quick Initialization: Initialize the c

Kunal Bagaria 4 Nov 7, 2023
Write Cross-platform application with React-like decralative UI framework and scalable ECS architecture all in Rust.

bevy_dioxus Dioxus Plugin for Bevy Write Cross-platform application with React-like decralative UI framework and scalable ECS architecture all in Rust

Junichi Sugiura 269 Dec 29, 2022
Simple macros to write colored and formatted text to a terminal. Based on `termcolor`, thus also cross-platform.
Simple macros to write colored and formatted text to a terminal. Based on `termcolor`, thus also cross-platform.

Bunt: simple macro-based terminal colors and styles bunt offers macros to easily print colored and formatted text to a terminal. It is just a convenie

Lukas Kalbertodt 202 Dec 22, 2022
Owner
z3r0yu
BugHunter@OverSpace / Penetration Tester@PolarisLab / CTFer@kn0ck
z3r0yu
GitHub
Command line utility to remove duplicates from the given input.

Command line utility to remove duplicates from the given input. Note that huniq does not sort the input, it just removes duplicates.

Karolin Varner 189 Dec 27, 2022
locdev is a handy CLI tool that simplifies the process of adding, removing, and listing entries in the hosts file.

locdev ??️ locdev is a handy CLI tool that simplifies the process of adding, removing, and listing entries in the hosts file. You no longer need to de

Nick Rempel 20 Jun 5, 2023
Turns lines of text into SVG files.

Sentences 2 svg Does what it says on the tin. This takes in a file with some sentences and outputs numbered svgs. There are 3 arguments to keep in min

Fallen 4 Eyes 1 Jan 1, 2022
Command-line tool that provides a workflow for extending, editing, diffing, and writing to vim-style grep lines.

Grug Grug is a command-line tool that provides a workflow for expanding, editing, diffing, and writing edits to files using vim-styled grep lines (suc

null 4 Apr 25, 2023
Shell Of A New Machine: Quickly configure new environments

Shell Of A New Machine soanm is a dead-simple tool for easily configuring new UNIX machines, with almost zero prerequisites on the target machine. All

Ben Weinstein-Raun 41 Dec 22, 2022
A little tool to create region-free openingTitle.arc files for New Super Mario Bros. Wii, or to convert them from one region to another

smallworld ...though the mountains divide and the oceans are wide... smallworld is a little tool that can create region-free openingTitle.arc files fo

NSMBW Community 7 Feb 6, 2023
Neovim plugin for moving lines up and down, written in Rust

Moveline.nvim Moveline is a simple plugin for moving lines up and down. It's written in Rust using my library nvim-utils. Installation Moveline can be

Will Hopkins 34 Mar 18, 2023
🦀 Stupid simple presentation of the number of words, characters and lines on your clipboard.

clipcount: Counting words from the clipboard content Why does this exist? Do you find yourself often needing to count the number of words in a piece o

Magnus Rødseth 3 Feb 23, 2024
A fast, resilient, isomorphic hacker news clone in ~1k lines of rust.

Hackernews sauron A hacker news clone in ~1k lines of rust. This is using sauron web-framework. Feature Isomorphic Completely identical server-side re

Jovansonlee Cesar 102 Nov 19, 2022
a cargo subcommand for counting lines of code in Rust projects

cargo-count Linux: A cargo subcommand for displaying line counts of source code in projects, including a niave unsafe counter for Rust source files. T

Kevin K. 125 Dec 1, 2022