A (aspiring to be fast) tool to find duplicate files.

Overview

find-duplicates

A decenly fast tool to find duplicate files. Handles symbolic and hard links and treats them seperately to duplicates.

Quickstart

Install rust and use cargo to compile the program.

$ cargo build --release

NOTE: it's important to use --release mode, because it will be noticably slower without optimizations.

use the program to find some duplicates:

$ ./target/release/find-duplicates -r ~/some-dir-with-maybe-duplicate-files

Help

USAGE: find-duplicates [flags] <input>
  where [flags] can be 0 or more of the following:
    -r, --recursive      include files in subdirectories,
                         search recursively.

    -v, --verbose        enable progress bars and other
                         extra output. cannot be used with
                         -q, --quiet.

    -q, --quiet          disable all non-essential output,
                         good for redirecting to files or
                         piping to other programs. cannot
                         be used with -v, --verbose

    -h, --help           print this message.

  and where <input> is a path to a directory.
You might also like...
Save cli commands and fuzzy find them later
Save cli commands and fuzzy find them later

crow - cli command memorizer What is crow? | Installation | Usage | FAQ What is crow? crow (command row) is a CLI tool to help you memorize CLI comman

Eslint - Find and fix problems in your JavaScript code.
Eslint - Find and fix problems in your JavaScript code.

ESLint Website | Configuring | Rules | Contributing | Reporting Bugs | Code of Conduct | Twitter | Mailing List | Chat Room ESLint is a tool for ident

🧠 A command-line utility for switching git branches more easily. Switch branches interactively or use a fuzzy search to find that long-forgotten branch name.
🧠 A command-line utility for switching git branches more easily. Switch branches interactively or use a fuzzy search to find that long-forgotten branch name.

git-smart-checkout A git command extension for switching git branches more efficiently. About Interactively switch branches or fuzzy search for that f

procs makes it easy to find and manage system processes

procs procs makes it easy to find and manage system processes. Right now, the main usage is finding processes by the ports it is listening on, but mor

Semantic find-and-replace using tree-sitter-based macro expansion

Semantic find-and-replace using tree-sitter-based macro expansion

try to find the correct word with only first letter and unknown letter count.

MOTUS Current dictionaries are provided in french and can contain some words not included in the official Motus dictionary. Additionally, dictionaries

Find and clean heavy build or cache directories.
Find and clean heavy build or cache directories.

ProjClean Find and clean heavy build or cache directories. ProjClean finds directories such as node_modules(node), target(rust), build(java) and their

Help Skelly to find bones, combine them to build his body back
Help Skelly to find bones, combine them to build his body back

Bones Collector Help Skelly to find bones, combine them to build his body back! Game made for the bevy Jam#2. Play it here in your browser: itch.io! R

Quickly find all blackhole directories with a huge amount of filesystem entries in a flat structure
Quickly find all blackhole directories with a huge amount of filesystem entries in a flat structure

findlargedir About Findlargedir is a tool specifically written to help quickly identify "black hole" directories on an any filesystem having more than

Comments
  • Make it faster!

    Make it faster!

    Though the rust version is faster than the python one, It's still way too slow, especially when comparing big files. Add in an incremental stage to reduce the amount of full checksums we need to compute as much as possible.

    opened by vintium 0
  • Add output options

    Add output options

    We wanna be able to have an output file, because when you have 1000s of files the terminal output is just way too big. also that file could be in a custom format (which would be documented and easy to parse) so that it could be used by other programs. Don't eliminate writing to stdout though, because it's convenient. Make it so you have to choose with an option whether you want to write to a file or stdout. Something like -o <file> for output to file, -s for output to stdout.

    opened by vintium 0
Owner
dylan
hi! (he they)
dylan
Filter, Sort & Delete Duplicate Files Recursively

Deduplicator Find, Sort, Filter & Delete duplicate files Usage Usage: deduplicator [OPTIONS] [scan_dir_path] Arguments: [scan_dir_path] Run Dedupl

Sreedev Kodichath 108 Jan 27, 2023
fd is a program to find entries in your filesystem. It is a simple, fast and user-friendly alternative to find

fd is a program to find entries in your filesystem. It is a simple, fast and user-friendly alternative to find. While it does not aim to support all of find's powerful functionality, it provides sensible (opinionated) defaults for a majority of use cases.

David Peter 25.9k Jan 9, 2023
fas stand for Find all stuff and it's a go app that simplify the find command and allow you to easily search everything you nedd

fas fas stands for Find all stuff and it's a rust app that simplify the find command and allow you to easily search everything you need. Note: current

M4jrT0m 1 Dec 24, 2021
A tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK.

FileQL - File Query Language FileQL is a tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK. Sampl

Amr Hesham 39 Mar 12, 2024
A CLI tool to find the dominant colours in an image 🎨

dominant_colours This is a command-line tool for finding the dominant colours of an image. It prints their hex codes to the terminal, along with a pre

Alex Chan 65 Nov 10, 2022
Tool that was built quickly for personal needs, you might find it useful though

COPYCAT Produced with stable-diffusion Clipboard (copy/paste) history buffer for terminal emulators, MAC OS gui and VIM* environment usage. Rrequireme

Dragan Jovanović 4 Dec 9, 2022
A CLI tool connected to GPT-3 to help find the right terminal command

Command Recall A CLI tool connected to GPT-3 to help find the right terminal command Install to install the cli: cargo install --git https://github.co

Camille Moatti 102 Feb 18, 2023
Quick, lightweight find and replace cli tool.

quick-replace A fast, lightweight find and replace tool Usage quick-replace [OPTIONS] <FROM> <TO> <path/to/file.txt> Options Flag Description -h Displ

Liam Watts 3 Apr 7, 2023
Voila is a domain-specific language launched through CLI tool for operating with files and directories in massive amounts in a fast & reliable way.

Voila is a domain-specific language designed for doing complex operations to folders & files. It is based on a CLI tool, although you can write your V

Guillem Jara 86 Dec 12, 2022
Intuitive find & replace CLI (sed alternative)

sd - s[earch] & d[isplace] sd is an intuitive find & replace CLI. The Pitch Why use it over any existing tools? Painless regular expressions sd uses r

Gregory 4k Jan 4, 2023