File Tree Fuzzer allows you to create a pseudo-random directory hierarchy filled with some number of files.

Related tags

Utilities ftzz
Overview

FTZZ

File Tree Fuzzer allows you to create a pseudo-random directory hierarchy filled with some number of files.

Installation

$ cargo +nightly install ftzz

Usage

Generate a reproducibly random tree in the current directory with about 1 million files:

$ ftzz g . -n 1M

Because FTZZ generates reproducible outputs, the generated directory will always have the same structure given the same inputs. To generate variations on a structure with the same parameters, add some entropy:

$ ftzz g . -n 1M --entropy $RANDOM

Other parameters can be found in the built-in docs:

$ ftzz help

Areas for improvement (a.k.a. limitations)

Exact # of files

FTZZ currently cannot generate an exact number of files. For my purposes, maximizing performance was the primary goal (FTZZ is extremely fast!) and I didn't need the outputs to be exact, but this could be an interesting feature.

I think the best way to implement it while still maximizing performance would be using a channel as an op queue: we precompute the target directory structure and then create the files/dirs. However, instead of doing this in two phases, we send a message to a coordinator thread each time we know the parameters for one directory which in turn spawns the blocking thread that actually creates files. The main complication is keeping track of dependencies so that create_dir doesn't fail, but I'm guessing it should be fine to use create_dir_all without causing a performance hit. The precomputing algorithm would stop if it reaches the target number of files, or sprinkle in the remaining files in randomly chosen directories (or maybe the leftovers could all be dumped in the root dir).

Comments
  • [Feature Request] Consider supporting option to fill files with specified data

    [Feature Request] Consider supporting option to fill files with specified data

    Consider an option (e.g. --fill-byte) that would fill files with the specified byte vs. random bytes. This would:

    1. potentially make generating the files even faster
    2. make the file content deterministic and compressible

    I have zero issues with the speed of ftzz as is. It's AMAZING! I'm requesting this for #2, but definitely curious about #1 Thank you!

    enhancement 
    opened by robm-gh 7
  • Update to public-api v0.21.0

    Update to public-api v0.21.0

    The dep bumps are (as you might remember) due to bumping cargo_toml to fix a crash (see https://github.com/Enselic/cargo-public-api/pull/186).

    Also bump rustdoc-json while we're at it.

    opened by Enselic 3
  • Build of 1.1.0 failed with cargo not finding cli-errors

    Build of 1.1.0 failed with cargo not finding cli-errors

    I tried to build and got the following errors:

    $ cargo install ftzz
        Updating crates.io index
      Installing ftzz v1.1.0
    error: failed to compile `ftzz v1.1.0`, intermediate artifacts can be found at `/tmp/cargo-installCV5Oh1`
    
    Caused by:
      no matching package named `cli-errors` found
      location searched: registry `crates-io`
      required by package `ftzz v1.1.0`
    

    I'm on:

    • CentOS Linux release 7.9.2009
    • rustc 1.64.0 (Red Hat 1.64.0-1.el7)
    • cargo 1.64.0
    opened by BaruchWeka 2
  • Look into supporting exact num files

    Look into supporting exact num files

    See https://github.com/SUPERCILEX/ftzz/tree/3cc6d10d3079775f6e5d7da4c22f86123e26c9f4#areas-for-improvement-aka-limitations

    Flag: -e, --files-exact, <num_files_exact>

    opened by SUPERCILEX 2
  • Purely use `rustdoc_json::Builder`

    Purely use `rustdoc_json::Builder`

    Looks like rustdoc_json hasn't sufficiently marked it's deprecated API as deprecated, because your code is more complicated than it needs to be :)

    (I'll try to improve on deprecation warnings in an upcoming rustdoc_json release)

    opened by Enselic 1
  • Download executables should be statically linked

    Download executables should be statically linked

    I was really hoping I wouldn't need to install the rust build env since there are downloadable executables but they are dynamically linked and the glibc you are using is likely newer than what I have.

    It would be nice to have statically linked executables to download and run.

    opened by BaruchWeka 0
  • Support generating random file contents

    Support generating random file contents

    Pass in total number of bytes to be split amongst generated files. Not respected exactly. Not counting metadata.

    Flag: -b, --total-bytes, <tree_size_bytes>

    opened by SUPERCILEX 0
  • Bump tokio from 1.23.0 to 1.23.1

    Bump tokio from 1.23.0 to 1.23.1

    Bumps tokio from 1.23.0 to 1.23.1.

    Release notes

    Sourced from tokio's releases.

    Tokio v1.23.1

    This release forward ports changes from 1.18.4.

    Fixed

    • net: fix Windows named pipe server builder to maintain option when toggling pipe mode (#5336).

    #5336: tokio-rs/tokio#5336

    Commits
    • 1a997ff chore: prepare Tokio v1.23.1 release
    • a8fe333 Merge branch 'tokio-1.20.x' into tokio-1.23.x
    • ba81945 chore: prepare Tokio 1.20.3 release
    • 763bdc9 ci: run WASI tasks using latest Rust
    • 9f98535 Merge remote-tracking branch 'origin/tokio-1.18.x' into fix-named-pipes-1.20
    • 9241c3e chore: prepare Tokio v1.18.4 release
    • 699573d net: fix named pipes server configuration builder
    • See full diff in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Fix the crappy probability distribution

    Fix the crappy probability distribution

    Problem: the number of files generated becomes proportionally unstable to the depth of the tree. That is, because we decide upon a target number of directories to generate up front and never update this number, if nodes closer to the root deviate significantly from the target, then the size of the tree will be completely bogus due to the exponential nature of its growth. Long winded way of saying: current dir gen number bad.

    I think this is fixable by dynamically adjusting the target number of directories generated. So if the parent node generated more dirs than expected, we'll take that into account and generate fewer directories such that the expected tree starting from the current root matches the remaining files to be generated according to our file to dir ratio distribution. Vice versa if too few dirs were generated.

    This should let us get rid of the log normal crap and widen the standard deviation to get more eccentric trees.


    Model with error/deviation from target and variance slack based on number of files generated from this sample.

    opened by SUPERCILEX 0
Releases(1.1.4)
Owner
Alex Saveau
All things 🔥base, Android, Gradle, and 🔓-source. Also, 🐤 builds.
Alex Saveau
A Rust library for random number generation.

A Rust library for random number generation.

null 1.3k Jan 6, 2023
Mount portable directory as consistent user directory.

PortableDesktop Mount portable directory as consistent user directory. PortableDesktopCli help PortableDesktopCli [options] <Target Path> <Link Path>

Kerwin Bryant 3 May 8, 2023
Automatically build a Rust module tree from the project directory structure

Hypermod An even lazier version of automod and supermod. Searches the src/ directory recursively for .rs files, then builds a module tree using the di

null 4 Aug 3, 2022
bevy_blender is a Bevy library that allows you to use assets created in Blender directly from the .blend file

bevy_blender bevy_blender is a Bevy library that allows you to use assets created in Blender directly from the .blend file.

Jerald Thomas 45 Jan 4, 2023
Rust library to scan files and expand multi-file crates source code as a single tree

syn-file-expand This library allows you to load full source code of multi-file crates into a single syn::File. Features: Based on syn crate. Handling

Vitaly Shukela 11 Jul 27, 2022
UnTeX is both a library and an executable that allows you to manipulate and understand TeX files.

UnTeX UnTeX is both a library and an executable that allows you to manipulate and understand TeX files. Usage Executable If you wish to use the execut

Jérome Eertmans 1 Apr 5, 2022
Count lines from files in a directory.

rust-cloc Count lines from files in a directory. Features Count the number of empty and non-empty lines in total from all files in a directory. Count

Daniel Liu 2 Apr 27, 2022
Create virtual serial ports, connect them to physical serial ports, and create routes between them all.

Virtual Serial Port Router (vsp-router) Create virtual serial ports, connect them to physical serial ports, and create routes between them all. vsp-ro

Rob Donnelly 3 Nov 24, 2022
Convert number like 42 to forty-two

num2words Convert number like 42 to forty-two Usage This crate can be either used as a library or a binary. Library Example usage: use num2words::Num2

Asperatus 5 Mar 30, 2022
1️⃣ el lisp number uno - one lisp to rule them all 🏆

luno el lisp number uno luno is the one lisp to rule them all. Still experimental, do not use it in production yet. goals embeddable small size simple

Eva Pace 3 Apr 25, 2022
A cli utility written in Rust that allows fetching all the labels of a project, save those as a YAML file

A cli utility written in Rust that allows fetching all the labels of a project, save those as a YAML file that you can easily edit or save as backup and apply a saved preset to new repositories.

Chevdor 4 May 5, 2022
This crate allows you to safely initialize Dynamically Sized Types (DST) using only safe Rust.

This crate allows you to safely initialize Dynamically Sized Types (DST) using only safe Rust.

Christofer Nolander 11 Dec 22, 2022
Twidge is a fresh approach to productivity. It integrates with your workflow and allows you to be your most productive self.

Twidge A productivity app which is an extension to your mind Twidge is a cross platform productivity app, powered by rust, tauri, prisma-client-rust T

Twidge 187 Jun 28, 2023
The lambda-chaos-extension allows you to inject faults into Lambda functions without modifying the function code.

Chaos Extension - Seamless, Universal & Lightning-Fast The lambda-chaos-extension allows you to inject faults into Lambda functions without modifying

AWS CLI Tools 5 Aug 2, 2023
Fusion is a cross-platform App Dev ToolKit build on Rust . Fusion lets you create Beautiful and Fast apps for mobile and desktop platform.

Fusion is a cross-platform App Dev ToolKit build on Rust . Fusion lets you create Beautiful and Fast apps for mobile and desktop platform.

Fusion 1 Oct 19, 2021
ᎩᎦᎨᎢ (IPA: [gigagei]) is a random quote fetching console utility. Written in Rust.

gigagei ᎩᎦᎨᎢ (IPA: [gigagei]) is a random quote fetching console utility. Written in Rust. Installing Use latest pre-built binary from releases Buildi

veleth 10 Jun 17, 2022
SubStrings, Slices and Random String Access in Rust

SubStrings, Slices and Random String Access in Rust This is a simple way to do it. Description Rust string processing is kind of hard, because text in

João Nuno Carvalho 2 Oct 24, 2021
A random.org client library for Rust

random.org A https://random.org client library. The randomness comes from atmospheric noise, which for many purposes is better than the pseudo-random

Victor Polevoy 7 Mar 10, 2022
A simple interpreter for the mathematical random-access machine

Random-access machine runner A simple Rust RAM program runner. Lexer/Parser Program executor Code formatter Web Compiled to WASM to run in the browser

Marcin Wojnarowski 5 Jan 14, 2023