Consider an option (e.g. --fill-byte) that would fill files with the specified byte vs. random bytes. This would:

1. potentially make generating the files even faster
2. make the file content deterministic and compressible

I have zero issues with the speed of ftzz as is. It's AMAZING! I'm requesting this for #2, but definitely curious about #1 Thank you!

The dep bumps are (as you might remember) due to bumping cargo_toml to fix a crash (see https://github.com/Enselic/cargo-public-api/pull/186).

Also bump rustdoc-json while we're at it.

I tried to build and got the following errors:

$ cargo install ftzz
    Updating crates.io index
  Installing ftzz v1.1.0
error: failed to compile `ftzz v1.1.0`, intermediate artifacts can be found at `/tmp/cargo-installCV5Oh1`

Caused by:
  no matching package named `cli-errors` found
  location searched: registry `crates-io`
  required by package `ftzz v1.1.0`

I'm on:

• CentOS Linux release 7.9.2009
• rustc 1.64.0 (Red Hat 1.64.0-1.el7)
• cargo 1.64.0

See https://github.com/SUPERCILEX/ftzz/tree/3cc6d10d3079775f6e5d7da4c22f86123e26c9f4#areas-for-improvement-aka-limitations

Flag: -e, --files-exact, <num_files_exact>

Looks like rustdoc_json hasn't sufficiently marked it's deprecated API as deprecated, because your code is more complicated than it needs to be :)

(I'll try to improve on deprecation warnings in an upcoming rustdoc_json release)

I was really hoping I wouldn't need to install the rust build env since there are downloadable executables but they are dynamically linked and the glibc you are using is likely newer than what I have.

It would be nice to have statically linked executables to download and run.

Pass in total number of bytes to be split amongst generated files. Not respected exactly. Not counting metadata.

Flag: -b, --total-bytes, <tree_size_bytes>

Bumps tokio from 1.23.0 to 1.23.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Problem: the number of files generated becomes proportionally unstable to the depth of the tree. That is, because we decide upon a target number of directories to generate up front and never update this number, if nodes closer to the root deviate significantly from the target, then the size of the tree will be completely bogus due to the exponential nature of its growth. Long winded way of saying: current dir gen number bad.

I think this is fixable by dynamically adjusting the target number of directories generated. So if the parent node generated more dirs than expected, we'll take that into account and generate fewer directories such that the expected tree starting from the current root matches the remaining files to be generated according to our file to dir ratio distribution. Vice versa if too few dirs were generated.

This should let us get rid of the log normal crap and widen the standard deviation to get more eccentric trees.

Model with error/deviation from target and variance slack based on number of files generated from this sample.