This is a lib and not a CLI, and can support basic known instantiations of index types. e.g. pufferfish + piscem. ModIndex should report / store only basic information, and have the info be split corresponding to k2u and u2pos data structures. CLIs that use mazu and need fine grain control over metadata should use the newtype idiom i.e. type MyIndex(ModIndex<...>) and handle metadata / provenance info as appropriate for MyIndex.

Use similar abstraction as ModIndex and write a struct like:

ModCCIndex<H: K2U, T> {
    k2u: H,
    colors: T,
    u2cc: BitVector,
}

or

ModCCIndex<H: K2U, T> {
    k2u: H,
    u2color: T, // containing bit-vector
}

And make it easy to implement / try different color compression schemes.

Current implementation has a pufferfish style hash for k-mers with minimizers that occur more than some threshold number of times. Which is different from the actual SSHash skew index that is more optimized.

• [ ] Point hits to skew index to super-k-mer (minimizer) IDs instead of positions on global unitig sequence. This would use fewer bits per k-mer in skew index. Current impl points k-mers directly to a position on useq.
• [ ] Use a level-by-level bucketing scheme. Current implementation uses one single bucket for all skew k-mers.
• [ ] #3

To-dos:

Must

• [ ] Basic usage / lib documentation (use rustdoc)
• [ ] Stabilize upstream dependencies, i.e., merge thejasonfan form of kme.rs back to combine-lab
• [ ] fix or remove num_bits (size on heap) measurements
• [ ] Note new canonical minimimizer definition

Nice-to-haves

• [ ] pufferfish2 sparse unitig table sampling
• [ ] RefSeqCollection with PolyNs. We can build from FASTA files but we can't 2-bit encode and represent RefSeqs with PolyN stretches right now.
• [ ] Officially supported modular serialization so constituent index components are easy to measure. But for now, we can let CLIs that use mazu handle provenance and serialization formats.

New algorithms / features

• [ ] Standardized index info as JSON/YAML for supported "default / official" indexes.
• [ ] Generic (or more boilerplate) for K2U implementations to cache and to skip rank/select queries (to retrieve unitig length and start position) for consecutive k-mers mapping to the same unitig.

Extras:

• [ ] Pufferfish (C++) to mazu conversion for Sparse and Dense indexes.
• [ ] #2
• [ ] Basic benchmarks for some atomic data structures EF, rank/sel, WM etc.
• [ ] #5

Benchmarking / Questions:

• [ ] Compare mazu PolyN handling vs. pufferfish's pseudorandom insertion for Ns
• [ ] Note and benchmark differences to SSHash (canonical minimizer parsing + a more naive skew index)