MASHI まし

A 100% no-std compatible Rust implementation of a PAQ style arithmetic coding, context mixing compressor. Its intended use case is compressing 64k demos, but whatever works for you!

Quick start

The main crate is mashi-core but I've provided a very bare-bones mashi-cli so you can test the compressor.

To build and use the CLI:

cargo build --release
target/release/mashi-cli compress tests/test.wasm
target/release/mashi-cli decompress tests/test.wasm.mashi

Cargo features

Use the encoder feature if you want to include the encoder
Use the std feature to enable std support (e.g. for running the unit tests)

Unscientific benchmarks

Compressing tests/test.wasm with various compressors:

Compressor	%	Size	Command-line
mashi	34.4%	12083	`mashi-cli compress`
zpaq	37.0%	13004	`zpaq a -m5`
xz	38.5%	13522	`xz --format=raw --lzma2=preset=9e`
zstd	41.3%	14536	`zstd --ultra -22`
gzip	44.5%	15629	`gzip --9`
original	100%	35151

License

Since the original PAQ compressors are GPL licensed be aware that Mashi is also GPL licensed.

Credits

The compression model is based on the PAQ work by Matt Mahoney et al.
The arithmetic coder is a port of the one in Crinkler, done by Mentor and Blueberry.
Wrapping my head around all of this was aided by the videos of Jeff Miller and the compression seminar by Ferris at Revision 2020.
Some of my porting efforts were eased by the work done by aufdj.

Rust-based toolset and library for Halo: Combat Evolved

Ringhopper Ringhopper is a pure-Rust library used for creating and manipulating tag data for Halo: Combat Evolved. It also contains the Invader toolse

4 Nov 19, 2023

The non-opinionated Rust-based commit message linter.

Documentation | Website git-sumi The non-opinionated Rust-based commit message linter Transform your commit practices with flexible linting for consis

6 Mar 1, 2024

Rust implementations of finite-field arithmetic based on big integers with configurable limb sizes

multiprecision TODO: rewrite readme implement CIOS for 16-bit limbs All operations are in little-endian form (where the digit in memory at the smalles

4 Mar 30, 2024

TodoX is a sophisticated Rust-based application designed to facilitate seamless todo management.

Rust Todo List App is a command-line tool written in Rust that allows users to manage their tasks efficiently. Whether you need to add, mark as done, edit, or clear tasks from your todo list, this app provides essential functionalities to streamline your task management process. Additionally, I have integrated sqlite3 using the rusqlite crate. The database stores the data and will persist indefinitely until you manually delete it.

3 Apr 4, 2024

Rust-based linux reverse shell listener.

Comments

Stream support

Would be very nice to be able to pipe data through this. Could you add this feature to your CLI? Also, it would be nice if you enhanced the CLI a bit, so it is usable in non-rust projects. Thanks!

opened by L3P3 0