A series of compact encoding schemes for building small and fast parsers and serializers

Overview

compact-encoding

Rust implementation of

Usage

use compact_encoding::*;

let mut state = State::new();

// use pre_encode to determine the required buffer size
Uint8::pre_encode(&mut state, 42);
//String::pre_encode(&mut state, "hi");

println!("State: {:#?}", state);

// allocate the buffer based on previous pre_encode() calls
state.alloc();

// actually encode the values into the buffer
Uint8::encode(&mut state, 42);
//String::encode(&mut state, "hi");

// for decoding, the decode() method is used (state.start should point to start of buffer)
state.start = 0;

let uint8 = Uint8::decode(&mut state).expect("could not decode");
println!("{:?}", uint8);
//let string = String::decode(&mut state).expect("could not decode"));
//println!("{:?}", string);

Encoder API

State

See [State]

enc.preencode(state, val)

Does a fast preencode dry-run that only sets state.end. Use this to figure out the required buffer size. Afterwards call state.alloc(); to create state.buffer.

enc.encode(state, val)

Encodes val into state.buffer at position state.start. Updates state.start to point after the encoded value when done.

val = enc.decode(state)

Decodes a value from state.buffer at position state.start. Updates state.start to point after the decoded value when done in the buffer.

Helpers

If you are just encoding to a buffer or decoding from one you can use the encode and decode helpers to reduce your boilerplate

//const buf = cenc.encode(cenc.bool, true)
//const bool = cenc.decode(cenc.bool, buf)

Spec

Details on the encoding are documented in spec.md.

Design

Details on design ideas are documented in design.md.

License

MIT

You might also like...
A fast, performant implementation of skip list in Rust.
A fast, performant implementation of skip list in Rust.

Subway A fast, performant implementation of skip list in Rust. A skip list is probabilistic data structure that provides O(log N) search and insertion

Pure Rust port of CRFsuite: a fast implementation of Conditional Random Fields (CRFs)

crfs-rs Pure Rust port of CRFsuite: a fast implementation of Conditional Random Fields (CRFs) Currently only support prediction, model training is not

Rust library for reading/writing numbers in big-endian and little-endian.

byteorder This crate provides convenience methods for encoding and decoding numbers in either big-endian or little-endian order. Dual-licensed under M

Crate to parse and emit EDN

edn-rs Near Stable no breaking changes expected. Crate to parse and emit EDN This lib does not make effort to conform the EDN received to EDN Spec. Th

pem-rs pem PEM jcreekmore/pem-rs [pem] — A Rust based way to parse and encode PEM-encoded data

pem A Rust library for parsing and encoding PEM-encoded data. Documentation Module documentation with examples Usage Add this to your Cargo.toml: [dep

Decode SCALE bytes into custom types using a scale-info type registry and a custom Visitor impl.

scale-decode This crate attempts to simplify the process of decoding SCALE encoded bytes into a custom data structure given a type registry (from scal

Free Rust-only Xbox ADPCM encoder and decoder

XbadPCM Safe (and optionally no-std) Rust crate for encoding and decoding Xbox ADPCM blocks. Decoding example Here is example code for decoding stereo

Decode Mode S and ADS-B signals in Rust

rs1090 rs1090 is a Rust library to decode Mode S and ADS-B messages. It takes its inspiration from the Python pyModeS library, and uses deku in order

Easy c̵̰͠r̵̛̠ö̴̪s̶̩̒s̵̭̀-t̶̲͝h̶̯̚r̵̺͐e̷̖̽ḁ̴̍d̶̖̔ ȓ̵͙ė̶͎ḟ̴͙e̸̖͛r̶̖͗ë̶̱́ṉ̵̒ĉ̷̥e̷͚̍ s̷̹͌h̷̲̉a̵̭͋r̷̫̊ḭ̵̊n̷̬͂g̵̦̃ f̶̻̊ơ̵̜ṟ̸̈́ R̵̞̋ù̵̺s̷̖̅ţ̸͗!̸̼͋

Rust S̵̓i̸̓n̵̉ I̴n̴f̶e̸r̵n̷a̴l mutability! Howdy, friendly Rust developer! Ever had a value get m̵̯̅ð̶͊v̴̮̾ê̴̼͘d away right under your nose just when

A Rust library for building modular, fast and compact indexes over genomic data

mazu A Rust library for building modular, fast and compact indexes over genomic data Mazu (媽祖)... revered as a tutelary deity of seafarers, including

The rust client for CeresDB. CeresDB is a high-performance, distributed, schema-less, cloud native time-series database that can handle both time-series and analytics workloads.

The rust client for CeresDB. CeresDB is a high-performance, distributed, schema-less, cloud native time-series database that can handle both time-series and analytics workloads.

The Polkadot Hackathon Global Series North America edition is the second in a series of hackathons that brings the cutting edge of blockchain to a global community.

Polkadot Hackathon Project We are translating Uniswap v2 to Ink. Dependencies Install cargo-contract for building Ink contracts: cargo install dylint-

Databento Binary Encoding (DBZ) - Fast message encoding and storage format for market data

dbz A library (dbz-lib) and CLI tool (dbz-cli) for working with Databento Binary Encoding (DBZ) files. Python bindings for dbz-lib are provided in the

Small, compact music player written with Rust.

trill Small, compact music player based on rodio. Usage Run the program with --help to see the available options. To just play a sound file: trill -p

Rust library for HTTP authentication. Parses challenge lists, responds to Basic and Digest challenges. Likely to be extended with server support and additional auth schemes.

Rust library for HTTP authentication. Parses challenge lists, responds to Basic and Digest challenges. Likely to be extended with server support and a

Count and convert between different indexing schemes on utf8 string slices

Str Indices Count and convert between different indexing schemes on utf8 string slices. The following schemes are currently supported: Chars (or "Unic

A tool to deserialize data from an input encoding, transform it and serialize it back into an output encoding.

dts A simple tool to deserialize data from an input encoding, transform it and serialize it back into an output encoding. Requires rust = 1.56.0. Ins

A pure-Rust implementation of various threshold secret sharing schemes

Threshold Secret Sharing Efficient pure-Rust library for secret sharing, offering efficient share generation and reconstruction for both traditional S

A pure-Rust implementation of various threshold secret sharing schemes

Threshold Secret Sharing Efficient pure-Rust library for secret sharing, offering efficient share generation and reconstruction for both traditional S

Owner
Manfred Kröhnert
Manfred Kröhnert
Encoding and decoding support for BSON in Rust

bson-rs Encoding and decoding support for BSON in Rust Index Overview of BSON Format Usage BSON Values BSON Documents Modeling BSON with strongly type

mongodb 304 Dec 30, 2022
Variable-length signed and unsigned integer encoding that is byte-orderable for Rust

ordered-varint Provides variable-length signed and unsigned integer encoding that is byte-orderable. This crate provides the Variable trait which enco

Khonsu Labs 7 Dec 6, 2022
Astro Format is a library for efficiently encoding and decoding a set of bytes into a single buffer format.

Astro Format is a library for efficiently transcoding arrays into a single buffer and native rust types into strings

Stelar Labs 1 Aug 13, 2022
Implementation of Bencode encoding written in rust

Rust Bencode Implementation of Bencode encoding written in rust. Project Status Not in active developement due to lack of time and other priorities. I

Arjan Topolovec 32 Aug 6, 2022
A Gecko-oriented implementation of the Encoding Standard in Rust

encoding_rs encoding_rs an implementation of the (non-JavaScript parts of) the Encoding Standard written in Rust and used in Gecko (starting with Fire

Henri Sivonen 284 Dec 13, 2022
Character encoding support for Rust

Encoding 0.3.0-dev Character encoding support for Rust. (also known as rust-encoding) It is based on WHATWG Encoding Standard, and also provides an ad

Kang Seonghoon 264 Dec 14, 2022
A HTML entity encoding library for Rust

A HTML entity encoding library for Rust Example usage All example assume a extern crate htmlescape; and use htmlescape::{relevant functions here}; is

Viktor Dahl 41 Nov 1, 2022
A TOML encoding/decoding library for Rust

toml-rs A TOML decoder and encoder for Rust. This library is currently compliant with the v0.5.0 version of TOML. This library will also likely contin

Alex Crichton 1k Dec 30, 2022
Entropy Encoding notebook. Simple implementations of the "tANS" encoder/decoder.

EntropyEncoding Experiments This repository contains my Entropy Encoding notebook. Entropy encoding is an efficient lossless data compression scheme.

Nadav Rotem 4 Dec 21, 2022
TLV-C encoding support.

TLV-C: Tag - Length - Value - Checksum TLV-C is a variant on the traditional [TLV] format that adds a whole mess of checksums and whatnot. Why, you as

Oxide Computer Company 3 Nov 25, 2022