Simple async codec for rkyv. Reuses streaming buffer for maximum speed

Related tags

Utilities rkyv_codec
Overview

rkyv_codec

docs.rs crates.io

Simple async codec for rkyv. Reuses streaming buffer for maximum speed!

This crate provides a makeshift adaptor for streaming &Archiveds from an AsyncRead using a reusable external buffer, as well as a futures::Sink implementation to serialize Objects to an AsyncWrite. It uses multiformat's unsigned_varint for variable-length length encoding by default, but allows for other kinds of length encoding through the LengthEncoding trait.

Examples

This crate has two examples: chat_client & chat_server. Run both of them at the same time to see a proof-of-concept Archive tcp stream in action.

To run:

cargo run --example chat_client

cargo run --example chat_server

Simple usage example:

::new(writer); codec.send(&value).await.unwrap(); // Reading let mut reader = &codec.inner()[..]; let mut buffer = AlignedVec::new(); // Aligned streaming buffer for re-use let data: &Archived = archive_stream::<_, Test, VarintLength>(&mut reader, &mut buffer).await.unwrap(); // This returns a reference into the passed buffer let value_received: Test = data.deserialize(&mut Infallible).unwrap(); assert_eq!(value, value_received);"> 
#[derive(Archive, Deserialize, Serialize, Debug, PartialEq, Clone)]
#[archive_attr(derive(CheckBytes, Debug))] // Checkbytes is required
struct Test {
    int: u8,
    string: String,
    option: Option<Vec<i32>>,
}
let value = Test {
    int: 42,
    string: "hello world".to_string(),
    option: Some(vec![1, 2, 3, 4]),
};
// Writing
let writer = Vec::new();
let mut codec = RkyvWriter::<_, VarintLength>::new(writer);
codec.send(&value).await.unwrap();
// Reading
let mut reader = &codec.inner()[..];
let mut buffer = AlignedVec::new(); // Aligned streaming buffer for re-use
let data: &Archived<Test> = archive_stream::<_, Test, VarintLength>(&mut reader, &mut buffer).await.unwrap(); // This returns a reference into the passed buffer
let value_received: Test = data.deserialize(&mut Infallible).unwrap();
assert_eq!(value, value_received);

Benchmarks

These are a set of benchmarks, each benchmark represents 50 test objects being either sent or received.

test tests::bench_archive_sink_50              ... bench:      10,388 ns/iter (+/- 1,603)
test tests::bench_archive_sink_prearchived_50  ... bench:       2,032 ns/iter (+/- 302)
test tests::bench_archive_stream_50            ... bench:       3,544 ns/iter (+/- 438)
test tests::bench_futures_cbor_sink_50         ... bench:      14,105 ns/iter (+/- 1,160)
test tests::bench_futures_cbor_stream_50       ... bench:       9,632 ns/iter (+/- 1,437)
test tests::bench_rkyv_futures_codec_sink_50   ... bench:       6,671 ns/iter (+/- 521)
test tests::bench_rkyv_futures_codec_stream_50 ... bench:       4,925 ns/iter (+/- 948)
test tests::bench_rkyv_writer_50               ... bench:       3,547 ns/iter (+/- 271)
test tests::bench_u64_length_encoding          ... bench:       4,226 ns/iter (+/- 327)
test tests::bench_varint_length_encoding       ... bench:       4,243 ns/iter (+/- 343)

The fastest real benchmark (full serialization and bytecheck) is using RkyvWriter for writing and archive_stream for reading. This is compared to the slowest benchmark: futures_codec library's CborCodec.

Feel free to contribute your own benchmarks!

You might also like...
single file, std only, async Rust executor

whorl - A single file, std only, async Rust executor whorl was created to teach you how async executors work in Rust. It is not the fastest executor n

Michael Gattozzi 459 Dec 29, 2022
BPF library for Async Rust, complementary for libbpf-rs.

libbpf-async A library for writing BPF programs in Async Rust, complementary for libbpf-rs, Rust wrapper for libbpf. Currently, this provides Async-fr

FUJITA Tomonori 13 Nov 9, 2022
Async `TryFrom/TryInto` traits

async-convert Async TryFrom/TryInto traits API Docs | Releases | Contributing Installation $ cargo add async-convert Safety This crate uses #![deny(un

Yosh 4 Mar 4, 2022
You can name anonymous Future from async fn without dyn or Box!

rename-future You can name anonymous Future from async fn without dyn or Box! PLEASE READ THIS THIS PROJECT NOT YET WELL TESTED! DON'T USE THIS IN PRO

Jun Ryung Ju 54 Sep 17, 2022
The most fundamental type for async synchronization: an intrusive linked list of futures

wait-list This crate provides WaitList, the most fundamental type for async synchronization. WaitList is implemented as an intrusive linked list of fu

Sabrina Jewson 7 Oct 26, 2022
Ector is an open source async, no-alloc actor framework for embedded devices

Ector is an open source async, no-alloc actor framework for embedded devices. Ector is an open source async, no-alloc actor framework for embedded dev

Drogue IoT 11 Dec 15, 2022
Async variant of Tonic's `interceptor` function

Tonic Async Interceptor This crate contains AsyncInterceptor, an async variant of Tonic's Interceptor. Other than accepting an async interceptor funct

Arcanyx Technical Wizardry LLC 4 Dec 20, 2022
Use winit like the async runtime you've always wanted

async-winit Use winit like the async runtime you've always wanted. winit is actually asynchronous, contrary to popular belief; it's just not async. It

John Nunley 17 Apr 23, 2023
Rust library for one-time async initialization

async-lazy An async version of once_cell::sync::Lazy, std::sync::OnceLock or lazy_static. Uses tokio's sychronization primitives. This crate offers an

Jules Bertholet 5 Jun 9, 2023
Comments
  • Fix varint decoding with very small archive sizes

    Fix varint decoding with very small archive sizes

    When the archived data is very small (< 9 bytes) the total buffer length will be too small to fill the buffer for the length prefix. This change copies as many bytes as possible into the length prefix decoding slice, then decodes the length prefix.

    opened by djkoloski 0
Releases(v0.3.0)
Owner
Zyansheep
Zyansheep
GitHub
A high-performance SPSC bounded circular buffer of bytes

Cueue A high performance, single-producer, single-consumer, bounded circular buffer of contiguous elements, that supports lock-free atomic batch opera

Thaler Benedek 38 Dec 28, 2022
Combine internet connections, increase your download speed

dispatch A SOCKS proxy that balances traffic between network interfaces. Should work on macOS, Windows, and Linux. Only tested on macOS for now. This

Alexandre Kirszenberg 90 Jan 7, 2023
Port of the fantastic Iconoir Icon Pack to Rust embedded devices, with a focus on speed, usability, and completeness.

embedded-iconoir - Icons for every device, ever. What is embedded-iconor? embedded-iconoir is a library that allows you to use Iconoir on embedded dev

null 9 Mar 25, 2023
Github mirror of codeberg repo. Monitor live bandwidth usage/ network speed on PC. Native version also available for Android, separately.

Netspeed Monitor Netspeed is a cross-platform desktop application that shows the live upload speed, download speed and day's usage as an overlay. Feat

Vishnu N K 16 May 3, 2023
Async executor for WebAssembly

There are a number of async task executors available in Rust's ecosystem. However, most (if not all?) of them rely on primitives that might not be available or optimal for WebAssembly deployment at the time.

wasm.rs 65 Dec 31, 2022
The feature-rich, portable async channel library

The feature-rich, portable async channel library > crates.io > docs.rs Why use Postage? Includes a rich set of channels. | barrier | broadcast | dispa

Austin Jones 221 Dec 26, 2022
Mix async code with CPU-heavy thread pools using Tokio + Rayon

tokio-rayon Mix async code with CPU-heavy thread pools using Tokio + Rayon Resources Documentation crates.io TL;DR Sometimes, you're doing async stuff

Andy Barron 74 Jan 2, 2023
Another Async IO Framework based on io_uring

kbio, the Async IO Framework based on io_uring, is used in KuiBaDB to implement async io. Features Support multi-threading concurrent task submission.

KuiBaDB 59 Oct 31, 2022
async-alloc-counter measures max allocations in a future invocation

async-alloc-counter measures max allocations in a future invocation see examples/ for usage This allocator can be used as follows: use async_alloc_cou

Geoffroy Couprie 2 Dec 3, 2021
Golang like WaitGroup implementation for sync/async Rust.

wg Golang like WaitGroup implementation for sync/async Rust.

Al Liu 8 Dec 6, 2022