A read-only, memory-mapped cache.

Overview

mmap-cache

A low-level API for a memory-mapped cache of a read-only key-value store.

Design

The [Cache] index is an [fst::Map], which maps from arbitrary byte sequences to [u64]. We use the [u64] as byte offsets into a memory-mapped file that stores arbitrary values. This read-only cache is very compact and simple to construct, requiring only constant memory for serializing arbitrarily large maps.

By using Finite State Transducers from the [fst] crate, we get a highly compressed mapping that performs key --> offset lookups in O(key length) time.

Example

use mmap_cache::{FileBuilder, MmapCache};

const INDEX_PATH: &str = "/tmp/mmap_cache_index";
const VALUES_PATH: &str = "/tmp/mmap_cache_values";

// Serialize to files. As required by the finite state transducer (FST) builder,
// keys must be provided in sorted (lexicographical) order.
let mut builder = FileBuilder::create_files(INDEX_PATH, VALUES_PATH)?;
builder.insert(b"abc", b"def")?;
builder.insert(b"foo", b"bar")?;
builder.finish()?;

// Map the files into memory.
let cache = unsafe { MmapCache::map_paths(INDEX_PATH, VALUES_PATH) }?;
let value = unsafe { cache.get_transmuted_value(b"foo") };
assert_eq!(value, Some(b"bar"));
You might also like...
Non-volatile, distributed file cache backed by content-addressed storage

blobnet A low-latency file server that responds to requests for chunks of file data. This acts as a non-volatile, over-the-network content cache. Inte

Rust type wrapper to cache hash of potentially large structures.

CachedHash For a type T, CachedHashT wraps T and implements Hash in a way that caches T's hash value. This is useful when T is expensive to hash (fo

Key-value cache RESP server with support for key expirations ⌛

BADER-DB (بادِر) Key-value cache RESP server with support for key expirations ⌛ Supported Features • Getting Started • Basic Usage • Cache Eviction •

A generational arena based LRU Cache implementation in 100% safe rust.

generational-lru Crate providing a 100% safe, generational arena based LRU cache implementation. use generational_lru::lrucache::{LRUCache, CacheError

Attribute-Level Caching in Heterogeneous In-Memory DBMS

Alchemy Attribute-Level Caching in Heterogeneous In-Memory DBMS Alchemy is a DRAM-PM hybrid database engine built from scratch to achieve high perform

memory-mapped registers for x86_64 systems

regmap some well-known and known-to-be-good computer architectures, such as the Microchip PIC product line, or many of the AVR processor family, were

Rust library for concurrent data access, using memory-mapped files, zero-copy deserialization, and wait-free synchronization.

mmap-sync mmap-sync is a Rust crate designed to manage high-performance, concurrent data access between a single writer process and multiple reader pr

Read-optimized cache of Cardano on-chain entities
Read-optimized cache of Cardano on-chain entities

Read-optimized cache of Cardano on-chain entities Intro Scrolls is a tool for building and maintaining read-optimized collections of Cardano's on-chai

(Read-only) Generate n-grams

N-grams Documentation This crate takes a sequence of tokens and generates an n-gram for it. For more information about n-grams, check wikipedia: https

Check the reproducibility status of your Arch Linux packages (read-only mirror)
Check the reproducibility status of your Arch Linux packages (read-only mirror)

arch-repro-status A CLI tool for querying the reproducibility status of the Arch Linux packages using data from a rebuilderd instance such as reproduc

EncyroHop an open source external CS2 read only kernel gameplay enhancer.

Download - Latest Release Encyro is a robust open-source external CS2 (Counter-Strike 2) read-only kernel gameplay enhancer, designed to augment the g

An object-relational in-memory cache, supports queries with an SQL-like query language.

qlcache An object-relational in-memory cache, supports queries with an SQL-like query language. Warning This is a rather low-level library, and only p

Key-Value based in-memory cache library which supports Custom Expiration Policies

Endorphin Key-Value based in-memory cache library which supports Custom Expiration Policies with standard HashMap, HashSet interface. use endorphin::H

Stretto is a Rust implementation for ristretto. A high performance memory-bound Rust cache.

Stretto is a Rust implementation for ristretto. A high performance memory-bound Rust cache.

A general-purpose distributed memory cache system compatible with Memcached

memcrsd memcached server implementation in Rust memcrsd is a key value store implementation in Rust. It is compatible with binary protocol of memcache

ConstDB - an in-memory cache store which aims at master-master replications

A redis-like cache store that implements CRDTs and active-active replications.

memory-profiler — A memory profiler for Linux
memory-profiler — A memory profiler for Linux

A memory profiler for Linux Features Can be used to analyze memory leaks, see where exactly the memory is being consumed, identify temporary allocatio

Custom memory allocator that helps discover reads from uninitialized memory

libdiffuzz: security-oriented alternative to Memory Sanitizer This is a drop-in replacement for OS memory allocator that can be used to detect uses of

Custom memory allocator that helps discover reads from uninitialized memory

libdiffuzz: security-oriented alternative to Memory Sanitizer This is a drop-in replacement for OS memory allocator that can be used to detect uses of

Owner
Duncan
Duncan
Key-Value based in-memory cache library which supports Custom Expiration Policies

Endorphin Key-Value based in-memory cache library which supports Custom Expiration Policies with standard HashMap, HashSet interface. use endorphin::H

Jun Ryung Ju 15 Oct 1, 2022
Stretto is a Rust implementation for ristretto. A high performance memory-bound Rust cache.

Stretto is a Rust implementation for ristretto. A high performance memory-bound Rust cache.

Al Liu 310 Dec 29, 2022
A general-purpose distributed memory cache system compatible with Memcached

memcrsd memcached server implementation in Rust memcrsd is a key value store implementation in Rust. It is compatible with binary protocol of memcache

null 274 Dec 14, 2022
ConstDB - an in-memory cache store which aims at master-master replications

A redis-like cache store that implements CRDTs and active-active replications.

null 27 Aug 15, 2022
Rust cache structures and easy function memoization

cached Caching structures and simplified function memoization cached provides implementations of several caching structures as well as a handy macro f

James Kominick 996 Jan 7, 2023
This is a Rust implementation for HashiCorp's golang-lru. This crate contains three LRU based cache, LRUCache, TwoQueueCache and AdaptiveCache.

This is a Rust implementation for HashiCorp's golang-lru. This crate contains three LRU based cache, LRUCache, TwoQueueCache and AdaptiveCache.

Al Liu 84 Jan 3, 2023
A native stateless cache implementation.

fBNC fBNC, Blockchain Native Cache. A native stateless storage library for block chain. Its value is to improve the stability and security of online s

Findora Foundation 1 Jan 12, 2022
A lightweight key-value cache system developed for experimental purposes

A lightweight key-value cache system developed for experimental purposes. It can also include distributed systems setup if I can.

Burak Selim Senyurt 8 Jul 23, 2022
A set of safe Least Recently Used (LRU) map/cache types for Rust

LruMap A set of safe Least-Recently-Used (LRU) cache types aimed at providing flexible map-like structures that automatically evict the least recently

Khonsu Labs 4 Sep 24, 2022
Turn your discord cache back to viewable images.

discache ?? Every time you view an Image in Discord, it gets saved in your cache folder as an unviewable file. Discache allows you to convert those fi

sam 2 Dec 14, 2022