A general-purpose distributed memory cache system compatible with Memcached

Last update: Dec 14, 2022

Overview

memcrsd memcached server implementation in Rust

memcrsd is a key value store implementation in Rust. It is compatible with binary protocol of memcached server.

Supported features and compatibility

To check compatibility with memcached server implementation memcrsd project is using memcapable tool from libmemcached library

Here is a capability status for memcrsd:

./memcapable -h 127.0.0.1  -b -p 11211                 
Hostname was provided.127.0.0.1
binary noop                             [pass]
binary quit                             [pass]
binary quitq                            [pass]
binary set                              [pass]
binary setq                             [pass]
binary flush                            [pass]
binary flushq                           [pass]
binary add                              [pass]
binary addq                             [pass]
binary replace                          [pass]
binary replaceq                         [pass]
binary delete                           [pass]
binary deleteq                          [pass]
binary get                              [pass]
binary getq                             [pass]
binary getk                             [pass]
binary getkq                            [pass]
binary incr                             [pass]
binary incrq                            [pass]
binary decr                             [pass]
binary decrq                            [pass]
binary version                          [pass]
binary append                           [pass]
binary appendq                          [pass]
binary prepend                          [pass]
binary prependq                         [pass]
binary stat                             [pass]
All tests passed

Bug reports

Feel free to use the issue tracker on github.

If you are reporting a security bug please contact a maintainer privately. We follow responsible disclosure: we handle reports privately, prepare a patch, allow notifications to vendor lists. Then we push a fix release and your bug can be posted publicly with credit in our release notes and commit history.

Website

https://memc.rs/

Testing

memcrsd project is tested using different types of tests:

unit testing,
fuzzy testing,
end-2-end tests

Fuzzy testing

At the moment decoding network packets is fuzzy tested.

Generating test coverage reporting

To generate test coverage there is a convenient shell script that compiles and executed unit tests:

cd memcrs
./coverage.sh
firefox ../target/debug/coverage/index.html

The plan in the future is to have coverage ~90%.

Integration testing

For end-to-end integration testing at the moment memcrsd project is using memcapable tool from libmemcache library. In the future memclt binary will be used to perform end-to-end tests.

Measuring performance

Measuring performance can be tricky, thats why to measure performance memcrsd project is using industry standard benchmarking tool for measuring performance of memcached server which is memtier_benchmark. This tool can be used to generate various traffic patterns. It provides a robust set of customization and reporting capabilities all wrapped into a convenient and easy-to-use command-line interface. More information about memtier benchmark tool can be found on RedisLabs blog.

Memtier benchmark installation

Memtier benchmark is available on github, it needs to be cloned and compiled:

git clone https://github.com/RedisLabs/memtier_benchmark.git
autoreconf -ivf
./configure
make
make install

Generating flamegraph

To be able to sample and resolve Kernel functions:

we need to expose kernel addresses(kptr_restrict)
grant unprivileged users access to performance events in kernel(perf_event_paranoid)

sudo sh -c " echo 0 > /proc/sys/kernel/kptr_restrict"
sudo sh -c " echo -1 >> /proc/sys/kernel/perf_event_paranoid"

Generating flamegraphs:

sudo apt install -y linux-tools-common linux-tools-generic
cargo install flamegraph
cd ~/projects/memix
cargo flamegraph
cargo flamegraph --bin memcrsd

Attaching perf

By default release profile is built with debug symbols, see Cargo.toml:

[profile.release]
opt-level = 3
debug=true

On Ubuntu install required packages:

sudo apt install -y linux-tools-common linux-tools-generic

We can start a server:

./target/release/memcrsd -v & perf record -F 99 -p `pgrep memcrsd`

First we run the memcrsd server and we send it to the background using the ampersand (&) symbol.
Next to it, so it executes immediately, we run perf that receives the process identifier (PID) courtesy of pgrep memcrsd.
The pgrep command returns the PID of a process by name.

After benchmarks are executed reports can be displayed using perf report:

perf report

You might also like...

Read-optimized cache of Cardano on-chain entities

Read-optimized cache of Cardano on-chain entities Intro Scrolls is a tool for building and maintaining read-optimized collections of Cardano's on-chai

58 Dec 2, 2022

Turn your discord cache back to viewable images.

discache 🦀 Every time you view an Image in Discord, it gets saved in your cache folder as an unviewable file. Discache allows you to convert those fi

2 Dec 14, 2022

Rust type wrapper to cache hash of potentially large structures.

CachedHash For a type T, CachedHashT wraps T and implements Hash in a way that caches T's hash value. This is useful when T is expensive to hash (fo

3 Dec 8, 2022

Key-value cache RESP server with support for key expirations ⌛

BADER-DB (بادِر) Key-value cache RESP server with support for key expirations ⌛ Supported Features • Getting Started • Basic Usage • Cache Eviction •

7 Apr 21, 2023

A generational arena based LRU Cache implementation in 100% safe rust.

generational-lru Crate providing a 100% safe, generational arena based LRU cache implementation. use generational_lru::lrucache::{LRUCache, CacheError

37 Dec 21, 2022

Hitbox is an asynchronous caching framework supporting multiple backends and suitable for distributed and for single-machine applications.

62 Dec 27, 2022

Attribute-Level Caching in Heterogeneous In-Memory DBMS

Alchemy Attribute-Level Caching in Heterogeneous In-Memory DBMS Alchemy is a DRAM-PM hybrid database engine built from scratch to achieve high perform

14 Jun 20, 2022

🍋: A General Lock following paper "Optimistic Lock Coupling: A Scalable and Efficient General-Purpose Synchronization Method"

Optimistic Lock Coupling from paper "Optimistic Lock Coupling: A Scalable and Efficient General-Purpose Synchronization Method" In actual projects, th

22 Oct 13, 2022

General purpose memory allocator written in Rust.

Memalloc Memory allocator written in Rust. It implements std::alloc::Allocator and std::alloc::GlobalAlloc traits. All memory is requested from the ke

35 Dec 25, 2022

Keyhouse is a skeleton of general-purpose Key Management System written in Rust.

Keyhouse Keyhouse is a skeleton of general-purpose Key Management System. Keyhouse is not an off-the-shelf system, and it's not ready for production.

148 Jan 1, 2023

A simplified general-purpose queueing system for Rust apps.

A simplified general-purpose queueing system for Rust apps. Example // Create a new Redeez object, and define your queues let mut queues = Redeez::new

11 Jan 16, 2023

Cleora AI is a general-purpose model for efficient, scalable learning of stable and inductive entity embeddings for heterogeneous relational data.

Cleora Cleora is a genus of moths in the family Geometridae. Their scientific name derives from the Ancient Greek geo γῆ or γαῖα "the earth", and metr

405 Dec 20, 2022

🍖 ham, general purpose programming language

🍖 ham, a programming language made in rust status: alpha Goals Speed Security Comfort Example fn calc(value){ if value == 5 { return 0

19 Nov 10, 2022

A feature-rich, production-ready, general purpose 2D/3D game engine written in Rust with a scene editor.

5.4k Jan 4, 2023

General-purpose asynchronous socket stream.

async-socket This crate implements a general-purpose asynchronous socket. The Socket implements AsyncRead, AsyncWrite, Stream and Clone traits and thu

3 Oct 20, 2021

Gecko is a high-level, general-purpose programming language built on top of the LLVM project.

Gecko is a high-level, general-purpose programming language built on top of the LLVM project. Gecko Technology & principles Gecko is a general-purpose

19 Oct 3, 2022

Ethereal - a general-purpose programming language that is designed to be fast and simple

Ethereal is a general-purpose programming language that is designed to be fast and simple. Heavly inspired by Monkey and written in Rust

21 Nov 25, 2022

A general-purpose, transactional, relational database that uses Datalog and focuses on graph data and algorithms

cozo A general-purpose, transactional, relational database that uses Datalog for query and focuses on graph data and algorithms. Features Relational d

1.9k Jan 9, 2023

General purpose client/server networking library written in Rust, built on top of the QUIC protocol which is implemented by quinn

Overview "This library stinks!" ... "Unless you like durian" durian is a client-server networking library built on top of the QUIC protocol which is i

92 Dec 31, 2022

Comments

make clippy happy

[N] ❯ cargo clippy
    Checking memcrs v0.0.1 (/Users/jun/Documents/workspace/memc-rs/memcrs)
warning: unused import: `super::*`
 --> memcrs/src/protocol/binary_codec/binary_encoder_tests.rs:1:5
  |
1 | use super::*;
  |     ^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

warning: you should consider adding a `Default` implementation for `MemcacheStoreBuilder`
  --> memcrs/src/memcache/builder.rs:24:5
   |
24 | /     pub fn new() -> MemcacheStoreBuilder {
25 | |         MemcacheStoreBuilder {}
26 | |     }
   | |_____^
   |
   = note: `#[warn(clippy::new_without_default)]` on by default
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#new_without_default
help: try adding this
   |
23 + impl Default for MemcacheStoreBuilder {
24 +     fn default() -> Self {
25 +         Self::new()
26 +     }
27 + }
   |

opened by junnplus 0

Releases(memcrsd-0.0.1f)

Owner

A key-value memory cache written in Rust and compatible with memcached.

GitHub https://memc.rs

Non-volatile, distributed file cache backed by content-addressed storage

blobnet A low-latency file server that responds to requests for chunks of file data. This acts as a non-volatile, over-the-network content cache. Inte

12 Dec 31, 2022

Key-Value based in-memory cache library which supports Custom Expiration Policies

Endorphin Key-Value based in-memory cache library which supports Custom Expiration Policies with standard HashMap, HashSet interface. use endorphin::H

15 Oct 1, 2022

Stretto is a Rust implementation for ristretto. A high performance memory-bound Rust cache.

310 Dec 29, 2022

ConstDB - an in-memory cache store which aims at master-master replications

A redis-like cache store that implements CRDTs and active-active replications.

27 Aug 15, 2022

A read-only, memory-mapped cache.

mmap-cache A low-level API for a memory-mapped cache of a read-only key-value store. Design The [Cache] index is an [fst::Map], which maps from arbitr

3 Jun 28, 2022

A lightweight key-value cache system developed for experimental purposes

A lightweight key-value cache system developed for experimental purposes. It can also include distributed systems setup if I can.

8 Jul 23, 2022

Rust cache structures and easy function memoization

cached Caching structures and simplified function memoization cached provides implementations of several caching structures as well as a handy macro f

996 Jan 7, 2023

This is a Rust implementation for HashiCorp's golang-lru. This crate contains three LRU based cache, LRUCache, TwoQueueCache and AdaptiveCache.

84 Jan 3, 2023

A native stateless cache implementation.

fBNC fBNC, Blockchain Native Cache. A native stateless storage library for block chain. Its value is to improve the stability and security of online s

1 Jan 12, 2022

A set of safe Least Recently Used (LRU) map/cache types for Rust

LruMap A set of safe Least-Recently-Used (LRU) cache types aimed at providing flexible map-like structures that automatically evict the least recently

4 Sep 24, 2022