Object Pool LockFree in Rust

Vaillant Etienne

Last update: Sep 7, 2022

Related tags

Foreign Function Interface lockfree-object-pool

Overview

Lock Free Object Pool

A thread-safe object pool collection with automatic return.

Some implementations are lockfree :

LinearObjectPool
SpinLockObjectPool

Other use std::Mutex :

MutexObjectPool

And NoneObjectPool basic allocation without pool.

Usage

[dependencies]
lockfree-object-pool = "0.1"

extern crate lockfree_object_pool;

Example

The general pool creation looks like this for

 let pool = LinearObjectPool::<u32>::new(
     ||  Default::default(), 
     |v| {*v = 0; });

And use the object pool

  let mut item = pool.pull();
  *item = 5;
  ...

At the end of the scope item return in object pool.

Interface

All implementations support same interface :

struct ObjectPool<T> {  
}

impl<T> ObjectPool<T> {
  // for LinearObjectPool, SpinLockObjectPool and MutexObjectPool
  // init closure used to create an element
  // reset closure used to reset element a dropped element
  pub fn new<R, I>(init: I, reset: R) -> Self
    where
        R: Fn(&mut T) + 'static + Send + Sync,
        I: Fn() -> T + 'static + Send + Sync + Clone,
    {
      ...
    }

  // for NoneObjectPool
  // init closure used to create an element
  pub fn new<I>(init: I) -> Self
    where
        I: Fn() -> T + 'static
    {
      ...
    }

  pub fn pull(&self) -> Reusable<T> {
    ...
  }

  pub fn pull_owned(self: &Arc<Self>) -> OwnedReusable<T> {
    ...
  }
}

struct Reusable<T> {  
}

impl<'a, T> DerefMut for Reusable<'a, T> {
    fn deref_mut(&mut self) -> &mut Self::Target {
        ...
    }
}

impl<'a, T> Deref for Reusable<'a, T> {
    type Target = T;

    fn deref(&self) -> &Self::Target {
        ...
    }
}

struct OwnedReusable<T> {  
}

impl<'a, T> DerefMut for OwnedReusable<'a, T> {
    fn deref_mut(&mut self) -> &mut Self::Target {
        ...
    }
}

impl<'a, T> Deref for OwnedReusable<'a, T> {
    type Target = T;

    fn deref(&self) -> &Self::Target {
        ...
    }
}

Multithreading

All implementation support allocation/desallocation from on or more thread. You only need to wrap the pool in a [std::sync::Arc] :

 let pool = Arc::new(LinearObjectPool::<u32>::new(
     ||  Default::default(), 
     |v| {*v = 0; }));

Performance

Global report.

Allocation

ObjectPool	Duration in Monothreading (us)	Duration Multithreading (us)
NoneObjectPool	1.2848	0.62509
MutexObjectPool	1.3107	1.5178
SpinLockObjectPool	1.3106	1.3684
LinearObjectPool	0.23732	0.38913
`crate 'sharded-slab'`	1.6264	0.82607
`crate 'object-pool'`	0.77533	0.26224

Report monothreading and multithreading

Forward Message between Thread

ObjectPool	1 Reader - 1 Writter (ns)	5 Reader - 1 Writter (ns)	1 Reader - 5 Writter (ns)	5 Reader - 5 Writter (ns)
NoneObjectPool	529.75	290.47	926.05	722.35
MutexObjectPool	429.29	207.17	909.88	409.99
SpinLockObjectPool	34.277	182.62	1089.7	483.81
LinearObjectPool	43.876	163.18	365.56	326.92
`crate 'sharded-slab'`	525.82	775.79	966.87	1289.2

Not supported by crate 'object-pool'

Report 1-1, 5-1, 1-5 , 5-5

Desallocation

ObjectPool	Duration in Monothreading (ns)	Duration Multithreading (ns)
NoneObjectPool	111.81	93.585
MutexObjectPool	26.108	101.86
SpinLockObjectPool	22.441	50.107
LinearObjectPool	7.5379	41.707
`crate 'sharded-slab'`	7.0394	10.283
`crate 'object-pool'`	20.517	44.798

Report monothreading and multithreading

Comparison with Similar Crates

crate 'sharded-slab': I like pull interface but i dislike
- Default / Reset trait because not enough flexible
- Performance
- create_owned method not use a reference on Self

crate 'object-pool': use a spinlock to sync and the performance are pretty good but i dislike :

need to specify fallback at each pull call :

use object_pool::Pool;
let pool = Pool::<Vec<u8>>::new(32, || Vec::with_capacity(4096);
// ...
let item1 = pool.pull(|| Vec::with_capacity(4096));
// ...
let item2 = pool.pull(|| Vec::with_capacity(4096));

no reset mechanism, need to do manually
no possiblity to forward data between thread

TODO

why the object-pool with spinlock has so bad performance compared to spinlock mutex use by crate 'object-pool'
impl a tree object pool (cf toolsbox)

Implementation detail

TODO

Licence

cf Boost Licence

Related Projects

crate 'object-pool' - A thread-safe object pool in rust with mutex
crate 'sharded-slab' - A lock-free concurrent slab
toolsbox - Some object pool implementation en c++

Bridge the gap between Haskell and Rust

Curryrs Curryrs (a play on the name of Haskell Curry, rs for Rust libraries, and it's pronunciation couriers) is a library for providing easy to use b

296 Oct 18, 2022

Rust in Haskell FFI Example

Provides an example for using Rust in Haskell. To use this you'll need cargo, rustc, cabal and GHC installed. To execute the example run the following

21 Oct 1, 2022

Run Java code from Rust!

Java Native Interface Bindings for Rust This library provides complete FFI bindings to the Java Native Interface, as well as a safe and intuitive wrap

66 Nov 28, 2022

Embedding Rust in Java

Java/Rust Example An example project showing how to call into Rust code from Java. OSX Linux Windows Requirements Java 7+ Rust (tested with 1.0, night

318 Jan 1, 2023

Rust-JDBC bindings

jdbc A Rust library that allows you to use JDBC and JDBC drivers. Usage First, add the following to your Cargo.toml: [dependencies] jdbc = "0.1" Next,

18 Feb 9, 2022

Lua 5.3 bindings for Rust

rust-lua53 Aims to be complete Rust bindings for Lua 5.3 and beyond. Currently, master is tracking Lua 5.3.3. Requires a Unix-like environment. On Win

150 Dec 14, 2022

Safe Rust bindings to Lua 5.1

124 Jan 5, 2023

Zero-cost high-level lua 5.3 wrapper for Rust

td_rlua This library is a high-level binding for Lua 5.3. You don't have access to the Lua stack, all you can do is read/write variables (including ca

47 May 4, 2022

Rust library to interface with Lua

hlua This library is a high-level binding for Lua 5.2. You don't have access to the Lua stack, all you can do is read/write variables (including callb

488 Dec 26, 2022

Comments

various soundness fixes and other improvements
First of all I wanted to say that I really like what you've done here especially the concept of a reset function to reset objects once they've been used.

This pr fixes various soundness holes including:

require init and reset to implement Send and Sync:

fix bounds for Send and Sync: Previously you just implemented Send and Sync regardless of whether the underlying type actually implements those traits: https://github.com/EVaillant/lockfree-object-pool/blob/cba781c8e6d7eb9a15773149a29511cefafb665f/src/linear_object_pool.rs#L86-L87 https://github.com/EVaillant/lockfree-object-pool/blob/cba781c8e6d7eb9a15773149a29511cefafb665f/src/spin_lock_object_pool.rs#L94-L95 https://github.com/EVaillant/lockfree-object-pool/blob/cba781c8e6d7eb9a15773149a29511cefafb665f/src/mutex_object_pool.rs#L93-L94 https://github.com/EVaillant/lockfree-object-pool/blob/cba781c8e6d7eb9a15773149a29511cefafb665f/src/non_object_pool.rs#L55-L56

fix unsound casts to mutable references: simply casting to a mutable reference is UB https://github.com/EVaillant/lockfree-object-pool/blob/cba781c8e6d7eb9a15773149a29511cefafb665f/src/linear_reusable.rs#L50-L52 this pr uses UnsafeCells in Page for creating mutable references to the objects

Other improvements:

use pub(crate) instead of #[doc(hidden)] pub

add "safety" comments for unsafe blocks

change Page::alloc to use the first available bit instead of checking each bit individually

use fetch_update in Page::alloc

use fetch_or in Page::free instead of looping with compare_exchange_weak

I also wanted to point out that the tests in test_generic_02 and test_generic_03 don't completely work as intended for MutexObjectPool or SpinLockObjectPool as they compare the address of the objects with happen to be on the stack for MutexReusable and SpinLockReusable. https://github.com/EVaillant/lockfree-object-pool/blob/cba781c8e6d7eb9a15773149a29511cefafb665f/src/mutex_reusable.rs#L24-L27 I didn't fix this though

It might be worth considering to use generics instead of trait objects for init and reset. This introduces a lot more generics to keep track of, but also allows the Send and Sync requirements on init and reset to be relaxed (if the functions don't implement Send or Sync the pool then also doesn't implement Send or Sync which might not be needed in most situations).
opened by Freax13 4
Add capability to dynamically shrink an object pool
It would be convenient to be able to periodically free the unused objects in the pool to deal with elastic workloads.

For example, let's say we periodically check if the unused objects comprise > X% of the total number of allocations for an extended period of time. If > N of these checks return true, it would be nice to then shrink the pool to reduce memory usage.

For instance, in LinearObjectPool, we

Tracked an additional pointer shrink_target representing the node representing X% in the list,

New Page's have a pointer to the shrink_target at the time they were added

we could implement the shrink by updating head -> shrink_target, shrink_target -> shrink_target's shrink_target
opened by carllin 1

Owner

Vaillant Etienne

GitHub

Omi - Object Mapping Intelligently

Omi - Object Mapping Intelligently A library that maps object structures to relational databases and provides interfaces for common queries and update

6 Dec 28, 2022

Slitter is a C- and Rust-callable slab allocator implemented primarily in Rust, with some C for performance or to avoid unstable Rust features.

Slitter is a less footgunny slab allocator Slitter is a classically structured thread-caching slab allocator that's meant to help write reliable long-