Rust bindings for LMDB

Related tags

Database lmdb-rs
Overview

lmdb-rs

Build status (master) Latest version

Rust bindings for LMDB

Documentation (master branch)

Building

LMDB is bundled as submodule so update submodules first:

git submodule update --init

And then

cargo build

Feedback

Feel free to ping me if you have a question or a suggestion how to make it better and idiomatic.

Comments
  • Can't use EnvCreateFlags

    Can't use EnvCreateFlags

    Hey! I have the following issue, when trying to create an environment with EnvCreateFlags, I get this error:

    thread '<main>' panicked at 'error opening environment: 22: Invalid argument', src/main.rs:38
    An unknown error occurred
    

    Code extract is:

    use lmdb::core::{EnvCreateReadOnly};
    let env_builder = EnvBuilder::new() 
      .map_size(256 * 1024 * 1024)      
      .max_dbs(5)                       
      .flags(EnvCreateReadOnly);        
    let env = env_builder.open(&path, 0o664).unwrap();
    

    Do you find anything wrong with that ? Thanks !

    Btw, great lib! I tried to discover the issue by myself but I'm still learning Rust :)

    opened by krakatoa 10
  • -O3 and SEGV

    -O3 and SEGV

    Just a heads up in case somebody else is struggling with the same issue.

    After a recent upgrade I've started seeing crashes in LMDB, they looked like this

    #0  0x00007fad06517cfc in mdb_cursor_put () from /usr/local/lib/libbyzon.so
    #1  0x00007fad0651af53 in mdb_put () from /usr/local/lib/libbyzon.so
    #2  0x00007fad06506698 in set_value_with_flags (self=<optimized out>, db=<optimized out>, key=...,
        value=..., flags=<optimized out>)
        at /home/grank/.cargo/registry/src/github.com-1ecc6299db9ec823/lmdb-rs-0.7.2/src/core.rs:1069
    #3  set_value (self=<optimized out>, db=<optimized out>, key=..., value=...)
        at /home/grank/.cargo/registry/src/github.com-1ecc6299db9ec823/lmdb-rs-0.7.2/src/core.rs:1061
    #4  set (self=<optimized out>, db=2, key=..., value=...)
        at /home/grank/.cargo/registry/src/github.com-1ecc6299db9ec823/lmdb-rs-0.7.2/src/core.rs:1079
    #5  lmdb_rs::core::{{impl}}::set (self=<optimized out>, key=..., value=...)
        at /home/grank/.cargo/registry/src/github.com-1ecc6299db9ec823/lmdb-rs-0.7.2/src/core.rs:414
    

    and only happened in the release builds. I've tried a couple of things, like adding global locks around LMDB operations and such, I've upgraded LMDB, but to no avail.

    Finally I've lovered the optimization level in order to better debug the issue and voila, the problem vanished.

    My conjecture is that LMDB is normally tested with -O2 optimization (cf), but the Rust package uses -O3 (cf), the level which is not supported by the LMDB code base and triggers some kind of a bug.

    opened by ArtemGr 9
  • strange segfault's occur related to lmdb-rs

    strange segfault's occur related to lmdb-rs

    I've encountered a rather bizarre segfault occurrence that is somehow linked to lmdb-rs. The segfault is odd and manifests in strangle places. In my project that I currently am experiencing issues with this it generally manifests when trying to transform data read from the db or move an Arc to the environment into a new thread. You can check out the code here.

    I created a repository purely for reference in helping solve this issue, which you can find here. I'm happy to help in fixing this if you can point me in the right direction. I've really enjoyed using this library and need it to work to complete my senior project.

    opened by code-ape 8
  • keyrange with excluding end

    keyrange with excluding end

    Hello,

    is there a nice way to iterate over keys starting with a given "from" key (including that) while ending the iteration at a given "end" key (this is, excluding that key from the iteration)? While keyrange_from/_to individually provide what I need, it seems that keyrange includes the "end" key on purpose. However, I fail to find a strong reason for this.

    P.

    opened by xitep 8
  • Cannot cross-compile liblmdb-sys

    Cannot cross-compile liblmdb-sys

    Unfortunalty, the libmbd Makefile does not allow to override CC and AR: https://github.com/LMDB/lmdb/blob/ad8488cfac644d7a289e428ab3c403c859d844cb/libraries/liblmdb/Makefile#L21-L22.

    I am still a novice to Rust. Maybe it's possible to patch the Makefile with build.rs?

    opened by joerg-krause 6
  • Make three Database methods use dynamic dispatch

    Make three Database methods use dynamic dispatch

    These methods are all thin wrappers around NativeTransaction methods that use dynamic dispatch so monomorphizing is probably a pessimization anyhow, and this lets these functions be called with dynamic dispatch (in our case we'd like to call the function with one of a variety of ToMdbValue concrete implementations, determined at runtime).

    opened by frew 6
  • FYI, there seem to be a problem with `keyrange_to` with MDB_INTEGERKEY

    FYI, there seem to be a problem with `keyrange_to` with MDB_INTEGERKEY

    Just FYI, I've encountered a persevering early termination problem iterating keys with keyrange_to. With the database defined as follows:

    let expiration_idx = try! (env.create_db ("expiration_idx", DbFlags::from_bits_truncate (lmdb_sys::MDB_DUPSORT | lmdb_sys::MDB_INTEGERKEY)));
    

    I had to rewrite the loop as follows:

      let mut hashes_found: Vec<u64> = Vec::new();  // NB: Must not change the index while iterating it.
      let till = time::get_time().sec as u64;
      // NB: Using `iter` because `keyrange_to` doesn't work as expected (misses some keys).
      for rec in try! (expiration_idx.iter()) {
        if rec.get_key::<u64>() > till {break}
        hashes_found.push (rec.get_value::<u64>());}
    

    Because the keyrange_to version terminated early and have been missing some keys.

    The problematic keyrange_to version:

      let till = time::get_time().sec as u64 + 1;
      for rec in try! (expiration_idx.keyrange_to (&till)) {
        hashes_found.push (rec.get_value::<u64>());}
    
    opened by ArtemGr 6
  • please publish a new version to crates.io

    please publish a new version to crates.io

    there is already a pull request, please press the merge button, currently this crate does not compile on crates.io (apparently a backwards incompatible compiler change) https://github.com/vhbit/lmdb-rs/pull/43

    opened by iqualfragile 5
  • Provide access to the `mdb_reader_check`.

    Provide access to the `mdb_reader_check`.

    This method should be called periodically to ensure that the crashed readers are removed from the lock table. cf. the caveats at http://symas.com/mdb/doc/index.html

    opened by ArtemGr 5
  • Add support for setting compare and dupsort functions

    Add support for setting compare and dupsort functions

    This is a bit icky since it exposes the ffi function header, but since lmdb doesn't pass a void* to the comparison function, I couldn't think of a good way to set up a wrapper function. I guess we could do a macro, but seems like that'd probably be more confusing than just needing to interact with the ffi layer. Thoughts?

    opened by frew 4
  • Support android

    Support android

    Make wip branch build (closing brace, tests not updated for master rust) Add android support (update mdb to a version that correctly builds with ndk, update makefile to support ndk-build).

    Makefiles are not my strong point and ndk-build is not supper cooperative, so the update may be pretty naive.

    opened by voloko 4
  • Update to Rust 2018 Edition

    Update to Rust 2018 Edition

    • liblmdb-sys: use cc instead if the deprecated gcc crate.
    • liblmdb-sys: suppress innocuous compiler warnings from mdb
    • lmdb-rs: update bitflags to version 1.3 with name-spacing [breaking change]
    • lmdb-rs: update log to version 0.4
    • lmdb-rs: replace try! macro usage with the ? operator
    • lmdb-rs: update to Rust 2018 edition and address all clippy warnings
    opened by jmaygarden 0
  • refactor(liblmdb-sys): use cc crate instead of gcc

    refactor(liblmdb-sys): use cc crate instead of gcc

    cc crate uses the native C compiler to compile C sources. Since gcc has been removed from android ndk, this commit helps to have a seamless compilation for android with cargo build.

    opened by elendilm 0
  • Assertion and panic with Cursor::to_next_key after Cursor::del_item

    Assertion and panic with Cursor::to_next_key after Cursor::del_item

    This seems likely to be caused by the bug listed here https://www.openldap.org/lists/openldap-bugs/201604/msg00046.html which has been since resolved. A new version of LMDB should correct it. It looks like lmdb-rs uses LMDB 0.9.18, but this was fixed in 0.9.19 (also visible in this PowerDNS ticket).

    opened by Taywee 0
  • Crashing due to mdb_freelist_save

    Crashing due to mdb_freelist_save

    lmdb is segment faulting in the function mdb_freelist_save

    When debugging with rust-gdb:

    
    [New Thread 0x7ffff47df700 (LWP 18986)]
    mdb/libraries/liblmdb/mdb.c:3067: Assertion 'pglast <= env->me_pglast' failed in mdb_freelist_save()
    
    Thread 62 "worker_0" received signal SIGABRT, Aborted.
    [Switching to Thread 0x7ffff47df700 (LWP 18986)]
    __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
    51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
    (gdb)
    

    anyone had the same problem?

    opened by flipchan 1
Releases(0.5.0)
  • 0.5.0(Sep 17, 2015)

    [breaking-change] Accessing items through mut borrow caused a lot of pain in the code, so instead the value is moved and can be unwrapped back later. This will require one more line of code like cursor = item.into_inner(), which shouldn't be a problem as cursor should be already mutable and previous code required creating artificial block anyway.

    Source code(tar.gz)
    Source code(zip)
Owner
Valerii Hiora
Valerii Hiora
SQLite3 Bindings for Rust

SQLite3 Bindings for Rust To compile use rustc src/sqlite3.rs or if you have Cargo installed cargo build. The interface is currently evolving both alo

null 74 Aug 6, 2022
leveldb LevelDB LevelDB skade/leveldb — LevelDB bindings

Rust leveldb bindings Almost-complete bindings for leveldb for Rust. Documentation Rust version policy leveldb is built and tested on stable releases

Florian Gilcher 161 Nov 20, 2022
Rustic bindings for sqlite3

Rust-Sqlite3 Rustic bindings for sqlite3. OBSOLETE in favor of jgallagher/rusqlite Copyright (c) 2014-2017 Dan Connolly and contributors Share and enj

Dan Connolly 57 Sep 26, 2022
"KakaoTalk" internal api bindings

talk-api-client "KakaoTalk" internal api bindings Examples Auth See examples/auth.rs To run example, clone this repository and run command cargo run -

storycraft 5 Sep 7, 2022
🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, SQLite, and MSSQL.

SQLx ?? The Rust SQL Toolkit Install | Usage | Docs Built with ❤️ by The LaunchBadge team SQLx is an async, pure Rust† SQL crate featuring compile-tim

launchbadge 7.6k Dec 31, 2022
Redis re-implemented in Rust.

rsedis Redis re-implemented in Rust. Why? To learn Rust. Use Cases rsedis does not rely on UNIX-specific features. Windows users can run it as a repla

Sebastian Waisbrot 1.6k Jan 6, 2023
A generic connection pool for Rust

r2d2 A generic connection pool for Rust. Documentation Opening a new database connection every time one is needed is both inefficient and can lead to

Steven Fackler 1.2k Jan 8, 2023
An ArangoDB driver for Rust

Rincon Rincon is an ArangoDB driver for Rust. It enables low level access to ArangoDB in a typesafe and Rust idiomatic manner. The name Rincon is deri

Innoave 35 Mar 21, 2021
Cassandra DB native client written in Rust language. Find 1.x versions on https://github.com/AlexPikalov/cdrs/tree/v.1.x Looking for an async version? - Check WIP https://github.com/AlexPikalov/cdrs-async

CDRS CDRS is looking for maintainers CDRS is Apache Cassandra driver written in pure Rust. ?? Looking for an async version? async-std https://github.c

Alex Pikalov 338 Jan 1, 2023
Cassandra (CQL) driver for Rust, using the DataStax C/C++ driver under the covers.

cassandra-cpp This is a maintained Rust project that exposes the DataStax cpp driver at https://github.com/datastax/cpp-driver/ in a somewhat-sane cra

null 93 Jan 7, 2023
CouchDB client-side library for the Rust programming language

Chill Chill is a client-side CouchDB library for the Rust programming language, available on crates.io. It targets Rust Stable. Chill's three chief de

null 35 Jun 26, 2022
Sofa - CouchDB for Rust

Sofa - CouchDB for Rust Documentation Here: http://docs.rs/sofa Installation [dependencies] sofa = "0.6" Description This crate is an interface to Cou

66 Origin 40 Feb 11, 2022
⚡🦀 🧨 make your rust types fit DynamoDB and visa versa

?? ?? dynomite dynomite makes DynamoDB fit your types (and visa versa) Overview Goals ⚡ make writing dynamodb applications in rust a productive experi

Doug Tangren 197 Dec 15, 2022
A Rust client for the ElasticSearch REST API

rs-es Introduction An ElasticSearch client for Rust via the REST API. Targetting ElasticSearch 2.0 and higher. Other clients For later versions of Ela

Ben Ashford 218 Dec 27, 2022
An Elasticsearch REST API client for Rust

elastic elastic is an efficient, modular API client for Elasticsearch written in Rust. The API is targeting the Elastic Stack 7.x. elastic provides st

null 249 Oct 18, 2022
An etcd client library for Rust.

etcd An etcd client library for Rust. etcd on crates.io Documentation for the latest crates.io release Running the tests Install Docker and Docker Com

Jimmy Cuadra 138 Dec 27, 2022
etcd for rust

etcd for Rust An etcd(API v3) client for Rust, and it provides async/await APIs backed by tokio and tonic. Documentation on the library can be found a

ccc 159 Dec 30, 2022
The official MongoDB Rust Driver

MongoDB Rust Driver This repository contains the officially supported MongoDB Rust driver, a client side library that can be used to interact with Mon

mongodb 1.1k Dec 30, 2022
Redis library for rust

redis-rs Redis-rs is a high level redis library for Rust. It provides convenient access to all Redis functionality through a very flexible but low-lev

Armin Ronacher 2.8k Jan 8, 2023