RefineDB - A strongly-typed document database that runs on any transactional key-value store.

Related tags

Database RefineDB
Overview

RefineDB

Build status

A strongly-typed document database that runs on any transactional key-value store.

Currently supported backends are:

  • FoundationDB for distributed deployment.
  • SQLite for single-machine deployment.
  • A simple in-memory key-value store for the web playground.

Try RefineDB on the Web Playground!

Warning: Not ready for production.

Motivation

Databases should be more scalable than popular SQL databases, more structured than popular NoSQL databases, and support stronger static type checking than any of the current databases. So I decided to build RefineDB as "the kind of database that I want to use myself".

RefineDB will be used as the database service of rusty-workers.

Architecture

Architecture

Getting started

Examples are a TODO but rdb-analyzer's tests and rdb-server (which uses RefineDB itself to store metadata) should give some basic insight on how the system works.

Schemas and the type system

In RefineDB, schemas are defined with types. For example, a part of a schema for a simple blog would look like:

type SiteConfig {
  site_name: string,
  registration_open: int64,
}

type BlogPost {
  @primary
  id: string,
  author_email: string,
  author_name: string,
  title: string,
  content: string,
  access_time: AccessTime,
}

type AccessTime {
  create_time: int64,
  update_time: int64,
}

export SiteConfig site_config;
export set posts;

The primitive types are:

  • int64: 64-bit signed integer.
  • double: IEEE 754 double-precision floating point number.
  • string: UTF-8 string.
  • bytes: Byte array.
  • set: A set with element type T.

Sum types are nice to have too, but I haven't implemented it yet.

Queries: the TreeWalker VM and RefineAsm

Queries in RefineDB are encoded as data flow graphs, and query execution is graph reduction.

The TreeWalker VM is a massively concurrent data flow virtual machine for running the queries, but I haven't written documentation on its internals.

RefineAsm is the textual representation of the query graph, with some syntactic sugar to make writing it easier.

An example RefineAsm script for adding a post to the above blog schema:

type PostMap = map {
  id: string,
  author_email: string,
  author_name: string,
  title: string,
  content: string,
  access_time: map {
    create_time: int64,
    update_time: int64,
  },
};
export graph add_post(root: schema, post: PostMap) {
  s_insert root.posts $ call(build_post) [post];
}
graph build_post(post: PostMap): BlogPost {
  return build_table(BlogPost)
    $ m_insert(access_time) (build_table(AccessTime) post.access_time) post;
}

Storage plan and schema migration

A storage plan is how a schema maps to entries in the key-value store. By separating schemas and storage plans, RefineDB's schemas are just "views" of the underlying keyspace and schema changes are fast.

During a migration, added fields are automatically assigned new storage keys, and removed fields will not be auto-deleted from the storage. This allows multiple schema versions to co-exist, enables the client to choose which schema version to use, and prevents unintended data deletion.

Storage design doc

License

MIT

Comments
  • SQLite backend is broken when running with multiple threads.

    SQLite backend is broken when running with multiple threads.

    Getting random data corruption when using SQLite backend to run tests concurrently. Limiting test threads to 1 makes these errors disappear.

    An example failure looks like:

    thread 'data::treewalker::exec_test::set_queries' panicked at 'internal error: entered unreachable code', rdb-analyzer/src/data/treewalker/exec_test.rs:254:10
    
    ---- data::treewalker::asm::asm_test::basic_exec stdout ----
    compile took 64.591668ms
    TwScript { graphs: [TwGraph { name: "main", exported: false, nodes: [(LoadParam(0), [], None), (GetField(0), [0], None), (LoadConst(0), [], None), (LoadConst(1), [], None), (CreateMap, [], None), (InsertIntoMap(4), [3, 4], None), (InsertIntoMap(3), [2, 5], None), (BuildTable(2), [6], None), (InsertIntoTable(1), [7, 1], None), (LoadConst(2), [], None), (InsertIntoTable(5), [9, 1], None)], output: None, param_types: [0], output_type: None }], entry: 0, consts: [Primitive(Int64(1)), Primitive(Int64(2)), Primitive(String("test_name"))], idents: ["some_item", "duration", "Duration<int64>", "start", "end", "name"], types: [Schema] }
    tyck took 123.214µs
    exec took 821.355µs
    Ok(None)
    compile took 47.039006ms
    TwScript { graphs: [TwGraph { name: "main", exported: false, nodes: [(LoadParam(0), [], None), (GetField(0), [0], None), (GetField(1), [1], None), (GetField(2), [1], None), (GetField(3), [1], None), (LoadConst(0), [], None), (Eq, [3, 5], None), (GetField(4), [4], Some(6)), (LoadConst(1), [], Some(6)), (Not, [6], None), (GetField(5), [4], Some(9)), (LoadConst(2), [], Some(9)), (Select, [7, 10], None), (Select, [8, 11], None), (GetField(6), [0], None), (LoadConst(3), [], None), (GetSetElement, [15, 14], None), (GetField(2), [16], None), (LoadConst(4), [], None), (GetSetElement, [18, 14], None), (GetField(2), [19], None), (LoadConst(5), [], None), (GetSetElement, [21, 14], None), (GetField(2), [22], None), (GetField(9), [1], None), (IsNull, [17], None), (Not, [25], None), (LoadConst(6), [], Some(25)), (Nop, [17], Some(26)), (Select, [27, 28], None), (CreateMap, [], None), (InsertIntoMap(13), [29, 30], None), (InsertIntoMap(12), [23, 31], None), (InsertIntoMap(11), [20, 32], None), (InsertIntoMap(10), [17, 33], None), (InsertIntoMap(9), [24, 34], None), (InsertIntoMap(8), [13, 35], None), (InsertIntoMap(7), [12, 36], None), (InsertIntoMap(2), [3, 37], None), (InsertIntoMap(1), [2, 38], None)], output: Some(39), param_types: [0], output_type: Some(1) }], entry: 0, consts: [Primitive(String("test")), Primitive(String("start")), Primitive(String("end")), Primitive(String("xxx")), Primitive(String("yyy")), Primitive(String("zzz")), Primitive(String("<unknown>"))], idents: ["some_item", "id", "name", "duration", "start", "end", "many_items", "value", "kind", "altname", "set_member_name_1", "set_member_name_2", "set_member_name_3", "set_member_name_1_nonnull"], types: [Schema, Map(RedBlackTreeMap { root: Some(Node { entry: Entry { key: "name", value: Primitive(String) }, color: Black, left: Some(Node { entry: Entry { key: "id", value: Primitive(String) }, color: Black, left: Some(Node { entry: Entry { key: "altname", value: Primitive(String) }, color: Red, left: None, right: None }), right: Some(Node { entry: Entry { key: "kind", value: Primitive(String) }, color: Red, left: None, right: None }) }), right: Some(Node { entry: Entry { key: "set_member_name_2", value: Primitive(String) }, color: Red, left: Some(Node { entry: Entry { key: "set_member_name_1", value: Primitive(String) }, color: Black, left: None, right: Some(Node { entry: Entry { key: "set_member_name_1_nonnull", value: Primitive(String) }, color: Red, left: None, right: None }) }), right: Some(Node { entry: Entry { key: "value", value: Primitive(Int64) }, color: Black, left: Some(Node { entry: Entry { key: "set_member_name_3", value: Primitive(String) }, color: Red, left: None, right: None }), right: None }) }) }), size: 9 })] }
    tyck took 186.836µs
    exec took 2.659092ms
    Ok(None)
    thread 'data::treewalker::asm::asm_test::basic_exec' panicked at 'called `Option::unwrap()` on a `None` value', rdb-analyzer/src/data/treewalker/asm/asm_test.rs:174:21
    
    ---- data::treewalker::asm::asm_test::list_ops stdout ----
    compile took 285.62346ms
    TwScript { graphs: [TwGraph { name: "main", exported: false, nodes: [(LoadParam(0), [], None), (LoadConst(0), [], None), (LoadConst(1), [], None), (LoadConst(2), [], None), (LoadConst(3), [], None), (LoadConst(4), [], None), (CreateList(2), [], None), (PrependToList, [5, 6], None), (PrependToList, [4, 7], None), (PrependToList, [3, 8], None), (PrependToList, [2, 9], None), (PrependToList, [1, 10], None), (CreateMap, [], None), (LoadConst(5), [], None), (Reduce(3, false), [12, 13, 11], None), (LoadConst(6), [], None), (Call(2), [15], None), (CreateMap, [], None), (LoadConst(5), [], None), (Reduce(3, false), [17, 18, 16], None), (CreateMap, [], None), (InsertIntoMap(3), [19, 20], None), (InsertIntoMap(2), [16, 21], None), (InsertIntoMap(1), [14, 22], None), (InsertIntoMap(0), [11, 23], None), (GetField(5), [0], None), (CreateMap, [], None), (CreateList(3), [], None), (Reduce(1, false), [26, 27, 11], None), (BuildSet, [28], None), (InsertIntoTable(4), [29, 25], None), (GetField(5), [0], None), (CreateMap, [], None), (CreateList(3), [], None), (Reduce(1, false), [32, 33, 16], None), (BuildSet, [34], None), (InsertIntoTable(6), [35, 31], None)], output: Some(24), param_types: [0], output_type: Some(1) }, TwGraph { name: "transform_numbers", exported: false, nodes: [(LoadParam(0), [], None), (LoadParam(1), [], None), (LoadParam(2), [], None), (CreateMap, [], None), (InsertIntoMap(8), [2, 3], None), (BuildTable(7), [4], None), (PrependToList, [5, 1], None)], output: Some(6), param_types: [4, 5, 2], output_type: Some(5) }, TwGraph { name: "gen_numbers", exported: false, nodes: [(LoadParam(0), [], None), (LoadConst(5), [], None), (Eq, [0, 1], None), (CreateList(2), [], Some(2)), (Not, [2], None), (LoadConst(4), [], Some(4)), (Sub, [0, 5], Some(4)), (Call(2), [6], Some(4)), (PrependToList, [0, 7], Some(4)), (Select, [3, 8], None)], output: Some(9), param_types: [2], output_type: Some(6) }, TwGraph { name: "sum", exported: false, nodes: [(LoadParam(0), [], None), (LoadParam(1), [], None), (LoadParam(2), [], None), (Add, [1, 2], None)], output: Some(3), param_types: [4, 2, 2], output_type: Some(2) }], entry: 0, consts: [Primitive(Int64(5)), Primitive(Int64(4)), Primitive(Int64(3)), Primitive(Int64(2)), Primitive(Int64(1)), Primitive(Int64(0)), Primitive(Int64(20))], idents: ["list1", "sum1", "list2", "sum2", "collection_a", "store", "collection_b", "Number<>", "value"], types: [Schema, Map(RedBlackTreeMap { root: Some(Node { entry: Entry { key: "list2", value: List(VmListType { ty: Primitive(Int64) }) }, color: Black, left: Some(Node { entry: Entry { key: "list1", value: List(VmListType { ty: Primitive(Int64) }) }, color: Black, left: None, right: None }), right: Some(Node { entry: Entry { key: "sum1", value: Primitive(Int64) }, color: Black, left: None, right: Some(Node { entry: Entry { key: "sum2", value: Primitive(Int64) }, color: Red, left: None, right: None }) }) }), size: 4 }), Primitive(Int64), Table(VmTableType { name: "Number<>" }), Map(RedBlackTreeMap { root: None, size: 0 }), List(VmListType { ty: Table(VmTableType { name: "Number<>" }) }), List(VmListType { ty: Primitive(Int64) })] }
    tyck took 367.834µs
    exec took 8.56912ms
    Ok(Some(Map(VmMapValue { elements: RedBlackTreeMap { root: Some(Node { entry: Entry { key: "sum1", value: Primitive(Int64(15)) }, color: Black, left: Some(Node { entry: Entry { key: "list2", value: List(VmListValue { member_ty: Primitive(Int64), node: List { head: Some(Node { value: Primitive(Int64(20)), next: Some(Node { value: Primitive(Int64(19)), next: Some(Node { value: Primitive(Int64(18)), next: Some(Node { value: Primitive(Int64(17)), next: Some(Node { value: Primitive(Int64(16)), next: Some(Node { value: Primitive(Int64(15)), next: Some(Node { value: Primitive(Int64(14)), next: Some(Node { value: Primitive(Int64(13)), next: Some(Node { value: Primitive(Int64(12)), next: Some(Node { value: Primitive(Int64(11)), next: Some(Node { value: Primitive(Int64(10)), next: Some(Node { value: Primitive(Int64(9)), next: Some(Node { value: Primitive(Int64(8)), next: Some(Node { value: Primitive(Int64(7)), next: Some(Node { value: Primitive(Int64(6)), next: Some(Node { value: Primitive(Int64(5)), next: Some(Node { value: Primitive(Int64(4)), next: Some(Node { value: Primitive(Int64(3)), next: Some(Node { value: Primitive(Int64(2)), next: Some(Node { value: Primitive(Int64(1)), next: None }) }) }) }) }) }) }) }) }) }) }) }) }) }) }) }) }) }) }) }), last: Some(Primitive(Int64(1))), length: 20 } }) }, color: Black, left: Some(Node { entry: Entry { key: "list1", value: List(VmListValue { member_ty: Primitive(Int64), node: List { head: Some(Node { value: Primitive(Int64(5)), next: Some(Node { value: Primitive(Int64(4)), next: Some(Node { value: Primitive(Int64(3)), next: Some(Node { value: Primitive(Int64(2)), next: Some(Node { value: Primitive(Int64(1)), next: None }) }) }) }) }), last: Some(Primitive(Int64(1))), length: 5 } }) }, color: Red, left: None, right: None }), right: None }), right: Some(Node { entry: Entry { key: "sum2", value: Primitive(Int64(210)) }, color: Black, left: None, right: None }) }), size: 4 } })))
    compile took 63.291593ms
    TwScript { graphs: [TwGraph { name: "main", exported: false, nodes: [(LoadParam(0), [], None), (CreateMap, [], None), (CreateList(2), [], None), (GetField(1), [0], None), (GetField(0), [3], None), (Reduce(1, false), [1, 2, 4], None), (CreateMap, [], None), (CreateList(2), [], None), (GetField(1), [0], None), (GetField(2), [8], None), (Reduce(1, false), [6, 7, 9], None), (CreateMap, [], None), (InsertIntoMap(4), [10, 11], None), (InsertIntoMap(3), [5, 12], None)], output: Some(13), param_types: [0], output_type: Some(1) }, TwGraph { name: "decompose_numbers", exported: false, nodes: [(LoadParam(0), [], None), (LoadParam(1), [], None), (LoadParam(2), [], None), (GetField(5), [2], None), (PrependToList, [3, 1], None)], output: Some(4), param_types: [3, 4, 5], output_type: Some(4) }], entry: 0, consts: [], idents: ["collection_a", "store", "collection_b", "list1", "list2", "value"], types: [Schema, Map(RedBlackTreeMap { root: Some(Node { entry: Entry { key: "list1", value: List(VmListType { ty: Primitive(Int64) }) }, color: Black, left: None, right: Some(Node { entry: Entry { key: "list2", value: List(VmListType { ty: Primitive(Int64) }) }, color: Red, left: None, right: None }) }), size: 2 }), Primitive(Int64), Map(RedBlackTreeMap { root: None, size: 0 }), List(VmListType { ty: Primitive(Int64) }), Table(VmTableType { name: "Number<>" })] }
    tyck took 289.364µs
    exec took 968.593µs
    Ok(Some(Map(VmMapValue { elements: RedBlackTreeMap { root: Some(Node { entry: Entry { key: "list2", value: List(VmListValue { member_ty: Primitive(Int64), node: List { head: None, last: None, length: 0 } }) }, color: Black, left: Some(Node { entry: Entry { key: "list1", value: List(VmListValue { member_ty: Primitive(Int64), node: List { head: None, last: None, length: 0 } }) }, color: Red, left: None, right: None }), right: None }), size: 2 } })))
    thread 'data::treewalker::asm::asm_test::list_ops' panicked at 'assertion failed: `(left == right)`
      left: `0`,
     right: `5`', rdb-analyzer/src/data/treewalker/asm/asm_test.rs:475:67
    
    
    failures:
        data::treewalker::asm::asm_test::basic_exec
        data::treewalker::asm::asm_test::list_ops
        data::treewalker::exec_test::basic_exec
        data::treewalker::exec_test::set_queries
    
    test result: FAILED. 30 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.13s
    
    
    
    opened by losfair 0
  • Configure Renovate

    Configure Renovate

    Mend Renovate

    Welcome to Renovate! This is an onboarding PR to help you understand and configure settings before regular Pull Requests begin.

    🚦 To activate Renovate, merge this Pull Request. To disable Renovate, simply close this Pull Request unmerged.


    Detected Package Files

    • rdb-analyzer/Cargo.toml (cargo)
    • rdb-pgsvc/Cargo.toml (cargo)
    • rdb-proto/Cargo.toml (cargo)
    • rdb-server/Cargo.toml (cargo)
    • rdbctl/Cargo.toml (cargo)
    • rdb-server.Dockerfile (dockerfile)
    • .github/workflows/ci.yml (github-actions)

    Configuration

    🔡 Renovate has detected a custom config for this PR. Feel free to ask for help if you have any doubts and would like it reviewed.

    Important: Now that this branch is edited, Renovate can't rebase it from the base branch any more. If you make changes to the base branch that could impact this onboarding PR, please merge them manually.

    What to Expect

    With your current configuration, Renovate will create 27 Pull Requests:

    chore(deps): update rust crate lalrpop to 0.19.8
    • Schedule: ["at any time"]
    • Branch name: renovate/lalrpop-0.x
    • Merge into: main
    • Upgrade lalrpop to 0.19.8
    fix(deps): update rust crate lalrpop-util to 0.19.8
    • Schedule: ["at any time"]
    • Branch name: renovate/lalrpop-util-0.x
    • Merge into: main
    • Upgrade lalrpop-util to 0.19.8
    chore(deps): update rust crate tonic-build to 0.8
    • Schedule: ["at any time"]
    • Branch name: renovate/tonic-build-0.x
    • Merge into: main
    • Upgrade tonic-build to 0.8
    fix(deps): update rust crate bumpalo to 3.11
    • Schedule: ["at any time"]
    • Branch name: renovate/bumpalo-3.x
    • Merge into: main
    • Upgrade bumpalo to 3.11
    fix(deps): update rust crate clap to 3.2.23
    • Schedule: ["at any time"]
    • Branch name: renovate/clap-3.x
    • Merge into: main
    • Upgrade clap to 3.2.23
    fix(deps): update rust crate console to 0.15.2
    • Schedule: ["at any time"]
    • Branch name: renovate/console-0.x
    • Merge into: main
    • Upgrade console to 0.15.2
    fix(deps): update rust crate dialoguer to 0.10
    • Schedule: ["at any time"]
    • Branch name: renovate/dialoguer-0.x
    • Merge into: main
    • Upgrade dialoguer to 0.10
    fix(deps): update rust crate foundationdb to 0.7
    • Schedule: ["at any time"]
    • Branch name: renovate/foundationdb-0.x
    • Merge into: main
    • Upgrade foundationdb to 0.7
    fix(deps): update rust crate indexmap to 1.9
    • Schedule: ["at any time"]
    • Branch name: renovate/indexmap-1.x
    • Merge into: main
    • Upgrade indexmap to 1.9
    fix(deps): update rust crate lru to 0.8
    • Schedule: ["at any time"]
    • Branch name: renovate/lru-0.x
    • Merge into: main
    • Upgrade lru to 0.8
    fix(deps): update rust crate petgraph to 0.6
    • Schedule: ["at any time"]
    • Branch name: renovate/petgraph-0.x
    • Merge into: main
    • Upgrade petgraph to 0.6
    fix(deps): update rust crate phf to 0.11
    • Schedule: ["at any time"]
    • Branch name: renovate/phf-0.x
    • Merge into: main
    • Upgrade phf to 0.11
    fix(deps): update rust crate prost to 0.11
    • Schedule: ["at any time"]
    • Branch name: renovate/prost-0.x
    • Merge into: main
    • Upgrade prost to 0.11
    fix(deps): update rust crate r2d2_sqlite to 0.21
    • Schedule: ["at any time"]
    • Branch name: renovate/r2d2_sqlite-0.x
    • Merge into: main
    • Upgrade r2d2_sqlite to 0.21
    fix(deps): update rust crate rpds to 0.12
    • Schedule: ["at any time"]
    • Branch name: renovate/rpds-0.x
    • Merge into: main
    • Upgrade rpds to 0.12
    fix(deps): update rust crate rusqlite to 0.28
    • Schedule: ["at any time"]
    • Branch name: renovate/rusqlite-0.x
    • Merge into: main
    • Upgrade rusqlite to 0.28
    fix(deps): update rust crate serde_yaml to 0.9
    • Schedule: ["at any time"]
    • Branch name: renovate/serde_yaml-0.x
    • Merge into: main
    • Upgrade serde_yaml to 0.9
    fix(deps): update rust crate sha2 to 0.10
    • Schedule: ["at any time"]
    • Branch name: renovate/sha2-0.x
    • Merge into: main
    • Upgrade sha2 to 0.10
    fix(deps): update rust crate sysinfo to 0.26
    • Schedule: ["at any time"]
    • Branch name: renovate/sysinfo-0.x
    • Merge into: main
    • Upgrade sysinfo to 0.26
    fix(deps): update rust crate tonic to 0.8
    • Schedule: ["at any time"]
    • Branch name: renovate/tonic-0.x
    • Merge into: main
    • Upgrade tonic to 0.8
    chore(deps): update actions/checkout action to v3
    • Schedule: ["at any time"]
    • Branch name: renovate/actions-checkout-3.x
    • Merge into: main
    • Upgrade actions/checkout to v3
    chore(deps): update ubuntu docker tag to v22
    • Schedule: ["at any time"]
    • Branch name: renovate/ubuntu-22.x
    • Merge into: main
    • Upgrade ubuntu to 22.04
    fix(deps): update rust crate async-recursion to v1
    • Schedule: ["at any time"]
    • Branch name: renovate/async-recursion-1.x
    • Merge into: main
    • Upgrade async-recursion to 1.0.0
    fix(deps): update rust crate clap to v4
    • Schedule: ["at any time"]
    • Branch name: renovate/clap-4.x
    • Merge into: main
    • Upgrade clap to 4.0.26
    fix(deps): update rust crate rmp-serde to v1
    • Schedule: ["at any time"]
    • Branch name: renovate/rmp-serde-1.x
    • Merge into: main
    • Upgrade rmp-serde to 1.1
    fix(deps): update rust crate similar to v2
    • Schedule: ["at any time"]
    • Branch name: renovate/similar-2.x
    • Merge into: main
    • Upgrade similar to 2
    fix(deps): update rust crate uuid to v1
    • Schedule: ["at any time"]
    • Branch name: renovate/uuid-1.x
    • Merge into: main
    • Upgrade uuid to 1.2

    🚸 Branch creation will be limited to maximum 2 per hour, so it doesn't swamp any CI resources or spam the project. See docs for prhourlylimit for details.


    ❓ Got questions? Check out Renovate's Docs, particularly the Getting Started section. If you need any further assistance then you can also request help here.


    This PR has been generated by Mend Renovate. View repository job log here.

    opened by renovate[bot] 0
  • GUI/TUI for viewing and modifying databases.

    GUI/TUI for viewing and modifying databases.

    Given a schema and its associated database, there is currently no way to navigate through the data other than writing a RefineAsm program and invoking it to do the query. It would be great if we have a user interface (GUI/TUI) to view and modify the structured data without writing any code.

    No one is currently working on this. Please comment if you're interested!

    good first issue help wanted 
    opened by losfair 0
  • Support HSE as a KV backend.

    Support HSE as a KV backend.

    HSE is a fast embeddable key-value store designed for SSDs and persistent memory built by Micron. It would be nice to support HSE as a KV backend in RefineDB.

    opened by losfair 1
  • Packed tables.

    Packed tables.

    Given a table like:

    type PostInfo {
      author: string,
      title: string,
      content: string,
      create_time: string,
    }
    

    It's often useful to store all fields in a single key-value entry to reduce query overhead.

    That's what the @packed attribute is for:

    type Post {
      @primary
      id: string,
      @packed
      info: PostInfo,
    }
    

    However currently the @packed attribute is only implemented on the syntax level, and not pushed down to either the storage plan or TreeWalker.

    Changes needed for @packed:

    • Storage planner: Treat packed fields as a leaf node.
    • TreeWalker: Keep track of the packed/unpacked kind of any set and table DFG nodes.
    • TreeWalker: Lens-like structure for looking inside a packed field.
    opened by losfair 0
Owner
Heyang Zhou
Doing distributed database @bytedance. I'm interested in both the mathematical and mechanical foundations of computing. Compilers / PLT / OS / HW design.
Heyang Zhou
General basic key-value structs for Key-Value based storages

General basic key-value structs for Key-Value based storages

Al Liu 0 Dec 3, 2022
Pure rust embeddable key-value store database.

MHdb is a pure Rust database implementation, based on dbm. See crate documentation. Changelog v1.0.3 Update Cargo.toml v1.0.2 Update Cargo.toml v1.0.1

Magnus Hirth 7 Dec 10, 2022
PickleDB-rs is a lightweight and simple key-value store. It is a Rust version for Python's PickleDB

PickleDB PickleDB is a lightweight and simple key-value store written in Rust, heavily inspired by Python's PickleDB PickleDB is fun and easy to use u

null 155 Jan 5, 2023
RedisLess is a fast, lightweight, embedded and scalable in-memory Key/Value store library compatible with the Redis API.

RedisLess is a fast, lightweight, embedded and scalable in-memory Key/Value store library compatible with the Redis API.

Qovery 145 Nov 23, 2022
Log structured append-only key-value store from Rust In Action with some enhancements.

riakv Log structured, append only, key value store implementation from Rust In Action with some enhancements. Features Persistent key value store with

Arindam Das 5 Oct 29, 2022
A LSM-based Key-Value Store in Rust

CobbleDB A LSM-based Key-Value Store in Rust Motivation There is no open-source LSM-based key-value store in Rust natively. Some crates are either a w

Yizheng Jiao 2 Oct 25, 2021
A sessioned Merkle key/value store

storage A sessioned Merkle key/value store The crate was designed to be the blockchain state database. It provides persistent storage layer for key-va

Findora Foundation 15 Oct 25, 2022
A rust Key-Value store based on Redis.

Key-Value Store A Key-Value store that uses Redis to store data. Built using an async web framework in Rust with a full Command-Line interface and log

Miguel David Salcedo 0 Jan 14, 2022
🔥🌲 High-performance Merkle key/value store

merk High-performance Merkle key/value store Merk is a crypto key/value store - more specifically, it's a Merkle AVL tree built on top of RocksDB (Fac

Nomic 189 Dec 13, 2022
A simple embedded key-value store written in rust as a learning project

A simple embedded key-value store written in rust as a learning project

Blobcode 1 Feb 20, 2022
Lightweight, Strongly Typed Xata Client written in Rust

xata-rs: Lightweight, Strongly Typed Xata Client xata-rs is a third party Xata client, allowing interaction with Xata's REST API. Adding xata-rs (WIP)

Alvaro Machuca Recalde 4 Jun 12, 2023
Immutable Ordered Key-Value Database Engine

PumpkinDB Build status (Linux) Build status (Windows) Project status Usable, between alpha and beta Production-readiness Depends on your risk toleranc

null 1.3k Jan 2, 2023
AgateDB is an embeddable, persistent and fast key-value (KV) database written in pure Rust

AgateDB is an embeddable, persistent and fast key-value (KV) database written in pure Rust. It is designed as an experimental engine for the TiKV project, and will bring aggressive optimizations for TiKV specifically.

TiKV Project 535 Jan 9, 2023
A simple key value database for storing simple structures.

Perdia-DB A simple key value database for storing simple structures. No nesting of structures is supported, but may be implemented in the future. Toke

Perdia 4 May 24, 2022
A fast and simple in-memory database with a key-value data model written in Rust

Segment Segment is a simple & fast in-memory database with a key-value data model written in Rust. Features Dynamic keyspaces Keyspace level control o

Segment 61 Jan 5, 2023
A "blazingly" fast key-value pair database without bloat written in rust

A fast key-value pair in memory database. With a very simple and fast API. At Xiler it gets used to store and manage client sessions throughout the pl

Arthur 16 Dec 16, 2022
An embedded, in-memory, immutable, copy-on-write, key-value database engine

An embedded, in-memory, immutable, copy-on-write, key-value database engine. Features In-memory database Multi-version concurrency control Rich transa

SurrealDB 7 May 5, 2024
A Distributed SQL Database - Building the Database in the Public to Learn Database Internals

Table of Contents Overview Usage TODO MVCC in entangleDB SQL Query Execution in entangleDB entangleDB Raft Consensus Engine What I am trying to build

Sarthak Dalabehera 38 Jan 2, 2024
A programmable document database inspired by CouchDB written in Rust

PliantDB PliantDB aims to be a Rust-written, ACID-compliant, document-database inspired by CouchDB. While it is inspired by CouchDB, this project will

Khonsu Labs 718 Dec 31, 2022