Seed your development database with real data ⚡️

Last update: Jun 21, 2022

replibyte logo

Seed Your Development Database With Real Data ⚡️

Replibyte is a blazingly fast tool to seed your databases with your production data while keeping sensitive data safe 🔥

MIT License stable badge stable badge Build and Tests Discord

Prerequisites

  • MacOSX / Linux / Windows
  • Nothing more! Replibyte is stateless and does not require anything special.

Usage

Create a dump

replibyte -c conf.yaml dump create

List all dumps

replibyte -c conf.yaml dump list

type          name                  size    when                    compressed  encrypted
PostgreSQL    dump-1647706359405    154MB   Yesterday at 03:00 am   true        true
PostgreSQL    dump-1647731334517    152MB   2 days ago at 03:00 am  true        true
PostgreSQL    dump-1647734369306    149MB   3 days ago at 03:00 am  true        true

Restore the latest dump in a local container

replibyte -c conf.yaml dump restore local -v latest -i postgres -p 5432

Restore the latest dump in a remote database

replibyte -c conf.yaml dump restore remote -v latest

Features

  • Support data dump and restore for PostgreSQL, MySQL and MongoDB
  • Replace sensitive data with fake data
  • Works on large database (> 10GB)
  • Database Subsetting: Scale down a production database to a more reasonable size 🔥
  • Start a local database with the prod data in a single command 🔥
  • On-the-fly data (de)compression (Zlib)
  • On-the-fly data de/encryption (AES-256)
  • Fully stateless (no server, no daemon) and lightweight binary 🍃
  • Use custom transformers

Here are the features we plan to support

  • Auto-detect and version database schema change
  • Auto-detect sensitive fields
  • Auto-clean backed up data

Getting Started

  1. How Replibyte works
  2. Initial setup:
    1. Install
    2. Configure
  3. Step-by-step guides:
    1. Create a dump
    2. Restore a dump
    3. Subset a dump
    4. Delete a dump
    5. Deploy Replibyte
      1. Container
      2. Qovery

Demo

What is RepliByte

Contributing

Check here.

Thanks

Thanks to all people sharing their ideas to make Replibyte better. We do appreciate it. I would also thank AirByte, a great product and a trustworthy source of inspiration for this project.


Replibyte is initiated and maintained by Qovery

GitHub

https://github.com/Qovery/Replibyte
Comments
  • 1. panic: Unterminated string literal in SQL instruction

    Hello, at Flexhire we are trying out this tool to seed our staging DB with production data.

    We are using replibyte v0.6 for Linux x86 and the dump is of a PostgreSQL 11 DB hosted on AWS RDS.

    When generating a dump from our production DB we are encountering the following crash. I included a stack backtrace generated with RUST_BACKTRACE=full

    We have text columns containing user entered text in the markdown format that might have characters such as '. The application uses rails so these characters should be escaped. Perhaps there is some issue with the escaping done by Replibyte? It looks like INSERT instructions are the ones causing the problem.

    thread 'main' panicked at 'TokenizerError { message: "Unterminated string literal", line: 1, col: 788 }', dump-parser/src/postgres/mod.rs:747:13
    stack backtrace:
       0:     0x7f147c81bc5d - std::backtrace_rs::backtrace::libunwind::trace::h081201764674ef17
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
       1:     0x7f147c81bc5d - std::backtrace_rs::backtrace::trace_unsynchronized::hebab37398c391bd7
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
       2:     0x7f147c81bc5d - std::sys_common::backtrace::_print_fmt::h301516df68ed24f9
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:66:5
       3:     0x7f147c81bc5d - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h8f5170f4f03a12c0
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:45:22
       4:     0x7f147c867fac - core::fmt::write::h5dc5601e8d9f6367
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/fmt/mod.rs:1190:17
       5:     0x7f147c8138c8 - std::io::Write::write_fmt::h5b19302eb99d9acf
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/io/mod.rs:1657:15
       6:     0x7f147c81e2e7 - std::sys_common::backtrace::_print::hd81cf53a75c8ae6a
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:48:5
       7:     0x7f147c81e2e7 - std::sys_common::backtrace::print::hb5aa882e87c2a0dc
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:35:9
       8:     0x7f147c81e2e7 - std::panicking::default_hook::{{closure}}::had913369af61b326
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:295:22
       9:     0x7f147c81dfb0 - std::panicking::default_hook::h37b06af9ee965447
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:314:9
      10:     0x7f147c81ea39 - std::panicking::rust_panic_with_hook::hf2019958d21362cc
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:698:17
      11:     0x7f147c81e727 - std::panicking::begin_panic_handler::{{closure}}::he9c06fdd592f8785
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:588:13
      12:     0x7f147c81c124 - std::sys_common::backtrace::__rust_end_short_backtrace::ha521b96560789310
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/sys_common/backtrace.rs:138:18
      13:     0x7f147c81e439 - rust_begin_unwind
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:584:5
      14:     0x7f147b94ba23 - core::panicking::panic_fmt::h28f1697d4e9394b4
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:143:14
      15:     0x7f147c267afe - dump_parser::postgres::get_tokens_from_query_str::h5c129d0e7d926086
      16:     0x7f147b982287 - replibyte::source::postgres::read_and_transform::{{closure}}::h38d9592830b06c69
      17:     0x7f147b97b9d6 - dump_parser::utils::list_sql_queries_from_dump_reader::h540f884dd1afeb0d
      18:     0x7f147b9dad0f - <replibyte::tasks::full_dump::FullDumpTask<S> as replibyte::tasks::Task>::run::ha2213c1574b9739f
      19:     0x7f147ba97a55 - replibyte::commands::dump::run::h86af3c9d6a23f4c9
      20:     0x7f147ba59b30 - replibyte::main::h880123c1e7bd6b08
      21:     0x7f147ba8b9b3 - std::sys_common::backtrace::__rust_begin_short_backtrace::h9dd32b852f85f8a3
      22:     0x7f147b9f1e89 - std::rt::lang_start::{{closure}}::h434a6ad0c3bb7757
      23:     0x7f147c81b3b4 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::hd127f27863548251
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ops/function.rs:259:13
      24:     0x7f147c81b3b4 - std::panicking::try::do_call::h926290883a1d024e
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:492:40
      25:     0x7f147c81b3b4 - std::panicking::try::hc74a3d1f4a4b6e5f
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:456:19
      26:     0x7f147c81b3b4 - std::panic::catch_unwind::h5eb7ded2df1a4d5f
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panic.rs:137:14
      27:     0x7f147c81b3b4 - std::rt::lang_start_internal::{{closure}}::h0736f9682f7c55ea
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/rt.rs:128:48
      28:     0x7f147c81b3b4 - std::panicking::try::do_call::h2772c479b1c89ef7
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:492:40
      29:     0x7f147c81b3b4 - std::panicking::try::h967ebbc371287391
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:456:19
      30:     0x7f147c81b3b4 - std::panic::catch_unwind::h41bcc02b28316856
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panic.rs:137:14
      31:     0x7f147c81b3b4 - std::rt::lang_start_internal::haf46799f55774d07
                                   at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/rt.rs:128:20
      32:     0x7f147ba5db02 - main
    
    Reviewed by fazo96 at 2022-05-10 10:56
  • 2. How to use on macos?

    hello! I was wondering the best way to use this on macos. From the docs, I used brew to install:

    brew install replibyte
    

    but when I restart my terminal and run replibyte, I get:

    zsh: command not found: replibyte
    

    Brew info:

    ✗ brew info replibyte
    qovery/replibyte/replibyte: stable 0.6.3
    Seed Your Development Database With Real Data ⚡️
    https://github.com/Qovery/replibyte
    /usr/local/Cellar/replibyte/0.6.3 (49 files, 275.3KB) *
      Built from source on 2022-05-16 at 10:56:08
    From: https://github.com/Qovery/homebrew-replibyte/blob/HEAD/Formula/replibyte.rb
    License: MIT
    

    brew --cellar replibyte

    ✗ brew --cellar replibyte
    /usr/local/Cellar/replibyte
    

    Is there a step that I'm missing with brew?

    Reviewed by brett-anderson at 2022-05-16 20:11
  • 3. feat: restore backup in local container

    This PR add support for restoring a backup inside a Docker container as discussed in #32.

    This is currently a wip, needs some tests and improvements

    Closes #32

    Reviewed by fabriceclementz at 2022-04-02 20:55
  • 4. mongo dump not working, terminal without output

    Hi, and thank you really much for your efforts. I'm trying to make an anonymous dump using replibyte and a mongo database but I think is not working as expected. I have my conf.yaml, really simple just to avoid interactions and I'm doing everything locally to avoid mistakes.

    conf.yaml

    source:
      connection_uri: mongodb://mongo:[email protected]:27017/test
      transformers: # optional - hide sensitive data
          - database: test
            table: employees
            columns:
              - name: surname
                transformer_name: random
              - name: first_name
                transformer_name: first-name
              - name: email
                transformer_name: email
              - name: phone_ext
                transformer_name: phone-number
    destination:
      connection_uri: mongodb://mongo:[email protected]:27017/test # you can use $DATABASE_URL
    datastore:
      local_disk:
        dir: .
    

    But everytime I run replibyte -c conf.yaml dump create I have no answer in the terminal, sometimes I can see this message "No such file or directory (os error 2)". I'm using mongo4.4 is not supported?

    image

    I just want to run the transformer but I'm not sure if I can. Thank you very much for your help.

    Kind regards

    Reviewed by victorgomezg93 at 2022-05-19 10:47
  • 5. Nomenclature / naming

    I have the feeling that "bridge" does not mean anything and it might be more accurate to rename it into something like "store". The idea is a place where to store the created dataset from the source database. A "bridge" does not really reflect this concept. What do you think?

    Reviewed by evoxmusic at 2022-05-02 19:57
  • 6. feat: custom transformer with wasm

    This is a draft of implementing #26

    I have written some boilerplate code which will eventually lead to the full implementation of custom transformer with Web Assembly (wasm) and I also included one simple test.

    Before I continue with the implementation, there are a few points that should be addressed:

    1. where should we receive the input for this transformer (the wasm bytes)? as a special new argument flag? or from replibyte.yaml file?

    2. wasm only supports 32/64 bits numbers, so maybe we should add a few more values to Column:

      pub enum Column {
           // ...
           NumberValuei32(String, i32),
           NumberValuei64(String, i64),
           NumberValuei128(String, i128),  // not compatible with wasm, unless we truncate
          // ...
      }
      
    3. the first try of compilation yielded the following error: *mut Ctx cannot be shared between threads safely within wasmer_runtime_core::instance::InstanceInner, the trait std::marker::Sync is not implemented for *mut Ctx

      This is due to fact that Transformer is Sync. as a temporary solution, I removed Sync from Transformer, of course I realize that this is not a long term solution but I wanted to make the compilation and example test work. we need to think about how we are going to address this issue.

    There will probably be more points to think about in the future, but for now these are the main things that bothered me. @evoxmusic what do you think?

    Reviewed by benny-n at 2022-04-03 23:23
  • 7. Enhancement: make RepliByte installable from Homebrew

    Everything is in the title :) Today, it's tedious to install RepliByte from MacOSX. It will be great to make it installable from Homebrew for MacOSX Intel and M1 processors. Anyone can help here? :)

    Reviewed by evoxmusic at 2022-03-30 07:01
  • 8. setup lib.rs and bin coexistence

    The main thing I'm trying to do in this pull-request is to add the file replibyte/tests/mysql.rs. In addition to allowing importing modules from tests folder, it also revealed that the current structure allows for 'clever' use statements such as

    // replibyte/src/config.rs
    use crate::{RandomTransformer, Transformer};
    
    // or replibyte/src/tasks.rs
    use crate::Source;
    

    which is saying "use RandomTransformer, Transformer that are imported from crate (main.rs)". If main.rs in the future doesn't use them anymore, replibyte/src/config.rs (and its dependent) would break.

    Please let me know what you think 🤓

    Reviewed by tbmreza at 2022-03-15 00:45
  • 9. Docker image not working

    Hi, I'm trying to run the docker image to test just following the guide:

    git clone https://github.com/Qovery/replibyte.git
    
    # Build image with Docker
    docker build -t replibyte -f Dockerfile .
    
    # Run RepliByte
    docker run -v $(pwd)/examples:/examples/ replibyte -c /examples/replibyte.yaml transformer list
    

    I'm doing it in Ubuntu 20.04 and when I try to make the docker run I have this error:

    No such file or directory (os error 2)
    

    Maybe some package is missing? If I make the installation from source or from linux the same message appear.

    Thank you very much for your help

    Reviewed by victorgomezg93 at 2022-06-02 13:36
  • 10. Rename backup to dump?

    As we rename the cli command from backup to dump, I think we could rename it to Dump in the datastore directory. WDYT?

    Example:

    // datastore/mod.rs
    pub struct Backup {
        pub directory_name: String,
        pub size: usize,
        pub created_at: u128,
        pub compressed: bool,
        pub encrypted: bool,
    }
    

    to

    pub struct Dump {
        pub directory_name: String,
        pub size: usize,
        pub created_at: u128,
        pub compressed: bool,
        pub encrypted: bool,
    }
    
    // datastore/mod.rs
    pub struct IndexFile {
        pub backups: Vec<Backup>,
    }
    

    to

    pub struct IndexFile {
        pub dumps: Vec<Dump>,
    }
    

    This update will lead to a renaming of the key backups to dumps inside the metadata.json file.

    Reviewed by fabriceclementz at 2022-05-14 20:51
  • 11. Feat add delete backup command

    This PR starts to implement the delete backup command as discussed in #66.

    I need to update the readme too and I will update the PR to add the following options:

    • [x] --older-than=2d option
    • [x] --keep-last=10 option

    Closes #66

    Reviewed by fabriceclementz at 2022-04-28 20:55
  • 12. fix: don't panic anymore when there is no column value matching in insert into

    Hi @evoxmusic I think this PR should fix the panic mentioned in #134 but I failed to reproduce this bug so I just removed the unwrap to gracefully handle the case where there is no matching value for a column.

    I let this PR in draft for now, maybe we can have the part of the dump causing this panic so we can validate It's not a problem in the dump-parser.

    Closes #134

    Reviewed by fabriceclementz at 2022-06-14 20:19
  • 13. Error on dump: "Unterminated string literal"

    Hi, I'm using replibyte with Postgres and GCP. When I run replibyte dump create, the script panics with the error:

    thread 'main' panicked at 'TokenizerError { message: "Unterminated string literal", line: 9, col: 64 }', dump-parser/src/postgres/mod.rs:761:13

    I previously saw this issue before, but it was fixed here: https://github.com/Qovery/Replibyte/issues/94. I was able to create a dump after that, but now I'm seeing the error again.

    Happy to provide the specific failing query over email!

    Reviewed by ederski at 2022-06-11 00:07
  • 14. Access to the whole row in custom transformers

    I am building custom transformers to transform some of our data

    The problem I am having is that inside the transformer I only have access to the column value, and nothing else, which makes it impossible for me to have transformers that change the value depending on something else.

    One example I have is I want to anonymize users unless they are staff users. I can do it easily with emails (replibyte/transformers/email-transformer/src/main.rs)) but I also want to keep their hashed password but since I only have access to the column value I have no way to know which password I am changing

    Reviewed by marcoacierno at 2022-06-10 09:50
  • 15. tokio: replace custom blocking tokio calls, with native tokio::runtime::Handle::block_on

    Hi, hope you are doing well. I noticed that replibyte uses a custom function and a runtime for getting some values that need to be awaited. We can completely remove the code in runtime.rs since tokio gives us the ability to do the same thing natively via tokio::runtime::Handle::block_on. There are only a few things that will need to be changed in the code.

    Firstly, we will need to add the #[tokio::main] macro and make the main function async.

    #[tokio::main]
    async fn main() {...}
    

    Then, in s3.rs, we would change the following implementation with a custom runtime caller:

    if let Some(profile) = profile {
        config_loader = config_loader
            .region(
                ProfileFileRegionProvider::builder()
                    .profile_name(profile.as_ref())
                    .build(),
            )
            .credentials_provider(
                ProfileFileCredentialsProvider::builder()
                    .profile_name(profile.as_ref())
                    .build(),
            )
            .retry_config(
                block_on(
                    ProfileFileRetryConfigProvider::builder()
                        .profile_name(profile.as_ref())
                        .build()
                        .retry_config_builder(),
                )?
                .build(),
            )
    }
    

    with the native tokio version:

    -                    block_on(
    -                        ProfileFileRetryConfigProvider::builder()
    -                            .profile_name(profile.as_ref())
    -                            .build()
    -                            .retry_config_builder(),
    -                    )?
    -                    .build(),
    +                    tokio::runtime::Handle::current()
    +                        .block_on(async move {
    +                            ProfileFileRetryConfigProvider::builder()
    +                                .profile_name(profile.as_ref())
    +                                .build()
    +                                .retry_config_builder()
    +                                .await
    +                        })?
    +                        .build(),
                     )
    

    This way, we can also get rid of additional crate dependencies, which in this case are lazy_static and ctrlc (and possibly more!). The usages of these packages are only limited to two files, so it's safe to say that replacing these should not be an issue. I consider replacing ctrlc with tokio's native tokio::signal::ctrl_c function, which can be awaited in a dedicated tokio::task via tokio::spawn. Having fewer dependencies is always a good practice.

    Thus, the new code for capturing ctrl+c would be:

    fn wait_until_ctrlc(msg: &str) {
        let (tx, mut rx) = tokio::sync::mpsc::channel::<()>(1);
        tokio::spawn(async move {
            match tokio::signal::ctrl_c().await {
                Ok(_) => tx.send(()).await.expect("cannot capture ctrl+c"),
                Err(error) => eprintln!("{}", error),
            }
        });
    
        println!("{}", msg);
        tokio::runtime::Handle::current().block_on(async move { rx.recv().await });
    }
    

    Also notice that the channel is bounded (channel::<()>(1)), and can only receive one input from the sender, after which, the sending channel would become closed.

    For your information, the current changeset can be found on the following pages:

    • #160 (all branches linked below, depend on this PR's branch)
    • https://github.com/michaelgrigoryan25/replibyte/tree/michaelgrigoryan25/tokio
    • https://github.com/michaelgrigoryan25/replibyte/tree/michaelgrigoryan25/tokio-ctrlc

    Let me know what you think about this proposal so that I can open a PR accordingly.

    Reviewed by michaelgrigoryan25 at 2022-06-08 21:47
  • 16. Create `replibyte debug` command

    Proposal

    It seems to be a common theme that when bug reports / help issues are filed, the author is asked for certain info like

    • replibyte version
    • contents of config file

    It would be helpful for people both authoring + triaging issues if a user could run a command like replibyte debug that would collect all that useful info automatically. Then they would just need to copy that output and past it in the issue.

    Other thoughts

    • at least it would be useful to output the locally available info like stated above (replibyte version + config file). But it would be cool if the command could try to connect to the configured database and retrieve extra information like the database version
    • this could work well for this other issue, as running this command could be one of the checklist items for filing a bug-type issue
    Reviewed by wtait1-ff at 2022-06-08 19:48
Materialize simplifies application development with streaming data. Incrementally-updated materialized views - in PostgreSQL and in real time. Materialize is powered by Timely Dataflow.
Materialize simplifies application development with streaming data. Incrementally-updated materialized views - in PostgreSQL and in real time. Materialize is powered by Timely Dataflow.

Materialize is a streaming database for real-time applications. Get started Check out our getting started guide. About Materialize lets you ask questi

Jun 22, 2022
Scalable and fast data store optimised for time series data such as financial data, events, metrics for real time analysis

OnTimeDB Scalable and fast data store optimised for time series data such as financial data, events, metrics for real time analysis OnTimeDB is a time

Apr 5, 2022
A Modern Real-Time Data Processing & Analytics DBMS with Cloud-Native Architecture, built to make the Data Cloud easy
A Modern Real-Time Data Processing & Analytics DBMS with Cloud-Native Architecture, built to make the Data Cloud easy

A Modern Real-Time Data Processing & Analytics DBMS with Cloud-Native Architecture, built to make the Data Cloud easy

Jun 25, 2022
Skybase is an extremely fast, secure and reliable real-time NoSQL database with automated snapshots and SSL
Skybase is an extremely fast, secure and reliable real-time NoSQL database with automated snapshots and SSL

Skybase The next-generation NoSQL database What is Skybase? Skybase (or SkybaseDB/SDB) is an effort to provide the best of key/value stores, document

Jun 21, 2022
Skytable is an extremely fast, secure and reliable real-time NoSQL database with automated snapshots and TLS
Skytable is an extremely fast, secure and reliable real-time NoSQL database with automated snapshots and TLS

Skytable is an effort to provide the best of key/value stores, document stores and columnar databases, that is, simplicity, flexibility and queryability at scale. The name 'Skytable' exemplifies our vision to create a database that has limitless possibilities. Skytable was previously known as TerrabaseDB (and then Skybase) and is also nicknamed "STable", "Sky" and "SDB" by the community.

Jun 20, 2022
The spatial message broker and database for real-time multiplayer experiences. Official Rust implementation.

WorldQL Server Rust implementation of WorldQL, the spatial message broker and database for real-time multiplayer experiences Setup Instructions ⚠️ Thi

Jun 14, 2022
A simple library for Firebase real-time database

Firerust A very simple library to implement the Firebase real-time database in your code with the best performance Instalation Add this to your Cargo.

Apr 15, 2022
🐸Slippi DB ingests Slippi replays and puts the data into a SQLite database for easier parsing.
🐸Slippi DB ingests Slippi replays and puts the data into a SQLite database for easier parsing.

The primary goal of this project is to make it easier to analyze large amounts of Slippi data. Its end goal is to create something similar to Ballchasing.com but for Melee.

Jun 12, 2022
postgres-ical - a PostgreSQL extension that adds features related to parsing RFC-5545 « iCalendar » data from within a PostgreSQL database

postgres-ical - a PostgreSQL extension that adds features related to parsing RFC-5545 « iCalendar » data from within a PostgreSQL database

Feb 23, 2022
Visualize your database schema

dbviz Visualize your database schema. The tool loads database schema and draws it as a graph. Usage $ dbviz -d database_name | dot -Tpng > schema.png

Feb 13, 2022
SubZero - a standalone web server that turns your database directly into a REST/GraphQL api

What is this? This is a demo repository for the new subzero codebase implemented in Rust. subZero is a standalone web server that turns your database

Jun 3, 2022
a tokio-enabled data store for triple data

terminusdb-store, a tokio-enabled data store for triple data Overview This library implements a way to store triple data - data that consists of a sub

Jun 17, 2022
Experimental blockchain database

A database for the blockchain. Design considerations API The database is a universal key-value storage that supports transactions. It does not support

Jun 20, 2022
Immutable Ordered Key-Value Database Engine

PumpkinDB Build status (Linux) Build status (Windows) Project status Usable, between alpha and beta Production-readiness Depends on your risk toleranc

May 26, 2022
Distributed transactional key-value database, originally created to complement TiDB
Distributed transactional key-value database, originally created to complement TiDB

Website | Documentation | Community Chat TiKV is an open-source, distributed, and transactional key-value database. Unlike other traditional NoSQL sys

Jun 21, 2022
small distributed database protocol

clepsydra Overview This is a work-in-progress implementation of a core protocol for a minimalist distributed database. It strives to be as small and s

Dec 2, 2021
A user crud written in Rust, designed to connect to a MySQL database with full integration test coverage.

SQLX User CRUD Purpose This application demonstrates the how to implement a common design for CRUDs in, potentially, a system of microservices. The de

Jun 22, 2022
Rust version of the Haskell ERD tool. Translates a plain text description of a relational database schema to dot files representing an entity relation diagram.

erd-rs Rust CLI tool for creating entity-relationship diagrams from plain text markup. Based on erd (uses the same input format and output rendering).

May 23, 2022
AgateDB is an embeddable, persistent and fast key-value (KV) database written in pure Rust

AgateDB is an embeddable, persistent and fast key-value (KV) database written in pure Rust. It is designed as an experimental engine for the TiKV project, and will bring aggressive optimizations for TiKV specifically.

Jun 22, 2022