Blazingly fast data generation & seeding for MongoDB

Valencian Digital

Last update: Jan 12, 2022

Related tags

Database rust database mongodb seeding data-generation

Overview

Planter

Blazingly fast and simple data generation & seeding for MongoDB

Installation

Use the package manager cargo to install planter. Add the following to your Cargo.toml

[dependencies]
planter = {git = "https://github.com/valencian-digital/planter", branch = "main"}# from online repo```

Usage

fn user_generator(_history: &planter::GeneratedData) -> bson::Document {
    return doc! {
        "_id": bson::Bson::ObjectId(bson::oid::ObjectId::new()),
        "name": "Jane Doe"
    };
}

...

let documents_per_collection = 1000;
let collections: Vec<(String, planter::EntityGenerator)> = vec![
    (String::from("users"), user_generator),
];

planter::seed_data(
    collections,
    planter::Configurations::new(documents_per_collection, planter::SeedMode::Disk),
);

Then to import that data into your running MongoDB database, execute the following command:

mongorestore --db=test data/

Look at the mongorestore documentation in order to see all of options

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

License

MIT

Implements the packet parser for Gran Turismo 7 telemetry data, allowing a developer to retrieve data from a running game.

gran-turismo-query Implements the packet parser for Gran Turismo 7 telemetry data, allowing a developer to retrieve data from a running game. Features

3 Dec 11, 2023

A fast and simple in-memory database with a key-value data model written in Rust

Segment Segment is a simple & fast in-memory database with a key-value data model written in Rust. Features Dynamic keyspaces Keyspace level control o

61 Jan 5, 2023

Efficient and fast querying and parsing of GTDB's data

xgt xgt is a Rust tool that enables efficient querying and parsing of the GTDB database. xgt consists of a collection of commands mirroring the GTDB A

7 Apr 1, 2023

Materialize simplifies application development with streaming data. Incrementally-updated materialized views - in PostgreSQL and in real time. Materialize is powered by Timely Dataflow.

Materialize is a streaming database for real-time applications. Get started Check out our getting started guide. About Materialize lets you ask questi

4.7k Jan 8, 2023

🐸Slippi DB ingests Slippi replays and puts the data into a SQLite database for easier parsing.

The primary goal of this project is to make it easier to analyze large amounts of Slippi data. Its end goal is to create something similar to Ballchasing.com but for Melee.

20 Jan 2, 2023

A Key-Value data storage system. - dorea db

Dorea DB 🛰 Dorea is a key-value data storage system. It is based on the Bitcask storage model Documentation | Crates.io | API Doucment 简体中文 | English

112 Dec 2, 2022

Hypergraph is data structure library to create a directed hypergraph in which a hyperedge can join any number of vertices.

224 Dec 21, 2022

Zenith substitutes PostgreSQL storage layer and redistributes data across a cluster of nodes

5.7k Jan 6, 2023

It's not a novel data sturcture just AVL and Btree for rust

This crate named as ABtree but this not means it is a novel data sturcture. It’s just AVL tree and Btree. For the Btree, what makes it different from

3 Jun 20, 2022

Comments

Improve Generation Performance (Consider Concurrency)
Things That Are Problems

convert_collection function (necessary to get the right writing format)

generate_collections function (takes the vast majority of the time still)
opened by LuisOsta 2
Bump crossbeam-utils from 0.8.5 to 0.8.8
Bumps crossbeam-utils from 0.8.5 to 0.8.8.

Release notes

Sourced from crossbeam-utils's releases.

crossbeam-utils 0.8.8

Fix a bug when unstable loom support is enabled. (#787)

crossbeam-utils 0.8.7

Add AtomicCell<{i*,u*}>::{fetch_max,fetch_min}. (#785)

Add AtomicCell<{i*,u*,bool}>::fetch_nand. (#785)

Fix unsoundness of AtomicCell<{i,u}64> arithmetics on 32-bit targets that support Atomic{I,U}64 (#781)

crossbeam-utils 0.8.6

Re-add AtomicCell<{i,u}64>::{fetch_add,fetch_sub,fetch_and,fetch_or,fetch_xor} that were accidentally removed in ~~0.8.0~~ 0.7.1 on targets that do not support Atomic{I,U}64. (#767)

Re-add AtomicCell<{i,u}128>::{fetch_add,fetch_sub,fetch_and,fetch_or,fetch_xor} that were accidentally removed in ~~0.8.0~~ 0.7.1. (#767)

Commits

5b4d808 Merge #800

6fa7246 Prepare for the next release

8efcd02 Merge #798

baedeb0 channel: Do not panic on very large timeout

113daf4 Merge #799

5cb7cac Update no_atomic.rs

0081fcc Merge #789 #797

7b2d65f Update to stabilized const_fn_trait_bound

bd75c3c Add force_push to ArrayQueue

b11f1a8 Merge #793

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0