Teach your PostgreSQL database how to speak MongoDB Wire Protocol

Overview

banner-small

β€œIf it looks like MongoDB, swims like MongoDB, and quacks like MongoDB, then it probably is PostgreSQL.” πŸ™ƒ

Discord | Online Demo | Intro Video | Quickstart

OxideDB

OxideDB is a translation layer that works as a MongoDB database server while using PostgreSQL's JSON capabilities as the underlying data store.

Audience

This project might be something that you could be interested on if:

  • You spend too much time managing; or too much money paying for a MongoDB instance, while only using it as a simple document store, without any sharding features
  • You already have a running PostgreSQL deployment, or prefer to manage it over MongoDB

On the other hand, if your use-case leverages MongoDB as a distributed database, then unfortunately this project might not be for you. At least right now supporting multi-sharding and scale-out deployments is not part of the roadmap.

Current status

The project was heavily inspired by FerretDB and is on its early days. The main difference is that there is no intention to support any database other than PostgreSQL (FerretDB is also supporting Tigris) and it's written in Rust, as opposed to Go.

In order to translate the MongoDB Query language - which is based on JSON - to SQL I have ported the mongodb-language-model library that was originally written in Node.js and PEG.js to Rust and pest.rs. It was an excellent opportunity to learn how parsers work in a bit more depth.

You can check it out here: mongodb-language-model-rust.

At this moment, it's being developed as a personal project, but contributors are highly welcomed. If that something you'd be interested on, be more than welcome to contact me.

Online demo

There's an online demo of the Web Interface live hosted at Railway.app.

Quickstart

Download the latest binary and run it. You will need to point it to a running PostgreSQL for Oxide to use as its backend.

> $ ./oxide --postgres-url "postgres://postgres:postgres@localhost:5432/test"
[2022-07-13T02:57:46Z INFO  oxide::server] Connecting to postgres://*****:*****@localhost:5432/test...
[2022-07-13T02:57:46Z INFO  oxide::server] OxideDB listening on 127.0.0.1:27017...

And with the database configuration set, you can use any MongoDB client to connect to OxideDB, like mongosh:

> $ mongosh
Current Mongosh Log ID:	62ce3531d10f489bc82520c4
Connecting to:		mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.5.0
Using MongoDB:		3.6.23
Using Mongosh:		1.5.0

For mongosh info see: https://docs.mongodb.com/mongodb-shell/

------
   The server generated these startup warnings when booting
   2022-07-12T18:56:41.654-0300:
   2022-07-12T18:56:41.654-0300: ** WARNING: Access control is not enabled for the database.
   2022-07-12T18:56:41.654-0300: **          Read and write access to data and configuration is unrestricted.
   2022-07-12T18:56:41.654-0300:
------

test> db.col.insertMany([{ name: "Felipe" }, { name: "Fernanda" }]);
{
  acknowledged: true,
  insertedIds: {
    '0': ObjectId("62ce3536d10f489bc82520c5"),
    '1': ObjectId("62ce3536d10f489bc82520c6")
  }
}
test> db.col.find({ "name": "Fernanda" })
[ { _id: ObjectId("62ce3536d10f489bc82520c6"), name: 'Fernanda' } ]

Quick intro

Intro to OxideDB

Web interface

Web UI

If you want to perform quick queries on your database, you can use the built in web interface, just start oxide with the --web flag:

> $ ./oxide --postgres-url postgres://postgres:postgres@localhost:5432/oxide
[2022-07-29T00:20:11Z INFO  oxide::server] Connecting to postgres://*****:*****@localhost:5432/oxide...
[2022-07-29T00:20:11Z INFO  oxide::server] OxideDB listening on 0.0.0.0:27017...

Now just navigate to http://localhost:5432 and you'll be able to perform quick queries.

Running options

You can also set the DATABASE_URL environment variable or even use a .env file.

By default oxide will bind to 127.0.0.1 and port 27017. You can change those settings using the following parameters:

> $ ./oxide --help
oxide 0.1.6
A database compatible with MongoDB Wire Protocol that uses PostgreSQL for backend storage.

USAGE:
    oxide [OPTIONS]
    oxide <SUBCOMMAND>

OPTIONS:
    -d, --debug                          Show debugging information
    -h, --help                           Print help information
    -l, --listen-addr <LISTEN_ADDR>      Listening address, defaults to 127.0.0.1
    -p, --port <PORT>                    Listening port, defaults to 27017
    -u, --postgres-url <POSTGRES_URL>    PostgreSQL connection URL
    -V, --version                        Print version information
    -w, --web                            Starts web interface
        --web-addr <WEB_ADDR>            Web binding address

SUBCOMMANDS:
    help    Print this message or the help of the given subcommand(s)
    web     Start OxideDB web interface

Running with Docker

Assuming you're running a local PostgreSQL instance, you can run OxideDB with Docker with the command below.

docker build -t oxide .
docker run -d -p 27017:27017 -p 8087:8087 -e DATABASE_URL=postgres://postgres:[email protected]:5432/test --name oxide oxide

Running from source

git clone https://github.com/fcoury/oxide.git
cd oxide
export DATABASE_URL="postgres://postgres:postgres@localhost:5432/test"
make start
Comments
  • Got unknown OP_MSG command: getlasterror

    Got unknown OP_MSG command: getlasterror

    ./oxide --postgres-url "postgres://postgres:postgres@localhost:5432/test" [2022-08-27T17:28:22Z INFO oxide::server] Connecting to postgres://***:***@localhost:5432/test... [2022-08-27T17:28:22Z INFO oxide::server] OxideDB listening on 0.0.0.0:27017... [2022-08-27T17:28:24Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})] [2022-08-27T17:28:24Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})] [2022-08-27T17:28:27Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})] [2022-08-27T17:28:42Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})] [2022-08-27T17:28:42Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})] [2022-08-27T17:28:42Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})] thread '' panicked at 'called Result::unwrap() on an Err value: ValueAccessError: field is not present', src/commands/insert.rs:20:66 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace [2022-08-27T17:28:46Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})] [2022-08-27T17:28:46Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})] [2022-08-27T17:28:46Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})] [2022-08-27T17:28:54Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})] [2022-08-27T17:29:46Z ERROR oxide::handler] Got unknown OP_MSG command: getlasterror [Document({"getlasterror": Int32(1), "$db": String("admin")})]

    bug 
    opened by whisper-bye 8
  • Unable to create index - ERROR: syntax error at or near

    Unable to create index - ERROR: syntax error at or near "."

    Hello! I'm working on testing OxideDB and FerretDB with Meteor. https://github.com/meteor/meteor/discussions/12150

    While testing OxideDB, I ran into an issue with creating an index.

    https://github.com/meteor/meteor/discussions/12150#discussioncomment-3469279

    postgres    | 2022-08-24 20:42:36.260 UTC [1] LOG:  database system is ready to accept connections
    setup       | CREATE SCHEMA
    setup exited with code 0
    oxide       | [2022-08-24T20:42:45Z INFO  oxide::server] OxideDB listening on 0.0.0.0:27017...
    meteor_app  | Successfully connected to MongoDB
    meteor_app  | Starting app...
    postgres    | 2022-08-24 20:42:46.281 UTC [80] ERROR:  syntax error at or near "." at character 41
    postgres    | 2022-08-24 20:42:46.281 UTC [80] STATEMENT:  CREATE UNIQUE INDEX IF NOT EXISTS emails.address_1 ON "meteor"."users" ((_jsonb->'emails.address'))
    oxide       | thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: db error: ERROR: syntax error at or near "."
    oxide       | 
    oxide       | Caused by:
    oxide       |     ERROR: syntax error at or near "."
    oxide       | 
    oxide       | Location:
    oxide       |     /rustc/e092d0b6b43f2de967af0887873151bb1c0b18d3/library/core/src/convert/mod.rs:550:9', src/commands/create_indexes.rs:33:18
    oxide       | note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    meteor_app  | 
    meteor_app  | /opt/bundle/bundle/programs/server/node_modules/fibers/future.js:313
    meteor_app  |                                           throw(ex);
    meteor_app  |                                           ^
    meteor_app  | errorClass [Error]: [An error occurred when creating an index for collection "users: connection 1 to 172.20.0.6:27017 closed]
    meteor_app  |     at Collection.createIndex (packages/mongo/collection.js:770:15)
    meteor_app  |     at setupUsersCollection (packages/accounts-base/accounts_server.js:1813:9)
    meteor_app  |     at new AccountsServer (packages/accounts-base/accounts_server.js:74:5)
    meteor_app  |     at packages/accounts-base/server_main.js:7:12
    meteor_app  |     at module (packages/accounts-base/server_main.js:19:1)
    meteor_app  |     at fileEvaluate (packages/modules-runtime.js:336:7)
    meteor_app  |     at Module.require (packages/modules-runtime.js:238:14)
    meteor_app  |     at require (packages/modules-runtime.js:258:21)
    meteor_app  |     at /opt/bundle/bundle/programs/server/packages/accounts-base.js:2395:15
    meteor_app  |     at /opt/bundle/bundle/programs/server/packages/accounts-base.js:2402:3
    meteor_app  |     at /opt/bundle/bundle/programs/server/boot.js:401:38
    meteor_app  |     at Array.forEach (<anonymous>)
    meteor_app  |     at /opt/bundle/bundle/programs/server/boot.js:226:21
    meteor_app  |     at /opt/bundle/bundle/programs/server/boot.js:464:7
    meteor_app  |     at Function.run (/opt/bundle/bundle/programs/server/profile.js:280:14)
    meteor_app  |     at /opt/bundle/bundle/programs/server/boot.js:463:13 {
    meteor_app  |   isClientSafe: true,
    meteor_app  |   error: 'An error occurred when creating an index for collection "users: connection 1 to 172.20.0.6:27017 closed',
    meteor_app  |   reason: undefined,
    meteor_app  |   details: undefined,
    meteor_app  |   errorType: 'Meteor.Error'
    meteor_app  | }
    meteor_app exited with code 1
    

    This may be related to the code here: https://github.com/fcoury/oxide/commit/192f076f6e65d1d41b4f6ce53a53db121051b3fe

    An example object for testing would be:

    {
      "_id": "5QGdzKp2nCKtwz9QJ",
      "emails": [
        {
          "address": "[email protected]",
          "verified": true
        }
      ],
      "profile": {
        "username": "wreiske",
        "name": "Will Reiske",
        "photo": "https://s.gravatar.com/avatar/f830b06583b46bd48115f506b98d5689?s=512&d=mp",
      }
    }
    

    Creating an index on emails.address fails.

    opened by wreiske 4
  • Add `findAndModify`

    Add `findAndModify`

    https://www.mongodb.com/docs/v6.0/reference/command/findAndModify/

    [2022-08-04T14:47:13Z ERROR oxide::handler] Got unknown OP_MSG command: findAndModify
        [Document({"findAndModify": String("pages"), "query": Document({"type": String("page"), "name": String("results")}), "new": Boolean(false), "remove": Boolean(false), "upsert": Boolean(true), "update": Document({"type": String("page"), "name": String("results"), "title": String("Results"), "queue": String("results"), "providers": Array([Document({"name": String("results"), "provides": String("order"), "main": Boolean(true)})]), "fields": Array([Document({"field": String("order.Patient.FirstName")}), Document({"field": String("order.Patient.LastName")}), Document({"type": String("date"), "label": String("DOB"), "field": String("order.Patient.Dob"), "metadata": Document({"destination": String("PRW")})}), Document({"type": String("select"), "label": String("Sex"), "field": String("order.Patient.Sex"), "values": Array([Document({"key": String("U"), "value": String("Unknown")}), Document({"key": String("M"), "value": String("Male")}), Document({"key": String("F"), "value": String("Female")})])}), Document({"field": String("order.Patient.Sex"), "label": String("Sex"), "type": String("select"), "values": Array([Document({"key": String("U"), "value": String("Unknown")}), Document({"key": String("M"), "value": String("Male")}), Document({"key": String("F"), "value": String("Female")})])}), Document({"field": String("order.Patient.Gender"), "label": String("Gender"), "type": String("select"), "values": Array([Document({"key": String("U"), "value": String("Unknown")}), Document({"key": String("CM"), "value": String("Man")}), Document({"key": String("CF"), "value": String("Woman")})])}), Document({"field": String("order.Results"), "label": String("Results")}), Document({"field": String("order.Result.id"), "type": String("text"), "label": String("")}), Document({"field": String("order.Result.mutation"), "type": String("text")}), Document({"field": String("order.Result.result"), "type": String("select"), "values": Array([Document({"key": String("Not Detected"), "value": String("Not Detected")}), Document({"key": String("Detected"), "value": String("Detected")})])}), Document({"field": String("order.Result.type"), "type": String("select"), "values": Array([Document({"key": String("D"), "value": String("Driver")}), Document({"key": String("P"), "value": String("Passenger")}), Document({"key": String(""), "value": String("-")})])}), Document({"field": String("order.Result.qtd"), "label": String("MM/mL")}), Document({"field": String("orders.SetPreview"), "label": String("Preview PDF"), "type": String("button-setter"), "severity": String("info"), "icon": String("fa-file-pdf-o"), "setValue": Array([Document({"field": String("isFaxPreviewVisible"), "expr": String("true")}), Document({"field": String("fax.PreviewRefreshTime"), "expr": String("new Date().getTime()")})])}), Document({"field": String("order.data.PDFPreview"), "label": String(""), "type": String("pdf"), "provider": String("result-preview"), "providerParams": Document({"order": String("order"), "template": Document({"expr": String("'patient-report'")}), "previewRefreshTime": String("fax.PreviewRefreshTime")}), "refreshCondition": String("!_.isEqual(data.providerParams.previewRefreshTime, data.providerParamsOld.previewRefreshTime)"), "height": Document({"_expr": String("window.innerHeight - 220")})})]), "display": Array([Array([Document({"type": String("column"), "scroll": Boolean(true), "height": String("calc(100vh - 8rem)"), "display": Array([Document({"type": String("group"), "title": String("Patient"), "display": Array([Array([String("order.Patient.FirstName"), String("order.Patient.LastName")]), Array([String("order.Patient.Dob"), String("order.Patient.Sex"), String("order.Patient.Gender")]), Document({"type": String("group-actions"), "alignment": String("right"), "display": Array([String("orders.SetPreview")])})])}), Document({"type": String("group"), "title": String("Results"), "display": Array([Document({"type": String("grid"), "source": String("order.Results"), "fieldIndex": String("id"), "columns": Array([String("id"), String("mutation"), String("result"), String("type"), String("qtd")])})])})])}), String("order.data.PDFPreview")])])}), "$db": String("cognito-product")})]
    
    enhancement 
    opened by fcoury 2
  • Got unknown OP_MSG command: count

    Got unknown OP_MSG command: count

    Hello! I'm working on testing OxideDB and FerretDB with Meteor. https://github.com/meteor/meteor/discussions/12150

    Thank you for your incredibly fast PR and fix for https://github.com/fcoury/oxide/issues/60. I've pulled in the latest changes and now I'm having this issue:

    oxide       | [2022-08-25T21:13:17Z INFO  oxide::server] OxideDB listening on 0.0.0.0:27017...
    meteor_app  | Successfully connected to MongoDB
    meteor_app  | Starting app...
    oxide       | [2022-08-25T21:13:18Z ERROR oxide::handler] Got unknown OP_MSG command: count
    oxide       |     [Document({"count": String("tasks"), "query": Document({}), "$db": String("meteor")})]
    meteor_app  | /opt/bundle/bundle/programs/server/node_modules/fibers/future.js:313
    meteor_app  |                                           throw(ex);
    meteor_app  |                                           ^
    meteor_app  | 
    meteor_app  | MongoServerError: no such command: 'count'
    meteor_app  |     at MessageStream.messageHandler (/opt/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:462:30)
    meteor_app  |     at MessageStream.emit (events.js:400:28)
    meteor_app  |     at MessageStream.emit (domain.js:475:12)
    meteor_app  |     at processIncomingData (/opt/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:108:16)
    meteor_app  |     at MessageStream._write (/opt/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:28:9)
    meteor_app  |     at writeOrBuffer (internal/streams/writable.js:358:12)
    meteor_app  |     at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    meteor_app  |     at Socket.ondata (internal/streams/readable.js:731:22)
    meteor_app  |     at Socket.emit (events.js:400:28)
    meteor_app  |     at Socket.emit (domain.js:475:12)
    meteor_app  |     at addChunk (internal/streams/readable.js:293:12)
    meteor_app  |     at readableAddChunk (internal/streams/readable.js:267:9)
    meteor_app  |     at Socket.Readable.push (internal/streams/readable.js:206:10)
    meteor_app  |     at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
    meteor_app  |   ok: 0,
    meteor_app  |   code: 59,
    meteor_app  |   codeName: 'CommandNotFound'
    meteor_app  | }
    meteor_app exited with code 1
    

    Thank you for all that you do!

    opened by wreiske 1
  • Spelling

    Spelling

    This PR corrects misspellings identified by the check-spelling action.

    The misspellings have been reported at https://github.com/jsoref/oxide/commit/c962f0aefd60a55fe79d03f6dc8254719cac03d0#commitcomment-80112315

    The action reports that the changes in this PR would make it happy: https://github.com/jsoref/oxide/commit/22e21092ec446b932081e31f481f8cbd0971eb87

    Note: this PR does not include the action. If you're interested in running a spell check on every PR and push, that can be offered separately.

    opened by jsoref 1
  • Handles multiple sections on insert message

    Handles multiple sections on insert message

    This addresses a bug handling messages with multiple sections: one kind 0 message containing the command information and a subsequent section of kind 1 with only the documents.

    This now works for insertions using IntelliJ IDEA Database console:

    image

    Fixes #66

    opened by fcoury 0
  • Matches arrays when searching for nested fields

    Matches arrays when searching for nested fields

    For a query like {"a.b.c": 1}, MongoDB matches a document where the field "b" is an array containing the object {"c": 1}.

    This case is handled by this PR.

    We also need to investigate the broader case where it should also match the following objects:

    {"a": [{"b": {"c": 1}}]} {"a": [{"b": [{"c": 1}]}]}

    Those other cases are being tracked on issue #67

    opened by fcoury 0
  • Implicit array matching

    Implicit array matching

    MongoDB queries always match arrays implicitly.

    For instance: if you run the query {"a.b.c": 1}, it should match a document where the field "b" is an array containing the object {"c": 1}.

    This query also should match at least the following documents:

    {"a": [{"b": {"c": 1}}]} {"a": [{"b": [{"c": 1}]}]}

    enhancement 
    opened by fcoury 0
  • Support `$addToSet` on updates

    Support `$addToSet` on updates

    First identified here - https://github.com/meteor/meteor/discussions/12150

    https://www.mongodb.com/docs/manual/reference/operator/update/addToSet/

    oxide       | [2022-08-26T01:23:32Z ERROR oxide::server] Error while handling: Err(InvalidUpdateError { reason: "Unknown modifier: $addToSet" })
    meteor_app  | Exception while invoking method 'login' MongoServerError: Err(InvalidUpdateError { reason: "Unknown modifier: $addToSet" })
    meteor_app  |     at MessageStream.messageHandler (/opt/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/connection.js:462:30)
    meteor_app  |     at MessageStream.emit (events.js:400:28)
    meteor_app  |     at MessageStream.emit (domain.js:475:12)
    meteor_app  |     at processIncomingData (/opt/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:108:16)
    meteor_app  |     at MessageStream._write (/opt/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/cmap/message_stream.js:28:9)
    meteor_app  |     at writeOrBuffer (internal/streams/writable.js:358:12)
    meteor_app  |     at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    meteor_app  |     at Socket.ondata (internal/streams/readable.js:731:22)
    meteor_app  |     at Socket.emit (events.js:400:28)
    meteor_app  |     at Socket.emit (domain.js:475:12)
    meteor_app  |     at addChunk (internal/streams/readable.js:293:12)
    meteor_app  |     at readableAddChunk (internal/streams/readable.js:267:9)
    meteor_app  |     at Socket.Readable.push (internal/streams/readable.js:206:10)
    meteor_app  |     at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
    
    enhancement 
    opened by fcoury 0
  • Implement `$skip` pipeline stage

    Implement `$skip` pipeline stage

    https://www.mongodb.com/docs/manual/reference/operator/aggregation/skip/

    Thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: CommandExecutionError { message: "Unrecognized pipeline stage name: '{ \"$skip\": 0 }'" }', src/commands/aggregate/mod.rs:38:44
    stack backtrace:
       0: _rust_begin_unwind
       1: core::panicking::panic_fmt
       2: core::result::unwrap_failed
    
    enhancement 
    opened by fcoury 0
  • Implement `$count` pipeline stage

    Implement `$count` pipeline stage

    They are missing and used on our product:

    thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: CommandExecutionError { message: "Unrecognized pipeline stage name: '{ \"$count\": \"count\" }'" }', src/commands/aggregate/mod.rs:38:44
       3: core::result::Result<T,E>::unwrap
                 at /private/tmp/rust-20220719-52938-vitvrw/rustc-1.62.1-src/library/core/src/result.rs:1078:23
       4: <oxide::commands::aggregate::Aggregate as oxide::commands::Handler>::handle
                 at ./src/commands/aggregate/mod.rs:38:19
       5: oxide::handler::run
                 at ./src/handler.rs:124:9
       6: oxide::handler::handle_op_msg
                 at ./src/handler.rs:208:16
       7: oxide::handler::route
                 at ./src/handler.rs:270:31
       8: oxide::handler::handle
                 at ./src/handler.rs:101:11
       9: oxide::server::handle_connection
                 at ./src/server.rs:116:42
      10: oxide::server::Server::start_with_pool::{{closure}}
                 at ./src/server.rs:68:17
      11: core::ops::function::FnOnce::call_once{{vtable.shim}}
                 at /private/tmp/rust-20220719-52938-vitvrw/rustc-1.62.1-src/library/core/src/ops/function.rs:248:5
      12: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
                 at /private/tmp/rust-20220719-52938-vitvrw/rustc-1.62.1-src/library/alloc/src/boxed.rs:1872:9
      13: oxide::threadpool::Worker::new::{{closure}}
                 at ./src/threadpool.rs:86:21
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    stack backtrace:
       0: _rust_begin_unwind
       1: core::panicking::panic_fmt
       2: core::result::unwrap_failed
       3: core::result::Result<T,E>::unwrap
                 at /private/tmp/rust-20220719-52938-vitvrw/rustc-1.62.1-src/library/core/src/result.rs:1078:23
       4: <oxide::commands::aggregate::Aggregate as oxide::commands::Handler>::handle
                 at ./src/commands/aggregate/mod.rs:38:19
       5: oxide::handler::run
                 at ./src/handler.rs:124:9
       6: oxide::handler::handle_op_msg
                 at ./src/handler.rs:208:16
       7: oxide::handler::route
                 at ./src/handler.rs:270:31
       8: oxide::handler::handle
                 at ./src/handler.rs:101:11
       9: oxide::server::handle_connection
                 at ./src/server.rs:116:42
      10: oxide::server::Server::start_with_pool::{{closure}}
                 at ./src/server.rs:68:17
      11: core::ops::function::FnOnce::call_once{{vtable.shim}}
                 at /private/tmp/rust-20220719-52938-vitvrw/rustc-1.62.1-src/library/core/src/ops/function.rs:248:5
      12: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
                 at /private/tmp/rust-20220719-52938-vitvrw/rustc-1.62.1-src/library/alloc/src/boxed.rs:1872:9
      13: oxide::threadpool::Worker::new::{{closure}}
                 at ./src/threadpool.rs:86:21
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    
    enhancement 
    opened by fcoury 0
  • Extract operations logic

    Extract operations logic

    Operations like $multiply are confined to the stage it's coded on ($group in this case).

    We should extract all operations and allow them to be called from many different places, like on $project phase.

    One example from MongoDB with our orders collection:

    test> db.orders.aggregate([{ $project: { array: [ '$name', { $multiply: [ '$price', '$quantity'] }] } }])
    [
      { _id: 0, array: [ 'Pepperoni', 190 ] },
      { _id: 1, array: [ 'Pepperoni', 400 ] },
      { _id: 2, array: [ 'Pepperoni', 630 ] },
      { _id: 3, array: [ 'Cheese', 180 ] },
      { _id: 4, array: [ 'Cheese', 650 ] },
      { _id: 5, array: [ 'Cheese', 140 ] },
      { _id: 6, array: [ 'Vegan', 170 ] },
      { _id: 7, array: [ 'Vegan', 180 ] }
    ]
    

    And the data:

    db.orders.insertMany([ { name: "Pepperoni", size: "small", price: 19, quantity: 10, date: ISODate("2021-03-13T08:14:30Z") }, { name: "Pepperoni", size: "medium", price: 20, quantity: 20, date: ISODate("2021-03-13T09:13:24Z") }, { name: "Pepperoni", size: "large", price: 21, quantity: 30, date: ISODate("2021-03-17T09:22:12Z") }, { name: "Cheese", size: "small", price: 12, quantity: 15, date: ISODate("2021-03-13T11:21:39.736Z") }, { name: "Cheese", size: "medium", price: 13, quantity: 50, date: ISODate("2022-01-12T21:23:13.331Z") }, { name: "Cheese", size: "large", price: 14, quantity: 10, date: ISODate("2022-01-12T05:08:13Z") }, { name: "Vegan", size: "small", price: 17, quantity: 10, date: ISODate("2021-01-13T05:08:13Z") }, { name: "Vegan", size: "medium", price: 18, quantity: 10, date: ISODate("2021-01-13T05:10:13Z") }]);
    
    enhancement refactor 
    opened by fcoury 0
  • Support remaining `$project` use cases

    Support remaining `$project` use cases

    • [x] Include Exising Fields - https://www.mongodb.com/docs/manual/reference/operator/aggregation/project/#include-existing-fields
    • [x] Exclude Fields - https://www.mongodb.com/docs/manual/reference/operator/aggregation/project/#exclude-fields
      • [ ] Exclude fields conditionally
    • [x] Suppress the _id field - https://www.mongodb.com/docs/manual/reference/operator/aggregation/project/#suppress-the-_id-field
    • [x] Add New Fields or Reset Existing Fields - https://www.mongodb.com/docs/manual/reference/operator/aggregation/project/#add-new-fields-or-reset-existing-fields
      • [x] Literal Values
      • [x] Field Rename
      • [x] New Array Fields
    • [ ] Embedded Document Fields - https://www.mongodb.com/docs/manual/reference/operator/aggregation/project/#embedded-document-fields
      • [ ] Path Collision Errors in Embedded Fields - https://www.mongodb.com/docs/manual/reference/operator/aggregation/project/#path-collision-errors-in-embedded-fields
    • [ ] Restrictions - https://www.mongodb.com/docs/manual/reference/operator/aggregation/project/#restrictions
    enhancement 
    opened by fcoury 0
  • Support all delete use cases

    Support all delete use cases

    Our implementation only deletes all records that match the query.

    We need to add proper support for deleteOne and ordered:

    https://www.mongodb.com/docs/manual/reference/method/db.collection.deleteOne/

    [Document({"delete": String("test_16deae6e-2afa-42a6-b129-ceef7d46c19b"), "deletes": Array([Document({"q": Document({"a": Int32(1)}), "limit": Int32(0)})]), "ordered": Boolean(true), "$db": String("db_test")})]
    
    enhancement 
    opened by fcoury 0
Releases(v0.1.7)
postgres-ical - a PostgreSQL extension that adds features related to parsing RFC-5545 Β« iCalendar Β» data from within a PostgreSQL database

postgres-ical - a PostgreSQL extension that adds features related to parsing RFC-5545 Β« iCalendar Β» data from within a PostgreSQL database

Edgar Onghena 1 Feb 23, 2022
A Distributed SQL Database - Building the Database in the Public to Learn Database Internals

Table of Contents Overview Usage TODO MVCC in entangleDB SQL Query Execution in entangleDB entangleDB Raft Consensus Engine What I am trying to build

Sarthak Dalabehera 38 Jan 2, 2024
The Solana AccountsDb plugin for PostgreSQL database.

The solana-accountsdb-plugin-postgres crate implements a plugin storing account data to a PostgreSQL database to illustrate how a plugin can be develo

Lijun Wang 3 Jun 16, 2022
The official MongoDB Rust Driver

MongoDB Rust Driver This repository contains the officially supported MongoDB Rust driver, a client side library that can be used to interact with Mon

mongodb 1.1k Dec 30, 2022
Blazingly fast data generation & seeding for MongoDB

Planter Blazingly fast and simple data generation & seeding for MongoDB Installation Use the package manager cargo to install planter. Add the followi

Valencian Digital 4 Jan 12, 2022
This is superseded by the official MongoDB Rust Driver

This Repository is NOT a supported MongoDB product MongoDB Rust Driver Prototype NOTE: This driver is superseded by the official MongoDB Rust driver,

MongoDB, Inc. Labs 382 Nov 15, 2022
The most efficient, scalable, and fast production-ready serverless REST API backend which provides CRUD operations for a MongoDB collection

Optimal CRUD Mongo Goals of This Project This is meant to be the most efficient, scalable, and fast production-ready serverless REST API backend which

Evaluates2 1 Feb 22, 2022
Automatically publish MongoDB changes to Redis for Meteor.

changestream-to-redis Warning The project is currently in its alpha phase. There are no production loads using it yet nor any large-scale tests were c

RadosΕ‚aw Miernik 8 Jul 29, 2023
Generate type-checked Rust from your PostgreSQL.

Cornucopia Generate type checked Rust from your SQL Install | Example Cornucopia is a small CLI utility resting on postgres designed to facilitate Pos

null 206 Dec 25, 2022
small distributed database protocol

clepsydra Overview This is a work-in-progress implementation of a core protocol for a minimalist distributed database. It strives to be as small and s

Graydon Hoare 19 Dec 2, 2021
tectonicdb is a fast, highly compressed standalone database and streaming protocol for order book ticks.

tectonicdb crate docs.rs crate.io tectonicdb tdb-core tdb-server-core tdb-cli tectonicdb is a fast, highly compressed standalone database and streamin

Ricky Han 525 Dec 23, 2022
Materialize simplifies application development with streaming data. Incrementally-updated materialized views - in PostgreSQL and in real time. Materialize is powered by Timely Dataflow.

Materialize is a streaming database for real-time applications. Get started Check out our getting started guide. About Materialize lets you ask questi

Materialize, Inc. 4.7k Jan 8, 2023
🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, SQLite, and MSSQL.

SQLx ?? The Rust SQL Toolkit Install | Usage | Docs Built with ❀️ by The LaunchBadge team SQLx is an async, pure Rust† SQL crate featuring compile-tim

launchbadge 7.6k Dec 31, 2022
Native PostgreSQL driver for the Rust programming language

Rust-Postgres PostgreSQL support for Rust. postgres Documentation A native, synchronous PostgreSQL client. tokio-postgres Documentation A native, asyn

Steven Fackler 2.8k Jan 8, 2023
PostgreSQL procedural language handler for Clojure via SCI

pl/sci Status This is very much an experiment and I'm open to feedback on where to take this next. Build Requirements lein GraalVM CE 20.3.0 Java 11 c

Michiel Borkent 45 Nov 28, 2022
A tool for automated migrations for PostgreSQL, SQLite and MySQL.

Models Models is an implementation for a SQL migration management tool. It supports PostgreSQL, MySQL, and SQLite. Quick Start install the CLI by runn

null 45 Nov 16, 2022
Zenith substitutes PostgreSQL storage layer and redistributes data across a cluster of nodes

Zenith substitutes PostgreSQL storage layer and redistributes data across a cluster of nodes

null 5.7k Jan 6, 2023
Rust library to parse, deparse and normalize SQL queries using the PostgreSQL query parser

This Rust library uses the actual PostgreSQL server source to parse SQL queries and return the internal PostgreSQL parse tree.

pganalyze 37 Dec 18, 2022
Rust - Build a CRUD API with SQLX and PostgreSQL

In this article, you'll learn how to build a CRUD API in Rust using SQLX, Actix-web, and PostgreSQL. Learning how to build a CRUD API as a developer will equip you with valuable skills for building robust, maintainable, and scalable applications.

CODEVO 5 Feb 20, 2023