SQLite clone from scratch in Rust

Related tags

Database rust_sqlite
Overview

Rust-SQLite (SQLRite)

Build Status dependency status Coverage Status Maintenance MIT licensed

Rust-SQLite, aka SQLRite , is a simple embedded database modeled off SQLite, but developed with Rust. The goal is get a better understanding of database internals by building one.

What I cannot create, I do not understand. — Richard Feynman

Read the series of posts about it:

What would SQLite look like if written in Rust?

The SQLite Architecture

Requirements

Before you begin, ensure you have met the following requirements:

Usage (TBD)

> ./rust_sqlite -- help
Rust-SQLite 0.1.0
Joao Henrique Machado Silva <[email protected]>
Light version of SQLite developed with Rust

USAGE:
    rust_sqlite

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

Project Progress

Not checked means I am currently working on.

  • CLI and REPL Interface
  • Parse meta commands and sql commands.
  • Execute simple commands
  • Standarized error handling
  • Generic validation structure for SQL Commands.
  • Create Table Command
  • Improve error handling with https://github.com/dtolnay/thiserror
  • Simple INSERT queries command parsing.
  • Serialization | Deserialization to and from binary encodings (Investigating).

Roadmap

Features that are in the roadmap of the project:

Ideally in order of priority, but nothing set in stone.

  • In memory BTreeMap indexes initially only for PRIMARY KEYS.
  • Simple SELECT queries (Single WHERE clause and no JOINS).
  • Storage engine.
  • Implement Open command to load database with a command .open
  • UNIQUE key constraints.
  • Joins
    • INNER JOIN (or sometimes called simple join)
    • LEFT OUTER JOIN (or sometimes called LEFT JOIN)
    • CROSS JOIN
    • The RIGHT OUTER JOIN and FULL OUTER JOIN are not supported in SQLite.
  • Pager Module
    • Implementing transactional ACID properties
    • Concurrency
    • Lock Manager
  • Indexing - cost and performance gain analysis
  • Benchmarking vs SQLite for comparison
  • Server Client / Connection Manager
  • Different implementations of storage engines and data structures to optimize for different scenarios
    • Write Heavy - LSM Tree && SSTable
    • Read Heavy - B-Tree

Contributing

Pull requests are warmly welcome!!!

For major changes, please open an issue first and let's talk about it. We are all ears!

If you'd like to contribute, please fork the repository and make changes as you'd like and shoot a Pull Request our way!

Please make sure to update tests as appropriate.

If you feel like you need it go check the GitHub documentation on creating a pull request.

Code of Conduct

Contribution to the project is organized under the terms of the Contributor Covenant, the maintainer of SQLRite, @joaoh82, promises to intervene to uphold that code of conduct.

Contact

If you want to contact me you can reach me at [email protected].

Inspiration
You might also like...
Using embedded database modeled off SQLite - in Rust
Using embedded database modeled off SQLite - in Rust

Rust-SQLite (SQLRite) Rust-SQLite, aka SQLRite , is a simple embedded database modeled off SQLite, but developed with Rust. The goal is get a better u

SQLite-based on-disk cache for Rust.

sqlite-cache SQLite-based on-disk cache for Rust. Usage let cache = Cache::new( CacheConfig::default(), rusqlite::Connection::open_in_memory()

Query is a Rust server for your remote SQLite databases and a CLI to manage them.

Query Query is a Rust server for your remote SQLite databases and a CLI to manage them. Table Of Contents Run A Query Server CLI Install Use The Insta

Simple and handy btrfs snapshoting tool. Supports unattended snapshots, tracking, restoring, automatic cleanup and more. Backed with SQLite.
Simple and handy btrfs snapshoting tool. Supports unattended snapshots, tracking, restoring, automatic cleanup and more. Backed with SQLite.

Description Simple and handy btrfs snapshoting tool. Supports unattended snapshots, tracking, restoring, automatic cleanup and more. Backed with SQLit

Some bunch of test scripts to generate a SQLite DB with 1B rows in fastest possible way

To find out the fastest way to create an SQLite DB with 1B random rows.

🐸Slippi DB ingests Slippi replays and puts the data into a SQLite database for easier parsing.
🐸Slippi DB ingests Slippi replays and puts the data into a SQLite database for easier parsing.

The primary goal of this project is to make it easier to analyze large amounts of Slippi data. Its end goal is to create something similar to Ballchasing.com but for Melee.

A tool for automated migrations for PostgreSQL, SQLite and MySQL.

Models Models is an implementation for a SQL migration management tool. It supports PostgreSQL, MySQL, and SQLite. Quick Start install the CLI by runn

Interface to SQLite

SQLite The package provides an interface to SQLite. Example Open a connection, create a table, and insert some rows: let connection = sqlite::open(":m

SQLite compiled to WASM with pluggable data storage

wasm-sqlite SQLite compiled to WASM with pluggable data storage. Useful to save SQLite in e.g. Cloudflare Durable Objects (example: https://github.com

Comments
  • Simple SELECT queries (Single WHERE clause and no JOINS).

    Simple SELECT queries (Single WHERE clause and no JOINS).

    Simple SELECT queries

    Expect

    Simple SELECT queries (Single WHERE clause and no JOINS).

    Recognition how to do it

    1. recognising how it is done in sqllite.
    opened by grzesiekb 3
  • Rethink of project logic

    Rethink of project logic

    Hey, I was just starting to implement the SELECT query of the engine and I guess I've been thinking about the overall approach. At the moment it seems to be very modular (which isn't actually bad when it comes to abstracting over the engine). Parse query -> Parse to structs -> Execute. I think we should remove the middle step and go straight to execution. This would mean fewer allocations along the line. So basically take sqlparser's output and just use that in execution I'd love to know what you think.

    enhancement 
    opened by joshbenaron 3
  • Using WASM instead of custom virtual machine?

    Using WASM instead of custom virtual machine?

    I was actually toying around with building a Rust database not too long ago & the interesting thought I had was about using WASM w/ a custom API instead of a VM to execute queries. The mental model I had was one of LLVM IR - that would make this suitable as a general database layer that's potentially accessible from any language. The JIT part of WASM is a nice "free" feature you get although I don't think it would meaningfully add performance but I think having a language-agnostic target API to build against could be interesting, letting any language target their queries against the runtime (i.e. write the database code in the language of your choice with domain-specific optimizations rather than using plain text SQL).

    The open design question I didn't explore is what does the API look like & do you put the plan optimizer as part of the SQL layer or if you have the WASM API be the plan optimizer. I think the latter is probably the easiest to bring up while the former offers the largest opportunity for exploring greenfield ideas in this space.

    I know there's exploration of integrating WASM for stored procedures, but you would get that implicitly for free if it was WASM at the lowest layers.

    On the other hand, this could be a terrible idea. Wonder what your thought on this architecture is.

    opened by vlovich 5
Owner
João Henrique Machado Silva
I do server stuff...
João Henrique Machado Silva
X-Engine: A SQL Engine built from scratch in Rust.

XNGIN (pronounced "X Engine") This is a personal project to build a SQL engine from scratch. The project name is inspired by Nginx, which is a very po

Jiang Zhe 111 Dec 15, 2022
🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, SQLite, and MSSQL.

SQLx ?? The Rust SQL Toolkit Install | Usage | Docs Built with ❤️ by The LaunchBadge team SQLx is an async, pure Rust† SQL crate featuring compile-tim

launchbadge 7.6k Dec 31, 2022
Simple document-based NoSQL DBMS from scratch

cudb (a.k.a. cuda++) Simple document-based noSQL DBMS modelled after MongoDB. (Has nothing to do with CUDA, has a lot to do with the Cooper Union and

Jonathan Lam 3 Dec 18, 2021
Ergonomic bindings to SQLite for Rust

Rusqlite Rusqlite is an ergonomic wrapper for using SQLite from Rust. It attempts to expose an interface similar to rust-postgres. use rusqlite::{para

Rusqlite 1.9k Jan 5, 2023
Provides a Rust-based SQLite extension for using Hypercore as the VFS for your databases.

SQLite and Hypercore A Rust library providing SQLite with an virtual file system to enable Hypercore as a means of storage. Contributing The primary r

Jacky Alciné 14 Dec 5, 2022
ChiselStore is an embeddable, distributed SQLite for Rust, powered by Little Raft.

ChiselStore ChiselStore is an embeddable, distributed SQLite for Rust, powered by Little Raft. SQLite is a fast and compact relational database manage

null 516 Jan 2, 2023
cogo rust coroutine database driver (Mysql,Postgres,Sqlite)

cdbc Coroutine Database driver Connectivity.based on cogo High concurrency,based on coroutine No Future<'q,Output=*>,No async fn, No .await , no Poll*

co-rs 10 Nov 13, 2022
Rusqlite is an ergonomic wrapper for using SQLite from Rust

Rusqlite Rusqlite is an ergonomic wrapper for using SQLite from Rust. It attempts to expose an interface similar to rust-postgres. use rusqlite::{para

Rusqlite 1.9k Jan 7, 2023
Build SQLite virtual file systems (VFS) by implementing a simple Rust trait.

sqlite-vfs Build SQLite virtual file systems (VFS) by implementing a simple Rust trait. Documentation | Example This library is build for my own use-c

Markus Ast 56 Dec 19, 2022
A Rust-based comment server using SQLite and an intuitive REST API.

soudan A Rust-based comment server using SQLite and an intuitive REST API. Soudan is built with simplicity and static sites in mind. CLI usage See sou

Elnu 0 Dec 19, 2022