Hiisi is a proof of concept libSQL written in Rust following TigerBeetle-style with deterministic simulation testing.

Related tags

Cryptography hiisi
Overview

Hiisi

Hiisi

Proof of concept libSQL server written in Rust with deterministic simulation testing.

MIT

Why Hiisi?

SQLite is a versatile database, but serverless apps, for example, don't have persistent state to have an in-process database. Hiisi is a database server for remote SQL execution on libSQL/SQLite databases written in Rust, but follows similar architecture as TigerBeetle to support deterministic simulation testing (DST).

Hiisi is an experimental proof-of-concept and is not suitable for production use.

Features

  • libSQL server supporting the wire protocol
  • Deterministic simulation testing (DST)

Getting Started

Simulator:

cd simulator && cargo run

Server:

cd server && cargo run

FAQ

How is Hiisi different from libSQL?

Hiisi is a proof-of-concept alternative to the libSQL server, which provides the same functionality for remote SQL execution for libSQL/SQLite databases. There is no hard dependency between the two projects.

Comments
  • JavaScript client test fails

    JavaScript client test fails

    Steps to reproduce:

    cd hiisi-server && cargo run

    cd testing/sql/javascript && npm i && node index.js

    Actual output:

    [2024-08-12T09:11:35Z INFO  hiisid] Listening for SQL HTTP requests on 127.0.0.1:8080
thread 'main' panicked at hiisi-server/src/server.rs:75:35:
called `Result::unwrap()` on an `Err` value: Token
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    bug help wanted good first issue 
    opened by penberg 3
  • Improve future executor

    Improve future executor

    The I/O loop supports executing futures for compatibility with async Rust. The current implementation just busy-polls on a future until it's done. Let's look at ways to improve this.

    enhancement help wanted performance 
    opened by penberg 1
  • When does SQLite release resources?

    When does SQLite release resources?

    We want to keep SQLite databases resident in memory, but let's make sure that SQLite releases all relevant resources after prepared statements are finalized. For example, to avoid leaking file descriptors over time.

    question 
    opened by penberg 0
  • generalize http server

    generalize http server

    This allows us to process arbitrary headers and handle errors.

    This is done in preparation for an admin server that will share this code, even if this code is extracted to a different file, but has value on itself.

    opened by glommer 0
  • Integrate libSQL with I/O dispatcher

    Integrate libSQL with I/O dispatcher

    Let's explore integrating libSQL with the I/O dispatcher. We need the following:

    • VFS implementation that submits I/O using the dispatcher
    • Light-weight userspace concurrency primitive

    Implementing a VFS interface is kind of straight-forward with the sqlite-vfs crate, for example. What you want is something like:

    struct VFS { io: IO }

impl Vfs for VFS {
    fn read_exact_at(&mut self, buf: &mut [u8], offset: u64) -> Result<(), io::Error> {
        self.pread(buf, offset);
    }
}

    However, since pread() only registers intent to read, we need to suspend execution here. That's because sqlite3_step() is a blocking interface and expects read_exact_at() to complete. (Note that Limbo solves this problem by never blocking and instead returning something like SQLITE_IO instead.)

    To solve this problem, let's explore using coroutines with, for example, the corosensei crate. What we want is something like:

    fn process(io: IO) {
    io.spawn(|| {
        sqlite3_step();
    });
}

impl IO {
  pub fn spawn(task_fn: TaskFn);

  pub fn current_task() -> TaskId;

  pub fn wake_task(id: TaskId);

  pub fn yield_current();
}

struct VFS { io: IO }

impl Vfs for VFS {
    fn read_exact_at(&mut self, buf: &mut [u8], offset: u64) -> Result<(), io::Error> {
        let task = io.current_task();
        self.pread(buf, offset, complete_pread);
        io.yield_current();
    }
}

pub fn complete_pread(task_id: TaskID) {
    io.wake_task(task_id);
}

    The complication here is passing around state. For example, we probably now need to make callbacks carry some state so that we can pass task_id around.

    enhancement help wanted question 
    opened by penberg 0
  • Smarter memory resident database limits

    Smarter memory resident database limits

    We have very rough knobs in place to control memory resident databases. Let's make it smarter and more dynamic by looking at memory pressure, for example.

    enhancement help wanted multitenancy 
    opened by penberg 0
Owner
Pekka Enberg
Founder and CTO at Turso. Software engineer interested in distributed systems, cloud/edge computing, and databases. Previously @ScyllaDB & Linux kernel.
Pekka Enberg
GitHub
OGC API & STAC - Proof of Concept

OAPI - POC Proof of concept (POC) to ingest geospatial datasets from MeteoSuisse into a SpatioTemporal Asset Catalog (STAC), expose as OGC API Feature

Camptocamp 32 Dec 1, 2022
JS Runtime proof-of-concept for interactions with AvdanOS

Important: we are migrating to a new Discord server .gg/avdanos What is this ? This repo aims to be a JavaScript environment where AvdanOS extensions

AvdanOS 4 Nov 26, 2022
Proof of concept implementation of ProtoGalaxy

protogalaxy-poc Proof of concept implementation of ProtoGalaxy (https://eprint.iacr.org/2023/1106.pdf) using arkworks. Experimental code, do not use i

null 30 Aug 13, 2023
Proof of concept implementation of Sigmabus

sigmabus-poc Proof of concept implementation of Sigmabus https://eprint.iacr.org/2023/1406, a cool idea by George Kadianakis and Mary Maller and Andri

arnaucube 5 Sep 30, 2023
Proof-of-concept Typst webapp alternative

Proof-of-Concept Typst Webapp Alternative With the following features: Collaborative editing (using operational-transform and referenced from ekzhang/

Matt Fellenz 3 Nov 7, 2023
gRPC client/server for zero-knowledge proof authentication Chaum Pederson Zero-Knowledge Proof in Rust

gRPC client/server for zero-knowledge proof authentication Chaum Pederson Zero-Knowledge Proof in Rust. Chaum Pederson is a zero-knowledge proof proto

Advaita Saha 4 Jun 12, 2023
AMCOS - A high-peformance parallel monte-carlo simulation for estimating the fair value of options in finance

antoons-monte-carlo-options-sim A high-peformance parallel monte-carlo simulation for estimating the fair value of options in finance. Written in Rust

null 4 Jun 18, 2022
Prototype risk modeling simulation for Portfolio using Arbiter.

proto-sim Prototype simulation using Arbiter as the simulation & agent engine. Build & Run build.sh cargo run Arbiter config The arbiter.toml config

Primitive 13 Aug 14, 2023
✨ rudimentary simulation of the three-body problem

✨ three_body a very rudimentary simulation of the three-body problem. i was curious how far we could get with just euler's method and a small time ste

carl 151 May 12, 2024
Proost - A small proof assistant written in Rust

A simple proof assistant written in Rust. The specification of the project may be found here, and the user manual here. The API documentation c

Proost 12 Mar 3, 2023
Alternative Mizar proof checker written in Rust

Mizar proof checker This is a (still experimental) proof checker for the Mizar language. To compile the project, get Rust 1.67 or later (https://rustu

Mario Carneiro 16 Apr 13, 2023
Generates Solidity code to verify EIP-712 style signatures

eip712 Generates Solidity code to verify EIP-712 style signatures. Usage First, create an abstract contract implementing the functionality you want: /

Sam Wilson 11 Dec 22, 2022
Implementation of Proof of Existence consensus using Substrate Framework, Frame, Pallets, RUST

Substrate Node Template A fresh FRAME-based Substrate node, ready for hacking ?? Getting Started Follow the steps below to get started with the Node T

Vijayendra Gaur 1 Jun 8, 2022
OpenZKP - pure Rust implementations of Zero-Knowledge Proof systems.

OpenZKP OpenZKP - pure Rust implementations of Zero-Knowledge Proof systems. Overview Project current implements ?? the Stark protocol (see its readme

0x 529 Jan 5, 2023
Rust implementation of Namada, a sovereign proof-of-stake blockchain that enables asset-agnostic private transfers

Namada Overview Namada is a sovereign proof-of-stake blockchain, using Tendermint BFT consensus, that enables multi-asset private transfers for any na

anoma 144 Jan 2, 2023
Simple automated proof assistant.

Esther is a work-in-progress, proof-of-concept automated theorem proof assistant based on Homotopy Type Theory. Acknowledgements Arend, Lean, Coq and

Aodhnait Étaín 5 Sep 14, 2021
Composable proof transcripts for public-coin arguments of knowledge

Merlin: composable proof transcripts for public-coin arguments of knowledge Merlin is a STROBE-based transcript construction for zero-knowledge proofs

dalek cryptography 99 Dec 22, 2022
Selendra is a multichains interoperable nominated Proof-of-Stake network for developing and running Substrate-based and EVM compatible blockchain applications.

Selendra An interoperable nominated Proof-of-Stake network for developing and running Substrate-based and EVM compatible blockchain applications. Read

Selendra 16 Nov 29, 2022
DAPOL+ Proof of Liabilities using Bulletproofs and Sparse Merkle trees

DAPOL+ implementation Implementation of the DAPOL+ protocol introduced in the "Generalized Proof of Liabilities" by Yan Ji and Konstantinos Chalkias A

Mysten Labs 5 Apr 9, 2023