A "blazingly" fast key-value pair database without bloat written in rust

Overview

Firefly

A fast key-value pair in memory database. With a very simple and fast API. At Xiler it gets used to store and manage client sessions throughout the platform.

CLI help menu

Installation

Cargo: $ cargo install ffly

AUR: $ paru -S ffly

Docker image: arthurdw/firefly

Performance comparison

Database ops
Firefly 167k
Skytable 143k
Redis 67k

(push_it scripts can be found in ffly-rs/examples/)

Query Language

Firefly only has three data operators, NEW, GET, DROP. We'll walk over each one.

Important note on values

ALL values must be valid ASCII, if not the server will reject the query.

Defining query types

The server may only accept one query type per TCP connection type. The default type is a string query. But this can be changed by using the QUERY TYPE keyword.

QUERY TYPE 'STRING' | 'BITWISE';

String queries

The simplest way to query something is by querying it using a string query. But because the parsing, and bandwidth of this is more than that is necessary we also provide bitwise queries.

String queries are very loosely defined, as it evaluates the query definition from left to right. And can derive that if you use e.g. GETV, you mean GET VALUE. This works for any query, identifiers are case-insensitive.

Create

You can create a new record by using the NEW keyword, the arguments should be wrapped within quotes. The first argument (after NEW) is the key, this should be unique throughout the db. If no TLL value is provided, the server will use 0 (aka no expiry).

NEW '{key}'
[ VALUE ] '{value}'
[ WITH TTL '{ttl}'];
Create examples
NEW 's2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y'
VALUE '86ebe1a0-11bf-11ed-aa8e-13602e2ad46b';
NEW 's2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y'
VALUE '86ebe1a0-11bf-11ed-aa8e-13602e2ad46b'
WITH TTL '604800';

Fetch

The GET keyword returns the value and TTL by default. But if you only want one of the two, you can specify this. You can only search by key!

GET [VALUE | TTL] '{key}';
Fetch examples
GET 's2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y';
GET TTL 's2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y';

Delete

Deleting one record is as straightforward as fetching one. You can only delete whole records. If required all records that have a value can also be deleted, but this action is very expensive and generally not recommended.

DROP '{key}';
DROP ALL '{value}';

Bitwise queries

Because string queries can consume more resources than what is required, there is a more efficient (less friendly) way to interact with Firefly. This is by sending the bits in a specific format. This section just describes the formats, if you want more information about the queries itself, then you can find it in the String queries section.

Please keep in mind that all the bitwise queries are very strict, and if a query is unrecognized it will discard it.

General notes

  • All values are delimited by a NUL character. (0x0)
  • The end of the query is assumed to be the last byte.
  • Queries start with their type, this is a numeric value
    • 0: NEW
    • 1: GET
    • 2: GET VALUE
    • 3: GET TTL
    • 4: DROP
    • 5: DROP ALL
    • 6: QUERY TYPE STRING
    • 7: QUERY TYPE BITWISE
  • The query type does not need to be delimited

Bitwise create

A with TTL must always be provided. If you don't want a TTL set this to 0.

0{key}0x0{value}0x0{ttl}

These are the two same create examples from the string queries: 0s2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y0x086ebe1a0-11bf-11ed-aa8e-13602e2ad46b0x0 0s2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y0x086ebe1a0-11bf-11ed-aa8e-13602e2ad46b0x0602800

Bitwise fetch

1s2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y 2s2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y 3s2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y

Bitwise delete

4s2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y 5s2.8eqYursP2McHeQvHB2bauyE6n3vptOj8M96PxmGAQMDfimeZ31WAzP3hSw5Ixv5Y

You might also like...
RedisLess is a fast, lightweight, embedded and scalable in-memory Key/Value store library compatible with the Redis API.

RedisLess is a fast, lightweight, embedded and scalable in-memory Key/Value store library compatible with the Redis API.

A simple embedded key-value store written in rust as a learning project

A simple embedded key-value store written in rust as a learning project

PickleDB-rs is a lightweight and simple key-value store. It is a Rust version for Python's PickleDB

PickleDB PickleDB is a lightweight and simple key-value store written in Rust, heavily inspired by Python's PickleDB PickleDB is fun and easy to use u

Log structured append-only key-value store from Rust In Action with some enhancements.

riakv Log structured, append only, key value store implementation from Rust In Action with some enhancements. Features Persistent key value store with

A LSM-based Key-Value Store in Rust

CobbleDB A LSM-based Key-Value Store in Rust Motivation There is no open-source LSM-based key-value store in Rust natively. Some crates are either a w

A rust Key-Value store based on Redis.

Key-Value Store A Key-Value store that uses Redis to store data. Built using an async web framework in Rust with a full Command-Line interface and log

CLI tool to work with Sled key-value databases.

sledtool CLI tool to work with Sled key-value databases. $ sledtool --help Usage: sledtool dbpath command [args] CLI tool to work with Sled da

A Key-Value data storage system. - dorea db

Dorea DB 🛰 Dorea is a key-value data storage system. It is based on the Bitcask storage model Documentation | Crates.io | API Doucment įŽ€äŊ“中文 | English

A sessioned Merkle key/value store

storage A sessioned Merkle key/value store The crate was designed to be the blockchain state database. It provides persistent storage layer for key-va

Comments
  • Docker image arthurdw/firefly is no accessible - autoclosed

    Docker image arthurdw/firefly is no accessible - autoclosed

    Docker image arthurdw/firefly seems to be private on dockerhub

    If I visit https://hub.docker.com/repository/docker/arthurdw/firefly I am redirected to the dockerhub login page

    configuration error 
    opened by BarbossHack 7
  • Unnecessary manual drop/free

    Unnecessary manual drop/free

    Rust should drop the stream automatically at that point, since it's no longer being referenced after. https://github.com/Arthurdw/firefly/blob/275c66d9e60a79406a1ba3852b7257a6fc24c852/ffly-rs/src/lib.rs#L96-L101

    enhancement 
    opened by Sculas 1
  • Configure Mend Bolt for GitHub

    Configure Mend Bolt for GitHub

    Welcome to Mend Bolt for GitHub (formerly WhiteSource). This is an onboarding PR to help you understand and configure settings before Mend starts scanning your repository for security vulnerabilities.

    :vertical_traffic_light: Mend Bolt for GitHub will start scanning your repository only once you merge this Pull Request. To disable Mend Bolt for GitHub, simply close this Pull Request.


    What to Expect

    This PR contains a '.whitesource' configuration file which can be customized to your needs. If no changes were applied to this file, Mend Bolt for GitHub will use the default configuration.

    Before merging this PR, Make sure the Issues tab is enabled. Once you merge this PR, Mend Bolt for GitHub will scan your repository and create a GitHub Issue for every vulnerability detected in your repository.

    If you do not want a GitHub Issue to be created for each detected vulnerability, you can edit the '.whitesource' file and set the 'minSeverityLevel' parameter to 'NONE'.


    :question: Got questions? Check out Mend Bolt for GitHub docs. If you need any further assistance then you can also request help here.

    opened by mend-bolt-for-github[bot] 0
  • Configure Mend Bolt for GitHub

    Configure Mend Bolt for GitHub

    Welcome to Mend Bolt for GitHub (formerly WhiteSource). This is an onboarding PR to help you understand and configure settings before Mend starts scanning your repository for security vulnerabilities.

    :vertical_traffic_light: Mend Bolt for GitHub will start scanning your repository only once you merge this Pull Request. To disable Mend Bolt for GitHub, simply close this Pull Request.


    What to Expect

    This PR contains a '.whitesource' configuration file which can be customized to your needs. If no changes were applied to this file, Mend Bolt for GitHub will use the default configuration.

    Before merging this PR, Make sure the Issues tab is enabled. Once you merge this PR, Mend Bolt for GitHub will scan your repository and create a GitHub Issue for every vulnerability detected in your repository.

    If you do not want a GitHub Issue to be created for each detected vulnerability, you can edit the '.whitesource' file and set the 'minSeverityLevel' parameter to 'NONE'.


    :question: Got questions? Check out Mend Bolt for GitHub docs. If you need any further assistance then you can also request help here.

    opened by mend-bolt-for-github[bot] 0
Releases(v0.0.2)
  • v0.0.2(Aug 14, 2022)

  • 0.0.1(Aug 14, 2022)

    🎉 Initial creation of the Firefly DB

    • [x] TCP Communication
    • [x] String query parsing
    • [x] Bitwise query parsing
    • [x] Saving to file (uses bincode for serialisation & deserialisation)
    • [x] Command line argument parsing (uses clap)
    • [x] Rust library for easy asynchronous query execution
    Source code(tar.gz)
    Source code(zip)
Owner
Arthur
19 y/o applied CS student @ Howest University
Arthur
AgateDB is an embeddable, persistent and fast key-value (KV) database written in pure Rust

AgateDB is an embeddable, persistent and fast key-value (KV) database written in pure Rust. It is designed as an experimental engine for the TiKV project, and will bring aggressive optimizations for TiKV specifically.

TiKV Project 535 Jan 9, 2023
A fast and simple in-memory database with a key-value data model written in Rust

Segment Segment is a simple & fast in-memory database with a key-value data model written in Rust. Features Dynamic keyspaces Keyspace level control o

Segment 61 Jan 5, 2023
Pure rust embeddable key-value store database.

MHdb is a pure Rust database implementation, based on dbm. See crate documentation. Changelog v1.0.3 Update Cargo.toml v1.0.2 Update Cargo.toml v1.0.1

Magnus Hirth 7 Dec 10, 2022
Immutable Ordered Key-Value Database Engine

PumpkinDB Build status (Linux) Build status (Windows) Project status Usable, between alpha and beta Production-readiness Depends on your risk toleranc

null 1.3k Jan 2, 2023
Distributed transactional key-value database, originally created to complement TiDB

Website | Documentation | Community Chat TiKV is an open-source, distributed, and transactional key-value database. Unlike other traditional NoSQL sys

TiKV Project 12.4k Jan 3, 2023
RefineDB - A strongly-typed document database that runs on any transactional key-value store.

RefineDB - A strongly-typed document database that runs on any transactional key-value store.

Heyang Zhou 375 Jan 4, 2023
A simple key value database for storing simple structures.

Perdia-DB A simple key value database for storing simple structures. No nesting of structures is supported, but may be implemented in the future. Toke

Perdia 4 May 24, 2022
LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values. Authors: Sanjay Ghem

Google 31.5k Jan 1, 2023
A Distributed SQL Database - Building the Database in the Public to Learn Database Internals

Table of Contents Overview Usage TODO MVCC in entangleDB SQL Query Execution in entangleDB entangleDB Raft Consensus Engine What I am trying to build

Sarthak Dalabehera 38 Jan 2, 2024
ForestDB - A Fast Key-Value Storage Engine Based on Hierarchical B+-Tree Trie

ForestDB is a key-value storage engine developed by Couchbase Caching and Storage Team, and its main index structure is built from Hierarchic

null 1.2k Dec 26, 2022