Rust client for Timeplus Proton, a fast and lightweight streaming SQL engine

Overview

Rust Client for Timeplus Proton

Rust client for Timeplus Proton.

Proton is a streaming SQL engine, a fast and lightweight alternative to Apache Flink, 🚀 powered by ClickHouse. It enables developers to solve streaming data processing, routing and analytics challenges from Apache Kafka, Redpanda and more sources, and send aggregated data to the downstream systems. Proton is the core engine of Timeplus, which is a cloud native streaming analytics platform.

The initial version(0.1.0) of the client was written by Marvin Hansen. Thanks for the contribution.

This client uses https://crates.io/crates/clickhouse as a dependency.

Install Proton

Please install Proton as a standalone server or via Docker. Make sure either port 8123 or 3218 is exposed for pront-rust-client to connect and run SQL.

  • 8123 is the port to run DDL and queries in historical mode
  • 3218 is the port to run DDL and quries in streaming mode by default. You can still wrap the stream with table(..) function to query them in historical mode.

Learn more

As a single binary

On Linux or Mac, you can install it via curl https://install.timeplus.com | sh

On Mac, you can also install it via brew install proton

After you get the proton binary, you can start the Proton server via proton server start

In a separate terminal, connect to the server via proton client (Note: If you encounter a 'connection refused' error, use: proton client --host 127.0.0.1)

As a Docker container

docker run -d --pull always --name proton -p 8123:8123 -p 8463:8463 ghcr.io/timeplus-io/proton:latest

Proton is automatically started with port 8123 and 8463 exposed. Open the terminal of the container, and run proton client

For detailed usage and more information, check out the documentation: https://docs.timeplus.com/proton

Install ProtonClient

Add the proton client to your project by running in a terminal:

cargo add proton_client

or by adding the following to your Cargo.toml:

[dependencies]

proton_client =  { version = "0.1.0"}

Use ProtonClient

use proton_client::prelude::{ProtonClient, Result};

const FN_NAME: &str = "[prepare]:";

#[tokio::main]
async fn main() -> Result<()> {
    println!("{}Start", FN_NAME);

    println!("{}Build client", FN_NAME);
    let client = ProtonClient::new("http://localhost:8123");

    println!("{}Create stream if not exists", FN_NAME);
    create_stream(&client)
        .await
        .expect("[main]: Failed to create Stream");

    println!("{}Stop", FN_NAME);
    Ok(())
}

pub async fn create_stream(client: &ProtonClient) -> Result<()> {
     client
         .execute_query("CREATE STREAM IF NOT EXISTS test_stream(no uint32, name string) ORDER BY no")
         .await
}

Run the examples

make example

Which shows:

-----------------------------------------
Select the number of the example to run:
-----------------------------------------
1) prepare: Prepare Proton for the examples: create table & load data
2) query: Query Proton with sample queries
3) remove: Cleanup Proton and delete streams
4) quit: Exit

-----------------------------------------
Make sure Proton is running
-----------------------------------------

Please select the number of the example to run. Also, first run the prepare example to prepare Proton for the query example. See the code of all examples here.

What's next?

To see more examples of using Proton, check out the examples folder in Proton repo.

Please note, by default the SQL queries are in streaming mode. Learn more about SQL syntax at Proton Documentation.

Documentation

You find the full documentation for Proton at docs.timeplus.com alongside documentation for the Timeplus (Cloud and BYOC) platform.

Contributing

We welcome your contributions!

Need help?

Join our Timeplus Community Slack to connect with Timeplus engineers and other Proton users.

For filing bugs, suggesting improvements, or requesting new features, see the open issues here on GitHub.

Licensing

Proton Rust Client uses Apache License 2.0. See details in the LICENSE.

Comments
  • Resolved issue #6: Stream query in release mode.

    Resolved issue #6: Stream query in release mode.

    Describe your changes

    Resolved previous issue #6

    • Added uncompressed client to handle streams in release mode
    • Added new method fetch_stream

    Issue ticket number and link

    https://github.com/timeplus-io/proton-rust-client/issues/6

    Code checklist before requesting a review

    • [x] All tests are passing when running make test?
    • [x] No errors or security vulnerabilities are reported by make check?

    For details on make, please see BUILD.md

    Note: The CI runs all of the above and fixing things before they hit CI speeds up the review and merge process. Thank you.

    enhancement 
    opened by marvin-hansen 2
  • Handover

    Handover

    Continue setting up the repo, as an early-stage Rust client lib for Proton.

    Check list

    https://github.com/timeplus-io/proton-rust-client/blob/main/Handover.md

    • [x] Update MAINTAINERS.md, CODEOWNERS, and CONTRIBUTING.md
    • [x] Update README
    • [x] Verify CI / Github actions

    The following actions to perform after this PR is merged to main branch

    • [x] Publish the client to crates.io
    • [x] Enable auto-release
    • [x] Remove the Handover file

    Routine check

    • [x] All tests are passing when running make test?
    • [x] No errors or security vulnerabilities are reported by make check?
    opened by jovezhong 2
  • chore: release v0.1.1

    chore: release v0.1.1

    🤖 New release

    • proton_client: 0.1.0 -> 0.1.1 (✓ API compatible changes)
    Changelog

    0.1.1 - 2024-02-14

    Other

    • Resolved issue #6: Stream query in release mode. (#7)
    • Updated Readme & Enabled release-plz (#4)
    • Update Cargo.toml, up to 5 keywords


    This PR was generated with release-plz.

    opened by github-actions[bot] 0
  • Updated Readme & Enabled release-plz

    Updated Readme & Enabled release-plz

    Describe your changes

    Updated Readme:

    • Added info how to install official crate

    Enabled release-plz

    Issue ticket number and link

    Code checklist before requesting a review

    • [x] All tests are passing when running make test?
    • [x] No errors or security vulnerabilities are reported by make check?

    For details on make, please see BUILD.md

    Note: The CI runs all of the above and fixing things before they hit CI speeds up the review and merge process. Thank you.

    opened by marvin-hansen 0
  • Code formatting, linting & script to run examples

    Code formatting, linting & script to run examples

    Describe your changes

    • Code formatting
    • Code linting
    • Added script to run all examples
    • Updated Readme to show how to use the script

    Issue ticket number and link

    Code checklist before requesting a review

    • [x] All tests are passing when running make test?
    • [x] No errors or security vulnerabilities are reported by make check?

    For details on make, please see BUILD.md

    Note: The CI runs all of the above and fixing things before they hit CI speeds up the review and merge process. Thank you.

    enhancement 
    opened by marvin-hansen 0
  • Refine examples

    Refine examples

    I tested the examples code. Changed the stream name from some to test_stream. Also keep the log message format consistent. Also updated README to inform the user to open port 8123 from Proton for this client.

    Also removed Handover.md

    opened by jovezhong 0
  • Streaming query does not work in release mode.

    Streaming query does not work in release mode.

    Describe what's wrong

    Proton has two sets of HTTP APIs, one is running on port 8123, this is the same as clickhouse. And the other is running on port 3218 which supports streaming queries. Simply put, the same query SELECT * FROM some_stream behaves differently on these two APIs. On port 8123 it returns immediately, but on port 3128, it will keep running and returnning fresh data.

    This isssue is about running streaming query on port 3128 with this library. It won't work in release mode (e.g. using cargo run --release). The reason why it worked for non-release mode is because of this:

    [dev-dependencies]
    clickhouse = { version = "0.11", features = ["test-util"] }
    

    with the test-util, the clickhouse crate does not use any compression by default (otherwise, it uses lz4), this is the tricky part. And with lz4, streaming query does not work. So, in order to make streaming query work, we need to disable compression when do streaming query:

    let client = Client::default()....;
    /// do non-streaming query with the `client` ...
    
    /// when you want to do streaming query, disable compression
    client.with_compression(Compression::None);
    client.query("<... my streaming query ...>").fetch ...
    

    How to reproduce

    Error message and/or stacktrace

    Additional context

    bug 
    opened by zliang-min 5
Releases(v0.1.1)
  • v0.1.1(Feb 14, 2024)

  • v0.1.0(Feb 12, 2024)

    What's Changed

    Initial release of Rust client for Timeplus Proton.

    Proton is a streaming SQL engine, a fast and lightweight alternative to Apache Flink, 🚀 powered by ClickHouse. It enables developers to solve streaming data processing, routing and analytics challenges from Apache Kafka, Redpanda and more sources, and send aggregated data to the downstream systems. Proton is the core engine of Timeplus, which is a cloud native streaming analytics platform.

    The initial version(0.1.0) of the client was written by Marvin Hansen. Thanks for the contribution.

    This client uses https://crates.io/crates/clickhouse as a dependency.

    New Contributors

    • @marvin-hansen made their first contribution in https://github.com/timeplus-io/proton-rust-client/pull/3
    • @jovezhong made their first contribution in https://github.com/timeplus-io/proton-rust-client/pull/1

    Full Changelog: https://github.com/timeplus-io/proton-rust-client/commits/v0.1.0

    Source code(tar.gz)
    Source code(zip)
RisingWave is a cloud-native streaming database that uses SQL as the interface language.

RisingWave is a cloud-native streaming database that uses SQL as the interface language. It is designed to reduce the complexity and cost of building real-time applications. RisingWave consumes streaming data, performs continuous queries, and updates results dynamically. As a database system, RisingWave maintains results inside its own storage and allows users to access data efficiently.

Singularity Data 3.7k Jan 2, 2023
🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, SQLite, and MSSQL.

SQLx ?? The Rust SQL Toolkit Install | Usage | Docs Built with ❤️ by The LaunchBadge team SQLx is an async, pure Rust† SQL crate featuring compile-tim

launchbadge 7.6k Dec 31, 2022
X-Engine: A SQL Engine built from scratch in Rust.

XNGIN (pronounced "X Engine") This is a personal project to build a SQL engine from scratch. The project name is inspired by Nginx, which is a very po

Jiang Zhe 111 Dec 15, 2022
A Rust SQL query builder with a pleasant fluent API closely imitating actual SQL

Scooby An SQL query builder with a pleasant fluent API closely imitating actual SQL. Meant to comfortably build dynamic queries with a little bit of s

Aleksei Voronov 100 Nov 11, 2022
Gh-sql - Query GitHub Projects (beta) with SQL

gh-sql: Query GitHub Projects (beta) with SQL Installation gh extension install KOBA789/gh-sql Features SELECT items DELETE items UPDATE item fields

Hidekazu Kobayashi 108 Dec 7, 2022
SQL validator tool for BigQuery standard SQL.

bqvalid What bqvalid does bqvalid is the SQL validator tool for BigQuery standard SQL. bqvalid fails with error message if there's the expression that

null 10 Dec 25, 2022
Streaming STOMP client for Rust

tokio-stomp An async STOMP client (and maybe eventually, server) for Rust, using the Tokio stack. It aims to be fast and fully-featured with a simple

null 7 Jun 15, 2022
tectonicdb is a fast, highly compressed standalone database and streaming protocol for order book ticks.

tectonicdb crate docs.rs crate.io tectonicdb tdb-core tdb-server-core tdb-cli tectonicdb is a fast, highly compressed standalone database and streamin

Ricky Han 525 Dec 23, 2022
SQL/JSON path engine in Rust.

sql-json-path SQL/JSON Path implementation in Rust. ?? Under development ?? Features Compatible with SQL/JSON Path standard and PostgreSQL implementat

RisingWave Labs 3 Nov 22, 2023
A Toy Query Engine & SQL interface

Naive Query Engine (Toy for Learning) ?? This is a Query Engine which support SQL interface. And it is only a Toy for learn query engine only. You can

谭巍 45 Dec 21, 2022
Materialize simplifies application development with streaming data. Incrementally-updated materialized views - in PostgreSQL and in real time. Materialize is powered by Timely Dataflow.

Materialize is a streaming database for real-time applications. Get started Check out our getting started guide. About Materialize lets you ask questi

Materialize, Inc. 4.7k Jan 8, 2023
Streaming Network Overlay Connection Arbitration Tunnel

SNOCAT Streaming Network Overlay Connection Arbitration Tunnel snocat is a framework for forwarding streams across authenticated, encrypted QUIC tunne

Microsoft 52 Nov 16, 2022
Lightweight async Redis client with connection pooling written in pure Rust and 100% memory safe

redi-rs (or redirs) redi-rs is a Lightweight Redis client with connection pooling written in Rust and 100% memory safe redi-rs is a Redis client writt

Oğuz Türkay 4 May 20, 2023
ReefDB is a minimalistic, in-memory and on-disk database management system written in Rust, implementing basic SQL query capabilities and full-text search.

ReefDB ReefDB is a minimalistic, in-memory and on-disk database management system written in Rust, implementing basic SQL query capabilities and full-

Sacha Arbonel 75 Jun 12, 2023
Lightweight, Strongly Typed Xata Client written in Rust

xata-rs: Lightweight, Strongly Typed Xata Client xata-rs is a third party Xata client, allowing interaction with Xata's REST API. Adding xata-rs (WIP)

Alvaro Machuca Recalde 4 Jun 12, 2023
Lightweight unifying client for RPC + BanksClient

Ellipsis Client Lightweight unifying client for RPC + BanksClient There are no good unifying traits for Solana Rust clients. EllipsisClient creates a

Ellipsis Labs 16 Dec 24, 2022
Async Lightweight HTTP client using system native library if possible. (Currently under heavy development)

Async Lightweight HTTP Client (aka ALHC) What if we need async but also lightweight http client without using such a large library like reqwest, isahc

SteveXMH 7 Dec 15, 2022
FeOphant - A SQL database server written in Rust and inspired by PostreSQL.

A PostgreSQL inspired SQL database written in Rust.

Christopher Hotchkiss 27 Dec 7, 2022
Rust library to parse, deparse and normalize SQL queries using the PostgreSQL query parser

This Rust library uses the actual PostgreSQL server source to parse SQL queries and return the internal PostgreSQL parse tree.

pganalyze 37 Dec 18, 2022