Blazingly fast framework for in-process microservices on top of Tower ecosystem

Overview

norpc = not remote procedure call

Crates.io documentation CI MIT licensed Tokei

Motivation

Developing an async application is often a very difficult task but building an async application as a set of microservices makes both designing and implementation much easier.

gRPC is a great tool in microservices. You can use this for communication over network but this isn't a good idea unless networking involves.

In such case, in-process microservices is a way to go. The services run on async runtime and communicate each other through in-memory async channel which doesn't occur serialization thus much more efficient than gRPC. I believe in-process microservices is a revolution for designing local async applications.

However, defining microservices in Rust does need a lot of coding for each services and they are mostly boilerplates. It will be helpful if these tedious tasks are swiped away by code generation.

tarpc is a previous work in this area however it is not a best framework for in-process microservices because it tries to support both in-process and networking microservices under the same abstraction. This isn't a good idea because the both implementation will because sub-optimal. In my opinion, networking microservices should use gRPC and in-process microservices should use dedicated framework for the specific purpose.

Also, tarpc doesn't use Tower's Service but define a similar abstraction called Serve by itself. This leads to reimplementing functions like rate-limiting and timeout which can be realized by just stacking Service decorators if depends on Tower. Since tarpc needs huge rework to become Tower-based, there is a chance to implement my own framework from scratch which will be much smaller and cleaner than tarpc because it only supports in-process microservices and is able to exploit the Tower ecosystem.

Architecture

スクリーンショット 2021-11-30 8 47 23

norpc utilizes Tower ecosystem. The core of the Tower ecosystem is an abstraction called Service which is like a function from Request to Response. The ecosystem has many decorators to add new behavior to an existing Service.

In the diagram, the client requests is coming from the top-left of the stacks and flows down to the bottom-right. The client and server is connected by async channel driven by Tokio runtime so there is no overhead for the serialization and copying because the message just "moves".

Here is how to generate codes for a simple service:

#[norpc::service]
trait HelloWorld {
    fn hello(s: String) -> String;
}

For more detail, please read Hello-World Example (README).

Performance (Compared to tarpc)

The RPC overhead is x1.7 lower than tarpc. With norpc, you can send more than 100k requests per second.

The benchmark program launches a noop server and send requests from the client. In measurement, Criterion is used.

noop request/1          time:   [8.9181 us 8.9571 us 9.0167 us]
noop request (tarpc)/1  time:   [15.476 us 15.514 us 15.554 us]

Author

Akira Hayakawa (@akiradeveloper)

Comments
  • Support Stream

    Support Stream

    #[norpc::service]
    trait Pub {
      fn message_stream(st: Stream<Elem>) -> Stream<Elem>;
    // or
      #[stream]
      fn message_stream() -> Elem; // output only?
    }
    

    yields fn message_stream(self) -> Pin<Box<dyn futures::stream::Stream<Item = Elem>;?

    I don't know. Need some experiments.

    enhancement 
    opened by akiradeveloper 5
  • Support !Send future

    Support !Send future

    Discussed in https://github.com/akiradeveloper/norpc/discussions/35

    Originally posted by akiradeveloper November 25, 2021 Adding (?Send) to async_trait macro will generate async function that yields non-Sync future.

    There are demands for non-Send futures.

    • https://github.com/google/tarpc/issues/338
    • https://github.com/hyperium/tonic/issues/844

    My understanding is Send is now required because the future may move between threads. So, if this doesn't happen in some particular case we can remove Sync constraint from them.

    Maybe we can use this to run all futures in the same thread? https://docs.rs/tokio/1.14.0/tokio/task/struct.LocalSet.html

    opened by akiradeveloper 4
  • Can't use full-path in neither input nor output types

    Can't use full-path in neither input nor output types

    v0.6.0

    The both should be accepted but failed to compilation due to "unexpected token".

    diff --git a/example/kvstore/tests/kvstore.rs b/example/kvstore/tests/kvstore.rs
    index 13c0abf..787cf6b 100644
    --- a/example/kvstore/tests/kvstore.rs
    +++ b/example/kvstore/tests/kvstore.rs
    @@ -9,7 +9,7 @@ trait KVStore {
         fn write(id: u64, s: String) -> ();
         fn write_many(kv: HashSet<(u64, String)>);
         // We can return a result from app to the client.
    -    fn noop() -> Result<bool, ()>;
    +    fn noop() -> std::result::Result<bool, ()>;
         // If app function fails error is propagated to the client.
         fn panic();
     }
    
    diff --git a/example/kvstore/tests/kvstore.rs b/example/kvstore/tests/kvstore.rs
    index 13c0abf..cc05bc6 100644
    --- a/example/kvstore/tests/kvstore.rs
    +++ b/example/kvstore/tests/kvstore.rs
    @@ -7,7 +7,7 @@ use tokio::sync::RwLock;
     trait KVStore {
         fn read(id: u64) -> Option<String>;
         fn write(id: u64, s: String) -> ();
    -    fn write_many(kv: HashSet<(u64, String)>);
    +    fn write_many(kv: std::collections::HashSet<(u64, String)>);
         // We can return a result from app to the client.
         fn noop() -> Result<bool, ()>;
         // If app function fails error is propagated to the client.
    
    bug 
    opened by akiradeveloper 2
  • Can not return explicit () value

    Can not return explicit () value

    Changing

    #[norpc::service]
    trait KVStore {
        fn read(id: u64) -> Option<String>;
        fn write(id: u64, s: String);
    

    to

    fn write(...) -> ()
    

    makes it uncompilable.

    bug 
    opened by akiradeveloper 2
  • Redesign for 0.8

    Redesign for 0.8

    I am thinking of generating functions that returns anyhow::Result<T> for definition that returns T.

    Reason one is, while implementing two applications with library I found that calling unwrap in async function is sometimes problematic. async function that panic can not be used with parallel execution tools like BoundedUnordered because failing one future in a row results in failing the entire execution. To work around I need to write this function to return Result. This is very weird.

    pub fn into_safe_future<T: Send + 'static>(
        fut: impl Future<Output = T> + Send + 'static,
    ) -> impl Future<Output = Result<T, JoinError>> {
        async move { tokio::spawn(fut).await }
    }
    

    Reason two is analogy to Tonic. Tonic generates a function that returns Result<T, tonic::Status> for a definition that returns T. And tonic::Status is just like (HttpCode, String). norpc is a library to define a in-process service and this is kind of in-process version of Tonic so it is not unnatural to norpc also generates a function that returns Result<T> as well.

    So the question what should be the Error type? I think Result<T, String> will suffice but anyhow::Result will be more convenient.

    opened by akiradeveloper 1
  • Implement an example that uses BoxService/BoxCloneService

    Implement an example that uses BoxService/BoxCloneService

    As of Rust 1.57 we can't define a struct field as like this

    struct T {
      svc: impl Service
    }
    

    The resulting concrete Service type from a ServiceBuilder may be very complex that's practically impossible to hold an ownership to the client that holds the Service.

    To workaround this, BoxService or BoxCloneService can be used as indirection layer.

    opened by akiradeveloper 1
  • Decorated client can't be cloned

    Decorated client can't be cloned

    This should work but it doesn't.

        let mut cli = RateLimitClient::new(chan);
        let mut cli = cli.clone();
    
    error[E0599]: the method `clone` exists for struct `RateLimitClient<tower::limit::RateLimit<ClientChannel<RateLimitRequest, RateLimitResponse>>>`, but its trait bounds were not satisfied
      --> example/rate-limit/tests/rate_limit.rs:35:23
       |
    7  | #[norpc::service]
       | -----------------
       | |
       | method `clone` not found for this
       | doesn't satisfy `_: Clone`
    ...
    35 |     let mut cli = cli.clone();
       |                       ^^^^^ method cannot be called on `RateLimitClient<tower::limit::RateLimit<ClientChannel<RateLimitRequest, RateLimitResponse>>>` due to unsatisfied trait bounds
       |
      ::: /home/akira.hayakawa/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.11/src/limit/rate/service.rs:14:1
       |
    14 | pub struct RateLimit<T> {
       | ----------------------- doesn't satisfy `_: Clone`
       |
       = note: the following trait bounds were not satisfied:
               `tower::limit::RateLimit<ClientChannel<RateLimitRequest, RateLimitResponse>>: Clone`
               which is required by `RateLimitClient<tower::limit::RateLimit<ClientChannel<RateLimitRequest, RateLimitResponse>>>: Clone`
       = help: items from traits can only be used if the trait is implemented and in scope
       = note: the following trait defines an item `clone`, perhaps you need to implement it:
               candidate #1: `Clone`
    

    The reason seems the tower::limit::RateLimit doesn't impl Clone. Without the line, it compiles.

        // let chan = ServiceBuilder::new()
        //     .rate_limit(1000, std::time::Duration::from_secs(1))
        //     .service(chan);
    

    Cloning decorated channel doesn't mean anything so this is ok but this is a bit convenient.

    opened by akiradeveloper 1
  • codegen: Generate `Result<T>` for convenience

    codegen: Generate `Result` for convenience

    If the code-generator generates Result<T> in the trait as Result<T> = Result<T, Self::Error> it will be very convenient but the feature (generic associated type) isn't supported by the compiler at the moment.

    #[norpc::async_trait]
    impl HelloWorld for HelloWorldApp {
        type Error = ();
        type Result<T> = Result<T, ()>;
    
    opened by akiradeveloper 1
  • Generate type alias to client

    Generate type alias to client

    To have a client in a struct field we need a complicated type like this.

    IdStoreClient<ClientChannel<IdStoreRequest, IdStoreResponse>>
    

    How about generate type alias like IdStoreClientT for convenience?

    good first issue 
    opened by akiradeveloper 1
  • Suppress warinng when there is only one method

    Suppress warinng when there is only one method

    If service has only one method, the unreachable!() arm will be warned by the compiler. Let's suppress this.

    warning: unreachable pattern
    --> /home/runner/work/norpc/norpc/target/debug/build/concurrent-message-05ff294aa07abe07/out/concurrent_message.rs:30:5
    |
    30 | _ => unreachable!(),
    | ^
    |
    = note: `#[warn(unreachable_patterns)]` on by default
    
    good first issue 
    opened by akiradeveloper 1
  • Support broadcasting

    Support broadcasting

    Currently, norpc only supports mpsc channel but in some case broadcasting is a good fit.

    https://docs.rs/tokio/1.16.1/tokio/sync/broadcast/index.html

    Typical use case is that a micro service receives a message from outside the process (by gRPC for example) and broadcast the message to N subscribers.

    enhancement 
    opened by akiradeveloper 0
  • Implement poll_ready for ClientService

    Implement poll_ready for ClientService

    I think this code is not correct. ClientService should poll the channel before calling call.

    impl<X: 'static + Send, Y: 'static + Send> crate::Service<X> for ClientService<X, Y> {
        type Response = Y;
        type Error = Error;
        type Future =
            std::pin::Pin<Box<dyn std::future::Future<Output = Result<Y, Self::Error>> + Send>>;
    
        fn poll_ready(
            &mut self,
            _: &mut std::task::Context<'_>,
        ) -> std::task::Poll<Result<(), Self::Error>> {
            Ok(()).into()
        }
    

    We could possibly use PollSender in tokio-util to make Sender pollable.

    opened by akiradeveloper 1
Owner
Akira Hayakawa
I am just a very good Rust programmer.
Akira Hayakawa
extremely-simplified top powered by ncurses.

xtop extremely-simplified top Depends xtop depends on below relatively primitive crates: ncurses: TUI sysconf: only to get a jiffy. signal-hook: to ha

smallkirby 4 Jan 28, 2022
A small random number generator hacked on top of Rust's standard library. An exercise in pointlessness.

attorand from 'atto', meaning smaller than small, and 'rand', short for random. A small random number generator hacked on top of Rust's standard libra

Isaac Clayton 1 Nov 24, 2021
mtop: top for Memcached

mtop mtop: top for Memcached. Features Display real-time statistics about your memcached servers such as Memory usage/limit Current/max connections Hi

Nick Pillitteri 6 Feb 27, 2023
Multithreaded, non-blocking Linux server framework in Rust

hydrogen Documentation hydrogen is a non-blocking socket server framework built atop epoll. It takes care of the tedious connection and I/O marshaling

Nathan Sizemore 358 Oct 31, 2022
Gomez - A pure Rust framework and implementation of (derivative-free) methods for solving nonlinear (bound-constrained) systems of equations

Gomez A pure Rust framework and implementation of (derivative-free) methods for solving nonlinear (bound-constrained) systems of equations. Warning: T

Datamole 19 Dec 24, 2022
Find files (ff) by name, fast!

Find Files (ff) Find Files (ff) utility recursively searches the files whose names match the specified RegExp pattern in the provided directory (defau

Vishal Telangre 310 Dec 29, 2022
Fast suffix arrays for Rust (with Unicode support).

suffix Fast linear time & space suffix arrays for Rust. Supports Unicode! Dual-licensed under MIT or the UNLICENSE. Documentation https://docs.rs/suff

Andrew Gallant 207 Dec 26, 2022
Rust edit distance routines accelerated using SIMD. Supports fast Hamming, Levenshtein, restricted Damerau-Levenshtein, etc. distance calculations and string search.

triple_accel Rust edit distance routines accelerated using SIMD. Supports fast Hamming, Levenshtein, restricted Damerau-Levenshtein, etc. distance cal

Daniel Liu 75 Jan 8, 2023
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility. Main features: Train new vocabularies and tok

Hugging Face 6.2k Jan 5, 2023
A fast, low-resource Natural Language Processing and Text Correction library written in Rust.

nlprule A fast, low-resource Natural Language Processing and Error Correction library written in Rust. nlprule implements a rule- and lookup-based app

Benjamin Minixhofer 496 Jan 8, 2023
A fast implementation of Aho-Corasick in Rust.

aho-corasick A library for finding occurrences of many patterns at once with SIMD acceleration in some cases. This library provides multiple pattern s

Andrew Gallant 662 Dec 31, 2022
Fast and easy random number generation.

alea A zero-dependency crate for fast number generation, with a focus on ease of use (no more passing &mut rng everywhere!). The implementation is bas

Jeff Shen 26 Dec 13, 2022
Vaporetto: a fast and lightweight pointwise prediction based tokenizer

?? VAporetto: POintwise pREdicTion based TOkenizer Vaporetto is a fast and lightweight pointwise prediction based tokenizer. Overview This repository

null 184 Dec 22, 2022
Composable n-gram combinators that are ergonomic and bare-metal fast

CREATURE FEATUR(ization) A crate for polymorphic ML & NLP featurization that leverages zero-cost abstraction. It provides composable n-gram combinator

null 3 Aug 25, 2022
A simple and fast linear algebra library for games and graphics

glam A simple and fast 3D math library for games and graphics. Development status glam is in beta stage. Base functionality has been implemented and t

Cameron Hart 953 Jan 3, 2023
Ultra-fast, spookily accurate text summarizer that works on any language

pithy 0.1.0 - an absurdly fast, strangely accurate, summariser Quick example: pithy -f your_file_here.txt --sentences 4 --help: Print this help messa

Catherine Koshka 13 Oct 31, 2022
Fast PDF password cracking utility equipped with commonly encountered password format builders and dictionary attacks.

PDFRip Fast PDF password cracking utility equipped with commonly encountered password format builders and dictionary attacks. ?? Table of Contents Int

Mufeed VH 226 Jan 4, 2023
🛥 Vaporetto is a fast and lightweight pointwise prediction based tokenizer. This is a Python wrapper for Vaporetto.

?? python-vaporetto ?? Vaporetto is a fast and lightweight pointwise prediction based tokenizer. This is a Python wrapper for Vaporetto. Installation

null 17 Dec 22, 2022
A lightning-fast Sanskrit toolkit. For Python bindings, see `vidyut-py`.

Vidyut मा भूदेवं क्षणमपि च ते विद्युता विप्रयोगः ॥ Vidyut is a lightning-fast toolkit for processing Sanskrit text. Vidyut aims to provide standard co

Ambuda 14 Dec 30, 2022