An RPC framework developing tutorial

Overview

Mini Lust 系列教程

好奇如何从零造出来一个 RPC 框架?本教程将带你一步一步写出来一个 Rust 版 Thrift RPC 框架。

教程说明

从第二章开始每章节都会附带代码。 这个代码是在上一章节的基础上写的,文档里一般会告诉你增加了哪些东西,但是如果你想详细地对比到底哪里变动了,可以自行 diff。

每章的代码会尽量通过测试保证代码是正确工作的,我们会通过 CI 来向你保证这一点。

教程结构

依次分几个章节:

  1. 前言部分,RPC 相关概念介绍
  2. Thrift IDL 介绍
  3. 序列化/反序列化的抽象
  4. Codec 和 Transport 抽象
  5. 客户端和服务端实现
  6. Thrift IDL 解析和代码生成
  7. 基于 tower 的服务发现和负载均衡
  8. 中间件支持

Credit

  • @dyxushuai: Lust project creator and helped mini-lust project a lot
  • @ihciah: Mini-lust tutorial creator and Lust project developer
  • @PureWhiteWu: Lust project developer
  • @LYF1999: Lust project developer
  • @Millione: Lust project developer

进度

  • 1~6 章节代码基本写完,教程尚需优化,部分细节需优化
  • CI 配置待补齐
You might also like...
Integra8 rust integration test framework Rust with a focus on productivity, extensibility, and speed.

integra8 Integra8 rust integration test framework Rust with a focus on productivity, extensibility, and speed. | This repo is in a "work in progress"

Incremental Program Analysis Framework
Incremental Program Analysis Framework

IncA Overview IncA is a program analysis framework. It comes with a DSL for the definition of program analyses and the runtime system evaluates progra

Rust bindings to the dos-like framework
Rust bindings to the dos-like framework

dos-like for Rust   This project provides access to Mattias Gustavsson's dos-like framework, so as to write DOS-like applications in Rust. How to use

A developer-friendly framework for building user interfaces in Rust
A developer-friendly framework for building user interfaces in Rust

Reading: "Fru" as in "fruit" and "i" as in "I" (I am). What is Frui? Frui is a developer-friendly UI framework that makes building user interfaces eas

A backend framework for building fast and flexible APIs rapidly.

Andromeda Andromeda is a backend framework for Rust, to simplify the development of the kinds of basic API services that we developers have to build s

🧪 The versatile and intuitive memory hacking framework.

🧪 hax 🤔 About hax is a Rust crate designed to make memory hacking, game hacking, cheat development, and any other low level memory based development

A minimal boilerplate for Astro / Vite with the Nannou creative framework (Rust → WASM). Supports multiple sketches + hot-reload.
A minimal boilerplate for Astro / Vite with the Nannou creative framework (Rust → WASM). Supports multiple sketches + hot-reload.

Astro x Nannou Starter astro-nannou-demo-1c.mov 🕹 Try it online! # 0a. Rust language tools open https://www.rust-lang.org/tools/install # 0b. wasm-p

Grebuloff is an experimental addon framework for Final Fantasy XIV.

Grebuloff Grebuloff is an experimental addon framework for Final Fantasy XIV. It introduces a new concept of what plugins can be, focusing on enabling

Indeed, an ORM library, not a framework, written in Rust

Ormlib Indeed, an ORM library, not a framework, written in Rust Features The main idea that I put into my ORM library is a minimum of stupid code and

Comments
  • Rust 的异步 I/O 是否适合在 DPI 的场景中使用?

    Rust 的异步 I/O 是否适合在 DPI 的场景中使用?

    参考 mini-lust/tutorials,我熟悉了异步 I/O tokio_util::codec::Decoder 的使用,但是想在 DPI 项目中使用它,便产生了下述疑问,希望您能帮我解惑

    Decoder 在 L7 Application Server 中如何使用

    当收到 data 时,调用 decoder 进行解析:

    • 如果 data 的数据足够长,decoder 可以完成解析,decoder 返回 Ok(Some()) 表示 data 被消费
    • 如果 data 的数据不够多,decoder 不能完成解析,decoder 返回 Ok(None()) 表示 data 未消费 (此时 data 由异步 I/O 缓存,等下次再有数据到来时,两次的数据拼接到一起再调用 decoder)

    这种处理模式在 L7 Application 的实现中是很 easy 的。

    Decoder 在 DPI 中如何使用

    当我们收到的 data 是 Eth frame 时就出现问题了 (例如 unix socket server 模拟 DPI 的应用场景)

    例如 unix client 将 http request 分两个 Eth frame 发送出去:

    • 第一次发送的数据为:Eth header + IP header + TCP header + "GET /index.html HTTP/1.1\r\n"
    • 第二次发送的数据为:Eth header + IP header + TCP header + "Host: www.test.com\r\n\r\n"
    1. unix server 第一次收到的数据是 Eth header + IP header + TCP header + "GET /index.html HTTP/1.1\r\n"
    2. 依次调用 Eth decoder + IP decoder + TCP decoder + HTTP decoder 进行解析,http decoder 不能完成解析,decoder 返回 Ok(None()) 表示 data 未消费。
    3. 此时第一次收到的数据 Eth header + IP header + TCP header + "GET /index.html HTTP/1.1\r\n" 被异步 I/O 缓存。
    4. unix server 第二次收到的数据是 Eth header + IP header + TCP header + "Host: www.test.com\r\n\r\n"
    5. 此时异步 I/O 缓存中的数据为 Eth header + IP header + TCP header + "GET /index.html HTTP/1.1\r\n"Eth header + IP header + TCP header + "Host: www.test.com\r\n\r\n"
    6. 此时 decoder 调用关系混乱,如果 IP header 再实现 IP 分片重组,TCP 乱序重组,decoder 的调用更加复杂,那么异步 I/O 是否适合在 DPI 的场景中使用?

    参考 Rust 的 DPI 开源项目 sniffglue 和 pcap-analyzer

    • https://github.com/kpcyrd/sniffglue
    • https://github.com/rusticata/pcap-analyzer 发现 sniffglue 和 pcap-analyzer 的实现中都没有使用异步 I/O,异步 I/O 是否适合在 DPI 的场景中使用?
    opened by l-w-p 0
Owner
null
High Assurance Rust - A free book about developing secure and robust systems software.

High Assurance Rust - A free book about developing secure and robust systems software.

Tiemoko Ballo 1.1k Jan 9, 2023
A project for developing Rust applications

rust_dev This is a project for developing Rust applications. The goal of this project is to provide a solid foundation for building robust and scalabl

Harsh agarwal 4 Feb 7, 2023
Raytracer tutorial for PPCA 2021, written in Rust.

Pseudo Photograph Company of ACM 工科和ACM的朋友们都已结课!看看这些了不起的艺术品: 工科 ACM ACM伪摄影公司,简称PPCA,于2021年成立 ?? 这个项目的主要工作是使用Rust语言实现一个光线追踪渲染器。以这个形式,你能通过学习一门新的(而且漂亮的)语

null 113 Dec 13, 2022
Rust serenity korean tutorial

?? 러스트 Serenity 디스코드 API 강좌 반갑습니다. 러스트 Serenity 디스코드 API 강좌에 오신것을 진심으로 환영합니다 ! Serenity는 러스트로 짜여진 디스코드 API 입니다. 독자 여러분들은 discord.py 를 해보셨습니까? 만약, disc

Fn79 3 Dec 17, 2021
Tutorial for parsing with nom 5.

Nom Tutorial Nom is a wonderful parser combinators library written in Rust. It can handle binary and text files. Consider it where you would otherwise

Benjamin Kay 265 Dec 11, 2022
LLVM tutorial in Rust language

Status Everything works. State corresponds to the Chapter 7 of the original tutorial (i.e. mutable variables implemented). TODO list. LLVM tutorial in

Jauhien Piatlicki 944 Dec 28, 2022
A tutorial of building an LSM-Tree storage engine in a week! (WIP)

LSM in a Week Build a simple key-value storage engine in a week! Tutorial The tutorial is available at https://skyzh.github.io/mini-lsm. You can use t

Alex Chi 870 Jan 3, 2023
A comprehensive and FREE Online Rust hacking tutorial utilizing the x64, ARM64 and ARM32 architectures going step-by-step into the world of reverse engineering Rust from scratch.

FREE Reverse Engineering Self-Study Course HERE Hacking Rust A comprehensive and FREE Online Rust hacking tutorial utilizing the x64, ARM64 and ARM32

Kevin Thomas 98 Jun 21, 2023
A recommender systems framework for Rust

Quackin Release the quackin! ?? Quackin is a recommender systems framework written in Rust. This is a young project, which means two things: There wil

Christopher Vittal 8 Dec 8, 2021
Safe Rust bindings to the DynamoRIO dynamic binary instrumentation framework.

Introduction The dynamorio-rs crate provides safe Rust bindings to the DynamoRIO dynamic binary instrumentation framework, essentially allowing you to

S.J.R. van Schaik 17 Nov 21, 2022