A highly customizable, full scale web backend for web-rwkv, built on axum with websocket protocol.

Related tags

Web programming web-rwkv-axum
Overview

web-rwkv-axum

A axum web backend for web-rwkv, built on websocket.

Supports BNF-constrained grammar, CFG sampling, etc., all streamed over network.

Still under heavy development, PRs and suggestions are welcome.

Testing

  • Run by cargo run --release ./config.toml. Wait for Model is loaded! to popup.
  • Run the /tests/curl_ws.py "{any prompt input}" in the tests folder.

Protocol

Since it's built based on websocket, and supports highly varied pipeline customizations including complex logits transformations and sampling methods, web-rwkv-axum is built on a new protocol.

For specification, please refer to the docs folder.

You might also like...
Layers, extractors and template engine wrappers for axum based Web MVC applications

axum-template Layers, extractors and template engine wrappers for axum based Web MVC applications Getting started Cargo.toml [dependencies] axum-templ

Altair Bueno 11 Dec 15, 2022
Starter template for use with the Leptos web framework and Axum.

Leptos Axum Starter Template This is a template for use with the Leptos web framework and the cargo-leptos tool using Axum. Creating your template rep

Leptos 10 Mar 4, 2023
Axum + Connect-Web = ♥️

Axum Connect-Web ⚠️ This project isn't even Alpha state yet. Don't use it. Brings the protobuf-based Connect-Web RPC framework to Rust via idiomatic A

Alec Thilenius 10 Mar 19, 2023
A customizable, simple and easy to use json REST API consumer

JACK is a generic JSON API client. It is useful to interact with APIs from multiple services such as Google and Twitter

Mente Binária 6 May 22, 2022
A full-featured and easy-to-use web framework with the Rust programming language.

Poem Framework A program is like a poem, you cannot write a poem without writing it. --- Dijkstra A full-featured and easy-to-use web framework with t

Poem Web 2.2k Jan 6, 2023
Implementation of the RealWorld backend API spec in Actix, Rust's powerful actor system and most fun web framework.
Implementation of the RealWorld backend API spec in Actix, Rust's powerful actor system and most fun web framework.

Actix codebase containing real world examples (CRUD, auth, advanced patterns, etc) that adheres to the RealWorld spec and API. ❗ (2021/05/13) This cod

Allen 475 Jan 2, 2023
Yew + Axum + blog = Yab

Yew + Axum + blog = Yab

STUDIO RSBM 13 Dec 5, 2022
Rust Axum+SQLx Sample

rust-axum-sqlx-sample Install git clone https://github.com/web3ten0/rust-axum-sqlx-1.git cd rust-axum-sqlx-1/local docker-compose up -d sh scripts/exe

web3ten0 5 Jul 9, 2022
🪪 Session-based user authentication for Axum.

axum-login 🪪 Session-based user authentication for Axum. 🎨 Overview axum-login is a Tower middleware providing session-based user authentication for

Max Countryman 99 Jan 5, 2023
Comments
  • Reworked the framework

    Reworked the framework

    • States are now all stored on GPU for faster swapping.
    • Added max_concurrency to limit max concurrent requests that can be sent to GPU at a time.
    • Implemented Terminal. A terminal can be used to determine the exit of an inference request. The client still needs to determine if to continue the generation by passing the last token or end the generation.
    • Cleaned up the AppState and reorganized many parts of the state for more readable code.
    opened by Prunoideae 0
  • Recent updates

    Recent updates

    • Almost completed the pipeline and framework to make it run.
    • Almost completed the docs.
    • Have a complete idea about redesigning the entire framework (mostly about AppState managing, state managing and swapping strategy, as well as sampling strategy)
    opened by Prunoideae 0
Owner
Li Junyu
PhD Student in Marine Environmental Science, HKUST.
Li Junyu
GitHub
example of a full stack web app (backend and frontend) wrtiten in Rust

rust-fullstack-example An example of creating a full stack web application (backend and frontend) using Rust. Backend Go to ./backend and start the se

mario 41 Dec 16, 2022
Implementation of the RWKV language model in pure WebGPU/Rust.

Web-RWKV This is an implementation of the language model of RWKV in pure WebGPU. Compile and Run Install Rust. Run cargo run --release --example gen t

研究社交 22 Jul 12, 2023
Rust Axum Full Course code.

Rust Axum Full Course source code. YouTube Full Course: https://youtube.com/watch?v=XZtlD_m59sM&list=PL7r-PXl6ZPcCIOFaL7nVHXZvBmHNhrh_Q MIT OR Apache,

null 27 Apr 19, 2023
The goal of this repository is to have a real world template of a Rust backend using Axum, Diesel, etc.

Axum Diesel Real-World Example A modular Rust backend template based on the Domain-Driven Design (DDD) architecture, utilizing the Axum and Diesel fra

Quentin Piot 11 Sep 22, 2023
Code template for a production Web Application using Axum: The AwesomeApp Blueprint for Professional Web Development.

AwesomeApp rust-web-app More info at: https://awesomeapp.dev/rust-web-app/ rust-web-app YouTube episodes: Episode 01 - Rust Web App - Course to Produc

null 45 Sep 6, 2023
A crate built on top of `axum-sessions`, implementing the CSRF Synchronizer Token Pattern

Axum Synchronizer Token Pattern CSRF prevention This crate provides a Cross-Site Request Forgery protection layer and middleware for use with the axum

null 3 Dec 15, 2022
Thalo is an event-sourcing framework for building large scale systems

Thalo Event sourcing framework for building microservices. Overview Thalo is an event-sourcing framework for building large scale systems based on the

null 548 Jan 3, 2023
Experiments with Rust CRDTs using Tokio web application framework Axum.

crdt-genome Synopsis Experiments with Rust CRDTs using Tokio web application framework Axum. Background Exploring some ideas of Martin Kleppmann, part

dougfort 3 Mar 18, 2022
Axum web framework tutorial for beginners.

Axum Tutorial For Beginners Hello web developers! This tutorial will cover how to write simple web applications in rust with axum framework. If you ar

Eray Karatay 46 Jan 5, 2023
Demo of Rust and axum web framework

Demo of Rust and axum web framework Demonstration of: Rust: programming language that focuses on reliability and stability. axum: web framework that f

Joel Parker Henderson 115 Dec 29, 2022