Rust Local-llm Resources

git-task Local-first task manager/bug tracker within your git repository which can import issues from GitHub. Installation Build a binary, then add it

cai - The fastest CLI tool for prompting LLMs Features Build with Rust 🦀 for supreme performance and speed! 🏎️ Support for models by Groq, OpenAI, A

COBOLEval: LLM Evaluation for COBOL COBOLEval is a dataset to evaluate the code generation abilities of Large Language Models on the COBOL programming

OpenLLaMA Overview A Rust LLaMA project to load, serve and extend LLM models. Key Objectives Support both GGML and HF(HuggingFace) models Support a st

Ollama Cloud Native Buildpack This buildpack builds an OCI image with Ollama and a large language model. Configure your model by an Ollama Modelfile o

FileQL - File Query Language FileQL is a tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK. Sampl

The Natural Language Shell integrates GPT4 or local GGUF-formatted models directly into the terminal experience, allowing operators to describe their tasks in either POSIX commands or fluent human language

An egui app for prompting a local offline LLM. Description coze is a small egui application for prompting a local offline LLM using the Huggingface ca

Table of Contents About The Project Getting Started Roadmap Contributing License Contact A rust interface for the OpenAI API and Llama.cpp ./server AP

smartcat (sc) Puts a brain behind cat! CLI interface to bring language models in the Unix ecosystem and allow power users to make the most out of llms

karyon An infrastructure for peer-to-peer, decentralized, and collaborative software. In molecular biology, a Karyon is essentially "a part of the cel

Oatmeal Terminal UI to chat with large language models (LLM) using different model backends, and integrations with your favourite editors! Overview In

Vault Vault will be a command line tool (if successful) similar to git which would have multiple features like brances etc etc. __ __ _ _

Auto Undiscord Did you hear that Discord will be blocking external websites from using images hosted on their servers? Did you host every image on you

candle-vllm Efficient platform for inference and serving local LLMs including an OpenAI compatible API server. Features OpenAI compatible API server p

Poly Poly is a versatile LLM serving back-end. What it offers: High-performance, efficient and reliable serving of multiple local LLM models Optional

A Slack chat bot written in Rust that allows the user to interact with a large language model. Creating an App on Slack, first steps Go to https://api

augre An LLM-powered (CodeLlama or OpenAI) local diff code review tool. Binary Usage Install Windows: $ iwr https://github.com/twitchax/augre/releases

Dreamer 💤 Does what you want, not what you say. Dreamer is a universal tool management system for any language, any build system, and any framework.

Make Cabana Route Utility that takes CSV formatted CAN log files and (optionally) accompanying videos, convert them to "routes" that can be opened in

Fast Vector Similarity Library Introduction The Fast Vector Similarity Library is designed to provide efficient computation of various similarity meas

CodeIndex CodeIndex is a local-first high performance codebase index engine designed for AI. It helps your LLM understand the structure and semantics

TP-Link L920 on/off script This is a (very) simple Rust binary that can turn on/off a TP-Link L920 led light strip in your local network. Installation

llama2-rs LLaMA2 port for Rust inspired by llama2.c. TODOs: Implement loading of the model Implement forward pass Implement generation Implement token

summarize-cli Attempt to summarize text from stdin, using a large language model (locally and offline), to stdout. cargo build --release target/releas

💯AI00 RWKV Server English | 中文 | 日本語 AI00 RWKV Server is an inference API server based on the RWKV model. It supports VULKAN inference acceleration a

Promptize Promptize attempts to solve the issues with context limits when working with AI systems. It allows a user to add an attribute to their struc

pg_vectorize under development The simplest implementation of LLM-backed vector search on Postgres. -- initialize an existing table select vectorize.i

clean-local A simple CLI to remove all the files and folders included in the .gitignore file. Written in RUST Supports glob and wildcard paths in .git

🐜 ANT Another Network Tunnel; A simple program for local/remote port forwarding over a SSH tunnel. Table of Contents Installation Pre-requisites Pre-

memex Super simple "memory" for LLM projects, semantic search, etc. Running the service Note that if you're running on Apple silicon (M1/M2/etc.), it'

A gateway that proxies signed requests to other APIs. Check the docs for more info. If you are looking for the managed version checkout this link http

Clipshare Do you ever have to work on multiple machines? Do you ever used your Github™ Gists just to send some text between then? Clipshare is here to

Multi-platform desktop app to download and run Large Language Models(LLM) locally in your computer 🔗 Download | Give it a Star ⭐ | Share it on Twitte

sothis Sothis is a tool for replaying historical state on a local anvil/hardhat testnet node. Usage Sothis currently has 2 modes. Live and historic. I

Opool: Fast lock-free concurrent and local object pool Opool is a high-performance Rust library that offers a concurrent and local object pool impleme

Auto Rust auto-rust is an experimental project that aims to automatically generate Rust code with LLM (Large Language Models) during compilation, util

llm-rs-python: Python Bindings for Rust's llm Library Welcome to llm-rs, an unofficial Python interface for the Rust-based llm library, made possible

Jumpstart your llm-chain projects with the llm-chain-template repository! This template provides a foundation for using the llm-chain library, complete with example code and instructions to get you started effortlessly.

pgvecto.rs pgvecto.rs is a Postgres extension that provides vector similarity search functions. It is written in Rust and based on pgrx. Features cosi

Tenere TUI interface for LLMs written in Rust 📸 Demo 💎 Supported LLMs Only ChatGPT is supported for the moment. But I'm planning to support more mod

SmartGPT SmartGPT is an experimental program meant to provide LLMs (particularly GPT-3.5 and GPT-4) with the ability to complete complex tasks without

Bolik monorepo Bolik Timeline is local-first software for keeping notes and files. This repo contains alpha-quality software. This means that we are e

llama-node Large Language Model LLaMA on node.js This project is in an early stage, the API for nodejs may change in the future, use it with caution.

Smol Rust RWKV What is it? A simple example of the RWKV approach to language models written in Rust by someone that knows basically nothing about math

llm-chain 🚀 llm-chain is a collection of Rust crates designed to help you work with Large Language Models (LLMs) more effectively. Our primary focus

Orphic A natural language shell interface for *nix systems. Overview Orphic is a CLI tool that uses GPT to translate complex tasks into shell commands

OFFFLIX A simple application to auto manage series on your local storage. Features Resume watching Play next episode Auto increment season Play random

axoasset This library offers read, write, and copy functions, for local and remote assets given a string that contains a relative or absolute local pa

with-thread-local A micro crate that simplifies a bit the use of the std macro thread_local!. extern crate regex; use with_thread_local::with_thread_

weird.one Local-first linkspage generator Usage You'll need Perseus and SurrealDB. Start the SurrealDB instance. surreal start -u root -p root --log d

clia-tracing-config A convenient tracing config and init lib, with symlinking and local timezone. Use these formats default, and can be configured: pr

clock-tui (tclock) A clock app in terminal. It support the following modes: Clock Timer Stopwatch Countdown Usage Install Install excutable by cargo:

TicTacToe Gui Game in rust Cross platform game using Macroquad - https://github.com/not-fl3/macroquad Planned to be Local and online multiplayer Quick

gitcp Copy files from Git repository to local. Install We are planning to add some installers support in the future. e.g. homebrew winget debian packa

mirrord lets you easily mirror traffic from your Kubernetes cluster to your development environment. It comes as both Visual Studio Code extension and

Figma Agent for Linux (a.k.a. Font Helper)

git-eq (aka git earthquake) Earthquakes are part of the daily life in many countries like in Taiwan. git-eq is a simple git command to quickly save yo

quad-storage This is the crate to save data in persistent local storage in miniquad/macroquad environment. In WASM the data persists even if tab or br

Remote Pseudoterminals Remote Pseudoterminals or "RPTY" is a Rust library which intercepts calls to the Linux kernel's TTY/PTY-related libc functions

Pup A command line tool aiming to upload the local image used in your markdown file to the GitHub repo and replace the local file path with the return

bore A modern, simple TCP tunnel in Rust that exposes local ports to a remote server, bypassing standard NAT connection firewalls. That's all it does:

CLEO Redux Начало Что такое CLEO Redux? Поддерживаемые языки Поддерживаемые релизы Связь с CLEO Library Запуск CLEO Redux как отдельного ПО Запуск CLE

Noir Pay v0 Built on Light Protocol Noir Pay will be directly built ontop of the Light Protocol SDK and provide users with a beautifully simple privat

Live Server - Launch a local network server with live reload feature for static pages

wolfram-app-discovery Discovery local installations of the Wolfram Language and Wolfram applications. This crate provides: The wolfram-app-discovery l

UniDirs Unified directories for different use cases of an application, providing standard directories for local development, when run as service or wh

udp53_lookup Test the interception/filter of UDP 53 of your local networks or hotspots. Inspired by BennyThink/UDP53-Filter-Type . What's the purpose?

tracing-local-time A patch to fix tracing LocalTime problem. Tracing-subscriber now has a bug in LocalTime, so build ourselves' to fix it. In this pat

Save Game Extractor | Download Save Game Extractor is a tool that automatically locates and copies save files for Windows games in public directories.

nauman A CI inspired approach for local job automation. Features • Installation • Usage • FAQ • Examples • Job Syntax About nauman is an easy-to-use j

age-plugin-remote TBD Design Notes Proxy identities act as bearer tokens. A proxy identity is only "active" if the authority provides it when starting

localzone Utility crate to figure out the IANA (Olson) timezone of the current machine. The IANA timezones have been largely established as the standa

git-chain Tool for rebasing a chain of local git branches. Motivation Suppose you have branches, each depending on a parent branch (usually called "st

WKSL - a wake and sleep utility An experiment in writing a small CLI utility in Rust. The program lets you wake a machine on your local network up fro

Rust Web Local Storage API A Rust implementation of the Web LocalStorage API, for use in non-browser contexts About the Web Local Storage API MDN docs

rizzy All in a tizzy over timestamps? rizzy is a UNIX filter that will convert UTC timestamps into the timestamp of your choosing. Example Before: $ c

Viceroy provides local testing for developers working with Compute@Edge. It allows you to run services written against the Compute@Edge APIs on your local development machine, and allows you to configure testing backends for your service to communicate with.

innisfree A tool to aid in self-hosting. Expose local services on your computer, via a public IPv4 address. Why? Most of the data I maintain is local,

Fastly VCL Interpreters in Rust A tree-walk interpreter for Fastly VCL Install project Install rust - curl https://sh.rustup.rs -sSf | sh Clone projec

Mousse is an Ethereum 2.0 emulator for local testing of Eth2 applications (mainly Rollups). HTTP Server The REST API definition can be found in the ht

TON OS Startup Edition Local blockchain for Free TON DApp development and testing. Have a question? Get quick help in our channel: TON OS Startup Edit