A Discord bot, written in Rust, that generates responses using the LLaMA language model.

Overview

llamacord

llamacord logo: a vaguely Discord Clyde-looking llama

A Discord bot, written in Rust, that generates responses using the LLaMA language model.

Built on top of llama-rs.

Setup

Model

  • Obtain the LLaMA weights from a reputable source (like Meta).
  • Convert and quantize them to GGML-q4 format using llama.cpp.

Bot

Discord

  • Create a Discord application and fill it out with your own details.
  • Go to Bot and create a new Bot.
    • Hit Reset Token, and copy the token it gives you somewhere.
  • Go to OAuth2 > URL Generator, select bot, then select Send Messages and Use Slash Commands.
    • Go to the URL it generates, and then invite it to a server of your choice.

Application

  • Install Rust 1.68 or above using rustup.
  • Run cargo run --release to start llamacord. This will auto-generate a configuration file, and then quit.
  • Fill in the configuration file with the required details, including the path to the quantized model.
  • You can then run llamacord to your heart's content.
You might also like...
A blazingly fast πŸ”₯ Discord bot written in Rust

rusty-bot πŸ¦€ A blazingly fast πŸ”₯ Discord bot written in Rust. Commands name use !rm count deletes old messages !meme subreddit sends a random meme

Powerfull Discord Raid Bot written in Rust, use VPN / Proxy because creating 200 channels in 10s Will ratelimit you.

Harakiri-Rust This the first Discord Raid Bot made in RustLang I recommend you use with a VPN or a Proxy to evade Discord Ratelimit. If bot doesn't st

Run LLaMA inference on CPU, with Rust πŸ¦€πŸš€πŸ¦™
Run LLaMA inference on CPU, with Rust πŸ¦€πŸš€πŸ¦™

LLaMA-rs Do the LLaMA thing, but now in Rust πŸ¦€ πŸš€ πŸ¦™ Image by @darthdeus, using Stable Diffusion LLaMA-rs is a Rust port of the llama.cpp project. Th

Run LLaMA inference on CPU, with Rust πŸ¦€πŸš€πŸ¦™
Run LLaMA inference on CPU, with Rust πŸ¦€πŸš€πŸ¦™

LLaMA-rs Do the LLaMA thing, but now in Rust πŸ¦€ πŸš€ πŸ¦™ Image by @darthdeus, using Stable Diffusion LLaMA-rs is a Rust port of the llama.cpp project. Th

A rusty interface to llama.cpp for rust

llama-cpp-rs Higher level API for the llama-cpp-sys library here: https://github.com/shadowmint/llama-cpp-sys/ A full end-to-end example can be found

A mimimal Rust implementation of Llama.c
A mimimal Rust implementation of Llama.c

llama2.rs Rust meets llama. A mimimal Rust implementation of karpathy's llama.c. Currently the code uses the 15M parameter model provided by Karpathy

High-level, optionally asynchronous Rust bindings to llama.cpp

llama_cpp-rs Safe, high-level Rust bindings to the C++ project of the same name, meant to be as user-friendly as possible. Run GGUF-based large langua

🦜 A hassle-free, highly performant, host it yourself Discord music bot built with Serenity in Rust. Powered by youtube-dl and Genius.

🦜 A hassle-free, highly performant and fast evolving Discord music bot built with Serenity in Rust. Deployment Usage Just create a bot account, copy

A discord bot that safely executes whatever rust you throw at it. Remote code execution as a service
A discord bot that safely executes whatever rust you throw at it. Remote code execution as a service

RustBot Bot is still under development and not ready for production use RustBot is a discord bot that executes whatever rust code you throw at it. In

Comments
  • Don't show the initial prompt in Alpaca mode

    Don't show the initial prompt in Alpaca mode

    The current logic is designed to show the entire prompt so that you can see how far the model has processed the prompt. Unfortunately, this gets quite noisy for Alpaca.

    Instead, it should immediately show the user's prompt with bold strikethrough. Once it gets to the prompt, it should unstrikethrough it as it gets processed. It should then wait until it actually starts generating new tokens to show anything else.

    enhancement 
    opened by philpax 0
  • Ability to cancel generation

    Ability to cancel generation

    Not sure how bots do this these days since I remember reactions having some problems with the API and rate-limiting, but a reaction to cancel prompt generation would be helpful if a message immediately goes in a direction you don't want.

    enhancement 
    opened by lmcintyre 0
Owner
Philpax
gamedev / systems programming / he/him
Philpax
OpenAI compatible API for serving LLAMA-2 model

Cria - Local llama OpenAI-compatible API The objective is to serve a local llama-2 model by mimicking an OpenAI API service. The llama2 model runs on

AmineDiro 66 Aug 8, 2023
Tiny Discord ticket support bot that utilizes the OpenAI GPT-3.5-turbo model.

BDFD AI Mod Our tiny Discord ticket support bot that utilizes the OpenAI GPT-3.5-turbo model. This project aims to help users by providing a very fast

NilPointer Software 5 Apr 20, 2023
A rust(serenity) based discord bot for the hacksquad discord server

A Discord Bot for Hacksquad How to Deploy? Requirements Docker Docker Compose Steps To Run Copy the docker-compose.yml and .env.example files to your

HackSquad 5 Jan 4, 2023
A Discord bot focused on addressing the inherent problems with Discord, to allow a more socialist/anarchist organization of servers.

ACABot A Discord bot focused on addressing the inherent problems with Discord, to allow a more socialist/anarchist organization of servers (or "guilds

null 4 May 3, 2022
Rewrite of the Discord Bot used for Managing the Infinity Bot List Servers.

Arcadia Rewrite of the Discord Bot used for Managing the Infinity Bot List Servers. Contributing Always run fmt.sh before making a Pull Request! MacOS

InfinityBotList 3 Dec 15, 2022
This is a Discord bot written in Rust to translate to and from the Bottom Encoding Standard using bottom-rs and Serenity.

bottom-bot This is a Discord bot written in Rust to translate to and from the Bottom Encoding Standard using bottom-rs and Serenity. Ever had this pro

Bottom Software Foundation 11 Dec 10, 2022
Rust bindings to llama.cpp, using metal on macOS

llama-rs Rust bindings to llama.cpp, for macOS, with metal support, for testing and evaluating whether it would be worthwhile to run an Llama model lo

Max Brunsfeld 7 Aug 31, 2023
An experimental Discord bot using Serenity.

VoidBot An experimental Discord bot using Serenity. Environment Variables Can be set with a .env file. DISCORD_TOKEN: The token for your bot. (require

null 1 May 21, 2022
A Discord bot for sending GeoGuessr challenge links that uses the GeoGuessr API written in rust.

GeoGuessr-bot-rs This is a simple implementation of a discord bot that send GeoGuessr-challenge links on demand. Features: Slash-commands Lightning-fa

Luukas PΓΆrtfors 6 Nov 18, 2022
A multipurpose (including music) Discord bot written in Rust

filloabot-rs A multipurpose (including music) Discord bot. This is a full rewrite using Rust of the original FilloaBot. Some features that were consid

FilloaBot 2 Oct 21, 2022