🧠 Motörhead is a memory and information retrieval server for LLMs.

Overview

Motörhead

License Metal License

Motörhead is a memory and information retrieval server for LLMs.

Why use Motörhead?

When building chat applications using LLMs, memory handling is something that has to be built every time. Motörhead is a server to assist with that process. It provides 3 simple APIs:

  • GET /sessions/:id/memory returns messages up to MAX_WINDOW_SIZE.
{
    "messages": [
        {
            "role": "AI",
            "content": "Electronic music and salsa are two very different genres of music, and the way people dance to them is also quite different.\n\nSalsa is a Latin dance style that originated in the Caribbean and is characterized by its lively, syncopated rhythm and intricate footwork. Salsa dancing often involves partners dancing in close proximity, with a lot of hip and body movement. The dance is often very sensual and requires a lot of coordination between partners.\n\nElectronic music, on the other hand, is often associated with more free-form, individualistic dancing. While there are certainly partner dances that can be done to electronic music, such as the \"shuffle\" or \"jacking\" styles, many people prefer to dance on their own and let the music guide their movements. Electronic music is often characterized by a strong, steady beat that can be danced to in a variety of ways, from energetic jumping and bouncing to more fluid, flowing movements.\n\nIn summary, while both salsa and electronic music can be great for dancing, they are very different styles of music with different dance styles and techniques."
        },
        {
            "role": "Human",
            "content": "how does it compare to salsa?"
        },
        {
            "role": "AI",
            "content": "Electronic music is a broad genre that encompasses many different styles, so there is no one \"right\" way to dance to it. However, there are some general tips that can help you get started:\n\n1. Feel the beat: Electronic music is often characterized by a strong, steady beat. Try to feel the rhythm of the music and let it guide your movements.\n\n2. Move your body: Electronic music is often associated with energetic, full-body movements. Don't be afraid to let loose and move your arms, legs, and hips.\n\n3. Experiment with different styles: Depending on the style of electronic music you're listening to, there may be different dance styles that are associated with it. For example, techno and house music are often associated with \"rave\" or \"club\" dancing, while breakbeat and drum and bass are often associated with more athletic, breakdancing-style movements.\n\n4. Watch others: If you're new to electronic music, it can be helpful to watch others dance to get a sense of the style and rhythm. You can also find many tutorials and instructional videos online that can help you learn specific dance moves.\n\nRemember, the most important thing is to have fun and let the music move you!"
        },
        {
            "role": "Human",
            "content": "how do you dance electronic music?"
        },
        {
            "role": "AI",
            "content": "Colombia has a vibrant electronic music scene, and there are many talented DJs and producers who have gained international recognition. Here are a few famous DJs from Colombia:\n\n1. Erick Morillo: Born in New York City to Colombian parents, Erick Morillo is a DJ and producer who has had a long and successful career in the electronic music industry. He is best known for his house music tracks and has won several awards for his work.\n\n2. Kali Uchis: Although not strictly a DJ, Kali Uchis is a Colombian-American singer and songwriter who has gained a large following for her unique blend of R&B, soul, and pop music. She has collaborated with many famous musicians, including Tyler, The Creator and Gorillaz.\n\n3. DJ Fresh: DJ Fresh is a Colombian DJ and producer who is known for his work in the drum and bass genre. He has released several albums and has won numerous awards for his music.\n\n4. Mario Ochoa: Mario Ochoa is a DJ and producer from Medellín, Colombia, who is known for his techno and tech-house tracks. He has released music on many famous labels, including Toolroom and 100% Pure.\n\nThese are just a few examples of the many talented DJs and producers from Colombia."
        },
        {
            "role": "Human",
            "content": "What are some famous djs from Colombia?"
        },
        {
            "role": "AI",
            "content": "Baum opened its doors in 2014 and has quickly become one of the most popular clubs for electronic music in Bogotá. It is known for its impressive sound system and its focus on underground techno and house music.\n\nVideo Club, on the other hand, has been around for much longer. It first opened in 1999 and has since become an institution in the Bogotá nightlife scene. It is known for its eclectic music selection, which includes everything from electronic music to rock and pop. Over the years, Video Club has hosted many famous DJs and musicians, including Daft Punk, Chemical Brothers, and LCD Soundsystem."
        }
    ],
    "context": "The conversation covers topics such as clubs for electronic music in Bogotá, popular tourist attractions in the city, and general information about Colombia. The AI provides information about popular electronic music clubs such as Baum and Video Club, as well as electronic music festivals that take place in Bogotá. The AI also recommends tourist attractions such as La Candelaria, Monserrate and the Salt Cathedral of Zipaquirá, and provides general information about Colombia's diverse culture, landscape and wildlife."
}
  • POST /sessions/:id/memory - you can send multiple messages to Motorhead to store.
curl --location 'localhost:8080/sessions/${SESSION_ID}/memory' \
--header 'Content-Type: application/json' \
--data '{
    "messages": [{ "role": "Human", "content": "ping" }, { "role": "AI", "content": "pong" }]
}'
  • DELETE /sessions/:id/memory - deletes the session's message list.

A max window_size is set for the LLM to keep track of the conversation. Once that max is hit, Motörhead will process (window_size / 2 messages) and summarize them. Subsequent summaries, as the messages grow, are incremental.

Config

  • MAX_WINDOW_SIZE (default:10) - Number of max messages returned by the server. When this number is reached, a job is triggered to halve it.
  • OPENAI_API_KEY (required)- Number of max messages returned by the server. When this number is reached, a job is triggered to halve it.

How to run

With Docker:

docker-compose build && docker-compose up

Examples

You might also like...
[WIP] An experimental Java-like language and it's virtual machine, for learning Java and JVM.

Sky VM An experimental Java-like language and it's virtual machine, for learning Java and JVM. Dependencies Rust (rust-lang/rust) 2021 Edition, dual-l

Robust and Fast tokenizations alignment library for Rust and Python
Robust and Fast tokenizations alignment library for Rust and Python

Robust and Fast tokenizations alignment library for Rust and Python

Some hacks and failed experiments surrounding nvidia's gamestream protocol and sunshine/moonlight implementations

Sunrise This repository contains a bunch of experiments surrounding the nvidia gamestream protocol and reimplementations in the form of sunshine and m

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility. Main features: Train new vocabularies and tok

Open deep learning compiler stack for cpu, gpu and specialized accelerators
Open deep learning compiler stack for cpu, gpu and specialized accelerators

Open Deep Learning Compiler Stack Documentation | Contributors | Community | Release Notes Apache TVM is a compiler stack for deep learning systems. I

Distributed compute platform implemented in Rust, and powered by Apache Arrow.
Distributed compute platform implemented in Rust, and powered by Apache Arrow.

Ballista: Distributed Compute Platform Overview Ballista is a distributed compute platform primarily implemented in Rust, powered by Apache Arrow. It

Tensors and differentiable operations (like TensorFlow) in Rust

autograd Differentiable operations and tensors backed by ndarray. Motivation Machine learning is one of the field where Rust lagging behind other lang

Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference
Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference

Sonos' Neural Network inference engine. This project used to be called tfdeploy, or Tensorflow-deploy-rust. What ? tract is a Neural Network inference

Cleora AI is a general-purpose model for efficient, scalable learning of stable and inductive entity embeddings for heterogeneous relational data.
Cleora AI is a general-purpose model for efficient, scalable learning of stable and inductive entity embeddings for heterogeneous relational data.

Cleora Cleora is a genus of moths in the family Geometridae. Their scientific name derives from the Ancient Greek geo γῆ or γαῖα "the earth", and metr

Comments
  • Schema change

    Schema change

    After some feedback from some folks, we're changing the schema.

    Instead of accepting/returning the raw text AI: .... Human:... we utilize { role: 'AI', content: '...'}, { role: 'Human', content: '...'}

    opened by Czechh 0
  • Add long-term memory retrieval

    Add long-term memory retrieval

    Right now, Motorhead only supports short-term memory + incremental summarization over time.

    We want to also tap to long-term memory by accessing Metal or another datastore.

    opened by Czechh 0
  • Create a ChatGPT Plugin

    Create a ChatGPT Plugin

    Motörhead can be the engine to power the memory of experiences on ChatGPT and other llm providers. An easy to use integration via a Plugin would be a big step in the right direction.

    opened by softboyjimbo 0
Rust-port of spotify/annoy as a wrapper for Approximate Nearest Neighbors in C++/Python optimized for memory usage.

Fareast This library is a rust port of spotify/annoy , currently only index serving is supported. It also provides FFI bindings for jvm, dotnet and da

Arthur·Thomas 13 Mar 10, 2022
LLaMa 7b with CUDA acceleration implemented in rust. Minimal GPU memory needed!

LLaMa 7b in rust This repo contains the popular LLaMa 7b language model, fully implemented in the rust programming language! Uses dfdx tensors and CUD

Corey Lowman 16 May 8, 2023
Orkhon: ML Inference Framework and Server Runtime

Orkhon: ML Inference Framework and Server Runtime Latest Release License Build Status Downloads Gitter What is it? Orkhon is Rust framework for Machin

Theo M. Bulut 129 Dec 21, 2022
Network-agnostic, high-level game networking library for client-side prediction and server reconciliation.

WARNING: This crate currently depends on nightly rust unstable and incomplete features. crystalorb Network-agnostic, high-level game networking librar

Ernest Wong 175 Dec 31, 2022
scalable and fast unofficial osu! server implementation

gamma! the new bancho server for theta! built for scalability and speed configuration configuration is done either through gamma.toml, or through envi

null 3 Jan 7, 2023
A Demo server serving Bert through ONNX with GPU written in Rust with <3

Demo BERT ONNX server written in rust This demo showcase the use of onnxruntime-rs on BERT with a GPU on CUDA 11 served by actix-web and tokenized wit

Xavier Tao 28 Jan 1, 2023
Narwhal and Tusk A DAG-based Mempool and Efficient BFT Consensus.

This repo contains a prototype of Narwhal and Tusk. It supplements the paper Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus.

Facebook Research 134 Dec 8, 2022
MesaTEE GBDT-RS : a fast and secure GBDT library, supporting TEEs such as Intel SGX and ARM TrustZone

MesaTEE GBDT-RS : a fast and secure GBDT library, supporting TEEs such as Intel SGX and ARM TrustZone MesaTEE GBDT-RS is a gradient boost decision tre

MesaLock Linux 179 Nov 18, 2022
Ecosystem of libraries and tools for writing and executing extremely fast GPU code fully in Rust.

Ecosystem of libraries and tools for writing and executing extremely fast GPU code fully in Rust.

Riccardo D'Ambrosio 2.1k Jan 5, 2023
Ecosystem of libraries and tools for writing and executing fast GPU code fully in Rust.

The Rust CUDA Project An ecosystem of libraries and tools for writing and executing extremely fast GPU code fully in Rust Guide | Getting Started | Fe

Rust GPU 2.1k Dec 30, 2022