Cloud Native Buildpack that builds an OCI image with Ollama and a large language model.

Related tags

Command-line ai buildpack llm ollama
Overview
ollama

Ollama Cloud Native Buildpack

CI

This buildpack builds an OCI image with Ollama and a large language model. Configure your model by an Ollama Modelfile or use any of the ready-made models available.

Note

This is an early version of this buildpack. Please report any issues you find so they can get fixed quickly.

Installing

The buildpack is currently only available on GitHub. Download the most recent version from the releases. This buildpack supports both amd64 and arm64 architectures.

Getting Started

All the buildpack requires is a Modelfile that defines the model. The examples directory has a couple of examples. Point pack to the Ollama buildpack and the directory containing the Modelfile to build your image:

pack build my-image -b ollama.cnb --path examples/llama2-cnb-praise

After the image has been built, run it with Docker. You also have to map the Ollama port (11434) from the container to your local machine to be able to connect to it:

docker run -p 11434:11434 my-image

Your Ollama instance is now ready to be used. You can either use the API directly with i.e. curl:

curl http://localhost:11434/api/generate -d '{"model": "custom_model", "prompt":"Hi!"}'

or use a more sophisticated client like oterm:

oterm screenshot

Building

This buildpack is written in Rust. Make sure you have a current version of Rust installed (i.e. via rustup) as well as libcnb-cargo (cargo install libcnb-cargo).

Running cargo libcnb package in the root of this repository will build the buildpack. Follow any instructions to install missing dependencies.

Use --target to build the buildpack for a different target OS and architecture.

You might also like...
auto-rust is an experimental project that aims to automatically generate Rust code with LLM (Large Language Models) during compilation, utilizing procedural macros.
auto-rust is an experimental project that aims to automatically generate Rust code with LLM (Large Language Models) during compilation, utilizing procedural macros.

Auto Rust auto-rust is an experimental project that aims to automatically generate Rust code with LLM (Large Language Models) during compilation, util

Minsky 6 May 14, 2023
Manage lightweight VMs created from OCI images
Manage lightweight VMs created from OCI images

krunvm krunvm is a CLI-based utility for managing lightweight VMs created from OCI images, using libkrun and buildah. Features Minimal footprint Fast

Containers 1.1k Jan 6, 2023
A diff-based data management language to implement unlimited undo, auto-save for games, and cloud-apps which needs to retain every change.

Docchi is a diff-based data management language to implement unlimited undo, auto-save for games, and cloud-apps which needs to save very often. User'

juzy 21 Sep 19, 2022
Codemod - Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention

Codemod - Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention. Codemod was developed at Facebook and released as open source.

Meta Archive 4k Dec 29, 2022
A rust crate for rendering large text to the terminal using font8x8 and ratatui.
A rust crate for rendering large text to the terminal using font8x8 and ratatui.

tui-big-text tui-big-text is a rust crate that renders large pixel text as a ratatui widget using the glyphs from the font8x8 crate. Installation carg

Josh McKinney 7 Sep 7, 2023
ChatGPT-Code-Review is a Rust application that uses the OpenAI GPT-3.5 language model to review code

ChatGPT-Code-Review is a Rust application that uses the OpenAI GPT-3.5 language model to review code. It accepts a local path to a folder containing code, and generates a review for each file in the folder and its subdirectories.

Greg P. 15 Apr 22, 2023
An ultra-fast CLI app that fixes JSON files in large codebase or folders

minosse An ultra fast CLI app that fixes json files in large codebase or folders USAGE: minosse [OPTIONS] input-dir FLAGS: -h, --help Prints

Antonino Bertulla 5 Oct 17, 2022
Framework for large distributed pipelines
Framework for large distributed pipelines

Rain Rain is an open-source distributed computational framework for processing of large-scale task-based pipelines. Rain aims to lower the entry barri

Substantic 705 Dec 27, 2022
Tool to allow parsing large JSON files without laoding into memory
Tool to allow parsing large JSON files without laoding into memory

Tool to allow parsing large JSON files without laoding into memory. Developed in Rust with adapters in other programming langauges for easy adoption

Salaah Amin 7 Jul 11, 2023
Owner
Manuel Fuchs
Geek. Father of two πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦. Into retro computing πŸ‘Ύ, functional programming Ξ», synths 🎹, and space stuff πŸ”­πŸš€ - JVM Languages Owner @salesforce @heroku.
Manuel Fuchs
GitHub
Attempt to summarize text from `stdin`, using a large language model (locally and offline), to `stdout`

summarize-cli Attempt to summarize text from stdin, using a large language model (locally and offline), to stdout. cargo build --release target/releas

null 4 Aug 23, 2023
Terminal UI to chat with large language models (LLM) using different model backends, and integrations with your favourite editors!

Oatmeal Terminal UI to chat with large language models (LLM) using different model backends, and integrations with your favourite editors! Overview In

Dustin Blackman 88 Dec 4, 2023
Slack chat bot written in Rust that allows the user to interact with a large language model.

A Slack chat bot written in Rust that allows the user to interact with a large language model. Creating an App on Slack, first steps Go to https://api

Marco Inacio 13 Nov 2, 2023
A container image builder tool for OCI (distrobox/toolbox, also podman/docker)

Distrobox Boost A container image builder tool for Open Container Initiative (distrobox/toolbox, also podman/docker). Distrobox is good enough in runn

xz-dev 6 Aug 15, 2023
Conference Monitoring Project based on Image Recognition that uses Rust Language and AWS Rekognition service to get the level of image similarity.

Conference Monitoring System based on Image Recognition in Rust This is a Conference Monitoring Project based on Image Recognition that uses Rust Lang

Pankaj Chaudhary 6 Dec 18, 2022
By mirroring traffic to and from your machine, mirrord surrounds your local service with a mirror image of its cloud environment.

mirrord lets you easily mirror traffic from your Kubernetes cluster to your development environment. It comes as both Visual Studio Code extension and

MetalBear 2.1k Jan 3, 2023
TMM is a Linux native game modding tool. it allows to install and depoly mods for Linux native and wine games.

Tux Mod Manager TMM is a Linux native mod manager made with the Tauri toolkit. It can install, load, remove and deploy mods for both Linux native and

Mathiew May 119 Dec 27, 2022
Macro to print variable(s) with values nicely (stripped from release builds)

log_macro Macro to print variable(s) with values nicely (stripped from release builds) Install cargo add log_macro Use Add this to top of file: #[mac

Nikita 3 Aug 22, 2023
πŸ¦€Rust + Large Language Models - Make AI Services Freely and Easily. Inspired by LangChain

llmchain: Modern Data Transformations with LLM ?? + Large Language Models, inspired by LangChain. Features Models: LLMs & Chat Models & Embedding Mode

Shafish Labs 63 Jun 22, 2023
Save image from your clipboard πŸ“‹ as an image file directly from your command line! πŸ”₯

Clpy ?? Save copied image from clipboard as an image file directly from your command line! Note It works only on windows as of now. I'll be adding sup

Piyush Suthar 13 Nov 28, 2022