Cloud Native Buildpack that builds an OCI image with Ollama and a large language model.

Last update: Mar 19, 2024

Related tags

Overview

Ollama Cloud Native Buildpack

This buildpack builds an OCI image with Ollama and a large language model. Configure your model by an Ollama Modelfile or use any of the ready-made models available.

Note

This is an early version of this buildpack. Please report any issues you find so they can get fixed quickly.

Installing

The buildpack is currently only available on GitHub. Download the most recent version from the releases. This buildpack supports both amd64 and arm64 architectures.

Getting Started

All the buildpack requires is a Modelfile that defines the model. The examples directory has a couple of examples. Point pack to the Ollama buildpack and the directory containing the Modelfile to build your image:

pack build my-image -b ollama.cnb --path examples/llama2-cnb-praise

After the image has been built, run it with Docker. You also have to map the Ollama port (11434) from the container to your local machine to be able to connect to it:

docker run -p 11434:11434 my-image

Your Ollama instance is now ready to be used. You can either use the API directly with i.e. curl:

curl http://localhost:11434/api/generate -d '{"model": "custom_model", "prompt":"Hi!"}'

or use a more sophisticated client like oterm:

Building

This buildpack is written in Rust. Make sure you have a current version of Rust installed (i.e. via rustup) as well as libcnb-cargo (cargo install libcnb-cargo).

Running cargo libcnb package in the root of this repository will build the buildpack. Follow any instructions to install missing dependencies.

Use --target to build the buildpack for a different target OS and architecture.

Save image from your clipboard 📋 as an image file directly from your command line! 🔥

Clpy 📋 Save copied image from clipboard as an image file directly from your command line! Note It works only on windows as of now. I'll be adding sup

13 Nov 28, 2022

Manage lightweight VMs created from OCI images

krunvm krunvm is a CLI-based utility for managing lightweight VMs created from OCI images, using libkrun and buildah. Features Minimal footprint Fast

1.1k Jan 6, 2023

A diff-based data management language to implement unlimited undo, auto-save for games, and cloud-apps which needs to retain every change.

Docchi is a diff-based data management language to implement unlimited undo, auto-save for games, and cloud-apps which needs to save very often. User'

21 Sep 19, 2022

Codemod - Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention

Codemod - Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention. Codemod was developed at Facebook and released as open source.

4k Dec 29, 2022

A rust crate for rendering large text to the terminal using font8x8 and ratatui.

Cloud Native Buildpack that builds an OCI image with Ollama and a large language model.

Related tags

Overview

Ollama Cloud Native Buildpack

Installing

Getting Started

Building

You might also like...

Save image from your clipboard 📋 as an image file directly from your command line! 🔥

Manage lightweight VMs created from OCI images

A diff-based data management language to implement unlimited undo, auto-save for games, and cloud-apps which needs to retain every change.

Codemod - Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention

A rust crate for rendering large text to the terminal using font8x8 and ratatui.

ChatGPT-Code-Review is a Rust application that uses the OpenAI GPT-3.5 language model to review code

An ultra-fast CLI app that fixes JSON files in large codebase or folders

Framework for large distributed pipelines

Tool to allow parsing large JSON files without laoding into memory

Releases(v1.0.0)

v1.0.0(Mar 14, 2024)

Owner

Manuel Fuchs

Attempt to summarize text from `stdin`, using a large language model (locally and offline), to `stdout`

Terminal UI to chat with large language models (LLM) using different model backends, and integrations with your favourite editors!

Slack chat bot written in Rust that allows the user to interact with a large language model.

A container image builder tool for OCI (distrobox/toolbox, also podman/docker)

Conference Monitoring Project based on Image Recognition that uses Rust Language and AWS Rekognition service to get the level of image similarity.

By mirroring traffic to and from your machine, mirrord surrounds your local service with a mirror image of its cloud environment.

TMM is a Linux native game modding tool. it allows to install and depoly mods for Linux native and wine games.

Macro to print variable(s) with values nicely (stripped from release builds)

🦀Rust + Large Language Models - Make AI Services Freely and Easily. Inspired by LangChain

auto-rust is an experimental project that aims to automatically generate Rust code with LLM (Large Language Models) during compilation, utilizing procedural macros.