Use HuggingFace's Candle with Go.

Overview

An example on using huggingface/candle with Golang. For educational purposes only. The implementation is thread-safe and uses multilingual-e5-large model.

On of the use cases for text embeddings is to provide semantic search:

> go test -run TestEmbeddings
top3 for 'query: burnouts and how to deal with them':
dist: 0.212324  'query: overworking leads to depression'
dist: 0.228119  'query: protein shakes and other stuff'
dist: 0.231076  'query: the cause of bad behaviour'
top3 for 'query: feline anatomy':
dist: 0.111701  'query: cat's body'
dist: 0.241103  'query: protein shakes and other stuff'
dist: 0.245675  'query: the works of Francis Bacon'
top3 for 'query: 16th century philosophers':
dist: 0.092926  'query: 18th century philosophers'
dist: 0.230078  'query: Critique of Pure Reason'
dist: 0.236497  'query: the works of Francis Bacon'
top3 for 'query: overworking leads to depression':
dist: 0.186588  'query: the cause of bad behaviour'
dist: 0.212324  'query: burnouts and how to deal with them'
dist: 0.239519  'query: what the reason for being not nice'
top3 for 'query: Critique of Pure Reason':
dist: 0.181644  'query: the cause of bad behaviour'
dist: 0.225016  'query: 18th century philosophers'
dist: 0.226892  'query: what the reason for being not nice'
top3 for 'query: the books of Immanuel Kant':
dist: 0.188276  'query: the works of Francis Bacon'
dist: 0.250316  'query: Critique of Pure Reason'
dist: 0.252901  'query: 18th century philosophers'
...

Usage

  1. Compile this project with Rust:
cargo build --release
  1. Navigate to go/ directory and build the Go binary there:
go build

The cgo code references Rust library in a relative path:

#cgo LDFLAGS: -L../target/release -ltxtvec

So the relative path should be the same for go build to work. 3. Now you have a single fat binary that can download and cache e5-large-multilingual model and serve the text embeddings:

curl -X POST -H "Content-Type: application/json" -d '["hello", "world"]' http://localhost:8080/embeddings

This requires 2.4 GB of RAM to serve the model.

You might also like...
Simple and minimalist forward auth service intended for use with reverse proxies (Traefik, Caddy, nginx, etc)
Simple and minimalist forward auth service intended for use with reverse proxies (Traefik, Caddy, nginx, etc)

nforwardauth nforwardauth is an extremely lightweight, blazing fast forward auth service that lets you use a single authentication middleware for all

Convert PNG image files to binary for use with AgonLight (TM)

image2agon Converts PNG files to binary data for AgonLight (TM) usage. This document is for version V1.0 of the program. V1.0 - initial upload NOTE: T

An attempt to start documenting the rust sdk for temporal and how to use it following some of the examples in typescript

This is an attempt to start documenting the rust sdk for temporal and how to use it following some of the examples in typescript.

derive(Code) simplifies error handling by providing an easy-to-use enumeration of error codes

enum-code Introduction enum-code is a derive macro for enum types. This library generates code that associates error codes with error types. It can be

Macros for candle-lora.

candle-lora-macro This library makes using candle-lora as simple as adding 2 macros to your model structs and calling a method! It is inspired by the

A simplified example in Rust of training a neural network and then using it based on the Candle Framework by Hugging Face.

candle-simplified-example A simplified example in Rust of training a neural network and then using it based on the Candle Framework by Hugging Face. H

Low rank adaptation (LoRA) for Candle.

candle-lora LoRA (low rank adaptation) implemented in Rust for use with Candle. This technique interchanges the fully-trainable layers of the model wi

Tutorial for Porting PyTorch Transformer Models to Candle (Rust)
Tutorial for Porting PyTorch Transformer Models to Candle (Rust)

Candle Tutorial - Convert Pytorch Models to Candle Candle is an ML framework written in rust that takes advantage of the speed and memory safety Rust

A rust chess implementation using a neural network scoring function built on huggingface/candle + rust + wasm

Rusty Chess What is it? Rusty Chess aims to be a high quality embeddable chess engine that runs entirely locally in the browser (no backend required).

rust+slint+candle+openchat3.5 demo
rust+slint+candle+openchat3.5 demo

Slint Chatbot Demo This is a demo of Rust + Slint + Candle + OpenChat LLM, it looks like this: Do it by yourself Make sure you have downloaded opencha

A Rust-based shell script to create a folder structure to use for a single class every semester. Mostly an excuse to use Rust.

A Rust Course Folder Shell Script PROJECT IN PROGRESS (Spring 2022) When completed, script will create a folder structure of the following schema: [ro

A simpler and 5x faster alternative to HashMap in Rust, which doesn't use hashing and doesn't use heap

At least 5x faster alternative of HashMap, for very small maps. It is also faster than FxHashMap, hashbrown, ArrayMap, and nohash-hasher. The smaller

Webpack loader for Rust files. DEPRECATED, use WasmPack instead

The project is in low maintance now Use WasmPack instead Webpack Rust loader Webpack loader for Rust Example add.rs #[no_mangle] pub fn add(a: i32, b:

A OAuth2 server library, for use in combination with actix or other frontends, featuring a set of configurable and pluggable backends.

oxide-auth A OAuth2 server library, for use in combination with common web servers, featuring a set of configurable and pluggable backends. About oxid

Easy to use cryptographic framework for data protection: secure messaging with forward secrecy and secure data storage. Has unified APIs across 14 platforms.
Easy to use cryptographic framework for data protection: secure messaging with forward secrecy and secure data storage. Has unified APIs across 14 platforms.

Themis provides strong, usable cryptography for busy people General purpose cryptographic library for storage and messaging for iOS (Swift, Obj-C), An

A easy-use client to influxdb

InfluxDBClient-rs A easy-use client to influxdb Overview This is an InfluxDB driver for Rust. Status This project has been able to run properly, PR is

Easy-to-use beanstalkd client for Rust (IronMQ compatible)

rust-beanstalkd Easy-to-use beanstalkd client for Rust (IronMQ compatible) Install Add this dependency to your Cargo.toml beanstalkd = "*" Documentati

An easy-to-use library for writing PDF in Rust

printpdf printpdf is a library designed for creating printable PDF documents. Crates.io | Documentation [dependencies] printpdf = "0.3.2" Features Cur

An easy-to-use, 2D GUI library written entirely in Rust.

Conrod An easy-to-use, 2D GUI library written entirely in Rust. Guide What is Conrod? A Brief Summary Screenshots and Videos Feature Overview Availabl

Owner
null
Rust library for hardware accelerated drawing of 2D shapes, images, and text, with an easy to use API.

Speedy2D Hardware-accelerated drawing of shapes, images, and text, with an easy to use API. Speedy2D aims to be: The simplest Rust API for creating a

null 223 Dec 26, 2022
Make and use playgrounds locally.

cargo playground Cargo playground opens a local playground in the editor of your choice. Install You can install it directly using cargo $ cargo insta

null 18 Jan 3, 2023
A tool to use docker / podman / oci containers with rust

contain-rs A tool to use docker / podman / oci containers with rust TODO improve error types improve error reporting handle std error for child proces

Felix Marezki 1 Jan 1, 2023
Small, clean, easy to use programming language

Thistle A modern, simplistic multi-paradigm language supporting object-oriented features Hello World! import IO object Main def main(): unit

null 7 Apr 13, 2022
A collection (eventually) of examples that use some non-beginner things.

nannou examples A collection (eventually) of examples that use some non-beginner things. Right now the only example combines nannou's standard draw AP

Alexis Andre 22 Oct 21, 2022
Demonstration of how to use the rust object_store crate

Introduction Demonstration of how to use the Rust object_store crate Example

Andrew Lamb 4 Aug 29, 2022
Game development practices with Rust programming language. I want to use different crates for this.

Hazır Oyun Motorlarını Kullanarak Rust Dili Yardımıyla Oyunlar Geliştirmek Rust programlama dilinde oyun geliştirmek için popüler birkaç hazır çatıyı

Burak Selim Senyurt 16 Dec 27, 2022
rust database for you to use and help me make!

Welcome To Rust Database! What is this? this is a database for you to git clone and use in your project! Why should i use it? It is fast and it takes

Carghai74 2 Dec 4, 2022
A dos attack for you to use!

Welcome To Rust Dos attacker! Why should I use it? It has unrivaled speeds because it is built in rust and hand optimized. It also comes with an AI mo

Carghai74 5 Dec 2, 2022
Use rust programming language to create a b+ tree.

Use rust programming language to create a b+ tree.

yangshijie 3 Jan 7, 2023