Rust implementation of @Qdrant/fastembed.

Overview

πŸ• Features

  • Supports synchronous usage. No dependency on Tokio.
  • Uses @huggingface/tokenizers for blazing-fast encodings.
  • Supports batch embedddings with parallelism using Rayon.

The default embedding supports "query" and "passage" prefixes for the input text. The default model is Flag Embedding, which is top of the MTEB leaderboard.

πŸ” Not looking for Rust?

πŸ€– Models

πŸš€ Installation

Run the following Cargo command in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

fastembed = "1"

πŸ“– Usage

use fastembed::{FlagEmbedding, InitOptions, EmbeddingModel, EmbeddingBase};

// With default InitOptions
let model: FlagEmbedding = FlagEmbedding::try_new(Default::default())?;

// With custom InitOptions
let model: FlagEmbedding = FlagEmbedding::try_new(InitOptions {
    model_name: EmbeddingModel::BGEBaseEN,
    show_download_message: true,
    ..Default::default()
})?;

let documents = vec![
    "passage: Hello, World!",
    "query: Hello, World!",
    "passage: This is an example passage.",
    // You can leave out the prefix but it's recommended
    "fastembed-rs is licensed under MIT"
    ];

 // Generate embeddings with the default batch size, 256
 let embeddings = model.embed(documents, None)?;

 println!("Embeddings length: {}", embeddings.len()); // -> Embeddings length: 4
 println!("Embedding dimension: {}", embeddings[0].len()); // -> Embedding dimension: 768

Supports passage and query embeddings for more accurate results

 // Generate embeddings for the passages
 // The texts are prefixed with "passage" for better results
 // The batch size is set to 1 for demonstration purposes
 let passages = vec![
     "This is the first passage. It contains provides more context for retrieval.",
     "Here's the second passage, which is longer than the first one. It includes additional information.",
     "And this is the third passage, the longest of all. It contains several sentences and is meant for more extensive testing."
    ];

 let embeddings = model.passage_embed(passages, Some(1))?;

 println!("Passage embeddings length: {}", embeddings.len()); // -> Embeddings length: 3
 println!("Passage embedding dimension: {}", embeddings[0].len()); // -> Passage embedding dimension: 768

 // Generate embeddings for the query
 // The text is prefixed with "query" for better retrieval
 let query = "What is the answer to this generic question?";

 let query_embedding = model.query_embed(query)?;

 println!("Query embedding dimension: {}", query_embedding.len()); // -> Query embedding dimension: 768

πŸš’ Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

  1. Quantized model weights
  2. ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Why light?

  1. No hidden dependencies via Huggingface Transformers

Why accurate?

  1. Better than OpenAI Ada-002
  2. Top of the Embedding leaderboards e.g. MTEB

πŸ“„ LICENSE

MIT Β© 2023

You might also like...
A naive DBSCAN implementation in Rust

DBSCAN Density-Based Spatial Clustering of Applications with Noise Wikipedia link DBSCAN is a density-based clustering algorithm: given a set of point

A random forest implementation in Rust

randomforest A random forest implementation in Rust. Examples use randomforest::criterion::Mse; use randomforest::RandomForestRegressorOptions; use ra

Rust implementation of multi-index hashing for neighbor searches on binary codes in the Hamming space

mih-rs Rust implementation of multi-index hashing (MIH) for neighbor searches on binary codes in the Hamming space, described in the paper Norouzi, Pu

kdtree implementation for rust.

kdtree-rust kdtree implementation for rust. Implementation uses sliding midpoint variation of the tree. More Info here Implementation uses single Vec

Rust implementation of user-based collaborative filtering

Rucommender Recommendation system written in Rust Overview An implementation in Rust of a collaborative filtering recommendations algorithm with a use

This repository features a simple Kalman filter and RTS smoother (KFS) implementation in Rust by using the ndarray library.
This repository features a simple Kalman filter and RTS smoother (KFS) implementation in Rust by using the ndarray library.

Kalman filter and RTS smoother in Rust (ndarray) This repository features a simple Kalman filter and RTS smoother (KFS) implementation in Rust by usin

Python+Rust implementation of the Probabilistic Principal Component Analysis model

Probabilistic Principal Component Analysis (PPCA) model This project implements a PPCA model implemented in Rust for Python using pyO3 and maturin. In

TopK algorithm implementation in Rust (Filtered Space-Saving)

TopK TopK algorithm implementation in Rust. This crate currently provides the Filtered Space-Saving algorithm. Version numbers follow the semver conve

Rust+OpenCL+AVX2 implementation of LLaMA inference code
Rust+OpenCL+AVX2 implementation of LLaMA inference code

RLLaMA RLLaMA is a pure Rust implementation of LLaMA large language model inference.. Supported features Uses either f16 and f32 weights. LLaMA-7B, LL

Comments
  • fix: MLE5 large url transform

    fix: MLE5 large url transform

    The MLE5Large model URL doesn't follow the same naming convention as the other models So, we tranform "fast-multilingual-e5-large" -> "intfloat-multilingual-e5-large" in the download URL The model directory name in the GCS storage is "fast-multilingual-e5-large", like the others

    released 
    opened by Anush008 1
  • Wrong onnxruntime.dll Picked Up When Running Integration Tests

    Wrong onnxruntime.dll Picked Up When Running Integration Tests

    Error

    ort 1.16 is not compatible with the ONNX Runtime binary found at `onnxruntime.dll`; expected GetVersionString to return '1.16.x', but got '1.10.0'
    

    How To Replicate

    For windows, fastembed does not include onnxruntime.dll in target/debug/deps (which is used by integration tests). Therefore if onnxruntime.dll exists in C:/Windows/System32/ then that version is used. Causing the issue above.

    To replicate, ensure a version of onnxruntime.dll exists in C:/Windows/System32/ that is not the same version as fastembed (currently 1.16.x). Adding a fastembed model to a library, like

    lazy_static! {
        pub static ref EMBEDDING_MODEL: FlagEmbedding = FlagEmbedding::try_new(InitOptions {
                model_name: EmbeddingModel::BGEBaseEN,
                show_download_message: true,
                ..Default::default()
            }).unwrap();
    }
    

    Then importing that module into an integration and attempting to use it that error will cause the error.

    How To fix by hand

    Add onnxruntime.dll to target/debug/deps

    Expected Behavior

    If fast fastembed runs correctly in a binary or library crate, it should also run correctly in their integration tests. (Note: I have not tested release mode or regular unit tests with fastembed yet.)

    Possible solution

    ort has a feature that could be used, but I have not tested. https://github.com/pykeio/ort/blob/4ab57859caa9490473bac3dfcd043dbb1b89d9a5/Cargo.toml#L44

    opened by mcmah309 2
Releases(v1.9.0)
Owner
Anush
Nocturnal back-end engineer.
Anush
Rust implementation of real-coded GA for solving optimization problems and training of neural networks

revonet Rust implementation of real-coded genetic algorithm for solving optimization problems and training of neural networks. The latter is also know

Yury Tsoy 19 Aug 11, 2022
Instance Distance is a fast pure-Rust implementation of the Hierarchical Navigable Small Worlds paper

Fast approximate nearest neighbor searching in Rust, based on HNSW index

Instant Domain Search, Inc. 135 Dec 24, 2022
A real-time implementation of "Ray Tracing in One Weekend" using nannou and rust-gpu.

Real-time Ray Tracing with nannou & rust-gpu An attempt at a real-time implementation of "Ray Tracing in One Weekend" by Peter Shirley. This was a per

null 89 Dec 23, 2022
A neural network, and tensor dynamic automatic differentiation implementation for Rust.

Corgi A neural network, and tensor dynamic automatic differentiation implementation for Rust. BLAS The BLAS feature can be enabled, and requires CBLAS

Patrick Song 20 Nov 7, 2022
Flexible, reusable reinforcement learning (Q learning) implementation in Rust

Rurel Rurel is a flexible, reusable reinforcement learning (Q learning) implementation in Rust. Release documentation In Cargo.toml: rurel = "0.2.0"

Milan Boers 60 Dec 29, 2022
A RustπŸ¦€ implementation of CRAFTML, an Efficient Clustering-based Random Forest for Extreme Multi-label Learning

craftml-rs A Rust implementation of CRAFTML, an Efficient Clustering-based Random Forest for Extreme Multi-label Learning (Siblini et al., 2018). Perf

Tom Dong 15 Nov 6, 2022
Barnes-Hut t-SNE implementation written in Rust.

bhtsne Barnes-Hut implementation of t-SNE written in Rust. The algorithm is described with fine detail in this paper by Laurens van der Maaten. Instal

Francesco Iannelli 48 Dec 27, 2022
An implementation of the Pair Adjacent Violators algorithm for isotonic regression in Rust

Pair Adjacent Violators for Rust Overview An implementation of the Pair Adjacent Violators algorithm for isotonic regression. Note this algorithm is a

Ian Clarke 3 Dec 25, 2021
Generic k-means implementation written in Rust

RKM - Rust k-means A simple Rust implementation of the k-means clustering algorithm based on a C++ implementation, dkm. This implementation is generic

Nick Sarten 8 Sep 25, 2021
Rust implementation for DBSCANSD, a trajectory clustering algorithm.

DBSCANSD Rust implementation for DBSCANSD, a trajectory clustering algorithm. Brief Introduction DBSCANSD (Density-Based Spatial Clustering of Applica

Nick Gu 2 Mar 14, 2021