A collection of serverless apps that show how Fermyon's Serverless AI

Related tags

Miscellaneous ai serverless llama spin fermyon
Overview

Fermyon Serverless AI Examples

This repository contains examples of using Fermyon Serverless AI.

Prerequisites

The following prerequisites are needed to build and run these examples:

  • Spin version 1.5.0 or newer. You can find the install and update instructions here.
  • The following plugins:
    • js2wasm 0.6.1 spin plugins install js2wasm.
    • py2wasm 0.3.2 spin plugins install py2wasm.
  • Optional, but highly recommended, is to use the Spin Cloud GPU component. This offloads inferencing to Fermyon Cloud GPUs, and thus requires a free account to Fermyon Cloud Serverless AI.
  • If you do not want to use the Cloud GPU component, the instructions on how to download the required models for inferencing and embedding can be found here.

Note that Fermyon Cloud support is currently in private beta. To apply for access to the private beta, please fill out this survey.

Examples overview

Here is a table of the following examples:

Example Name Spin SDK Inferencing Embedding Vector DBs
blog-recommendation-ts TypeScript ✅ ✅
code-generator-rs Rust ✅
haiku-generator-rs Rust ✅
haiku-generator-ts TypeScript ✅
newsfeeder-ts Typescript ✅
openapi-rs Rust ✅
sentiment-analysis-py Python ✅
sentiment-analysis-rs Rust ✅
sentiment-analysis-ts Typescript ✅
silly-walk-ts Typescript ✅

To get started building your own applications, follow these instructions to make sure you have installed the correct version of Spin and the necessary SDKs. If you'd like to learn more about the API, visit the API guide here.

Comments
  • Parse sentiment and show if we're unsure

    Parse sentiment and show if we're unsure

    This PR changes how we do sentiment analysis:

    • Changes the prompt to something the worked better for me in my experiments (using the llama <> and tags.
    • Parse the sentiment analysis so we can have more fine grained control on the logic
    • Have the front-end show "unsure" if the backend did not return one of the three possible results
    • Only ask for 6 tokens back to avoid getting longer responses than we'll ever use
    • Only save to the cache if the parsing of the sentiment was successful
    • Trim the input
    opened by rylev 1
  • Update readmes to consolidate prerequisites.

    Update readmes to consolidate prerequisites.

    • I've consolidated the prerequisites in the root README and referenced from all the READMEs in the individual folders.
    • Added recommendation to use Cloud GPU
    • Updated overview table
    • One small code change (see if you can spot it!!!)
    opened by mikkelhegn 0
  • Adding an OpenAPI component example

    Adding an OpenAPI component example

    Hi! 👋 Just wanted to add my example of how to create a wrapper around the LLM SDK into a re-usable component. Eventually this will get it's own repository similar to other re-usable spin components like the static fileserver.

    opened by jpflueger 0
  • Improve the Rust sentiment analysis example

    Improve the Rust sentiment analysis example

    Makes a few improvements to the Rust sentiment analysis example

    • Use [INST] for the prompt instead of the incorrect <INST>.
    • cargo fmt
    • Remove unnecessary second call to fetch cached sentiment
    • Move json formatting into helper function
    opened by rylev 0
  • Add example for a code generator

    Add example for a code generator

    This is the start of a simple example that uses code-llama for generating code for a given prompt in either python or bash. I'm not sure the prompt is ideal, but maybe it's good enough?

    opened by rylev 0
  • Added two TypeScript examples

    Added two TypeScript examples

    This adds two examples:

    • Silly walk is a Monty Python inspired example that is VERY VERY simple
    • newsfeeder shows how to build a prompt from an RSS feed and then run an inference
    opened by technosophos 0
  • Provide an Open AI API-compatible API component :-)

    Provide an Open AI API-compatible API component :-)

    I love this sample here 👏🏼: https://github.com/fermyon/ai-examples/tree/main/openapi-rs

    Wouldn't it be even greater if we had an Open AI Chat completions API-compatible API component? Then, we could use any tools out there that support Open AI.

    .cc @jpflueger

    opened by ChristianWeyer 2
Owner
Fermyon
Pioneering the next wave of cloud computing.
Fermyon
GitHub https://developer.fermyon.com/cloud/serverless-ai
Displays the Show/Movie you're currently watching on Discord!

Jellyfin-RPC Program used to display what you're currently watching on discord. Jellyfin-RPC uses the API to check what you're currently watching, thi

Radical 3 Jan 8, 2023
Keep distractive apps at bay allowing you to focus!

Focus Keep distractive apps at bay allowing you to focus! What it does. The apps specified are killed automatically after a period of 5 seconds till t

null 3 May 11, 2022
Discord Rich Presence for Creative Apps

Show your friends what you're working on, be it in Adobe Suite, Autodesk Suite, Cinema 4D or many more! Currently supported: C4D, Adobe Suite, Davinci Resolve, Maya, 3Ds Max, Sony Vegas, Substance suite, Isotropix suite, FL, Ableton, Blender, Cavalry,

null 6 Nov 30, 2022
A pure-Rust serverless discord chatbot hosted on Cloudflare Workers.

About A pure-Rust serverless discord chatbot hosted on Cloudflare Workers. With a free account you have up to 100k requests per day. For storing state

Mike Dallas 31 Nov 21, 2022
A collection of exponentially-smoothed camera controllers for the Bevy Engine.

smooth-bevy-cameras A collection of exponentially-smoothed camera controllers for the Bevy Engine. Look Transform All controllers are based on a LookT

Duncan 122 Dec 24, 2022
Collection of "surprising behaviours" in Rust. In the same spirit of wtfpython and wtfjs.

wtfrust Collection of "surprising behaviours" in Rust. In the same spirit of wtfpython and wtfjs. This document avoids common and general problems in

张实唯 3 Oct 29, 2021
A collection of compilers based around compiling a high level language to a Brainfuck dialect.

tf A collection of compilers based around compiling a high level language to a Brainfuck dialect. Built at, and for, the VolHacks V hackathon during O

adam mcdaniel 6 Nov 25, 2021
Blueboat is an open-source alternative to Cloudflare Workers. The monolithic engine for serverless web apps.

Blueboat Blueboat is an open-source alternative to Cloudflare Workers. Blueboat aims to be a developer-friendly, multi-tenant platform for serverless

Heyang Zhou 1.8k Jan 9, 2023
The most efficient, scalable, and fast production-ready serverless REST API backend which provides CRUD operations for a MongoDB collection

Optimal CRUD Mongo Goals of This Project This is meant to be the most efficient, scalable, and fast production-ready serverless REST API backend which

Evaluates2 1 Feb 22, 2022
The Rust Compiler Collection is a collection of compilers for various languages, written with The Rust Programming Language.

rcc The Rust Compiler Collection is a collection of compilers for various languages, written with The Rust Programming Language. Compilers Language Co

null 2 Jan 17, 2022
Subcommand to show result of macro expansion

cargo-expand Once installed, the following command prints out the result of macro expansion and #[derive] expansion applied to the current crate. $ ca

David Tolnay 1.8k Jan 4, 2023
Slideo: This tool uses OpenCV to automatically synchronize slides with videos that show these slides.

This tool matches video frames against PDF pages by using computer vision. It also ships a web app in which you can click on a PDF page to play the video from the first frame showing the page. Its primary use-case is to quickly play a recorded lecture from a given slide.

Henning Dieterichs 78 Oct 7, 2022
Show puffin profiler flamegraph in-game using egui

Show puffin profiler flamegraph in-game using egui puffin is an instrumentation profiler where you opt-in to profile parts of your code: fn my_functio

Emil Ernerfeldt 44 Jun 3, 2022
belt is a command line app that can show your time from a list of selected time zones

A CLI app to show your time from a list of selected time zones, and a rust lib to parse dates in string formats that are commonly used.

Rollie Ma 23 Nov 4, 2022
A simple program to show a histogram on the terminal.

A simple program to show a histogram on the terminal.

依云 4 Aug 10, 2021
A third-party iTunes addon to show the lyrics on the desktop.

iTunes Desktop Lyrics for Windows A third-party iTunes addon to show the lyrics on the desktop. Preview Prerequisites iTunes Usage Download the binary

Jiahao Lu 86 Dec 20, 2022
A third-party iTunes addon to show the lyrics on the desktop.

iLyrics A third-party iTunes addon to show the lyrics on the desktop. Preview Prerequisites iTunes Usage Download the binary or build from source. Run

Jiahao Lu 86 Dec 20, 2022
GitHub CLI extension to show & rename the default branch.

gh-default-branch GitHub CLI extension to show & rename the default branch. rename subcommand was inspired by this gist. ⚠️ Caution The rename subcomm

Daido Shota 8 Sep 22, 2022
Show unused code from multi-crate Rust projects

Warnalyzer Remove unused code from multi-crate Rust projects. The dead_code lint family of rustc is limited to one crate only and thus can't tell whet

null 75 Dec 27, 2022
A library of to show data (in browser, evcxr_jupyter) as table, chart...

showata A library of to show data (in browser, evcxr_jupyter) as table, chart.... The crate provides display for: image vector and slice (as table) nd

Procyon 20 Dec 12, 2022