Realtime audio processing / synthesis using Rust/WASM in the browser.

Overview

Rust Audio

About

This repo is my investigation into using Rust for creative audio coding on various platforms (e.g. desktop, web, etc.), but especially for targeting the web.

Granular Synthesizer Demo

Granular Synthesizer Demo

Todos:

  • Update licenses / cargo tomls, directory names

  • Show hoverable tooltips for icon buttons

  • Enable sampling from multi-channel audio input

  • split granular synth into its own repo / separate website entry before adding more features?

  • Show recording buffer visualization (or at least some sort of indication recording is taking place)

  • Interpolate changes in Gain

  • Name downloaded file something more like: "name_of_audio_granulated.wav" or similar

  • Optimizations:

    • Add finished grains to a hashmap when finished to prevent the need for searching for spent grains on every frame
    • Improve which rand function we're using for better efficiency
  • Use newtype-style units to disambiguate calculations

  • Move audio PROCESSING into a Web Worker thread so that cpal merely has to request audio data at the appropriate time

    • Keeps main thread / audio processing from getting locked up by UI updates and vice versa
    • Keeps WebAudio context on the main thread (where it has to be)
  • Share a single audio context that is initialized (?) at init time?

  • Memoize decoded audio from previous files? To prevent stutter on change?

  • Enable dragging the current buffer selection window?

  • Web

    • Refactor visual representation of current audio buffer:
      • use an svg element?
      • probably would be best to use a canvas to do this
  • Common

    • Clean up any unused files in /common
    • Move audio handles in there?
  • CLI

    • Re-sample any audio files that don't match the current sample rate.
  • Native

    • Make a Native app using Tauri that relies on current web view with Rust running natively under the hood?
    • Use serde_wasm_bindgen instead of message passing between backend and front end to prevent JSON-ifying buffers?
  • More audio tools / effects:

    • Recording
    • Reverb
    • Delay

Visual effects: - WebGL: particles that react / correspond to audio grains - Show audio output as a sample window? - Or just show current amplitude output with simple bars


General Fixes:

  • Correct sample rate for mp3 audio for all environments.
You might also like...
Rust bindings for the Wasm spec interpreter.

wasm-spec-interpreter This project shows how to use ocaml-interop to call into the Wasm spec interpreter. There are several steps to making this work:

Spine runtime for Rust (and wasm!) transpiled from the official C Runtime.

rusty_spine Spine runtime for Rust (and wasm!) transpiled from the official C Runtime. Supports Spine 4.1. [dependencies] rusty_spine = "0.4.0" Onlin

Simple devcontainer for Rust + WASM development

Devcontainer WASM-Rust Simple devcontainer for Rust development Usage Github Codespaces Just click the button: Visual Studio Code Note this assumes th

Create WASM plugins easily in Rust.

Scotch Library for creating WASM plugins with Rust. Scotch allows you to pass complex types to/from functions in WASM plugins. It achieves that by enc

Fast regex in Rust for Apache Arrow, compiled to WASM

Rust regex in wasm I have been looking for a fast regular expression library in Javascript that runs on Apache Arrow for a few years. Arrow uses UTF-8

Facilitating high-level interactions between Wasm modules and JavaScript

wasm-bindgen Facilitating high-level interactions between Wasm modules and JavaScript. Guide | API Docs | Contributing | Chat Built with 🦀 🕸 by The

Install `wasm-pack` by downloading the executable

wasm-pack-action Install wasm-pack by downloading the executable (much faster than cargo install wasm-pack, seconds vs minutes). Usage - uses: jetli/w

`wasm-snip` replaces a WebAssembly function's body with an `unreachable`

wasm-snip wasm-snip replaces a Wasm function's body with an unreachable instruction. API Docs | Contributing | Chat Built with 🦀 🕸 by The Rust and W

gc-sections for wasm

wasm-gc Note: you probably don't need to use this project. This project is no longer necessary to run by hand, nor do you need the wasm-gc executable

Comments
  • VST Plug-in Version?

    VST Plug-in Version?

    Hi there, I love the granular synth demo you built. Do you intend to create a VST version of it? If not, what's the license under which the code is licensed? I would be interested in creating a fork to continue the development of the plugin as a VST3 / Clap plugin (MIT Licensed, open source).

    opened by AlexW00 2
Owner
Austin Theriot
GitLab making it look like I don't code anymore.
Austin Theriot
Parametric surfaces drawn using the Rust + WASM toolchain with WebGL, React, and TypeScript.

Parametric Surfaces in the Browser My.Movie.3.mp4 Wanted to experiment with WebGL using the Rust + WASM toolchain, with React and TypeScript to glue e

Benji Nguyen 45 Oct 21, 2022
Low level tooling for WebAssembly in JavaScript using wasm-tools

js-wasm-tools js-wasm-tools compiles some of the API of wasm-tools to JavaScript and WebAssembly via wasm-bindgen. This offers low level tooling for W

Dominic Elm 59 Dec 19, 2022
RDF playground using WASM-compiled Sophia

SoWasm: an RDF playground based on Sophia This started as an experiment of compiling Sophia into WebAssembly, and grew into a (hopefully) useful playg

Pierre-Antoine Champin 4 Dec 19, 2023
Build frontend browser apps with Rust + WebAssembly. Supports server side rendering.

Percy Build frontend browser apps with Rust + WebAssembly. Supports server side rendering. The Percy Book This README gives a light introduction to Pe

Chinedu Francis Nwafili 2.1k Jan 1, 2023
📦✨ your favorite rust -> wasm workflow tool!

?? ✨ wasm-pack Your favorite Rust → Wasm workflow tool! Docs | Contributing | Chat Built with ?? ?? by The Rust and WebAssembly Working Group About Th

Rust and WebAssembly 4.8k Jan 5, 2023
Gun port in rust & wasm

gun-rs-wasm Rust & WASM port of Gun. For a non-wasm version, check out gun-rs Example (source) Use npm install rusty-gun import { Node as Gun } from "

Martti Malmi 39 Dec 19, 2022
📝 A template for creating WASM + Typescript + Rust workflow libraries.

Create Rust + TypeScript libraries with ease! PR'S WELCOMED! ✨ Inspiration I wanted to create a WebAssembly/Rust library with additional JS features,

Shaoru Ian Huang 25 Dec 24, 2022
Rust implementation of the Mina protocol, targeting Wasm and ARM architectures.

Mina-rs An implementation of Mina protocol in Rust, with focus on web and Wasm compatibility ** As you can probably tell this is a WIP! Don't use for

ChainSafe 157 Dec 12, 2022
A self-guided learning project that includes Rust + Wasm together

A self-guided learning project that includes Rust + Wasm together. Who knows, maybe Typescript and React joins too..

M.Yavuz Yagis 1 Feb 14, 2022
WASM bindings for React - enables you to write and use React components in Rust

This library enables you to write and use React components in Rust, which then can be exported to JS to be reused or rendered.

Yichuan Shen 55 Dec 24, 2022