Rust implementation of Andrej Karpathy's micrograd for purposes of learning both ML and Rust.

Overview

micrograd_rs

Rust implementation of Andrej Karpathy's micrograd for purposes of learning both ML and Rust.

Main takeaways

Basically the same takeaways that everybody goes through it seems.

  • ML is not easy, but nothing magical eigher :)
  • Writing the same thing in Rust takes multiple times the effort than in Python (at least at the learning stage).
  • Analyzing Rust code is multiple times easier than Python because of the strict static typing, and explicitness. I'm not talking about the general understanding of what the code does, but understanding what's really going on.
  • Rust compiler error messages and hints are enough to understand the problem in most situations.
  • Rust is much faster than Python (which isn't surprising, so is C), and it is an enormous advantage for ML, as training on huge datasets can take a lot of time. For example, on my machine the Python micrograd demo optimization loop took around 82 seconds, while Rust implementation only around 1.5 seconds in release mode šŸ¤Æ And around 20 seconds in debug, so don't forget to build your crates with --release, kids.

Demo

See the provided .ipynb notebooks for petgraph visualizations, and example training with usage of TanH and ReLU functions.

The TanH example is from Karpathy's "Neural Networks: Zero to Hero" notes, and the ReLU example is from micrograd demo notebook.

In order to run the notebooks locally, add Rust Jupyter Kernel with evcxr_jupyter.

rustup component add rust-src
cargo install evcxr_jupyter
~/.cargo/bin/evcxr_jupyter --install
jupyter notebook

Value operations

Operations with both owned and borrowed Value are supported, as well as with f64. The pow, exp, tanh, and relu functions are also supported.

use micrograd_rs::engine::Value;

let a = Value::new(-4.0);
let b = Value::new(2.0);
let mut c = &a + &b;
let mut d = &(&a * &b) + &b.pow(3.0);
c += &c + 1.0;
c += 1.0 + &c + (-&a);
d += &d * 2.0 + (&b + &a).relu();
d += 3.0 * &d + (&b - &a).relu();
let e = c - d;
let f = e.pow(2.0);
let mut g = &f / 2.0;
g += 10.0 / f;
g.backward();

println!("{:.4}", g.get_data());  // 24.7041, the outcome forward pass.
println!("{:.4}", a.get_grad());  // 138.8338, the numerical value of dg/da.
println!("{:.4}", b.get_grad());  // 645.5773, the numerical value of dg/db.
You might also like...
fast rust implementation of online nonnegative matrix factorization as laid out in the paper "detect and track latent factors with online nonnegative matrix factorization"

ONMF status: early work in progress. still figuring this out. code still somewhat messy. api still in flux. fast rust implementation of online nonnega

A fast lean and clean modern constraint programming solver implementation (in rust)

MaxiCP-rs This project aims at implementing a fast, and clean constraint programming solver with a focus on correctness, simplicity, maintainability a

An implementation of a predicative polymorphic language with bidirectional type inference and algebraic data types

Vinilla Lang Vanilla is a pure functional programming language based on System F, a classic but powerful type system. Merits Simple as it is, Vanilla

A radix tree implementation for router, and provides CRUD operations.

radixtree A radix tree implementation for router, and provides CRUD operations. Radixtree is part of treemux, on top of which updates and removes are

Majestic Lisp book and implementation, in Brazillian Portuguese.

Majestic Lisp Criado e desenvolvido por Lucas S. Vieira lucasvieira at protonmail dot com. Seja bem-vindo(a) a Majestic Lisp, um dialeto de Lisp cuj

Ray Tracing: The Next Week implementation in Rust
Ray Tracing: The Next Week implementation in Rust

rttnw Ray Tracing: The Next Week implementation in Rust How to run Install Rust: Link here. Run project git clone https://github.com/luliic2/rttnw cd

Rust implementation of ĀµKanren, a featherweight relational programming language.

ĀµKanren-rs This is a Rust implementation of ĀµKanren, a featherweight relational programming language. See the original Scheme implementation here for

Pure Rust Implementation of secp256k1.

SECP256K1 implementation in pure Rust Cargo Documentation SECP256K1 implementation with no_std support. Currently we have implementation for: Convert

A Rust implementation of generic prefix tree (trie) map with wildcard capture support

prefix_tree_map A Rust implementation of generic prefix tree (trie) map with wildcard capture support. Design Trie is a good data structure for storin

Owner
null
The source code that accompanies Hands-on Rust: Effective Learning through 2D Game Development and Play by Herbert Wolverson

Hands-on Rust Source Code This repository contains the source code for the examples found in Hands-on Rust. These are also available from my publisher

Herbert 259 Dec 4, 2022
Rust language from simple to deep, and then to strengthen learning in multiple module

Rust Library This project is used to learn rust language from simple to deep, and then to strengthen learning in multiple module. It is used to help n

Summer 1 Sep 19, 2022
A bunch of links to blog posts, articles, videos, etc for learning Rust

rust-learning A bunch of links to blog posts, articles, videos, etc for learning Rust. Feel free to submit a pull request if you have some links/resou

Camille TJHOA 8.8k Nov 29, 2022
My journey of learning rust as a web developer.

My journey of learning rust Low-level languages seem very interesting to me. I always wanted to learn a low-level language but never had the chance. T

null 1 Oct 21, 2021
A repo for learning rust

learnrust A repo for learning rust Topics: print basic print function. datatypes get to know what is datatypes. variables variables!!!. consts consts

Krisna Pranav 1 Dec 19, 2021
Rust Language Learning material

RustMaterial Rust Language Learning material Rust Rust is blazingly fast systems programming language that prevents segfaults and guarantees thread sa

Udhay Prakash Pethakamsetty 1 Jan 6, 2022
A learning project/fun experiment in internet protocol

Piper a learning project/fun experiment in internet protocol Version 0.4.0 (SEMVER) Goals Piper is Simple. A page is a page. There are no secondary re

null 13 Oct 27, 2022
Bril: A Compiler Intermediate Representation for Learning

Bril: A Compiler Intermediate Representation for Learning Bril (the Big Red Intermediate Language) is a compiler IR made for teaching CS 6120, a grad

Lesley Lai 1 Nov 12, 2021
An embedded key-value storage for learning purpose, which is based on the idea of SSTable / LSM-tree.

Nouzdb An embedded key-value storage for learning purpose, which is based on the idea of SSTable / LSM-tree. Plan Implement a memtable. Implement the

Nouzan 1 Dec 5, 2021
An implementation of Olm and Megolm in pure Rust.

A Rust implementation of Olm and Megolm vodozemac is a Rust implementation of libolm, a cryptographic library used for end-to-end encryption in Matrix

matrix.org 60 Nov 18, 2022