Tensorken: A Fun, Hackable, GPU-Accelerated, Neural Network library in Rust, Written by an Idiot

(work in progress)

Understanding deep learning from the perspective of a programmer, by building a deep learning framework from the ground up, in the spirit of tinygrad and micrograd.

• Fun and hackable: most importantly Tensorken doesn't take itself too seriously. It's meant to be small, hackable, easy to understand and change above all else. If you want something usable for real work, look elsewhere.
• GPU-Accelerated: For the moment Tensorken runs on the GPU via wgpu (Rust's implementation of WebGPU), but the "accelerated" comes with a large grain of salt.
• Neural network: Very much aspirational at the moment. There are just basic tensor operations that run on CPU and GPU. I have a prototype autodiff implementation on top, but have not added it to this repo yet.
• Rust: No particular reason other than that I'm learning Rust.
• Written by an idiot: Hi there! I know nothing about neural network or GPU programming. As a result, anything and everything in here may be slow, backward, wrong, or stupid, and that's not an exclusive or exhaustive list.

"Tensorken"?

The suffix -ke means "small" in Flemish. So tensorke is a small tensor. Some Flemish dialects append an extra n, which sounds better in English.

The shoulders of giants

Some cool stuff I looked at while figuring out how to build this.

• micrograd
• tinygrad
• MiniTorch
• JAX
• DiffSharp

Getting started

Just clone the repo and run cargo run --example tour. Then explore the code - it's not big!

Emerging posts with more explanation:

• Fun and Hackable Tensors in Rust, From Scratch