Neural Network from Scratch

Neural network implementations from scratch in Rust.

Setup & Run

Dataset used is mnist. Download the 4 archives and extract them into "datasets/mnist" folder.

A cargo run will setup the network, train it on a subset of the data while testing the result after each epoch infinitely until the target accuracy is reached. Currently, params resulting after training is not cached. Also, for now random seeds are used to produce reproducable and consistent results. Accuracy reached with the current params is >95%.

Only substantial external crate used is nalgebra for simple matrix operations.

This is running on the CPU right now, so it's not very fast. Ideally would want to make use of GPU computation.

Current implementations

• MLP: A multi-layer perceptron neural network.

Usage

Following are simple usages for the current implementations to play with.

MLP:

let train_dataset = ...;
let test_dataset = ...;

// Create
let mut network = mlp::Network</* input neurons */ 100, /* output neurons */ 10>::new(/* hidden layers */ &[60, 40]);

// Train
network.train(&train_dataset);

// Test
for sample in test_dataset {
    let result = network.run(&sample);
}

Ideas

Upcoming implementations:

• CNN
• RNN (LSTM)
• Transformer

Improvements

• More configuration (choose activation, additional layers such as dropout, etc)
• Serialize/Deserialize params to save training result
• Split lib crates
• GPU