Deep Learning Test

Implementing deep learning in Rust using just a linear algebra library (nalgebra). The neural network (4 hidden layers, 32 neurons per layer) attempts to reconstruct a 512x512 image. It takes a 2D position as input, and outputs an RGB value.

Results

Training time is around one minute, which is suprisingly quick considering that no parallelism or GPU acceleration is involved. Here is the reference image:

and here is the reconstruction from the neural network:

Note that I implemented Fourier features, as described here to improve reconstruction quality. Without this, here's what the reconstruction looks like:

Notes

• An optimizer is critical for deep learning. Otherwise, the earlier layers barely get trained at all. I used Adam Optimization, as described here.
• Remember to reserve one column in each weight matrix for bias. This modestly improves training time and reconstruction quality.