Neural Networks by Hand

Forward passes, backprop, optimisers, convolutions and recurrent cells, the pieces inside every framework.

Intermediate · 24 problems · 0 passed · Background reading: Neural Network Fundamentals

1. 01Dense Layer Forward Passeasy
2. 02ReLU Backward Gradient Passeasy
3. 03Numerically Stable Softmaxeasy
4. 04Categorical Cross-Entropy Losseasy
5. 05Mean Squared Error Loss Gradienteasy
6. 06Kaiming / He Initialization Varianceeasy
7. 07Conv2D Output Spatial Dimension Calculatoreasy
8. 082D Max Pooling Forward Passeasy
9. 09Bounding Box Intersection over Union (IoU)easy
10. 10Dense Layer Backward Passmedium
11. 11Softmax Gradient (Vector-Jacobian Product)hard
12. 12SGD with Momentum Parameter Stepeasy
13. 13Adam Optimizer Step with Bias Correctionmedium
14. 14Inverted Dropout Forward Passeasy
15. 151D Batch Normalization Forward Passmedium
16. 16Global Gradient Norm Clippingmedium
17. 172D Convolution From Scratchmedium
18. 18Multi-Channel 2D Convolutionmedium
19. 192D Max Pooling Backward Pass Gradient Routermedium
20. 20ResNet Basic Residual Block Forward Passmedium
21. 21Vanilla Elman RNN Step Forwardhard
22. 22LSTM Cell Forward Passhard
23. 23Conv2D Backward Pass Gradient w.r.t. Kernel Weightshard
24. 24Two-Layer MLP Training Stephard

Next track: Inside an LLM