ML Coding
Where "I understand how gradient descent works" gets tested by actually asking you to implement it, from scratch, without a framework doing the hard part for you.
What Gets Asked
- Implement a classical algorithm from scratch: k-means, k-NN, logistic regression, a decision tree split criterion — using only basic array operations, no scikit-learn. Forces you to actually understand the algorithm's mechanics (see Supervised Learning and Unsupervised Learning), not just how to call
.fit(). - Implement a neural network's forward/backward pass manually: no PyTorch autograd — write out the matrix multiplications and apply the chain rule by hand (see Calculus & Optimization and Neural Network Fundamentals). This is the single most common "prove you actually understand backprop" exercise -- the exact real computation to reproduce:
x = 1.200
h = w1·x + b1
a = σ(h)
y = w2·a + b2
L = ½(y−target)²
dL/dy
dL/da = dL/dy · w2
dL/dh = dL/da · σ'(h)
dL/dw1 = dL/dh · x
dL/dw2 = dL/dy · a
forward 1
This is why it's called BACKWARD propagation -- the forward pass computes left to right, then the chain rule walks right to left, reusing each already-computed local gradient rather than recomputing from scratch.
Forward: x = 1.2000 (w1=0.8, b1=-0.2, w2=1.5, b2=0.1, x=1.2, target=1)
- Implement attention from scratch: given Q, K, V matrices, compute scaled dot-product attention manually (see Attention & Transformers) — increasingly common given how central Transformers are to the field now:
Q: what am I looking for?
K: what do I contain, for others to find?
V: what do I actually offer if selected?
One embedding, three independent learned projections. Hover or tap a branch to see its formula.
How to Prepare
- Work through each implementation without looking at a reference solution first — struggling through the matrix shapes yourself is where the actual learning happens.
- Pay close attention to tensor/matrix shapes at every step — shape mismatches are the most common bug in this kind of exercise, and correctly reasoning about shapes signals real understanding of the underlying linear algebra.
- Practice explaining why each step is there, not just reproducing the code from memory — interviewers frequently ask "why did you do X" mid-exercise.
Next: ML/GenAI Knowledge Q&A — the rapid-fire conceptual questions that test breadth.