Neural Mastery
← Practice
Classical Momentum Gradient Descent Stepmedium

Classical Momentum Gradient Descent Step

medium Math for ML 10-15 min

NumPy allowed · plain Python earns 10 bonus points

Gradient descent step 1/4

Loss moves downhill

The update rule changes the parameters on each step.
Open lesson →
01parameters
02prediction
03loss
04gradient
θ = 4.000loss = 16.000∂L/∂θ = 8.000

θ ← θ − 0.14 · ∂L/∂θ

Implement a single parameter update under Polyak momentum acceleration.

Task

Implement momentum_gd_step(w, grad, v, lr=0.01, beta=0.9): compute v_next = beta * v + lr * grad and w_next = w - v_next. Return {"w": list[float], "v": list[float]} rounded to 4 decimals.

Function Signature

momentum_gd_step(w: list[float], grad: list[float], v: list[float], lr: float = 0.01, beta: float = 0.9) -> dict[str, list[float]]

Examples

Example 1: Zero Initial Velocity
Input: w = [1, 2] grad = [0.5, -0.5] v = [0, 0] lr = 0.1 beta = 0.9
Output: {'w': [0.95, 2.05], 'v': [0.05, -0.05]}
Explanation: First step with zero momentum.
Example 2: Continuing Momentum
Input: w = [0.95, 2.05] grad = [0.4, -0.4] v = [0.05, -0.05] lr = 0.1 beta = 0.9
Output: {'w': [0.865, 2.135], 'v': [0.085, -0.085]}
Explanation: Second step accumulating momentum.

Constraints

  • w, grad, v have matching length d >= 1.
  • 0 <= beta < 1.0, lr > 0.
Python3Saved ✓
def momentum_gd_step(w: list[float], grad: list[float], v: list[float], lr: float = 0.01, beta: float = 0.9) -> dict[str, list[float]]:
"""Compute single step of gradient descent with classical Polyak momentum.
v_next = beta * v + lr * grad; w_next = w - v_next. Return {'w': list, 'v': list}."""
# Your implementation here
pass

Case 1: Zero Initial Velocity
Input: w = [1, 2] grad = [0.5, -0.5] v = [0, 0] lr = 0.1 beta = 0.9
Expected: {'w': [0.95, 2.05], 'v': [0.05, -0.05]}
Case 2: Continuing Momentum
Input: w = [0.95, 2.05] grad = [0.4, -0.4] v = [0.05, -0.05] lr = 0.1 beta = 0.9
Expected: {'w': [0.865, 2.135], 'v': [0.085, -0.085]}