Neural Mastery
← Practice
Linear Regression Gradient Descent Stepeasy

Linear Regression Gradient Descent Step

easy Classical ML from Scratch 10–15 min

NumPy allowed · plain Python earns 10 bonus points

Gradient descent step 1/4

Loss moves downhill

The update rule changes the parameters on each step.
Open lesson →
01parameters
02prediction
03loss
04gradient
θ = 4.000loss = 16.000∂L/∂θ = 8.000

θ ← θ − 0.14 · ∂L/∂θ

Execute an explicit analytical gradient descent parameter update on weights and intercept for linear regression with Mean Squared Error loss.

Task

Implement linear_regression_gd_step(X, y, weights, bias, lr): compute predictions y_hat_i = w^T x_i + b, gradients grad_w = (1/N) X^T (y_hat - y) and grad_b = (1/N) sum(y_hat - y). Return [*new_w, new_b] where param_new = param - lr * grad.

Function Signature

linear_regression_gd_step(X: list[list[float]], y: list[float], weights: list[float], bias: float, lr: float) -> list[float]

Examples

Example 1: 1D Single Feature Update
Input: X = [[1], [2], [3]] y = [2, 4, 6] weights = [0] bias = 0 lr = 0.1
Output: [0.933333, 0.4]
Example 2: Zero Residual Exact Fit
Input: X = [[1, 0], [0, 1]] y = [1, 2] weights = [1, 2] bias = 0 lr = 0.05
Output: [1, 2, 0]

Constraints

  • Libraries allowed · Pure Python earns +10 bonus XP
  • len(X) == len(y) >= 1; len(weights) == len(X[0]).
  • Learning rate lr > 0.
Python3Saved ✓
def linear_regression_gd_step(X: list[list[float]], y: list[float], weights: list[float], bias: float, lr: float) -> list[float]:
"""Perform one gradient descent update step on weights and bias for MSE loss.

grad_w = (1/N) * sum_i (y_pred_i - y_i) * X[i]
grad_b = (1/N) * sum_i (y_pred_i - y_i)
Return [*new_weights, new_bias]."""
# Your implementation here
pass

Case 1: 1D Single Feature Update
Input: X = [[1], [2], [3]] y = [2, 4, 6] weights = [0] bias = 0 lr = 0.1
Expected: [0.933333, 0.4]
Case 2: Zero Residual Exact Fit
Input: X = [[1, 0], [0, 1]] y = [1, 2] weights = [1, 2] bias = 0 lr = 0.05
Expected: [1, 2, 0]