Neural Mastery
← Practice
Batch Gradient Descent for Linear Regressionmedium

Batch Gradient Descent for Linear Regression

medium Classical ML from Scratch 15–20 min

NumPy allowed · plain Python earns 10 bonus points

Gradient descent step 1/4

Loss moves downhill

The update rule changes the parameters on each step.
Open lesson →
01parameters
02prediction
03loss
04gradient
θ = 4.000loss = 16.000∂L/∂θ = 8.000

θ ← θ − 0.14 · ∂L/∂θ

Optimize linear regression weights using batch gradient descent.

Related Skills

Task

Implement batch_gradient_descent_linear(X, y, lr, epochs): start with zero weights, iteratively compute predictions X*w, errors pred - y, and gradient (2/n) * X^T * err, updating weights each epoch.

Function Signature

batch_gradient_descent_linear(X: list[list[float]], y: list[float], lr: float, epochs: int) -> list[float]

Examples

Example 1: 1D Simple Trend y = 2x
Input: X = [[1], [2], [3]] y = [2, 4, 6] lr = 0.1 epochs = 50
Output: [2]
Example 2: 2D Features
Input: X = [[1, 0], [0, 1], [1, 1]] y = [1, 2, 3] lr = 0.1 epochs = 50
Output: [1.0159, 1.9841]

Constraints

  • 1 <= len(X) <= 1000
  • 1 <= len(X[0]) <= 20
  • 0.0001 <= lr <= 0.5
  • 1 <= epochs <= 1000
Python3Saved ✓
def batch_gradient_descent_linear(X: list[list[float]], y: list[float], lr: float, epochs: int) -> list[float]:
"""X: feature matrix (n x d).
y: targets of length n.
lr: learning rate.
epochs: number of gradient descent iterations.
Weights w are initialized to all 0.0 (length d, no explicit bias).
MSE loss: (1/n) * sum((X*w - y)^2). Gradient: (2/n) * X^T * (X*w - y).
Update weights w = w - lr * grad.
Return final weights rounded to 4 decimal places."""
# Your implementation here
pass

Case 1: 1D Simple Trend y = 2x
Input: X = [[1], [2], [3]] y = [2, 4, 6] lr = 0.1 epochs = 50
Expected: [2]
Case 2: 2D Features
Input: X = [[1, 0], [0, 1], [1, 1]] y = [1, 2, 3] lr = 0.1 epochs = 50
Expected: [1.0159, 1.9841]