Neural Mastery
← Practice
Coordinate Descent for Lasso Regressionhard

Coordinate Descent for Lasso Regression

hard Classical ML from Scratch 25–30 min

NumPy allowed · plain Python earns 10 bonus points

Gradient descent step 1/4

Loss moves downhill

The update rule changes the parameters on each step.
Open lesson →
01parameters
02prediction
03loss
04gradient
θ = 4.000loss = 16.000∂L/∂θ = 8.000

θ ← θ − 0.14 · ∂L/∂θ

Fit an L1-regularized sparse linear model using cyclical coordinate descent with soft thresholding.

Related Skills

Task

Implement lasso_coordinate_descent(X, y, alpha, max_iter): iteratively update each coordinate w_j using soft thresholding on partial residuals, inducing feature sparsity.

Function Signature

lasso_coordinate_descent(X: list[list[float]], y: list[float], alpha: float, max_iter: int) -> list[float]

Examples

Example 1: One Informative and One Noise Feature
Input: X = [[1, 0.1], [2, -0.1], [3, 0.2], [4, -0.2]] y = [2, 4, 6, 8] alpha = 1 max_iter = 50
Output: [1.9667, 0]
Example 2: High Penalty Setting All to Zero
Input: X = [[1], [2]] y = [1, 1] alpha = 100 max_iter = 10
Output: [0]

Constraints

  • 1 <= len(X) <= 500
  • 1 <= len(X[0]) <= 20
  • alpha > 0.0
  • 1 <= max_iter <= 500
Python3Saved ✓
def lasso_coordinate_descent(X: list[list[float]], y: list[float], alpha: float, max_iter: int) -> list[float]:
"""X: design matrix (n x p).
y: target vector of length n.
alpha: L1 regularization penalty parameter.
max_iter: number of full coordinate descent sweeps.
Objective: 0.5 * ||y - X*w||_2^2 + alpha * ||w||_1.
For each feature j, compute rho_j = sum_i(X[i][j] * (y[i] - sum_{k != j} X[i][k]*w[k])).
Update w[j] via soft-thresholding S(rho_j, alpha) / sum_i(X[i][j]^2).
Return weights w rounded to 4 decimal places."""
# Your implementation here
pass

Case 1: One Informative and One Noise Feature
Input: X = [[1, 0.1], [2, -0.1], [3, 0.2], [4, -0.2]] y = [2, 4, 6, 8] alpha = 1 max_iter = 50
Expected: [1.9667, 0]
Case 2: High Penalty Setting All to Zero
Input: X = [[1], [2]] y = [1, 1] alpha = 100 max_iter = 10
Expected: [0]