Neural Mastery
← Practice
Logistic Regression: Forward Pass + Log Lossmedium

Logistic Regression: Forward Pass + Log Loss

medium Classical ML from Scratch 15–20 min

NumPy allowed · plain Python earns 10 bonus points

Gradient descent step 1/4

Loss moves downhill

The update rule changes the parameters on each step.
Open lesson →
01parameters
02prediction
03loss
04gradient
θ = 4.000loss = 16.000∂L/∂θ = 8.000

θ ← θ − 0.14 · ∂L/∂θ

Implement logistic regression's full forward pass and cost function -- the actual thing being minimized during training, distinct from just the sigmoid function itself.

Task

Implement logistic_regression_loss(X, y, weights, bias): for every row, compute z = w.x + b, squash it with sigmoid, then average the binary cross-entropy loss -(y*log(p) + (1-y)*log(1-p)) across all rows.

Function Signature

logistic_regression_loss(X: list[list[float]], y: list[int], weights: list[float], bias: float) -> float

Examples

Example 1: Zero Logit (Maximum Uncertainty)
Input: X = [[0]] y = [1] weights = [0] bias = 0
Output: 0.6931471805599453
Explanation: z=0 gives p=0.5; loss = -ln(0.5) = ln(2)
Example 2: Symmetric Confident Pair
Input: X = [[2], [-2]] y = [1, 0] weights = [1] bias = 0
Output: 0.12692801104297258
Explanation: Both rows are equally (and correctly) confident by symmetry

Constraints

  • Libraries (NumPy) are allowed and accepted normally; A plain-Python solution earns 10 bonus points.
  • Clip p into [1e-15, 1 - 1e-15] before calling log(), so a perfectly confident (and wrong) prediction never produces log(0).
  • Return the MEAN loss across all rows, not the sum.
Python3Saved ✓
import math

def logistic_regression_loss(X: list[list[float]], y: list[int], weights: list[float], bias: float) -> float:
"""X: list of feature rows. y: list of 0/1 labels. weights, bias:
model parameters. Return the mean binary cross-entropy (log) loss
over the dataset: for each row, z = dot(x, weights) + bias, p =
sigmoid(z), loss_i = -(y*log(p) + (1-y)*log(1-p)). Clip p to
[1e-15, 1-1e-15] before taking a log, to avoid log(0)."""
# Your implementation here
pass

Case 1: Zero Logit (Maximum Uncertainty)
Input: X = [[0]] y = [1] weights = [0] bias = 0
Expected: 0.6931471805599453
Case 2: Symmetric Confident Pair
Input: X = [[2], [-2]] y = [1, 0] weights = [1] bias = 0
Expected: 0.12692801104297258