Neural Mastery
← Practice
SVM Hinge Loss & Marginmedium

SVM Hinge Loss & Margin

medium Classical ML from Scratch 15–20 min

NumPy allowed · plain Python earns 10 bonus points

Gradient descent step 1/4

Loss moves downhill

The update rule changes the parameters on each step.
Open lesson →
01parameters
02prediction
03loss
04gradient
θ = 4.000loss = 16.000∂L/∂θ = 8.000

θ ← θ − 0.14 · ∂L/∂θ

Implement the soft-margin SVM objective directly -- the L2 margin-maximizing regularizer plus the hinge loss that only penalizes points inside (or on the wrong side of) the margin, distinct from every other loss function on this site because correctly-classified points beyond the margin contribute exactly zero.

Task

Implement svm_hinge_loss(X, y, weights, bias, C=1.0): compute 0.5 * sum(w_j^2) (the margin-maximizing regularizer) plus C times the mean hinge loss max(0, 1 - y_i*(w.x_i + b)) across all rows. Labels are +1/-1, not 0/1.

Function Signature

svm_hinge_loss(X: list[list[float]], y: list[int], weights: list[float], bias: float, C: float = 1.0) -> float

Examples

Example 1: One Correctly-Classified, One Violating
Input: X = [[1, 1], [2, 2]] y = [1, -1] weights = [1, 0] bias = 0 C = 1
Output: 2
Explanation: Row 2 lands on the wrong side of the boundary and dominates the hinge term
Example 2: Symmetric Violations
Input: X = [[1, 0], [-1, 0]] y = [1, -1] weights = [2, 0] bias = 0 C = 1
Output: 2

Constraints

  • Labels are +1 or -1 (the standard SVM convention), not 0/1.
  • The hinge term is 0 for any point already correctly classified with margin >= 1 -- only margin violations (including misclassifications) contribute.
  • Average the per-row hinge loss (divide by the number of rows) before multiplying by C.
Python3Saved ✓
def svm_hinge_loss(X: list[list[float]], y: list[int], weights: list[float], bias: float, C: float = 1.0) -> float:
"""X: feature rows. y: labels, each -1 or +1 (NOT 0/1). weights, bias,
C: model parameters. Return the soft-margin SVM primal objective:
0.5 * ||weights||^2 + C * mean_i(max(0, 1 - y_i * (w.x_i + b)))."""
# Your implementation here
pass

Case 1: One Correctly-Classified, One Violating
Input: X = [[1, 1], [2, 2]] y = [1, -1] weights = [1, 0] bias = 0 C = 1
Expected: 2
Case 2: Symmetric Violations
Input: X = [[1, 0], [-1, 0]] y = [1, -1] weights = [2, 0] bias = 0 C = 1
Expected: 2