Neural Mastery
← Practice
Categorical Cross-Entropy Losseasy

Categorical Cross-Entropy Loss

easy Neural Networks by Hand 15 min

NumPy allowed · plain Python earns 10 bonus points

Gradient descent step 1/4

Loss moves downhill

The update rule changes the parameters on each step.
Open lesson →
01parameters
02prediction
03loss
04gradient
θ = 4.000loss = 16.000∂L/∂θ = 8.000

θ ← θ − 0.14 · ∂L/∂θ

Calculate negative log-likelihood loss for multi-class classification from predicted softmax probability distributions.

Task

Implement categorical_cross_entropy(probs, targets, eps=1e-12): for each sample i, retrieve the predicted probability p = max(probs[i][targets[i]], eps) for ground truth class index targets[i]. The sample loss is -log(p). Return the average loss across all N samples rounded to 4 decimal places.

Function Signature

categorical_cross_entropy(probs: list[list[float]], targets: list[int], eps: float = 1e-12) -> float

Examples

Example 1: Confident correct predictions
Input: probs = [[0.7, 0.2, 0.1], [0.05, 0.9, 0.05]] targets = [0, 1]
Output: 0.231
Example 2: Uniform predictions 3 classes
Input: probs = [[0.3333, 0.3333, 0.3334]] targets = [2]
Output: 1.0984
Example 3: Low probability target
Input: probs = [[0.98, 0.01, 0.01]] targets = [1]
Output: 4.6052

Constraints

  • len(probs) == len(targets) >= 1
  • 0 <= targets[i] < len(probs[0])
  • probs entries are valid probabilities
Python3Saved ✓
def categorical_cross_entropy(probs: list[list[float]], targets: list[int], eps: float = 1e-12) -> float:
"""Compute average categorical cross-entropy loss given softmax probabilities and class targets.

Args:
probs: 2D probability matrix of shape (N, C).
targets: 1D list of integer class labels of length N (0 <= targets[i] < C).
eps: Small float to prevent log(0).

Returns:
Mean loss float rounded to 4 decimal places.
"""
pass

Case 1: Confident correct predictions
Input: probs = [[0.7, 0.2, 0.1], [0.05, 0.9, 0.05]] targets = [0, 1]
Expected: 0.231
Case 2: Uniform predictions 3 classes
Input: probs = [[0.3333, 0.3333, 0.3334]] targets = [2]
Expected: 1.0984
Case 3: Low probability target
Input: probs = [[0.98, 0.01, 0.01]] targets = [1]
Expected: 4.6052