Neural Mastery
← Practice
1D Batch Normalization Forward Passmedium

1D Batch Normalization Forward Pass

medium Neural Networks by Hand 20 min

NumPy allowed · plain Python earns 10 bonus points

Stabilize deep network training by standardizing mini-batch feature distributions across samples.

Related Skills

Task

Implement batch_norm1d_forward(...): in training=True mode, compute sample mean mu_B and variance sigma_B^2 across batch rows N for each feature column. Update running statistics: running_mean = (1 - momentum) * running_mean + momentum * mu_B and running_var = (1 - momentum) * running_var + momentum * sigma_B^2. Normalize inputs using mu_B and sigma_B^2. In training=False mode, use running_mean and running_var directly without updating them. Return {'out': out, 'running_mean': ..., 'running_var': ...}.

Function Signature

batch_norm1d_forward(x: list[list[float]], gamma: list[float], beta: list[float], running_mean: list[float], running_var: list[float], eps: float = 1e-5, momentum: float = 0.1, training: bool = True) -> dict

Examples

Example 1: Training mode batch of 2
Input: x = [[1, 10], [3, 20]] gamma = [1, 1] beta = [0, 0] running_mean = [0, 0] running_var = [1, 1] eps = 0.00001 momentum = 0.1 training = True
Output: {'out': [[-1, -1], [1, 1]], 'running_mean': [0.2, 1.5], 'running_var': [1, 3.4]}
Example 2: Eval mode using running stats
Input: x = [[2, 15]] gamma = [1, 1] beta = [0, 0] running_mean = [2, 15] running_var = [1, 25] eps = 0.00001 momentum = 0.1 training = False
Output: {'out': [[0, 0]], 'running_mean': [2, 15], 'running_var': [1, 25]}
Example 3: Eval mode with non-trivial gamma/beta
Input: x = [[3, 5]] gamma = [2, -1] beta = [1, 0.5] running_mean = [1, 5] running_var = [4, 1] eps = 0.00001 momentum = 0.1 training = False
Output: {'out': [[3, 0.5]], 'running_mean': [1, 5], 'running_var': [4, 1]}

Constraints

  • N >= 1
  • D >= 1
  • len(gamma) == len(beta) == len(running_mean) == len(running_var) == D
  • 0.0 <= momentum <= 1.0
Python3Saved ✓
def batch_norm1d_forward(x: list[list[float]], gamma: list[float], beta: list[float], running_mean: list[float], running_var: list[float], eps: float = 1e-5, momentum: float = 0.1, training: bool = True) -> dict:
"""Compute 1D Batch Normalization forward pass in training or inference mode.

Args:
x: 2D input batch of shape (N, D).
gamma: Scale parameter vector of length D.
beta: Shift parameter vector of length D.
running_mean: Running mean vector of length D.
running_var: Running variance vector of length D.
eps: Numerical stability constant.
momentum: Exponential moving average momentum factor.
training: Boolean flag indicating train vs test mode.

Returns:
Dictionary with 'out', 'running_mean', and 'running_var'.
"""
pass

Case 1: Training mode batch of 2
Input: x = [[1, 10], [3, 20]] gamma = [1, 1] beta = [0, 0] running_mean = [0, 0] running_var = [1, 1] eps = 0.00001 momentum = 0.1 training = True
Expected: {'out': [[-1, -1], [1, 1]], 'running_mean': [0.2, 1.5], 'running_var': [1, 3.4]}
Case 2: Eval mode using running stats
Input: x = [[2, 15]] gamma = [1, 1] beta = [0, 0] running_mean = [2, 15] running_var = [1, 25] eps = 0.00001 momentum = 0.1 training = False
Expected: {'out': [[0, 0]], 'running_mean': [2, 15], 'running_var': [1, 25]}
Case 3: Eval mode with non-trivial gamma/beta
Input: x = [[3, 5]] gamma = [2, -1] beta = [1, 0.5] running_mean = [1, 5] running_var = [4, 1] eps = 0.00001 momentum = 0.1 training = False
Expected: {'out': [[3, 0.5]], 'running_mean': [1, 5], 'running_var': [4, 1]}