Neural Mastery
← Practice
Vectorized Layer Normalization with Learnable Affinehard

Vectorized Layer Normalization with Learnable Affine

hard NumPy & Data Wrangling 20-25 min

NumPy allowed · plain Python earns 10 bonus points

Linear algebra step 1/4

A vector transforms

Matrix multiplication is the first computation behind every model.
Open lesson →
01vector
02matrix
03transform
04result
[2, −1] × [[1, 2], [−.5, 1]] = [0, −2]

Computed matrix × vector transformation

Normalize activations across feature dimensions per sample with learnable scale and bias.

Task

Implement layer_norm(features, gamma, beta, eps): for each sample row x of dimension D, compute mean mu = sum(x) / D and variance sigma^2 = sum((x - mu)^2) / D. Standardize and apply affine transform y = ((x - mu) / sqrt(sigma^2 + eps)) * gamma + beta. Round each element to 4 decimals.

Function Signature

layer_norm(features: list[list[float]], gamma: list[float], beta: list[float], eps: float = 1e-5) -> list[list[float]]

Examples

Example 1: Batch Size 2 Dim 2 Unit Affine
Input: features = [[2, 4], [10, 20]] gamma = [1, 1] beta = [0, 0] eps = 0.00001
Output: [[-1, 1], [-1, 1]]
Explanation: Layer norm across features with identity affine parameters.
Example 2: Scaled and Shifted Affine
Input: features = [[1, 2, 3]] gamma = [2, 0.5, 1] beta = [-1, 0, 1] eps = 0.00001
Output: [[-3.4495, 0, 2.2247]]
Explanation: Non-trivial learnable gamma and beta.

Constraints

  • len(gamma) == len(beta) == D
  • 1 <= len(features) <= 1000
  • eps > 0.0
Python3Saved ✓
def layer_norm(features: list[list[float]], gamma: list[float], beta: list[float], eps: float = 1e-5) -> list[list[float]]:
"""Compute layer normalization over feature dimension.
y = ((x - mean) / sqrt(var + eps)) * gamma + beta.
Round entries to 4 decimals."""
# Implement solution logic here
pass

Case 1: Batch Size 2 Dim 2 Unit Affine
Input: features = [[2, 4], [10, 20]] gamma = [1, 1] beta = [0, 0] eps = 0.00001
Expected: [[-1, 1], [-1, 1]]
Case 2: Scaled and Shifted Affine
Input: features = [[1, 2, 3]] gamma = [2, 0.5, 1] beta = [-1, 0, 1] eps = 0.00001
Expected: [[-3.4495, 0, 2.2247]]