Neural Mastery
← Practice
Conv2D Backward Pass Gradient w.r.t. Kernel Weightshard

Conv2D Backward Pass Gradient w.r.t. Kernel Weights

hard Neural Networks by Hand 25 min

NumPy allowed · plain Python earns 10 bonus points

Gradient descent step 1/4

Loss moves downhill

The update rule changes the parameters on each step.
Open lesson →
01parameters
02prediction
03loss
04gradient
θ = 4.000loss = 16.000∂L/∂θ = 8.000

θ ← θ − 0.14 · ∂L/∂θ

Derive and compute analytical gradients for convolutional filter weights during reverse-mode automatic differentiation.

Task

Implement conv2d_backward_weights(...). For each kernel weight position (ki, kj) in [0..kernel_size-1] x [0..kernel_size-1]: grad_W[ki][kj] = sum_{i=0..H_out-1, j=0..W_out-1} grad_output[i][j] * input_map[i * stride + ki][j * stride + kj]. Round each value to 4 decimal places.

Function Signature

conv2d_backward_weights(input_map: list[list[float]], grad_output: list[list[float]], kernel_size: int = 3, stride: int = 1) -> list[list[float]]

Examples

Example 1: 3x3 input 2x2 kernel stride 1
Input: input_map = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] grad_output = [[1, 1], [1, 1]] kernel_size = 2 stride = 1
Output: [[12, 16], [24, 28]]
Example 2: Stride 2 gradient accumulation
Input: input_map = [[1, 0, 2], [0, 3, 0], [4, 0, 5]] grad_output = [[2, 1], [0.5, 3]] kernel_size = 1 stride = 2
Output: [[21]]
Example 3: Unit grad output 1x1 filter
Input: input_map = [[2, 3], [4, 5]] grad_output = [[1, 2], [3, 4]] kernel_size = 1 stride = 1
Output: [[40]]

Constraints

  • 1 <= kernel_size <= 7
  • stride >= 1
  • Dimensions match valid forward convolution without padding
Python3Saved ✓
def conv2d_backward_weights(input_map: list[list[float]], grad_output: list[list[float]], kernel_size: int = 3, stride: int = 1) -> list[list[float]]:
"""Compute gradient of loss w.r.t. convolution kernel weights.

Args:
input_map: 2D list of shape (H_in, W_in).
grad_output: 2D list of incoming gradients of shape (H_out, W_out).
kernel_size: Spatial size of kernel (K x K).
stride: Stride used in forward convolution.

Returns:
2D list of kernel weight gradients of shape (kernel_size, kernel_size).
"""
pass

Case 1: 3x3 input 2x2 kernel stride 1
Input: input_map = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] grad_output = [[1, 1], [1, 1]] kernel_size = 2 stride = 1
Expected: [[12, 16], [24, 28]]
Case 2: Stride 2 gradient accumulation
Input: input_map = [[1, 0, 2], [0, 3, 0], [4, 0, 5]] grad_output = [[2, 1], [0.5, 3]] kernel_size = 1 stride = 2
Expected: [[21]]
Case 3: Unit grad output 1x1 filter
Input: input_map = [[2, 3], [4, 5]] grad_output = [[1, 2], [3, 4]] kernel_size = 1 stride = 1
Expected: [[40]]