Neural Mastery
← Practice
Vanilla Elman RNN Step Forwardhard

Vanilla Elman RNN Step Forward

hard Neural Networks by Hand 25 min

NumPy allowed · plain Python earns 10 bonus points

Compute recurrent state transitions for sequence modeling in a classic Elman recurrent neural network.

Task

Implement vanilla_rnn_step_forward(x, h_prev, Wxh, Whh, bh): for each batch sample i, compute the new hidden state h_next[i] = tanh(x[i] @ Wxh + h_prev[i] @ Whh + bh). Return h_next of shape (N, H) with all elements rounded to 4 decimal places.

Function Signature

vanilla_rnn_step_forward(x: list[list[float]], h_prev: list[list[float]], Wxh: list[list[float]], Whh: list[list[float]], bh: list[float]) -> list[list[float]]

Examples

Example 1: Single sample 1D input 1D hidden
Input: x = [[1]] h_prev = [[0.5]] Wxh = [[0.8]] Whh = [[0.2]] bh = [0.1]
Output: [[0.7616]]
Example 2: Zero initial state 2D hidden
Input: x = [[1, -1]] h_prev = [[0, 0]] Wxh = [[0.5, -0.5], [1, 0]] Whh = [[0.1, 0.2], [-0.1, 0.3]] bh = [0, 0]
Output: [[-0.4621, -0.4621]]
Example 3: Batch of 2 recurrent steps
Input: x = [[0.5], [-0.5]] h_prev = [[0.2, -0.2], [0.8, -0.8]] Wxh = [[1, -1]] Whh = [[0.5, 0], [0, 0.5]] bh = [0.1, -0.1]
Output: [[0.6044, -0.6044], [0, 0]]

Constraints

  • 1 <= N <= 50
  • 1 <= D_in, H <= 64
  • len(bh) == H
  • Whh is of shape (H, H)
Python3Saved ✓
def vanilla_rnn_step_forward(x: list[list[float]], h_prev: list[list[float]], Wxh: list[list[float]], Whh: list[list[float]], bh: list[float]) -> list[list[float]]:
"""Compute forward pass for a single timestep of a vanilla Elman RNN cell.

Args:
x: Input matrix of shape (N, D_in).
h_prev: Previous hidden state matrix of shape (N, H).
Wxh: Input-to-hidden weight matrix of shape (D_in, H).
Whh: Hidden-to-hidden weight matrix of shape (H, H).
bh: Hidden bias vector of length H.

Returns:
Updated hidden state matrix h_next of shape (N, H) rounded to 4 decimals.
"""
pass

Case 1: Single sample 1D input 1D hidden
Input: x = [[1]] h_prev = [[0.5]] Wxh = [[0.8]] Whh = [[0.2]] bh = [0.1]
Expected: [[0.7616]]
Case 2: Zero initial state 2D hidden
Input: x = [[1, -1]] h_prev = [[0, 0]] Wxh = [[0.5, -0.5], [1, 0]] Whh = [[0.1, 0.2], [-0.1, 0.3]] bh = [0, 0]
Expected: [[-0.4621, -0.4621]]
Case 3: Batch of 2 recurrent steps
Input: x = [[0.5], [-0.5]] h_prev = [[0.2, -0.2], [0.8, -0.8]] Wxh = [[1, -1]] Whh = [[0.5, 0], [0, 0.5]] bh = [0.1, -0.1]
Expected: [[0.6044, -0.6044], [0, 0]]