Neural Mastery
← Practice
Bellman Equation for Value Iterationmedium

Bellman Equation for Value Iteration

mediumReinforcement Learning⏱ 15 min
⚡ Libraries allowed · Pure Python earns +10 bonus XP
🎯 Mission
Implement state value update V*(s) = max_a sum_{s'} P(s'|s,a)[R + gamma * V(s')].

Task

Implement `bellman_update(state, transitions, V, gamma)` returning updated float state value.

Function Signature

bellman_update(state: int, transitions: dict, V: list[float], gamma: float) -> float

Examples

Example 1: 2 Action MDP
Input: {"state":0,"transitions":{"0":{"a1":[[0.8,1,10],[0.2,0,0]],"a2":[[1,0,2]]}},"V":[0,50],"gamma":0.9}
Output: 44

Constraints

  • For each action, sum probability * (reward + gamma * V[next_s]). Return maximum over actions.
Python3Saved ✓
def bellman_update(state, transitions, V, gamma):
"""
Computes updated V(s) = max_a sum_{s'} P(s'|s,a) * [ R(s,a,s') + gamma * V(s') ].
"""
# Your implementation here
pass

Case 1: 2 Action MDP
Input: {"state":0,"transitions":{"0":{"a1":[[0.8,1,10],[0.2,0,0]],"a2":[[1,0,2]]}},"V":[0,50],"gamma":0.9}
Expected: 44