Recover From Malformed and Failing Tool Callsmedium

Recover From Malformed and Failing Tool Calls

medium Build an Agent 15–20 min

Standard library only · runs in your browser

Your stage-4 loop crashes the first time the model emits broken JSON, and production models do that a few times in every thousand calls. Feed the error back so the model can fix itself, but give up on a model that keeps failing.

Related Skills

ReAct agents in production

Task

The starter code is your stage-4 loop. Extend it:

If parse_tool_call raises ValueError as e, don't crash. Append {"role": "user", "content": f"error: could not parse your tool call ({e}). Reply with a JSON tool call or a final answer."} and continue to the next step.
A dispatch result that starts with error: also counts as an error.
Track errors (total) and consecutive errors. A successful tool result resets the consecutive count to 0.
After a step, if consecutive errors reach max_consecutive_errors, return with "stop_reason": "too_many_errors" and "steps" set to the current step.
Every returned dict now also includes "errors", the total error count.

Function Signature

run_agent(task, script, max_steps=5, max_consecutive_errors=3) -> dict

Examples

Example 1: Recovers from malformed JSON
Input: task = '1+2?'
script = ['{"tool": "add", "arguments": {"a": 1,}}', '{"tool": "add", "arguments": {"a": 1, "b": 2}}', 'The sum is {last_tool_result}.']
Output: {'answer': 'The sum is 3.', 'steps': 3, 'tools_called': ['add'], 'errors': 1, 'stop_reason': 'final_answer'}

Example 2: The model receives your error message
Input: task = '1+2?'
script = ['{"tool": "add" "arguments": {}}', 'I saw: {last_feedback}']
Output: {'answer': 'I saw: error: could not parse your tool call (invalid JSON in tool call: Expecting \',\' delimiter). Reply with a JSON tool call or a final answer.', 'steps': 2, 'tools_called': [], 'errors': 1, 'stop_reason': 'final_answer'}

Example 3: Three failures in a row stops the run
Input: task = 'Find flights'
script = ['{"tool": "search_web", "arguments": {"query": "cheap flights"}}', '{"tool": "search_web", "arguments": {"query": "cheap flights"}}', '{"tool": "search_web", "arguments": {"query": "cheap flights"}}', 'never reached']
Output: {'answer': None, 'steps': 3, 'tools_called': ['search_web', 'search_web', 'search_web'], 'errors': 3, 'stop_reason': 'too_many_errors'}

Example 4: A success resets the consecutive count
Input: task = 'maths'
script = ['{"tool": "add", "arguments": {"a": 1,}}', '{"tool": "add", "arguments": {"a": 1,}}', '{"tool": "add", "arguments": {"a": 1, "b": 2}}', '{"tool": "add", "arguments": {"a": 1,}}', '{"tool": "add", "arguments": {"a": 1,}}', 'done']
max_steps = 8
Output: {'answer': 'done', 'steps': 6, 'tools_called': ['add'], 'errors': 4, 'stop_reason': 'final_answer'}

Constraints

Use the exact feedback message above. The scripted model echoes it back via `{last_feedback}`.
A parse error doesn't add anything to `tools_called`, but a tool that returns an error does.
`max_steps` still applies, and a final answer still wins immediately.

Python3Saved ✓

import json

import math

import re

def run_agent(task, script, max_steps=5):

llm = make_llm(script)

messages = [{"role": "system", "content": SYSTEM_PROMPT},

{"role": "user", "content": task}]

tools_called = []

for step in range(1, max_steps + 1):

reply = llm(messages)

messages.append({"role": "assistant", "content": reply})

call = parse_tool_call(reply)

if call is None:

return {"answer": reply, "steps": step, "tools_called": tools_called,

"stop_reason": "final_answer"}

tools_called.append(call["tool"])

messages.append({"role": "tool", "content": dispatch(call)})

return {"answer": None, "steps": max_steps, "tools_called": tools_called,

"stop_reason": "max_steps"}

# TODO (stage 5):

# - catch ValueError from parse_tool_call and feed the error back as a user message

# - count errors (parse errors + tool results starting with "error:")

# - stop with "too_many_errors" after max_consecutive_errors in a row

# - add "errors" to every returned dict, and accept max_consecutive_errors=3

# =============================================================================

# PROVIDED BELOW -- the sandbox and your earlier stages. Read it, don't edit it.

# =============================================================================

# ---------------------------------------------------------------------------

# The sandbox world your agent lives in (provided -- you don't edit this).

# Four deterministic tools, each with a JSON-Schema-style argument spec,

# exactly the shape you'd hand a real model's tool-calling API.

# ---------------------------------------------------------------------------

Case 1: Recovers from malformed JSON

Input: task = '1+2?'
script = ['{"tool": "add", "arguments": {"a": 1,}}', '{"tool": "add", "arguments": {"a": 1, "b": 2}}', 'The sum is {last_tool_result}.']

Expected: {'answer': 'The sum is 3.', 'steps': 3, 'tools_called': ['add'], 'errors': 1, 'stop_reason': 'final_answer'}

Case 2: The model receives your error message

Input: task = '1+2?'
script = ['{"tool": "add" "arguments": {}}', 'I saw: {last_feedback}']

Expected: {'answer': 'I saw: error: could not parse your tool call (invalid JSON in tool call: Expecting \',\' delimiter). Reply with a JSON tool call or a final answer.', 'steps': 2, 'tools_called': [], 'errors': 1, 'stop_reason': 'final_answer'}

Case 3: Three failures in a row stops the run

Input: task = 'Find flights'
script = ['{"tool": "search_web", "arguments": {"query": "cheap flights"}}', '{"tool": "search_web", "arguments": {"query": "cheap flights"}}', '{"tool": "search_web", "arguments": {"query": "cheap flights"}}', 'never reached']

Expected: {'answer': None, 'steps': 3, 'tools_called': ['search_web', 'search_web', 'search_web'], 'errors': 3, 'stop_reason': 'too_many_errors'}

Case 4: A success resets the consecutive count

Input: task = 'maths'
script = ['{"tool": "add", "arguments": {"a": 1,}}', '{"tool": "add", "arguments": {"a": 1,}}', '{"tool": "add", "arguments": {"a": 1, "b": 2}}', '{"tool": "add", "arguments": {"a": 1,}}', '{"tool": "add", "arguments": {"a": 1,}}', 'done']
max_steps = 8

Expected: {'answer': 'done', 'steps': 6, 'tools_called': ['add'], 'errors': 4, 'stop_reason': 'final_answer'}