Guardrails: Loop Detection and Token Budgetsmedium

Guardrails: Loop Detection and Token Budgets

medium Build an Agent 20–25 min

Standard library only · runs in your browser

The most expensive agent bug is the one that never errors: the model calls the same tool with the same arguments forever, and every call succeeds. Add the two guardrails every production agent has, repeat-call detection and a context budget.

Task

The starter is the stage-5 loop. Add:

Loop detection. For every successfully parsed tool call, build a signature: the tool name followed by json.dumps(arguments, sort_keys=True). If this call's signature equals the previous two calls' signatures (three identical in a row), don't run it. Return with "stop_reason": "loop_detected" and "steps": step. The repeated call isn't added to tools_called.

Token budget. At the start of each step, before calling the model, compute context_tokens = sum(count_tokens(m["content"]) for m in messages). If max_tokens is set and context_tokens > max_tokens, return with "stop_reason": "token_budget" and "steps": step - 1 (the model calls actually made).

Every returned dict also gets "context_tokens", the sum over messages at the moment you return. Defaults change to max_steps=8.

Function Signature

run_agent(task, script, max_steps=8, max_consecutive_errors=3, max_tokens=None) -> dict

Examples

Example 1: Same call three times in a row
Input: task = 'Weather in Paris?'
script = ['{"tool": "get_weather", "arguments": {"city": "Paris"}}', '{"tool": "get_weather", "arguments": {"city": "Paris"}}', '{"tool": "get_weather", "arguments": {"city": "Paris"}}', 'never']
Output: {'answer': None, 'steps': 3, 'tools_called': ['get_weather', 'get_weather'], 'errors': 0, 'stop_reason': 'loop_detected', 'context_tokens': 89}

Example 2: Different arguments are not a loop
Input: task = 'Weather everywhere'
script = ['{"tool": "get_weather", "arguments": {"city": "Paris"}}', '{"tool": "get_weather", "arguments": {"city": "Tokyo"}}', '{"tool": "get_weather", "arguments": {"city": "Lima"}}', 'All three: {last_tool_result}']
Output: {'answer': 'All three: 16C, overcast', 'steps': 4, 'tools_called': ['get_weather', 'get_weather', 'get_weather'], 'errors': 0, 'stop_reason': 'final_answer', 'context_tokens': 98}

Example 3: Context budget stops a long run
Input: task = 'Read the shipping and refund docs, then summarise.'
script = ['{"tool": "lookup_doc", "arguments": {"doc_id": "shipping"}}', '{"tool": "lookup_doc", "arguments": {"doc_id": "refund-policy"}}', 'Summary']
max_tokens = 75
Output: {'answer': None, 'steps': 2, 'tools_called': ['lookup_doc', 'lookup_doc'], 'errors': 0, 'stop_reason': 'token_budget', 'context_tokens': 102}

Example 4: Happy path reports context_tokens
Input: task = 'Weather in Tokyo?'
script = ['{"tool": "get_weather", "arguments": {"city": "Tokyo"}}', 'Tokyo: {last_tool_result}']
Output: {'answer': 'Tokyo: 24C, clear', 'steps': 2, 'tools_called': ['get_weather'], 'errors': 0, 'stop_reason': 'final_answer', 'context_tokens': 61}

Constraints

Parse errors don't add a signature and don't break a streak of identical calls.
`sort_keys=True` matters: `{"a": 1, "b": 2}` and `{"b": 2, "a": 1}` are the same call.
`count_tokens` is provided (about 4 characters per token).

Python3Saved ✓

import json

import math

import re

def run_agent(task, script, max_steps=5, max_consecutive_errors=3):

llm = make_llm(script)

messages = [{"role": "system", "content": SYSTEM_PROMPT},

{"role": "user", "content": task}]

tools_called = []

errors = 0

consecutive = 0

for step in range(1, max_steps + 1):

reply = llm(messages)

messages.append({"role": "assistant", "content": reply})

try:

call = parse_tool_call(reply)

except ValueError as e:

errors += 1

consecutive += 1

messages.append({"role": "user", "content":

f"error: could not parse your tool call ({e}). "

"Reply with a JSON tool call or a final answer."})

else:

if call is None:

return {"answer": reply, "steps": step, "tools_called": tools_called,

"errors": errors, "stop_reason": "final_answer"}

tools_called.append(call["tool"])

result = dispatch(call)

messages.append({"role": "tool", "content": result})

if result.startswith("error:"):

errors += 1

consecutive += 1

else:

consecutive = 0

if consecutive >= max_consecutive_errors:

Case 1: Same call three times in a row

Input: task = 'Weather in Paris?'
script = ['{"tool": "get_weather", "arguments": {"city": "Paris"}}', '{"tool": "get_weather", "arguments": {"city": "Paris"}}', '{"tool": "get_weather", "arguments": {"city": "Paris"}}', 'never']

Expected: {'answer': None, 'steps': 3, 'tools_called': ['get_weather', 'get_weather'], 'errors': 0, 'stop_reason': 'loop_detected', 'context_tokens': 89}

Case 2: Different arguments are not a loop

Input: task = 'Weather everywhere'
script = ['{"tool": "get_weather", "arguments": {"city": "Paris"}}', '{"tool": "get_weather", "arguments": {"city": "Tokyo"}}', '{"tool": "get_weather", "arguments": {"city": "Lima"}}', 'All three: {last_tool_result}']

Expected: {'answer': 'All three: 16C, overcast', 'steps': 4, 'tools_called': ['get_weather', 'get_weather', 'get_weather'], 'errors': 0, 'stop_reason': 'final_answer', 'context_tokens': 98}

Case 3: Context budget stops a long run

Input: task = 'Read the shipping and refund docs, then summarise.'
script = ['{"tool": "lookup_doc", "arguments": {"doc_id": "shipping"}}', '{"tool": "lookup_doc", "arguments": {"doc_id": "refund-policy"}}', 'Summary']
max_tokens = 75

Expected: {'answer': None, 'steps': 2, 'tools_called': ['lookup_doc', 'lookup_doc'], 'errors': 0, 'stop_reason': 'token_budget', 'context_tokens': 102}

Case 4: Happy path reports context_tokens

Input: task = 'Weather in Tokyo?'
script = ['{"tool": "get_weather", "arguments": {"city": "Tokyo"}}', 'Tokyo: {last_tool_result}']

Expected: {'answer': 'Tokyo: 24C, clear', 'steps': 2, 'tools_called': ['get_weather'], 'errors': 0, 'stop_reason': 'final_answer', 'context_tokens': 61}