Neural Mastery

State, Continuity & Memory Compaction

A typical production engineering task requires 15 to 40 distinct tool invocations across multiple hours. If an agent process crashes, restarts, or switches context, how does it know what was already done and what remains?

In naked model architectures, a restarted session begins with total amnesia. The agent inspects files it already edited, re-implements already finished modules, or accidentally reverts its own work.

Imagine working on a multi-day construction project. If you wake up every morning with total memory loss and have to guess what was built by wandering around the site, you'll end up tearing down finished walls and forgetting the plumbing.

Instead, the foreman leaves a waterproof clipboard on the job site: a checklist of completed rooms, pending tasks, and architectural blueprints (feature_list.json). No matter who starts the shift, they check the clipboard and get straight to work.

1. The Context Compaction Pipeline

Inspect how a harness compresses tens of thousands of tokens of accumulated logs while preserving 100% of the active task state:

Strategy
Context Health: Optimal Focus
CONTEXT USAGE: 17,700 / 128,000 TOKENS (13.8%)WINDOW LIMIT: 128K
System Rules (3500 tok)Tool Definitions (2200 tok)State Artifacts (2400 tok)Working Memory (9600 tok)
EFFECTIVE ATTENTION RETENTION
98.5% (High)
Model attends sharply to current task constraints.
ESTIMATED COST PER STEP
$0.04 / turn
Fixed predictable token budget.
Harness Compaction: Old tool traces and raw file reads are pruned into a structured state artifact (feature_list.json) and recent working memory. Token usage stays bounded under 18k tokens across 20+ turns, preserving sharp model attention.
Predict First
When an agent's conversational history is aggressively compacted by 70% to fit within budget, what is the single most reliable mechanism to prevent it from repeating work it completed 20 turns ago?

2. Anthropic's State-Externalization Pattern: feature_list.json

In Anthropic's research on long-running application development, agents that rely on conversational memory inevitably suffer from amnesia or context anxiety.

The solution is the State-Externalization Pattern: keeping task state outside the volatile LLM context window in structured on-disk files.

Observe how externalized state guides coordination across agent roles:

Multi-Agent Supervisor Protocol
Anthropic Triad: Planner → Generator → Evaluator
1. Planner AgentACTIVE
Owns global decomposition, budget tracking, and atomic step definitions.
2. Generator Agent
Writes pure code and test edits. Operates strictly inside the isolated sandbox.
3. Evaluator Agent
Deterministic referee. Runs pytest, verifies invariants, signs off on commits.
Step 1: Planner ScopingStep 1 of 5

Planner ingests the user request, inspects repository state, and writes an atomic work order to feature_list.json.

TASK_DISPATCH: Implement exponential backoff in retry_policy.py (tests/test_retry.py required)
External State Artifact: feature_list.json
⏳Feature: Add exponential backoff algorithm with max jitter cap
◻Feature: Add Redis distributed lock provider
◻Feature: Telemetry Prometheus metrics exporter
Anthropic's long-running agent architecture separates concerns into specialized agents. The Generator never marks its own work as complete; only the independent Evaluator can sign off on verification and state externalization.
{
  "task": "build-python-attention-kernel",
  "version": "1.0.0",
  "checklist": [
    {
      "id": "qkv_projection",
      "label": "Implement Q/K/V linear projections with PyTorch",
      "status": "passed",
      "tests": ["pytest tests/unit/test_attention.py::test_qkv_shapes"]
    },
    {
      "id": "causal_masking",
      "label": "Generate autoregressive lower-triangular causal mask",
      "status": "passed",
      "tests": ["pytest tests/unit/test_attention.py::test_causal_mask"]
    },
    {
      "id": "flash_attention_kernel",
      "label": "Tiled online softmax kernel with Triton / PyTorch C++",
      "status": "in_progress",
      "tests": ["pytest tests/unit/test_flash.py"]
    },
    {
      "id": "fastapi_serving_endpoint",
      "label": "FastAPI streaming inference route with healthcheck",
      "status": "pending",
      "tests": ["pytest tests/e2e/test_api.py"]
    }
  ],
  "active_working_files": [
    "src/models/flash_attention.py",
    "tests/unit/test_flash.py"
  ]
}

3. Python Compaction Algorithm: Live Interactive Runner

Below is a production compaction engine in Python. Run this live in-browser demo to see how old, verbose tool outputs are pruned while keeping recent turns and state checklists completely intact:

Run it yourself
from dataclasses import dataclass
from typing import List

@dataclass
class Message:
role: str
content: str
is_tool_trace: bool = False

def estimate_tokens(text: str) -> int:
# Approximate rule of thumb: ~4 characters per token
return max(1, len(text) // 4)

def compact_conversation(messages: List[Message], budget_tokens: int = 150) -> List[Message]:
total_tokens = sum(estimate_tokens(m.content) for m in messages)
print(f"Initial Token Count: {total_tokens} tokens (Budget: {budget_tokens})")
if total_tokens <= budget_tokens:
return messages

# Compaction Strategy:
# 1. Keep the initial system instruction (turn 0)
# 2. Keep the last 2 turns intact
# 3. Collapse old tool outputs in between into compact semantic markers
compacted: List[Message] = []
recent_cutoff = len(messages) - 2

for i, msg in enumerate(messages):
if i > 0 and i < recent_cutoff and msg.is_tool_trace:
# Prune noisy 500-token tool logs into a 5-token marker
compacted.append(Message(
role=msg.role,
content="[Tool log pruned: verified pass by pytest]",
is_tool_trace=True
))
else:
State Persistence Principle

Treat the filesystem as the source of truth, not the context window. Always write task status to feature_list.json and human milestones to progress.txt. When context is cleared or refreshed, reloading these two files restores full autonomous momentum.

Next in Curriculum

Learn how autonomous agents govern multi-step execution in Autonomous Loops & Supervisors.

Last updated Oct 29, 2026Edit this pageReport an issue