Neural Mastery

Context & Instruction Architecture

In modern AI engineering, the way documentation and rules are provided to an agent is as critical as the code itself. Developers often believe that more context is always better—dumping thousands of lines of documentation, architecture manuals, and code snippets into an instruction file.

In practice, this causes instruction rot and attention dispersion. Frontier transformers suffer from the "lost-in-the-middle" effect: the larger the context, the less reliably the model obeys individual constraints.

Imagine asking someone to assemble a piece of furniture, and instead of giving them a 1-page quickstart diagram, you hand them a 400-page encyclopedia detailing the chemical composition of the screws and the history of woodworking. They get overwhelmed, lose track of the core instructions, and make silly mistakes.

In Harness Engineering, we keep instructions razor-sharp. We give the agent a compact Map of where things live, and let it pull detailed manuals only when it specifically needs them.

1. The Instruction Budget & Map vs Manual

A production harness treats token space like CPU cache. Every instruction carries an opportunity cost:

OpenAI & Anthropic Context Architecture
Instruction Budget & Map vs Manual Simulator
Context Strategy
10,700 / 32,000 tokens (33%)
10 files (Microservice)100 files (Standard Application)300 files (Enterprise Monorepo)
Context Window Pressure (32k Token Ceiling):
● Core AGENTS.md Rules: 3,500 tok
● AST Repository Map: 2,700 tok
● Verification State: 2,000 tok
● Dynamic Tool Headroom: 21,300 tok
Naive Context Strategy

Model receives the complete contents of source files. When the codebase exceeds 15 files, attention decay kicks in: the model skips edge cases, ignores project conventions in `AGENTS.md`, and incurs high API costs.

The Map vs Manual Strategy

The harness parses Python AST to extract signatures, docstrings, and exports into an index. The agent navigates this map and requests detailed file reads strictly on demand. Savings: 93% token economy.

The Map vs Manual architecture prevents 'Context Anxiety' and attention dispersion. In an agent-first world, dumping raw files into the prompt wastes 95% of your token budget on syntax noise. An AST-extracted repository skeleton provides spatial navigation in under 4,000 tokens.
Predict First
If an engineer dumps an entire 80,000-token repository into Claude 3.7 or GPT-4o's 128k/200k context window before asking for a bugfix, what is the most common observed failure mode?

2. Dynamic Compaction & Token Pressure

As an agent edits code, executes tests, and receives compiler outputs, its working memory fills rapidly. A robust harness continuously compacts conversational history into structured external state:

Strategy
Context Health: Optimal Focus
CONTEXT USAGE: 17,700 / 128,000 TOKENS (13.8%)WINDOW LIMIT: 128K
System Rules (3500 tok)Tool Definitions (2200 tok)State Artifacts (2400 tok)Working Memory (9600 tok)
EFFECTIVE ATTENTION RETENTION
98.5% (High)
Model attends sharply to current task constraints.
ESTIMATED COST PER STEP
$0.04 / turn
Fixed predictable token budget.
Harness Compaction: Old tool traces and raw file reads are pruned into a structured state artifact (feature_list.json) and recent working memory. Token usage stays bounded under 18k tokens across 20+ turns, preserving sharp model attention.

3. The AGENTS.md Standard

Industry-standard harnesses use structured markdown files placed in the repository root to establish the project map:

# Repository Map for AI Agents (Python Stack)

## Core Tech Stack
- Backend & ML: Python 3.12 + FastAPI + PyTorch + Pydantic v2
- Tooling: uv / poetry for environment isolation
- Verification: pytest (`pytest -v`), mypy (`mypy --strict`), ruff (`ruff check .`)

## Build & Verification Commands
- Static audit & lint: `ruff check . && ruff format --check .`
- Type validation: `mypy --strict src/`
- Unit test suite: `pytest tests/unit/ -v`
- Integration / smoke test: `pytest tests/e2e/ -v`

## Code Invariants
- NEVER use bare `try/except Exception: pass` to swallow errors.
- All function parameters and return types must carry explicit PEP 484 type annotations.
- NEVER edit files in `tests/` to manufacture artificial passes without user authorization.
- Always execute the 3-layer verification pipeline before committing changes.

4. AST-Extracted Repository Mapping in Python

Instead of dumping raw source code, a production harness uses Python's built-in ast module to dynamically extract a lightweight architectural skeleton.

Run this live in-browser Python implementation to see how an AST parser extracts an 85% compressed map:

Run it yourself
import ast
import textwrap

sample_code = """
class MultiHeadAttention:
def __init__(self, d_model: int = 512, n_heads: int = 8):
self.d_model = d_model
self.n_heads = n_heads

def forward(self, q, k, v):
pass
"""

def extract_ast_skeleton(code: str) -> str:
clean_code = textwrap.dedent(code).strip()
tree = ast.parse(clean_code)
lines = []
for node in tree.body:
if isinstance(node, ast.ClassDef):
lines.append(f"class {node.name}:")
for item in node.body:
if isinstance(item, ast.FunctionDef):
args = [a.arg for a in item.args.args]
lines.append(f" def {item.name}({', '.join(args)}): ...")
return "\n".join(lines)

print("=== Raw Code ===")
print(f"Raw character length: {len(sample_code)}")

skeleton = extract_ast_skeleton(sample_code)
print("\n=== AST-Extracted Map (Compact Index) ===")
print(skeleton)
print(f"\nSkeleton character length: {len(skeleton)} -> {(1 - len(skeleton)/len(sample_code))*100:.1f}% token reduction!")

Architectural Heuristic

Always prefer structured AST skeletons over raw code inlining. By feeding the agent function signatures, type annotations, and docstrings, the agent gains full spatial awareness of the entire repository in under 3,000 tokens.

Next in Curriculum

See how the harness catches and corrects regressions in The 3-Layer Verification Engine.

Last updated Oct 29, 2026Edit this pageReport an issue