Specify, Verify, Debug

The work left when an agent writes the code: turn requests into tests, debug production failures, measure LLM output, and decide what ships.

Intermediate · 12 problems · 0 passed · Background reading: LLM, RAG & Agent Evaluation

One system, twelve situations. Helpdesk is a support bot whose code a coding agent wrote, and every stage is something that goes wrong with it, or a decision about it. Each stage stands on its own, and the capstone uses stage 3.

1. 01Turn the Request Into Testseasy
2. 02The Code Is Right, the Rule Is Wrongeasy
3. 03Read the Logs Before the Codemedium
4. 04Retries Refund Twicemedium
5. 05Stale Answers After a Policy Changemedium
6. 06Measure Retrieval Before Tuning Itmedium
7. 07Write a Check That Agrees With Peoplemedium
8. 08A Judge You Can Trusthard
9. 09Will It Hold 50 Requests a Second?medium
10. 10A Document Tells the Bot to Refundhard
11. 11Ship It or Roll It Backhard
12. 12Capstone: The Incident Reviewhard

Next track: Python for AI