An LLM that only answers questions is a chatbot. An LLM that can look things up, run code, and take action based on what it finds is an agent.
Mode
Same model underneath — the loop plus the ability to act on the outside world is the entire distinction.
The model can decide to call a tool, observe the result, and decide what to do next -- potentially many times -- before producing a final answer. The loop is the entire difference.
Neural Mastery
What Makes Something an Agent
A single LLM call takes an input, produces an output, and stops. An agent wraps an LLM in a loop: the model can decide to call a tool, observe the result, and decide what to do next — potentially many times — before producing a final answer. The defining feature isn't the model itself, it's the loop plus the ability to take actions that affect (or query) the outside world.
Step 1 of 5: Model decides
Given the input + history so far, decide: call a tool, or answer now?
Neural Mastery
Tool Use / Function Calling
Modern LLMs are trained to recognize when a task needs external information or action, and to emit a structured request (function name + arguments) instead of a plain text answer. The calling application executes the actual function, and feeds the result back into the model's context to continue.
"What's the weather in Tokyo?"
→
get_weather(city="Tokyo", unit="celsius")
● name — which tool● arguments — matching the declared schema
Click a part of the call below.
Neural Mastery
Here's what that looks like as an actual multi-step trace, one tool result shaping the next decision:
user"Is it a good day to fly a drone in Austin?"
Turn 1 of 6
Two tool calls in sequence, each observed result shaping the next decision -- the model only produces a final answer once it has both pieces of information it needed.
Neural Mastery
Designing tools well matters enormously:
Clear names and descriptions — the model chooses which tool to call based on these, exactly like a person choosing from a menu of documented functions.
Tool design
get_stock_price(ticker: string)
"Get the current stock price for a given ticker symbol."
Clear names/descriptions, well-specified schemas, narrow scope — each independently reduces malformed or wrong tool calls.
Model calls this confidently and correctly for any price question — the name, description, and schema all point at exactly one use.
Neural Mastery
Well-specified argument schemas — ambiguous or overly flexible schemas lead to malformed calls.
Every field is typed and required — there's exactly one valid shape for a call, so the model produces it correctly.
Neural Mastery
Narrow, composable tools beat one giant do-everything tool — easier for the model to reason about which one it needs.
Decomposition
Same 5 capabilities, two decompositions.
Each tool has one job — the model picks the right one directly, the same way choosing from a menu of documented functions works.
Neural Mastery
Agent Memory
Short-term memory: whatever fits in the current context window — the conversation so far, retrieved documents, tool results.
Long-term memory: information that needs to persist across sessions or beyond what fits in context — typically implemented as an external store (a database or vector store) that the agent can write to and query, effectively turning memory into a retrieval problem (see RAG).
Same agent, two structurally different memory stores with different lifetimes.
Hover a store.
Neural Mastery
Write (session 1)
Query (session 2+)
Same store, same embed step — long-term memory is retrieval with extra steps.
Session 1: something worth remembering gets embedded and written into the store — this is the exact same embed step RAG uses for indexing documents.
Neural Mastery
The core challenge: context windows are finite, but useful agent tasks often span far more information than fits in one window — so real agent systems need explicit strategies for what to keep, what to summarize, and what to offload to external memory.
● system prompt● conversation so far● retrieved documents● tool results
70 / 100 units used
Still fits. Keep dragging turns forward and watch the same 4 categories outgrow the fixed window.
Neural Mastery
Next: Context Engineering — the actual discipline of deciding what belongs in that finite window at every step.