Multi-Agent Systems

When one agent isn't enough — and the coordination problems that come with adding more.

Coordination Patterns

The three topologies, drawn as actual graphs rather than described in prose — the shape difference is the entire tradeoff between debuggability and flexibility:

• Hierarchical (orchestrator/sub-agent): one agent breaks down and delegates work to specialized sub-agents, then combines results — see Agent Architectures. Easiest to reason about and debug, since there's a clear owner of the overall task. The supervisor pattern is the more dynamic version of this — looping and re-routing between sub-agents based on intermediate results, rather than a fixed one-shot delegation plan.
• Peer-to-peer: agents communicate directly with each other as equals, without a central coordinator — more flexible, but harder to guarantee the task actually converges to a result, and harder to debug when something goes wrong.
• Blackboard: agents don't talk to each other directly at all — they read and write to a shared state/workspace ("the blackboard"), and react to changes in it. Useful when the set of contributing agents isn't fixed in advance.

Shared State and Conflict Resolution

When multiple agents can read and write shared state (a document, a database, a plan), you need a strategy for conflicts — two agents editing the same resource, or reaching contradictory conclusions. Common approaches: locking/turn-taking (only one agent acts on a resource at a time), a designated "resolver" agent that reconciles conflicting outputs, or simply designing the task decomposition so agents own disjoint pieces of state and never need to reconcile.

When Multi-Agent Actually Helps

Multi-agent systems earn their complexity when:

• Different sub-tasks genuinely need different tools, instructions, or models (e.g. a coding sub-agent vs. a research sub-agent)
• The task is large enough that a single agent's context window becomes a bottleneck
• Sub-tasks can run in parallel, and wall-clock time matters — a real greedy-scheduling simulation of exactly that speedup:

They don't help — and often actively hurt — when a single well-prompted agent with good tools was already handling the task fine. Coordination overhead, extra latency, and extra points of failure are real costs, not free architecture upgrades.

Agent Swarms: Scaling to Many Agents

The three topologies above assume a small, deliberately-designed set of agents — two specialists, three sub-agents under an orchestrator. "Swarm" specifically implies something different: potentially dozens to millions of largely homogeneous, lightweight agents, often coordinated in a more decentralized way — the term borrows directly from biological swarm intelligence (ant colonies foraging, birds flocking), where simple individual behavior produces sophisticated collective behavior without any single agent holding the full plan.

This is a distinct, later question from Single-Agent vs. Multi-Agent's "should I even split this into multiple agents" decision — a swarm architecture assumes that question is already answered yes, and asks the one that comes after it: once there are many agents, how do you actually coordinate them at scale?

OpenAI's Swarm (2024) is worth knowing even though OpenAI itself has since replaced it with the production-ready Agents SDK — the repository itself is explicitly labeled "experimental, educational," and that's exactly why it's a good teaching example: it strips the idea down to two concepts, nothing more. An Agent bundles a routine — a system prompt (instructions) plus a list of callable functions — and control passes between agents via a handoff: literally a function that returns a different Agent object instead of a normal value. That one mechanism (return a different agent, and control transfers to it) is the entire basis for "a team of specialized agents passing a conversation between themselves."

Swarms (kyegomez/swarms) is the production-oriented framework people actually mean today when they say "agent swarms" — an enterprise multi-agent orchestration library built around a large catalog of prebuilt coordination structures rather than one fixed topology: SequentialWorkflow and ConcurrentWorkflow for linear and parallel execution, GraphWorkflow for agents arranged as nodes in a DAG, HierarchicalSwarm for a director agent that plans and distributes work to others, MixtureOfAgents (parallel expert agents whose outputs get synthesized into one), and GroupChat (agents that collaborate through a shared conversational interface), among others — plus a SwarmRouter that exposes all of them through a single interface. It's built to be model-agnostic across providers, so a GPT-based agent, a Claude-based agent, and a locally-hosted open-weight agent can sit inside the same swarm.

What actually motivates reaching for this many agents, concretely: a content pipeline (research → writing → editorial review → fact-checking, each stage a different agent), a business-intelligence pipeline (data collection → analysis → reporting → visualization), or a customer-service pipeline (inquiry classification → response generation → escalation → follow-up) — each one just a sequential- or parallel-execution instance of the coordination patterns already covered above, run at a scale where naming and managing each agent by hand stops being practical.

What the Research Actually Shows

Frameworks explain how to build a swarm; three real results establish why — and when — adding more agents actually helps, rather than just sounding like it should.

The null-hypothesis version, and it still works. "More Agents Is All You Need" (Li et al., 2024) tests the simplest possible swarm: sample $N$ independent responses from the same model, no roles, no communication, no coordination between agents at all — just majority voting over the outputs ("Agent Forest"). Performance still measurably scales with $N$, and — matching the intuition that a genuinely hard problem benefits more from a second opinion than an easy one does — "the degree of enhancement is correlated to the task difficulty." This is the cleanest available evidence that "just add more agents" is a real technique with a real mechanism behind it, not hand-waving: even zero coordination produces a measurable gain.

Real coordination does better, with a citable number. Mixture-of-Agents (Wang et al., 2024) adds actual structure: agents are organized into layers, and every agent in layer $N{+}1$ receives all of layer $N$'s outputs as context before producing its own response — each layer genuinely refining on the last, not just re-voting. The result is concrete enough to check yourself: MoA built entirely from open-source models scored 65.1% on AlpacaEval 2.0, beating GPT-4 Omni's 57.5% — open-source agents, coordinated, outperforming a single frontier closed model by 7.6 points.

The honest limit. Society of HiveMind (Mamie & Rao, 2025) — explicitly framed around biological swarm behavior and evolutionary theory, the most literal "swarm" of the three — reports a real, task-dependent split: a significant improvement on tasks requiring intensive logical reasoning, but negligible benefit on tasks that mainly require retrieving existing knowledge. This is the same "coordination overhead is a real cost, not a free upgrade" honesty from earlier on this page, now with the specific dividing line: swarms help you reason better collectively; they don't help you know more than any one agent already does.

Further reading: "Multi-Agent Collaboration Mechanisms: A Survey of LLMs" (Tran et al., 2025) covers the broader landscape — participating agents, interaction types, organizational structures, and coordination strategies — in one place.

Common Problems & SOTA Solutions

• Agent loses context over long tasks → explicit memory systems, periodic summarization of progress-so-far, offloading state to an external store rather than keeping everything in the context window
• Agent gets stuck in loops (repeating the same failed action) → hard step limits, reflection checkpoints that force the agent to reassess, escalation to a human when stuck
• Tool-calling hallucination (calling tools that don't exist, or with malformed arguments) → strict schema validation on tool calls, clear error messages fed back to the model so it can self-correct, retry logic with backoff
• Runaway cost/latency in multi-agent chains → caching repeated sub-results, using smaller/cheaper models for simple sub-tasks and reserving the largest model for the parts that need it, parallelizing independent sub-agent calls instead of running them sequentially

Next: Agentic Coding Assistants — a real, named, everyday application of everything above: the loop, the tool palette, and the safety/reversibility tradeoffs, in tools you're probably already using.