LLM / Agent Frameworks

The tools that implement the concepts from LLMs & GenAI and Agents so you're not wiring up RAG pipelines and tool-calling loops from scratch every time.

When You Need a Framework at All

Worth asking before reaching for any of these: HuggingFace's Agents Course puts it plainly — an agentic framework isn't always needed. If the flow is simple (a chain of prompts), plain code gives full control with no abstraction to learn; frameworks earn their cost once the workflow gets complex enough (function-calling loops, multiple agents) that managing it by hand becomes error-prone. Underneath any of the frameworks below, the same real components recur: an LLM engine, a list of tools, a parser for extracting tool calls from the model's output, a system prompt kept in sync with that parser, a memory system, and error-logging/retry for when the model gets a tool call wrong.

LangChain

A framework for composing LLM applications out of reusable building blocks: chains (sequences of steps — prompt → LLM call → parse output → next step), retrievers (a standard interface over any vector store or search backend, see RAG), and agents (LLM-driven loops that decide which tool to call, implementing patterns like ReAct). Its value is standardizing the plumbing — swapping an LLM provider, a vector database, or a retrieval strategy without rewriting application logic.

LlamaIndex

Started as, and is still strong at, the data indexing and retrieval side of RAG — connecting to many data sources (documents, APIs, databases), chunking them (see RAG — Chunking Strategies), building indexes over them, and providing query interfaces on top. But it's also a real agent framework in its own right, not just a retrieval layer: FunctionAgent for straightforward tool-calling, AgentWorkflow for coordinating multiple agents, plus ReActAgent and CodeActAgent for different reasoning styles. A tool is just a typed, docstring-described Python function:

def multiply(a: float, b: float) -> float:
    """Multiply two numbers and returns the product"""
    return a * b

from llama_index.core.agent.workflow import FunctionAgent
workflow = FunctionAgent(tools=[multiply], llm=llm, system_prompt="...")
response = await workflow.run(user_msg="What is 20*4?")

The natural fit when an agent's main job really is connecting to and reasoning over many data sources — the same strength the RAG side already has.

Five More, Each on a Different Axis

Beyond LangChain/LangGraph and LlamaIndex, five more frameworks show up often enough to be worth knowing specifically — not as a longer feature checklist, but because each one is genuinely built around a different core idea, not a variation on the same one:

smolagents

Code-as-action

Microsoft Agent Framework

Production successor to AutoGen

Google ADK

Graph-based workflow runtime

DSPy

Programs, not prompts

LlamaIndex (as an agent framework)

Data-connected agents

HuggingFace's framework: the agent writes and executes real Python as its action, rather than emitting a JSON tool call that gets dispatched to a function. Its own README frames the distinction precisely: "agents being used to write code" (most frameworks) vs. "agents that think in code" (smolagents). Sandboxed execution via Docker, E2B, or Modal for the obvious reason.

smolagents (HuggingFace) writes its actions as executable code rather than JSON tool calls — its own docs put it as "agents that think in code," not "agents used to write code." The whole agent loop fits in roughly 1,000 lines; sandboxing (Docker, E2B, Modal) matters more here specifically because the action really is arbitrary code execution, not a fixed set of pre-registered functions.

Microsoft Agent Framework is what to reach for where you might expect AutoGen — AutoGen is officially in maintenance mode, and Microsoft's own migration guide points existing users at Agent Framework instead. It adds the things AutoGen didn't have: graph-based orchestration patterns (sequential, concurrent, handoff, group), checkpointing and time-travel, built-in OpenTelemetry tracing, and first-class .NET alongside Python — aimed specifically at the prototype-to-production gap.

Google ADK (Agent Development Kit) is a code-first, graph-based workflow runtime — routing, fan-out/fan-in, loops, and retry as explicit Workflow primitives, plus a Task API for structured agent-to-agent delegation. Model-agnostic, optimized for Gemini, with a local dev UI (adk web) for interactively testing an agent before deploying it.

DSPy (Stanford NLP) sits on a different axis from all four of the others: it isn't about orchestrating multi-step control flow, it's about not writing prompts by hand at all. Declare a typed Signature, run it through a Module (Predict, ChainOfThought, ReAct), and an optimizer — a "teleprompter" — tunes the actual prompt text against a metric on real training data:

class ExtractEvent(dspy.Signature):
    """Extract event details from an email."""
    email: str = dspy.InputField()
    event_name: str = dspy.OutputField()
    date: str = dspy.OutputField()

extract = dspy.Predict(ExtractEvent)
optimizer = dspy.GEPA(metric=accuracy, auto="medium")
optimized = optimizer.compile(extract, trainset=labeled_emails)

The prompt becomes a compiled, versioned artifact instead of a string you hand-edit — genuinely useful once a prompt has enough labeled examples to optimize against, and genuinely unnecessary before that point.

MCP SDKs

Building an MCP server — exposing your own tools/data to any MCP-compatible client — is done with an official SDK (Python or TypeScript). The SDK handles the protocol details (the message format, the connection lifecycle) so you just define your tools' logic and let the SDK expose them correctly.

Agent Orchestration Frameworks

LangGraph (built on LangChain) models an agent's control flow as an explicit graph of nodes and edges — useful when an agent's logic is more complex than a simple loop (conditional branches, cycles, human-in-the-loop checkpoints), and you want that structure to be visible and debuggable rather than implicit in prompt text.

CrewAI is built specifically around the multi-agent pattern of assigning different agents distinct roles (a "researcher," a "writer") and having them collaborate on a shared task — a higher-level abstraction than LangGraph, trading some flexibility for faster setup of common multi-agent patterns.

A simplified but real LangGraph-style control-flow graph — step through it and watch a genuine cycle (looping back to Act when confidence hasn't cleared the threshold yet) and a genuine conditional branch (Human review only when required):

Confidence threshold0.75

Human approval required

Click Step to run the agent through the graph one transition at a time.

Choosing between them: for a single agent with a fairly linear flow, you often don't need any of these — a direct loop calling the LLM API is simpler and easier to debug. Reach for an orchestration framework once the control flow itself (not just the prompting) becomes complex enough that managing it by hand gets error-prone. See also Agentic Coding Assistants for a specific, real application built on top of this same family of ideas.

References

HuggingFace Agents Course, Unit 2 — the "when do you need a framework at all" framing and the real component checklist (LLM engine, tools, parser, system prompt, memory, error handling) referenced above.
smolagents — HuggingFace's code-as-action agent framework.
Microsoft Agent Framework and AutoGen — AutoGen's own README states it's in maintenance mode and points new users to Agent Framework.
Google ADK (adk-python) — Agent Development Kit.
DSPy — programming, not prompting, language models.
LlamaIndex agent docs — FunctionAgent/AgentWorkflow and LlamaIndex's agent-building APIs.

Next: Serving & MLOps/LLMOps — getting any of this into production reliably.