APIs, HTTP & Communication Patterns

An API (Application Programming Interface) is a contract: a defined way for one piece of software to ask another for something, without needing to know how the other side is implemented internally. That definition sounds abstract until you notice how much of modern AI engineering is API calls, end to end:

• Calling an LLM (Anthropic, OpenAI, any hosted model) is an API call, and its streaming response is a specific HTTP communication pattern (below), not magic.
• An agent's tool use is, mechanically, the model producing a request that your code turns into an API call.
• MCP — the protocol that lets an agent talk to tools — runs over HTTP as one of its two standard transports.
• A model-serving endpoint you deploy (see Model Serving) is a web API you write, and the framework you write it in is almost certainly the one covered at the end of this page.

This page is the foundation those four things build on: how HTTP actually works, how REST structures an API around it, how authentication and authorization work, the handful of communication patterns that cover every way two systems exchange data over a network, and FastAPI — the framework you'll actually use to build one of these.

The Request/Response Cycle

Every HTTP interaction is a client sending a request and a server sending back a response — the same basic shape whether it's a browser loading a page or your code calling an LLM API:

A request has a method (what kind of action), a URL (what resource), headers (metadata — content type, auth token, etc.), and an optional body (the actual payload, e.g. your prompt as JSON). A response has a status code (did it work, and how), its own headers, and a body (the result). POST /v1/messages with your prompt in a JSON body, getting back a 200 with the model's reply in JSON, is this exact cycle — nothing more exotic is happening underneath an LLM API call.

That exact cycle, as a real request:

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model": "<model-id>", "max_tokens": 256, "messages": [{"role": "user", "content": "Explain HTTP in one sentence."}]}'

HTTP Methods

• GET: retrieve a resource, no side effects. Safe to retry, safe to cache.
• POST: create something, or trigger an action that has side effects (like generating a completion). Not safe to blindly retry — retrying a POST can create a duplicate.
• PUT: replace a resource entirely with what you send.
• PATCH: partially update a resource — send only the fields that changed.
• DELETE: remove a resource.

Idempotency — whether calling an operation twice has the same effect as calling it once — is the property that actually matters in practice: GET, PUT, and DELETE are idempotent (safe to retry on a network timeout, since you're not sure if the first attempt landed); POST generally is not (retrying a payment or a completion request can duplicate it), which is exactly why production API clients need explicit retry logic that accounts for this, not blanket "retry on failure."

Status Codes

The families matter more than memorizing every code: 2xx means it worked, 3xx means go look elsewhere, 4xx means your request was the problem (and is worth fixing before retrying — 400 malformed request, 401 missing/invalid auth, 403 authenticated but not allowed, 404 not found, 429 rate limited), 5xx means the server's problem (worth retrying with backoff, since it might be transient). Every LLM API integration needs to handle 429 gracefully — it's not an edge case, it's the normal shape of using a shared, rate-limited service under load.

import httpx, time

def call_with_backoff(payload: dict, max_attempts: int = 5) -> dict:
    for attempt in range(max_attempts):
        response = httpx.post(API_URL, json=payload, headers=headers)
        if response.status_code == 429:
            retry_after = float(response.headers.get("retry-after", 2 ** attempt))
            time.sleep(retry_after)
            continue
        response.raise_for_status()   # 4xx/5xx other than 429 -- fail loudly, don't silently swallow
        return response.json()
    raise RuntimeError("exhausted retries against a rate-limited endpoint")

REST: Structuring an API Around Resources

REST (Representational State Transfer) is the dominant convention for structuring HTTP APIs: model your data as resources identified by URLs (/users/123, /conversations/456/messages), and act on them using the standard HTTP methods above rather than inventing a custom verb per action:

This is a convention, not a protocol enforced by anything — it's worth briefly contrasting with the alternatives it's usually chosen over: RPC-style APIs (/createUser, /sendMessage) model actions directly as function calls instead of resources, which can be more natural for operations that don't map cleanly to CRUD. Most model-serving and AI-tooling APIs (including this site's own examples) are REST or REST-adjacent, which is why it's the focus here — but two real alternatives are common enough in production to be worth understanding precisely, not just by name.

REST vs. GraphQL vs. gRPC

GraphQL exposes a single endpoint and lets the client specify exactly which fields it needs in one query, instead of the server dictating a fixed response shape per endpoint. This directly fixes two real REST failure modes: over-fetching (an endpoint returns a whole object when the client needed one field from it) and under-fetching (rendering one screen needs data from three separate REST endpoints, so the client pays three round trips). The cost isn't free: because every query can request an arbitrary shape, a naive resolver is prone to the N+1 query problem — fetching a list of $N$ items, then issuing one additional backend query per item to resolve a nested field, turning what looked like one request into $N{+}1$ real ones underneath (solved with request-scoped batching, e.g. the DataLoader pattern, but it's a real implementation burden REST's fixed-shape endpoints don't have). GraphQL also breaks HTTP-level caching cleanly, covered below.

gRPC is the production-grade version of the RPC-style approach mentioned above: a strict, code-generated contract instead of a loose one. Services and message types are defined once in a .proto file, and the protoc compiler generates typed client and server code directly from it, so both sides agree on every call's exact shape before either is written. Messages serialize as Protocol Buffers — a compact binary format — rather than REST's typically-JSON text, meaning smaller payloads and cheaper (de)serialization at high request volume. Built on HTTP/2, gRPC supports four call shapes as first-class parts of the protocol itself: unary (one request, one response, like REST), server streaming, client streaming, and bidirectional streaming — real two-way streaming built in, not layered on top the way SSE/WebSockets sit on top of HTTP for REST. The real tradeoff: browsers can't natively speak gRPC (it needs a proxy layer, grpc-web, to be called from client-side JavaScript), so it's overwhelmingly used for internal service-to-service calls inside a backend, not a public API a browser calls directly — the opposite deployment target from REST and GraphQL, which both work natively with a browser's fetch().

REST caching, made concrete. GET being cacheable (stated above) is a quantifiable performance lever, not just a nice property — the improvement comes from a simple weighted average. If a fraction $h$ of requests are served from a cache (a CDN/edge cache, or an in-process one) instead of reaching the origin server, average latency is:

$\bar{L} = h \cdot L_{\text{hit}} + (1-h) \cdot L_{\text{miss}}$

Plugging in illustrative but realistic numbers — an edge cache hit at $L_{\text{hit}} \approx 5\text{ms}$, versus a cache miss that reaches the origin and queries a database at $L_{\text{miss}} \approx 150\text{ms}$:

• No caching ($h=0$): $\bar L = 150\text{ms}$.
• A realistic hit ratio for read-heavy, mostly-stable data ($h=0.8$): $\bar L = 0.8(5) + 0.2(150) = 34\text{ms}$ — a 4.4× reduction in average latency from caching alone.
• Aggressive caching of very stable data ($h=0.95$): $\bar L = 0.95(5) + 0.05(150) = 12.25\text{ms}$ — over 12× faster on average than serving every request uncached.

This is exactly why REST's fixed, URL-identified resources (GET /users/123) are so cache-friendly: an HTTP cache (browser, CDN, reverse proxy) keys directly on the URL, and Cache-Control/ETag response headers tell it how long a cached copy stays valid and how to revalidate without re-fetching the whole body. GraphQL's single POST /graphql endpoint breaks this cleanly — every query is a distinct POST body, not a distinct cacheable URL — which is exactly why GraphQL clients (Apollo Client, Relay) need their own normalized client-side caching machinery instead of getting HTTP caching for free the way REST does.

Authentication vs. Authorization

Two questions that get conflated constantly, and shouldn't be — they're answered by different mechanisms and fail in different ways:

Authentication ("who are you?") establishes identity — an API key, a username/password, a signed token. Authorization ("what are you allowed to do?") is a separate decision made after identity is established — an authenticated user can still be authorized to read but not delete, or scoped to only their own resources. A system can authenticate perfectly and still be badly broken if authorization is missing (any logged-in user can delete any other user's data) — this is a real, common vulnerability class, not a theoretical distinction.

API Keys and OAuth 2.0

The simplest authentication mechanism is an API key — a long secret string sent with every request (typically in a header), proving the caller is who they claim to be. It's what most LLM provider APIs use directly. For delegated access — "let this third-party app act on my behalf, without giving it my password" — OAuth 2.0 is the standard:

The authorization-code flow: the user is redirected to the provider (e.g. "Sign in with Google") and approves access there, the provider redirects back with a short-lived authorization code, and the requesting app exchanges that code (server-to-server, using its own client secret) for an access token it can then use to call the API on the user's behalf. The user's password is never seen by the requesting app at any point.

JWTs

A JSON Web Token is a common way to represent an access token itself — a compact, self-contained, cryptographically signed blob that carries claims (who this is, what they're allowed to do, when it expires) that any server can verify without a database lookup:

Three base64url-encoded parts separated by dots: a header (which signing algorithm), a payload (the actual claims — user ID, roles, expiry), and a signature (proof the token wasn't tampered with, computed over the header and payload using a secret or private key only the issuer holds). Critically: the payload is only encoded, not encrypted — anyone can decode and read a JWT's contents, they just can't forge a valid signature for altered contents without the signing key. Never put secrets in a JWT payload expecting them to stay hidden.

Communication Patterns

This is the part that determines how two systems actually stay in sync over time, and it's exactly the layer LLM streaming, webhooks, and real-time agent systems all live in. Five patterns cover essentially every real design:

• Synchronous request/response: the basic cycle above — ask, wait, get one answer. Simple, but the client has to block or poll for anything that takes a while.
• Short polling: the client repeatedly asks "is it ready yet?" on a timer. Simple to implement, wasteful (most requests get "not yet"), and adds latency up to one poll interval.
• Long polling: the client asks once, and the server holds the request open until there's something to report (or a timeout), then the client immediately reopens it. Less wasteful than short polling, still fundamentally a pull.
• Webhooks: inverts the control flow entirely — instead of the client asking, the server calls the client (a URL you registered in advance) the moment something happens. No polling at all, but requires your client to be a reachable server itself, and you have to handle retries/ordering/verification (was this request really from who it claims to be) on the receiving end.
• Server-Sent Events (SSE): one persistent HTTP connection, server streams events to the client as they happen, one direction only (server → client). This is exactly what LLM token streaming is — every token arriving as it's generated, over one open connection, is SSE (or a close variant) under the hood, at both Anthropic and OpenAI.
• WebSockets: a persistent, full-duplex connection — both sides can push messages at any time. More powerful than SSE, but more complex to set up and operate (stateful connections, needs its own scaling story) — the right choice specifically when the client also needs to push updates mid-stream, not just receive them (voice/realtime agent sessions being a common AI use case).

Picking among these is a real design decision, not a default: if you just need occasional updates and simplicity beats latency, poll. If you're building the receiving side of a third-party integration (payment confirmations, CI status), webhooks are usually the provider's only option and you build to receive them. If you're streaming a model's output token by token, SSE is the standard answer — it's simpler than WebSockets and the traffic is one-directional anyway. If the client needs to interrupt, send audio, or otherwise talk back mid-stream, WebSockets earns its extra complexity.

Building APIs with FastAPI

FastAPI is the dominant Python framework for building exactly this kind of API — serving models, building MCP servers, agent backends — because it's built around the same tools this whole page has covered: Pydantic models for request/response validation, native async support for the I/O-bound workloads AI serving involves, and automatic OpenAPI documentation generated from your code rather than written by hand.

A path operation is a Python function decorated with the HTTP method and URL it handles (@app.post("/v1/generate")). Its parameters can be typed with Pydantic models describing the expected request body — FastAPI validates incoming JSON against that model automatically and returns a clean 422 if it doesn't match, before your function body even runs; the return value is validated against a response model the same way, so what you send back is guaranteed to match what you documented. Endpoints declared async def let FastAPI handle many concurrent in-flight requests on one process without blocking — the right default for endpoints that spend most of their time waiting on a model call or a downstream API, not burning CPU.

Dependency Injection

FastAPI's Depends() mechanism lets a path operation declare "I need this" (a database session, the current authenticated user, a rate limiter check) and have FastAPI resolve and inject it automatically — including sharing one resolved dependency across multiple routes that need the same thing:

This is where authentication actually gets enforced in a real FastAPI service: a get_current_user dependency decodes and validates the JWT/API key from the request, and any route that declares it as a dependency automatically requires valid auth — the authentication vs. authorization distinction from earlier, implemented as one reusable, testable function instead of copy-pasted checks in every handler.

Everything above, together, in one real endpoint -- a Pydantic request model, a Depends()-enforced auth check, and streaming the response back over Server-Sent Events:

from fastapi import FastAPI, Depends, Header, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel

app = FastAPI()

class GenerateRequest(BaseModel):
    prompt: str

def get_current_user(authorization: str = Header(...)) -> str:
    token = authorization.removeprefix("Bearer ")
    if not is_valid_token(token):
        raise HTTPException(status_code=401, detail="invalid or expired token")
    return decode_user_id(token)

@app.post("/v1/generate")
async def generate(req: GenerateRequest, user_id: str = Depends(get_current_user)):
    async def token_stream():
        async for token in model.stream(req.prompt):
            yield f"data: {token}\n\n"   # the SSE wire format -- "data: <payload>\n\n" per event
    return StreamingResponse(token_stream(), media_type="text/event-stream")

Where This Shows Up Elsewhere on This Site

• MCP (Model Context Protocol) uses HTTP with Server-Sent Events as one of its two standard transports — the streaming pattern above, applied to tool calls instead of chat tokens.
• Every LLM provider's streaming chat completion — the token-by-token output you see in a chat UI — is Server-Sent Events over the request/response cycle covered above.
• Model Serving covers what sits behind the API you now understand the shape of: batching, autoscaling, and the infrastructure serving these endpoints at scale.
• LLM Inference Optimization covers TTFT (time-to-first-token) and inter-token latency — metrics that only make sense once you know these are being measured over a streaming SSE connection, not a single request/response round trip.

Next: Linux, Git & Developer Tooling — the daily tools every AI engineering job assumes fluency in.