LLM Inference Flow Visualizer
The full text-to-token pipeline as a clickable graph, from raw text through tokenization, embeddings, the Transformer block stack, and out through the LM head to a sampled token fed back in autoregressively. The sampling playground at the bottom is real math, not a mockup: real softmax with temperature scaling, real top-k truncation, real top-p (nucleus) truncation, over a small fixed set of next-token candidates.
Interactive
LLM Inference Flow Visualizer
Sampling: Turns the probability distribution into one actual next token -- temperature, top-k, and top-p all shape exactly how. Try the playground below.
Sampling playground
"the cat sat on the ___" -- next-token candidates and their logits
mat69.2%
chair22.5%
table8.3%
floor (cut)0.0%
roof (cut)0.0%
moon (cut)0.0%
Click any stage of the pipeline for what it does, and use the sampling playground to see how temperature / top-k / top-p actually reshape a probability distribution -- real softmax and truncation math over a small fixed candidate set.
What to Try
- Click through every node top to bottom, including the four nodes inside the dashed loop-back arrow — that loop is the Transformer Block × N repeating, not four separate one-time steps.
- Set Temperature near 0.1 and watch the distribution collapse almost entirely onto "mat" (the highest-logit candidate) — low temperature makes the model nearly deterministic. Push it toward 2.0 and watch the distribution flatten out instead.
- Set Top-k to 1 — no matter what temperature or top-p are set to, only one token can ever be sampled. This is exactly greedy decoding.
- Bring Top-p down to ~0.5 and watch how many candidates get cut varies with temperature — top-p's candidate set size adapts to how peaked the distribution already is, which is why it's generally preferred over a fixed top-k in production.
- Notice the autoregressive feedback arrow from Token back to Embeddings — the newly sampled token doesn't re-enter through the tokenizer, it's already a token ID; it just needs a fresh embedding lookup before flowing through the block stack again for the next token.
Back to Visual Lab Overview.