Inference
Prefill
Prefill processes all prompt tokens to initialize attention state before decoding.
How it works
The phase can exploit parallel computation across the prompt. Its workload and bottlenecks differ from one-token-at-a-time decode.
A concrete example
Illustrative: a 4,096-token prompt is processed in one prefill phase before token 4,097 is sampled.
Do not confuse it with Decode
prefill processes the prompt in parallel; decode extends the sequence incrementally.