Inference
Model serving
Model serving exposes a trained model through a reliable runtime interface.
How it works
A serving system handles request validation, scheduling, execution, observability, and failure recovery. Model quality alone does not provide these operational guarantees.
A concrete example
Illustrative: 4 replicas each sustain an illustrative 25 requests per second, giving 100 requests per second before coordination overhead.
Do not confuse it with Deployment
deployment installs a release; serving is the runtime system answering requests.