You've marked 0 of 6 pages in ML System Design understood. View your progress →

ML System Design — Roadmap

How to take a model and turn it into a working, scalable production system. Framed around a repeatable 9-step formula plus worked problem categories.

The 9-Step Framework

Click a step. Full depth on each lives in the 9-Step Framework.

Translate a business need into an ML problem.

Problem formulation — translate a business need into an ML problem
Metrics — offline metrics (what you optimize in dev) vs online metrics (what you measure in production, e.g. CTR, retention)
Architectural components — sketch the MVP: data → features → model → serving
Data collection & preparation — sourcing, labeling, cleaning
Feature engineering — what signals actually predict the target
Model development & offline evaluation — pick a model class, validate offline
Prediction service — batch vs online serving, latency budgets
Online testing & deployment — A/B tests, shadow deployment, canary rollout
Scaling, monitoring, and updates — drift detection, retraining cadence, rollback plans

Recommendation systems — candidate generation + ranking, cold start
Search & ranking — retrieval, learning-to-rank, relevance
News feed / ads ranking
NLP systems — classification, extraction pipelines at scale
Computer vision systems — detection/segmentation pipelines in production
GenAI / LLM systems (2026-era) — RAG pipeline design, agent system design, LLM serving infra

Model categories and where each is covered on this site
The model card attribute checklist (architecture, params, context, license, benchmarks, hardware, and more)
Benchmark dimensions by system type: inference, retrieval, general quality

Training-serving skew

→

Shared feature pipelines, feature stores

Cold start

→

Content-based fallback, exploration strategies

Feedback loops biasing the model

→

Randomized exploration, counterfactual evaluation

Model staleness / concept drift

→

Monitoring dashboards, scheduled retraining, online learning

Scaling inference to millions of requests

→

Caching, batching, model distillation, horizontal scaling

5 recurring failure modes, 5 SOTA fix patterns.

Click a problem -- every one of these eventually shows up in a production ML system regardless of domain; the fix pattern is what's reusable.

Training-serving skew → shared feature pipelines, feature stores
Cold start (new users/items) → content-based fallback, exploration strategies
Feedback loops biasing the model → randomized exploration, counterfactual evaluation
Model staleness / concept drift → monitoring dashboards, scheduled retraining, online learning
Scaling inference to millions of requests → caching, batching, model distillation, horizontal scaling