Machine Learning — Roadmap

Classical ML: the algorithms, the workflow around them, and the problems that show up constantly in practice.

Not sure which algorithm fits your problem? Answer a few questions:

1. ML Workflow Fundamentals

• Supervised vs unsupervised vs semi-supervised vs self-supervised learning
• Train / validation / test splits, cross-validation
• Data leakage — what it is and how it silently ruins models
• Feature engineering & feature selection
• Handling missing data, outliers, imbalanced classes
• Sampling strategies (stratified, bootstrap, SMOTE)

2. Supervised Learning

• Linear & logistic regression (and their assumptions)
• Decision trees, Random Forests, Gradient Boosting (XGBoost, LightGBM)
• Support Vector Machines
• k-Nearest Neighbors
• Naive Bayes
• Ensemble methods: bagging vs boosting vs stacking
• Survival analysis: Kaplan-Meier, Cox Proportional Hazards, censoring
• Recommender systems: matrix factorization, two-tower embeddings, retrieval->rank->re-rank
• Learning-to-rank: pointwise/pairwise/listwise, NDCG/MRR

3. Unsupervised Learning

• k-Means, hierarchical clustering, DBSCAN
• Dimensionality reduction: PCA, t-SNE, UMAP
• Anomaly detection

4. Model Evaluation & Metrics

• Bias-variance tradeoff, underfitting vs overfitting
• Regularization: L1, L2, dropout, early stopping
• Classification metrics: precision, recall, F1, ROC-AUC, PR-AUC
• Regression metrics: MSE, MAE, R²
• Calibration, threshold optimization (cost-based, not just default 0.5)
• Statistical significance of model improvements

4.5. Hyperparameter Optimization

• Grid vs. random search -- the real distinct-values-tried argument, not just intuition
• Bayesian optimization: Gaussian Process surrogate, the real Expected Improvement formula
• Successive halving and Hyperband

4.6. Uncertainty Estimation & Conformal Prediction

• Prediction intervals (quantile regression) and ensemble-based uncertainty
• Conformal prediction: the real score-function/calibration-quantile recipe and the real coverage theorem
• Why conformal prediction's guarantee holds regardless of model quality -- and what that does and doesn't buy you

5. Common Problems & Their State-of-the-Art Solutions

• Class imbalance → resampling, class weighting, focal loss
• Overfitting on small data → regularization, data augmentation, transfer learning
• Concept drift in production → monitoring, periodic retraining, online learning
• Curse of dimensionality → feature selection, PCA, regularization
• Slow training on huge tabular data → gradient boosting libraries (XGBoost/LightGBM), distributed training
• Explainability → SHAP, LIME, feature importance