Classical ML Interview Questions

A decision tree with no depth limit scores 99% on training data and 71% on validation. What do you try first?

A 10-class image classifier gets 64% on training data and 63% on validation. People score 95% on the same task. What's the problem?

You run 5-fold cross-validation on 10,000 rows. How many rows does each of the five models train on?

A demand forecast scores 0.92 R² with random 5-fold cross-validation and 0.55 in its first month in production. What's the most likely cause?

In a churn model, the most important feature by far is account_closure_reason. Validation AUC is 0.99. What do you conclude?

A colleague oversamples the minority class with SMOTE, then splits the data into train and test. Test recall is excellent. What's wrong?

Out of 1,000 transactions, 20 are fraud. Your model flags 50 transactions, and 15 of those are fraud. What is its precision?

Same model: 20 frauds in the data, 15 of them flagged. What is its recall?

Same model, precision 0.3 and recall 0.75. What is its F1 score? (Three decimals is fine.)

1% of users are fraudulent and your model has 99% accuracy. What should you report instead?

A test catches 99% of a rare condition and wrongly flags 1% of healthy people. The condition affects 1 in 1,000. If someone tests positive, what's the probability they have it? (As a decimal.)

The fraud team can review 200 cases a day. How do you choose the model's threshold?

Your model has a ROC-AUC of 0.95, and its predicted probabilities will be used to set insurance prices. What else do you check?

A model predicts 0.9 for an example that turns out positive. What is the log loss for that one prediction? (Natural log.)

You have 500 features and suspect only about 20 matter. You want a linear model that uses few of them. Which regularisation?

Your features range from 0–1 (a ratio) to 0–1,000,000 (income). Which of these models doesn't care about the difference in scale?

A random forest's built-in feature importance ranks zip_code (30,000 distinct values) at the top. Should you believe it?

k-nearest neighbours works well on 5 features but poorly once you add 195 more mostly-irrelevant ones. Why?

Two features in a linear regression have a correlation of 0.98. Their coefficients flip sign between training runs on resampled data. What's happening?

Income is missing for 30% of loan applicants, and applicants with missing income default twice as often. How do you handle it?

A model's real-world performance seems to have dropped, but true labels arrive 30 days late. What can you monitor today?

A feature has two buckets. In training, 30% of rows fall in bucket A and 70% in B. This week it's 70% A and 30% B. What's the population stability index (PSI)? (Natural log.)

An A/B test: control converts 10% of 5,000 users, the variant converts 11% of 5,000. What's the two-proportion z-score? (Two decimals.)

A product manager checks the A/B dashboard every morning and plans to stop the test the first day p is below 0.05. What's the problem?