Machine Learning Engineer Interview Questions and Answers

Questions and model answers

9 questions you are likely to hear.

1. 1

   Design a recommendation system for an ecommerce homepage.

   Why they ask: This is the most common ML system design question and tests end-to-end thinking.

   Sample answer: I would use a two-stage design. Candidate generation retrieves a few hundred items per user using collaborative filtering embeddings, recent views and popular items, served through an approximate nearest neighbour index. A ranking model, such as gradient boosting or a small neural network, scores candidates using user, item and context features from a feature store, optimising for click and purchase probability. Business rules handle diversity and out-of-stock items. Ranking must return in under 100 milliseconds, so features are precomputed. I would evaluate offline with NDCG and recall at k, then online with an A/B test on revenue per session, and retrain daily.

2. 2

   What is training-serving skew and how do you prevent it?

   Why they ask: Skew is a frequent cause of models underperforming after deployment.

   Sample answer: Training-serving skew happens when features or preprocessing differ between training and production, so the model sees different inputs live than it learned from. Causes include separate code paths for offline and online feature computation, different data sources, and time leakage in training data. I prevent it by sharing feature logic through a feature store or a single library used by both pipelines, using point-in-time correct joins for training data, and logging served features to compare against training distributions. On a pricing model, logging served features revealed a currency conversion bug in the online path.

3. 3

   How do you monitor a model in production?

   Why they ask: Interviewers want to see that you treat models as living systems.

   Sample answer: I monitor at three levels. Operational metrics include latency, error rates and throughput. Data metrics track input feature distributions and missing values against the training baseline, using measures like population stability index. Model metrics track prediction distributions and, when labels arrive, actual performance such as precision or error. Alerts go to the team when thresholds are crossed, and dashboards show trends by segment. For a credit risk model, label delay was about 60 days, so prediction drift and feature drift were our early warnings, and we scheduled regular retraining reviews.

4. 4

   How would you reduce inference latency for a deep learning model?

   Why they ask: Serving performance is core to ML engineering.

   Sample answer: I first profile to see whether time goes to preprocessing, model compute or network. For the model, I try quantisation to INT8 or FP16, exporting to ONNX or TensorRT, and distillation into a smaller model if accuracy allows. On the serving side, I use dynamic batching with Triton or a similar server, keep the model warm and close to the caller, and cache results for repeated inputs. For a text classification service, ONNX Runtime with INT8 quantisation cut p95 latency from about 120 milliseconds to under 40 with a negligible accuracy drop.

5. 5

   When would you fine-tune an LLM instead of using retrieval-augmented generation?

   Why they ask: This tests current, practical judgement on generative AI architectures.

   Sample answer: I use RAG when the model needs up-to-date or private knowledge, such as company documents, because I can update the index without retraining and cite sources. I consider fine-tuning when I need consistent style, a specific output format or better performance on a narrow task where prompting falls short, and when I have enough quality examples. Often the best answer is both: RAG for knowledge and light fine-tuning, such as LoRA, for format and tone. I always start with a strong prompt and retrieval baseline, and measure against an evaluation set before investing in fine-tuning.

6. 6

   How do you set up CI/CD for machine learning models?

   Why they ask: MLOps maturity is a key differentiator for ML engineers.

   Sample answer: Code changes run unit tests for data processing and feature logic, plus a small training run on sample data. Training pipelines are versioned and reproducible, logging data versions, parameters and metrics to MLflow. A candidate model is registered and must pass evaluation gates against the current production model on a fixed test set and key segments. Deployment goes first to a shadow or canary environment, comparing predictions and latency with production, before full rollout. Rollback is a registry version change. This setup let my team move from quarterly to weekly model updates safely.

7. 7

   How do you handle data drift versus concept drift?

   Why they ask: Understanding drift types determines the right response.

   Sample answer: Data drift is a change in the input distribution, like a new customer segment appearing, while the relationship between inputs and outcome may stay the same. Concept drift is when that relationship itself changes, such as fraud patterns evolving. Data drift can be detected immediately from inputs; concept drift often needs labels. For data drift I check whether performance is actually affected before retraining. For concept drift, I retrain on recent data, sometimes weight recent samples more, and review features. In a fraud system, we retrained weekly because fraud tactics changed quickly.

8. 8

   Write a Python function to compute precision and recall from predictions without libraries.

   Why they ask: Coding rounds check fundamentals and clean code.

   Sample answer: I would loop through paired true labels and predicted labels, counting true positives where both are 1, false positives where prediction is 1 and truth is 0, and false negatives where prediction is 0 and truth is 1. Precision is true positives divided by true positives plus false positives, and recall is true positives divided by true positives plus false negatives. I would guard against division by zero by returning 0 when the denominator is 0, and validate that both lists have the same length. I would mention that for large arrays I would vectorise this with NumPy.

9. 9

   Tell me about an ML system you took to production.

   Why they ask: This reveals real-world experience and ownership.

   Sample answer: At a logistics startup, I built a delivery time prediction service. I worked with data engineers on features like distance, traffic patterns and restaurant prep time, trained a LightGBM model and tracked experiments in MLflow. I served it with FastAPI on Kubernetes with autoscaling, keeping p95 latency under 50 milliseconds, and added monitoring for feature drift and prediction error. After an A/B test, the more accurate estimates reduced customer complaints about late deliveries noticeably. I also set up weekly retraining through Airflow, so the model adapted to seasonal patterns.

Keep going.

• Data Scientist Interview Questions and Answers
• Python Developer Interview Questions and Answers
• Data Engineer Interview Questions and Answers
• Machine Learning Job Support Online for ML Engineers
• System Design Interview Basics
• AI Copilot for Technical Interviews