AI & Data Interview questions

MLOps / ML Platform Engineer interview questions

These MLOps Engineer interview questions come from our vetting, where 3.4% of applicants pass 5 gates. You get 20 questions with answer hints, a task, red flags and a scorecard.

Updated: October 2026Remote Pro Ai · AI & Data20 questions

Build my squad

How we assess

Every applicant goes through 5 gates: application, an AI-assisted screen, a live technical assessment scored by a practice lead, a client-fit interview, and reference and credential checks. The questions below come from the live assessment. We score answers on reasoning and trade-offs, not on recalling definitions.

20 technical questions with model-answer hints

Questions 1–⁠10 (role-specific)

  1. What does a model CI/CD pipeline test that an app pipeline does not?

    Model-answer hint

    data validation, reproducible training, evaluation thresholds, bias checks, model signatures.

  2. How do you share GPUs fairly across teams?

    Model-answer hint

    quotas, priority classes, dedicated node pools, job queueing, preemption rules.

  3. Why use a feature store, and when is it overkill?

    Model-answer hint

    train-serve consistency and point-in-time correctness; overkill for a few simple models.

  4. How do you reproduce a model trained 6 months ago?

    Model-answer hint

    versioned code, data snapshot, environment image, registry metadata, lineage.

  5. Canary, shadow or A/B: when do you use each for a model?

    Model-answer hint

    shadow for risk-free comparison, canary for gradual exposure, A/B for business impact.

  6. How do you autoscale bursty inference traffic?

    Model-answer hint

    scale on concurrency or queue depth, warm pools, GPU sharing, cold-start trade-offs.

  7. What triggers retraining, and who approves it?

    Model-answer hint

    drift or performance thresholds, or schedule; automated pipeline with a human approval gate.

  8. How do you attribute and cut inference cost?

    Model-answer hint

    per-model cost tagging, batching, quantisation, right-sized instances, spot capacity for training.

  9. How do you secure a model registry?

    Model-answer hint

    RBAC, signed and immutable artefacts, encryption, audit logs.

  10. How do you release a prompt change for an LLM app safely?

    Model-answer hint

    versioned prompts, offline eval gate, staged rollout, quality monitoring, instant rollback.

Questions 11–⁠20 (AI & Data core)

  1. How do you decide between rules, classic ML and an LLM for a problem?

    Model-answer hint

    start from decision value and error cost; choose the simplest option that meets the metric.

  2. How do you detect training-serving skew?

    Model-answer hint

    shared feature code, logged inference inputs, distribution checks between training and live data.

  3. How do you build an evaluation set that predicts production results?

    Model-answer hint

    representative sampling, slice coverage, leakage checks, labelled with measured agreement, refreshed on a schedule.

  4. What goes on a model-health dashboard, and who acts on each signal?

    Model-answer hint

    quality, drift, latency, cost and the business KPI, each with an owner and threshold.

  5. How do you handle personal data in a training pipeline under PDPL or GDPR?

    Model-answer hint

    minimisation, lawful basis, pseudonymisation, residency, retention limits, DPIA.

  6. How do you roll back a model or prompt change that degrades results?

    Model-answer hint

    versioned registry, canary or shadow release, automated gates, one-step rollback.

  7. When does fine-tuning beat retrieval or better prompting?

    Model-answer hint

    behaviour, format and domain style favour tuning; fresh facts favour retrieval; weigh data and cost.

  8. Describe a data-quality incident you caught.

    Model-answer hint

    how it was detected, root cause, the contract or test added, the alert that now exists.

  9. How do you forecast and cap the monthly cost of an AI workload?

    Model-answer hint

    tokens or GPU hours, caching, batching, model routing, budgets with alerts.

  10. Explain a modelling trade-off you made to a non-technical executive.

    Model-answer hint

    plain language, business impact, the option rejected and why.

Interviewer tip: Ask for a real example after every answer. Strong candidates describe what they built, what broke and what they changed.

Practical task

Take a trained scikit-learn model and sample data, containerise it and deploy it to a local Kubernetes cluster behind an autoscaled endpoint. Register it in MLflow, then add a drift check that fails the pipeline when feature distributions shift past a threshold.

How to run it Share the brief 24 hours ahead or run it live. Ask the candidate to talk through trade-offs as they work; reasoning matters more than a finished artefact.

Red flags our practice leads watch for

  • Treats MLOps as plain DevOps and ignores data and model validation.
  • Cannot trace a model version back to its training data.
  • Has never owned monitoring or on-call for a model in production.
  • Proposes a heavyweight platform before asking about team size and model count.
  • Universal flag: Vague answers that never reach a concrete system, number or decision the candidate personally owned.

The scorecard we use

Score each dimension 1–⁠4 (1 = no evidence, 2 = partial, 3 = solid, 4 = exceptional):

DimensionWeightWhat a 4 looks like
Technical depth, production judgement and security40%Explains how and why, designs for failure and sensitive data, and knows each tool’s limits
Practical task and problem solving30%Delivers a working solution with tests and documented decisions
Communication and documentation20%Explains clearly to technical and non-technical audiences, in English or Arabic
References and regional context10%References confirm the claimed scope; knows the frameworks and data-residency rules that apply

Pass bar: weighted average of 3.0 out of 4 or higher, with no dimension below 2. The practice lead approves the final result.

Skip the interview loop

Every specialist on our shortlist has already passed this assessment. Working in 4 working days.

Frequently asked questions

Yes. The questions, hints, practical task and scorecard are free to use. They work best when every interviewer scores the same dimensions independently and compares notes afterwards. Ask for a concrete example after each answer, because recalled definitions say little about how someone performs on a live system.

They come from the same live technical assessment, but practice leads rotate and adapt questions so candidates cannot rehearse them. What stays fixed is the scope being tested, the practical task format and the scorecard, which is why the published version is safe to share.

Plan for a technical conversation covering 8 to 10 of the questions, plus the practical task either live or as a short take-home. Use the remaining questions as follow-ups. If you would rather skip the loop, Remote Pro Ai shortlists already-assessed specialists within 48 hours.

Our pass bar is a weighted average of 3.0 out of 4 or higher across the 4 dimensions, with no dimension below 2. Production judgement and security, scored inside the 40% technical dimension, often separate strong candidates from those who interview well but struggle once they own a live system.

The task mirrors the first week of real work in this role and never uses client data. It runs live or as a short take-home of a few hours, and the candidate then explains the trade-offs behind each decision. Reasoning and production habits count for more than a polished final artefact.