How we assess
Every applicant goes through 5 gates: application, an AI-assisted screen, a live technical assessment scored by a practice lead, a client-fit interview, and reference and credential checks. The questions below come from the live assessment. We score answers on reasoning and trade-offs, not on recalling definitions.
20 technical questions with model-answer hints
Questions 1–10 (role-specific)
Question 1: Classify an AI tool that screens job applicants under the EU AI Act.
Model-answer hint
employment use is high-risk; risk management, data governance, logging and human oversight apply.
Question 2: Provider or deployer: why does the distinction matter?
Model-answer hint
different obligations; substantial modification can turn a deployer into a provider.
Question 3: What are the core parts of an ISO/IEC 42001 management system?
Model-answer hint
context, leadership, planning with risk and impact assessment, operation, evaluation, improvement.
Question 4: How do you test a credit model for bias?
Model-answer hint
define groups, choose fairness metrics, compare error rates, investigate proxies, document trade-offs.
Question 5: What belongs in a model card?
Model-answer hint
intended use, data, performance by slice, limitations, owners, version history.
Question 6: How do you design meaningful human oversight?
Model-answer hint
competent reviewers, real ability to override, thresholds, automation-bias safeguards, logging.
Question 7: How does PDPL affect training a model on customer data?
Model-answer hint
lawful basis, purpose limitation, minimisation, transfer rules, retention.
Question 8: What is an AI incident, and how is it handled?
Model-answer hint
harm or near-miss from AI behaviour; detect, contain, notify, find root cause, update controls.
Question 9: How do you govern a third-party foundation model you cannot inspect?
Model-answer hint
supplier assessment, contract terms, your own evaluations, monitoring, fallback plans.
Question 10: How do you stop governance from blocking delivery?
Model-answer hint
risk-tiered process, templates, embedded in sprints, fast lane for low-risk cases.
Questions 11–20 (AI & Data core)
Question 11: How do you decide between rules, classic ML and an LLM for a problem?
Model-answer hint
start from decision value and error cost; choose the simplest option that meets the metric.
Question 12: How do you detect training-serving skew?
Model-answer hint
shared feature code, logged inference inputs, distribution checks between training and live data.
Question 13: How do you build an evaluation set that predicts production results?
Model-answer hint
representative sampling, slice coverage, leakage checks, labelled with measured agreement, refreshed on a schedule.
Question 14: What goes on a model-health dashboard, and who acts on each signal?
Model-answer hint
quality, drift, latency, cost and the business KPI, each with an owner and threshold.
Question 15: How do you handle personal data in a training pipeline under PDPL or GDPR?
Model-answer hint
minimisation, lawful basis, pseudonymisation, residency, retention limits, DPIA.
Question 16: How do you roll back a model or prompt change that degrades results?
Model-answer hint
versioned registry, canary or shadow release, automated gates, one-step rollback.
Question 17: When does fine-tuning beat retrieval or better prompting?
Model-answer hint
behaviour, format and domain style favour tuning; fresh facts favour retrieval; weigh data and cost.
Question 18: Describe a data-quality incident you caught.
Model-answer hint
how it was detected, root cause, the contract or test added, the alert that now exists.
Question 19: How do you forecast and cap the monthly cost of an AI workload?
Model-answer hint
tokens or GPU hours, caching, batching, model routing, budgets with alerts.
Question 20: Explain a modelling trade-off you made to a non-technical executive.
Model-answer hint
plain language, business impact, the option rejected and why.
Interviewer tip: Ask for a real example after every answer. Strong candidates describe what they built, what broke and what they changed.
Practical task
Review a one-page description of a GenAI claims-triage assistant for an insurer. Produce a risk classification, the top 5 risks with controls, a human-oversight design and the documentation required before launch.
How to run it Share the brief 24 hours ahead or run it live. Ask the candidate to talk through trade-offs as they work; reasoning matters more than a finished artefact.
Red flags our practice leads watch for
- Recites principles but cannot name concrete controls or evidence.
- Treats governance as a one-off sign-off rather than lifecycle monitoring.
- Cannot interpret basic performance or fairness metrics.
- Confuses general data-protection duties with AI-specific obligations.
- Universal flag: Vague answers that never reach a concrete system, number or decision the candidate personally owned.
The scorecard we use
Score each dimension 1–4 (1 = no evidence, 2 = partial, 3 = solid, 4 = exceptional):
| Dimension | Weight | What a 4 looks like |
|---|---|---|
| Technical depth, production judgement and security | 40% | Explains how and why, designs for failure and sensitive data, and knows each tool’s limits |
| Practical task and problem solving | 30% | Delivers a working solution with tests and documented decisions |
| Communication and documentation | 20% | Explains clearly to technical and non-technical audiences, in English or Arabic |
| References and regional context | 10% | References confirm the claimed scope; knows the frameworks and data-residency rules that apply |
Pass bar: weighted average of 3.0 out of 4 or higher, with no dimension below 2. The practice lead approves the final result.