How we assess
Every applicant goes through 5 gates: application, an AI-assisted screen, a live technical assessment scored by a practice lead, a client-fit interview, and reference and credential checks. The questions below come from the live assessment. We score answers on reasoning and trade-offs, not on recalling definitions.
20 technical questions with model-answer hints
Questions 1–10 (role-specific)
Question 1: When is async a bad idea in FastAPI?
Model-answer hint
CPU-bound work and blocking libraries; uses thread pools or workers; understands the event loop.
Question 2: How do you validate and version API inputs?
Model-answer hint
Pydantic models, explicit versioning, backwards compatibility, OpenAPI as the contract.
Question 3: Design a document Q&A feature over 50,000 PDFs.
Model-answer hint
Ingestion, chunking, embeddings, permission filters, citations, an evaluation set.
Question 4: How do you stop one user from running up a large LLM bill?
Model-answer hint
Per-user rate limits, token budgets, caching, spend monitoring and alerts.
Question 5: Your LLM provider times out for 10 minutes. What happens in your design?
Model-answer hint
Timeouts, retries with backoff, fallback model or graceful degradation, circuit breaker.
Question 6: How do you handle long-running jobs triggered by an API call?
Model-answer hint
Return 202 with a job id, queue via Celery, idempotency, status endpoint or webhook.
Question 7: How do you keep personal data out of logs and prompts?
Model-answer hint
Masking, allow-listed fields, regional endpoints, retention policies.
Question 8: How do you know whether a RAG answer is good?
Model-answer hint
Retrieval precision and recall, groundedness, human review, regression evaluation set.
Question 9: What is your approach to typing in Python?
Model-answer hint
Type hints throughout, mypy or Pyright in CI, Pydantic at boundaries; pragmatic about gradual typing.
Question 10: Describe a performance problem you fixed in a Python service.
Model-answer hint
Profiled first; N+1 queries, serialisation cost or caching; measured result.
Questions 11–20 (Mobile & Web core)
Question 11: How do you estimate a feature when requirements are still unclear?
Model-answer hint
break it down, name the unknowns, run a short spike, give a range and re-estimate as you learn.
Question 12: How do you integrate national-ID login such as Nafath or UAE Pass safely?
Model-answer hint
standard redirect or app-to-app flow, state and nonce checks, secure token storage, expiry and failure paths.
Question 13: What goes into your definition of done for a user-facing feature?
Model-answer hint
tests, accessibility checks, analytics events, error states, documentation, feature flag.
Question 14: How do you set and enforce a performance budget?
Model-answer hint
Core Web Vitals or startup targets measured on mid-range devices, with CI checks that fail on regression.
Question 15: How do you release safely to millions of users?
Model-answer hint
feature flags, staged rollout, crash and error monitoring, fast rollback, store review planning.
Question 16: How do you handle offline or unreliable connectivity?
Model-answer hint
local caching, optimistic updates with conflict handling, retries with backoff, clear user feedback.
Question 17: What do you check before an app starts handling payments?
Model-answer hint
PCI scope, tokenisation, idempotency, webhooks, reconciliation, provider sandbox tests.
Question 18: How do you keep dependencies and SDKs secure and current?
Model-answer hint
lockfiles, automated update pull requests, vulnerability scanning, licence checks, removing unused packages.
Question 19: A design cannot be built as drawn. What do you do?
Model-answer hint
early feasibility review, alternatives with trade-offs, design-system components, a shared decision with the designer.
Question 20: How do you review a colleague’s pull request?
Model-answer hint
correctness, readability, tests and security; small PRs; clear blocking versus non-blocking comments.
Interviewer tip: Ask for a real example after every answer. Strong candidates describe what they built, what broke and what they changed.
Practical task
Timebox 3 hours. Build a FastAPI service that ingests 20 provided bilingual FAQ documents, stores embeddings in pgvector and exposes an /ask endpoint that returns an answer with cited sources. Include per-key rate limiting, a mocked LLM client for tests, 5 pytest cases and a short note on how you would evaluate answer quality before launch.
How to run it Share the brief 24 hours ahead or run it live. Ask the candidate to talk through trade-offs as they work; reasoning matters more than a finished artefact.
Red flags our practice leads watch for
- Calls LLM APIs inside request handlers with no timeouts.
- Logs full prompts that contain personal data.
- Cannot explain the difference between threads, processes and async.
- Judges AI quality only by trying a few prompts manually.
- Universal flag: Vague answers that never reach a concrete system, number or decision the candidate personally owned.
The scorecard we use
Score each dimension 1–4 (1 = no evidence, 2 = partial, 3 = solid, 4 = exceptional):
| Dimension | Weight | What a 4 looks like |
|---|---|---|
| Technical depth, production judgement and security | 40% | Explains how and why, designs for failure and sensitive data, and knows each tool’s limits |
| Practical task and problem solving | 30% | Delivers a working solution with tests and documented decisions |
| Communication and documentation | 20% | Explains clearly to technical and non-technical audiences, in English or Arabic |
| References and regional context | 10% | References confirm the claimed scope; knows the frameworks and data-residency rules that apply |
Pass bar: weighted average of 3.0 out of 4 or higher, with no dimension below 2. The practice lead approves the final result.