AI Engineer interview prep

Cover the user problem, why an LLM was the right tool, the architecture (model, retrieval, tools), how you evaluated it, what failed and how you fixed it, and the result in numbers: accuracy, latency, cost or user adoption. Interviewers dig into decisions, so be ready to defend each one.

What they look for: Real decisions, measurement and honest failures.

ML engineers typically train, fine-tune and serve models, working with datasets, training pipelines and model architecture. AI engineers mostly build applications on top of existing foundation models: prompts, retrieval, agents, evals and product integration. The roles overlap, especially for fine-tuning and evaluation.

What they look for: A clear, non-dismissive distinction.

Describe a system, not a feed: you follow a few primary sources (model release notes, papers or engineering blogs), and when something new appears you test it on your own eval set before adopting it. Give an example of a model switch you made or rejected based on your evals.

What they look for: Evidence-based adoption.

Connect the company's AI product or clients to problems you have worked on. Mention something specific you tried or noticed in their product and one idea you would explore.

What they look for: Specific interest and preparation.

Tokens are the chunks of text a model reads and writes; one word is often one to two tokens, and many non-English scripts use more. Pricing is per input and output token, and output tokens are generated one at a time, so long outputs increase latency. Shorter prompts, smaller contexts and concise outputs cut both cost and latency.

What they look for: Linking tokens to cost, latency and languages.

Temperature controls how random the model's choice of next token is. Low values make outputs more consistent, which suits extraction, classification and evals. Higher values add variety for brainstorming or creative text. Even at zero, outputs can vary slightly, so do not rely on exact repeatability.

What they look for: Correct intuition and practical defaults.

It is the maximum number of tokens the model can consider at once, including the prompt, retrieved documents, conversation history and the answer. If you exceed it, the request fails or content must be truncated. Even within the limit, very long contexts can cost more and reduce attention to important details, so retrieve and summarise selectively.

What they look for: Limits plus the quality trade-off of long contexts.

Use prompting first, add retrieval when the model needs your facts, and fine-tune when you need consistent format, style or a narrow skill that prompting cannot reach, or a smaller, cheaper model to match a larger one on one task. Fine-tuning needs good labelled examples and an eval set to prove it helped. It is not the right way to add frequently changing knowledge.

What they look for: Ordered decision-making with evidence.

Use the provider's structured-output or function-calling feature with a schema where available, validate the output against the schema in code, retry with the validation error if it fails, and keep the schema simple. Log failures to improve prompts.

What they look for: Schema plus validation, not trust.

Embeddings are lists of numbers that represent the meaning of text, so similar meanings end up close together. They power semantic search, retrieval for RAG, clustering, deduplication and recommendations. Quality depends on the embedding model and on how you chunk the text.

What they look for: Clear definition with applications.

Ingest documents with metadata (source, team, date, access group). Chunk by structure (sections, headings) with some overlap, embed and store in a vector index, ideally alongside keyword search (hybrid). At query time: rewrite the question if needed, retrieve with permission filters, rerank, and pass the top chunks to the model with instructions to cite sources and say "I don't know" when unsupported. Evaluate retrieval (did the right chunk come back?) and answers (correct and grounded?) separately, and keep the index fresh with incremental updates.

What they look for: Permissions, hybrid retrieval, citations and two-level evaluation.

Follow the document's structure: split on headings or sections rather than fixed character counts, keep chunks small enough to be specific but large enough to carry context, add overlap or parent-section context, and attach metadata. Then test: compare retrieval hit rate on your eval questions for two or three strategies.

What they look for: Structure-aware and empirically tested.

Possible causes: the relevant passage was cut off or ranked low within the context, too many irrelevant chunks distracted the model, the prompt did not force use of sources, the question needed reasoning across several chunks, or the source itself was ambiguous. Fix with reranking, fewer but better chunks, clearer instructions and a citation requirement, then confirm on the eval set.

What they look for: Separating retrieval failure from generation failure.

Hybrid search combines keyword search (exact terms such as product codes, names and error messages) with vector search (meaning). Each catches what the other misses. Results are merged and often reranked by a cross-encoder or a model.

What they look for: Knowing why exact matching still matters.

Track document versions and timestamps, re-embed only changed documents, delete stale chunks, store the effective date as metadata and prefer the latest version at query time. Monitor index freshness as a metric.

What they look for: Incremental indexing and freshness.

Tools: look up order, check refund policy, issue refund (with limits), escalate to human. Flow: identify the customer and order, check eligibility against policy, refund automatically under a threshold and ask for human approval above it, then confirm to the customer. Log every tool call, cap the number of steps, and evaluate on historical tickets for correct decisions and policy compliance.

What they look for: Bounded autonomy, approvals and evaluation on real cases.

When the steps are known and fixed, a deterministic workflow with one or two model calls is cheaper, faster and easier to test. Agents add value when the path depends on what is discovered along the way. Start simple and add autonomy only where it measurably improves results.

What they look for: Pragmatism over hype.

Limit steps and budget, give it least-privilege tools, require confirmation for irreversible or high-value actions, validate tool inputs in code, detect repeated identical calls, and provide a clear "stop and escalate" path. Test with adversarial scenarios.

What they look for: Guardrails in code, not just in the prompt.

MCP is an open standard for connecting AI applications to tools and data sources. An MCP server exposes capabilities, such as querying a database or creating a ticket, that any compatible AI client can use. It reduces custom integration work and makes permissions and logging consistent across apps.

What they look for: Accurate definition and why it matters.

Define end-to-end tasks with known correct outcomes, run the agent in a sandbox, and score success, the number of steps, cost, time and any policy violations. Review traces of failures to see which step went wrong, and keep the task set as a regression suite.

What they look for: Task-level success plus trace review.

Collect real inputs (logs, support tickets, user requests), cover common, hard and edge cases, write expected outputs or a grading rubric with domain experts, and include examples that should be refused. Start with 50–200 examples, grow it with every production failure, and run it on every prompt or model change.

What they look for: Real data, rubrics and continuous growth.

Yes, with care. Give the grader a specific rubric, ask for reasoning plus a score, check its agreement with human labels on a sample, and watch for bias towards longer or similar-sounding answers. Use code-based checks wherever possible (exact match, schema, citations present) and keep humans in the loop for subjective quality.

What they look for: Calibrated model grading.

Measure each step first. Then: stream the response, cut prompt and context size, run retrieval and tool calls in parallel, cache frequent answers, use a smaller model for simple requests or first drafts, and pre-compute where possible. Track time to first token and p95 latency.

What they look for: Measure first, then several concrete levers.

Break cost down by feature, model, customer and token type. Common causes: a prompt that grew, longer conversation history, retries in a loop, a switch to a pricier model or abuse. Fix with caching, context limits, routing easy requests to cheaper models, batch processing for non-urgent jobs and per-user limits.

What they look for: Attribution before optimisation.

Treat all retrieved or user-supplied content as untrusted data, separate it clearly from instructions, limit the tools and data the model can reach, require confirmation for sensitive actions, filter outputs that could leak secrets, and keep attack examples in your eval suite. No single defence is enough.

What they look for: Defence in depth.

Run the full eval suite on the new model, compare quality, latency and cost by task, review differences manually, then roll out gradually behind a flag with monitoring and a quick rollback. Pin model versions in production so behaviour does not change unexpectedly.

What they look for: Eval-gated, staged rollout.

Take the text, a chunk size and an overlap. Move a window forward by (size − overlap) each step and slice, preferably on sentence or paragraph boundaries rather than mid-word. Handle edge cases: overlap greater than or equal to size, empty text and a final short chunk. Mention that in production you would count tokens rather than characters.

What they look for: Correct windowing and edge cases.

Wrap the call in a loop with a maximum number of attempts. On rate-limit or server errors, wait base × 2^attempt plus random jitter, then retry; on client errors such as invalid requests, fail immediately. Respect any retry-after header, set an overall timeout and log each attempt.

What they look for: Knowing which errors to retry.

Describe the failure, how you detected it (ideally monitoring, not a customer complaint), the root cause, the fix, and what you added so it cannot recur: an eval case, an alert or a guardrail.

What they look for: Ownership and systemic fixes.

A strong answer shows judgement: the task was better solved with rules, search or a simple model, or the risk was too high. Explain the evidence you used and the outcome.

What they look for: Judgement over enthusiasm.