AI Product Manager interview prep

Keep it to 90 seconds and make it a line, not a list: where you started, the thread that led you to AI products, and the proof you can do the job.

Example shape: "I started as a software engineer building data pipelines, moved into product two years ago, and for the last year I have owned an LLM feature that drafts replies for our support team. We took acceptance of drafts from 31% to 58% by building an eval set from real tickets. I want to do that at a bigger scale, which is why I am here."

What they look for: A clear narrative, one concrete AI result with a number, and a reason for this role.

Tie it to the work, not the hype. Good answers name what is different about AI products (measuring quality with evals, designing for uncertainty, cost per task) and why you enjoy those problems, backed by something you have built or shipped.

What they look for: Genuine interest in the hard parts, not "AI is the future".

Pick one you use weekly. Say who it is for, the job it does, what it gets right, and one specific failure you have seen. Then propose one improvement with the metric it would move and how you would test it.

Example: a meeting-notes assistant gets summaries right but misses action-item owners. Improve owner detection using calendar attendees, measure the share of action items with a correct owner on a labelled set of 200 meetings, then A/B test on edit rate.

What they look for: User empathy, a specific failure, and a measurable fix.

Use problem, approach, evaluation, launch, result. Spend most of the time on the decisions: why AI at all, which model or approach, how you defined good output, what guardrails you added, and what you learned when it met real users.

What they look for: Ownership of decisions and honest lessons, including what did not work.

Explain the split: you own the problem, the success criteria and the eval set; they own the approach and the model. Mention shared rituals (weekly error review, eval dashboards), and give an example where you changed the spec after seeing failure cases.

What they look for: Respect for the craft and a practical working rhythm.

Clarify the goal (faster, more accurate resolutions, not replacing agents) and the users (agents, team leads, customers).

Top pain points: looking up orders across tools, writing refunds policy-correctly, long angry-customer chats. Prioritise "suggested reply with the right refund action" because it is frequent and measurable.

Solution: the assistant reads the order, delivery timeline and policy, drafts a reply and proposes an action the agent approves. Guardrails: never issues refunds without approval, cites the policy line.

Metrics: average handle time, first-contact resolution, policy-correct refund rate on an eval set, agent acceptance of drafts, CSAT.

What they look for: Clear user, prioritised pain point, human in the loop, and metrics for quality and impact.

Start from the rep's day: research before calls, logging notes, writing follow-ups and updating the pipeline. Logging is hated and hurts data quality, so start there: auto-summarise calls into CRM fields and next steps for the rep to confirm.

Measure field completeness, time spent on admin per week, forecast accuracy and adoption by rep. Phase two: account research briefs before calls.

What they look for: Workflow thinking and a wedge that improves data the business depends on.

Segment: Class 11–12 students preparing for engineering entrance exams, often in Hindi or Hinglish, on low-cost Android phones and patchy networks.

Core job: getting unstuck on a problem at 11 p.m. without a teacher. Feature: photo of a problem leads to step-by-step hints, not answers, in the student's language, then a similar practice question.

Risks: wrong solutions (verify against a solution bank), dependency (hint-first design), cost (cache common problems). Metrics: problems solved after hints, weekly retention, accuracy of solutions on a graded set, cost per active student.

What they look for: Local context, pedagogy over answers, and cost awareness.

Ask what job users hire notes for (finding things later). Check evidence: search usage, failed searches, retention of heavy users. If search failure is high, "ask your notes" with citations is a strong fit. Risks: privacy concerns, hallucinated answers, cost for free users. Recommend a limited beta for paid users, measure answer usefulness and repeat usage, and show sources for every answer.

What they look for: A decision grounded in data, with risks and a test plan.

First-time users do not know what to ask and churn after one generic answer. Ideas: starter tasks based on the user's stated goal, showing what the assistant can do with files and tools, and a follow-up nudge the next day.

Pick one, define activation (for example, three successful tasks in week one) and measure day-7 retention in an A/B test.

What they look for: A crisp activation metric and one well-argued idea.

An LLM is trained on a huge amount of text to predict the next word. That simple task forces it to learn grammar, facts and patterns of reasoning. When you ask a question, it writes the most likely continuation one piece at a time. That is why it is fluent, why it can be wrong with confidence, and why giving it the right context in the prompt matters so much.

What they look for: Simple, accurate, and it explains the failure modes.

Prompting first: it is cheapest and fastest to iterate. Use retrieval-augmented generation (RAG) when the model needs facts it does not have, such as your docs or a customer's data, especially if they change often or need citations. Consider fine-tuning when you need a consistent format, tone or a narrow skill at lower cost and latency, and you have enough good examples. Many products combine RAG with a lightly fine-tuned or well-prompted model.

What they look for: A cost-and-data-driven decision, not a favourite technique.

Ground the model in retrieved sources and ask it to cite them; tell it to say "I don't know" when sources are missing; constrain outputs with structured formats; add checks (for example, verify numbers against the database); and design the UI so users can see sources and correct mistakes. Measure the hallucination rate on an eval set before and after each change.

What they look for: Layered defences and measurement.

Collect 100–300 real, representative inputs including hard and edge cases. Define what good looks like with a rubric (correct, complete, safe, right format). Label expected outputs or criteria with domain experts. Score with a mix of automated checks, model-graded rubrics checked against human labels, and human review for a sample. Track the score per release and look at failures by category, not just the average.

What they look for: Representative data, a rubric, and a habit of reading failures.

Check whether quality gains justify it for this task. Options: stream the response, use the larger model only for hard cases (routing), shorten prompts and context, cache repeated work, or run steps in parallel. Decide with data: quality on the eval set versus latency versus cost per task, and the effect on user metrics.

What they look for: Trade-off thinking and several concrete levers.

An agent is a model that can plan and take actions with tools (search, APIs, code) over several steps. It fits open-ended tasks where steps vary. It is the wrong choice when a fixed workflow does the job, because each extra step adds cost, latency and chances to fail. Start with the simplest workflow and add autonomy only where it measurably helps.

What they look for: Pragmatism about reliability and cost.

Prompt injection is when content the model reads (a web page, an email, a document) contains instructions that try to override yours. Protections: treat external content as data, limit what tools and permissions the model has, require confirmation for risky actions, filter inputs and outputs, and test with known attack examples as part of your eval set.

What they look for: Awareness of security as a product responsibility.

Cost per task = calls per task × (input tokens × input price + output tokens × output price), plus retrieval and infrastructure. Multiply by tasks per user and users. Compare with what the customer pays or the value created. Then look for savings: smaller models for easy steps, caching, shorter context, batch processing for non-urgent work.

What they look for: A clear formula and margin awareness.

North star: issues resolved without a human, confirmed by the customer. Supporting: containment rate, escalation rate, CSAT for bot conversations, repeat contact within 7 days (a sign of false resolution). Quality: accuracy on an eval set and policy compliance. Guardrails: harmful or off-policy answers, cost per conversation, latency.

What they look for: A north star plus metrics that catch fake wins.

Confirm the data first (logging change, a new segment of users, one big customer). Check what changed: model version, prompt, retrieval index, product release. Slice by language, task type, customer and platform. Read 50 thumbs-down examples and group the failures. Fix the top cause, add those cases to the eval set, and monitor.

What they look for: Structured root-cause analysis that ends in reading real examples.

Randomise users, not requests, and run long enough to get stable results. Use user-level outcome metrics (task completion, retention) rather than single-response ratings. Fix the model version and settings during the test, log every output for later review, and pair the experiment with offline evals so you know quality before exposing users.

What they look for: Experiment hygiene adapted to AI.

Agree launch criteria up front: an eval score threshold on key task types, zero critical safety failures in red-team tests, latency and cost within budget, a fallback when the model fails, and monitoring in place. Launch to a small percentage first with a clear rollback plan.

What they look for: Pre-agreed criteria and staged rollout.

Find the driver: more calls per task, longer contexts, a more expensive model, or abuse. Fixes in order of effort: cap context length, cache common answers, route easy requests to a smaller model, rate-limit heavy free users, and revisit pricing if heavy usage is also your most valuable usage.

What they look for: Diagnosis before cutting, and a link to pricing.

Default to APIs to learn fast. Consider training or fine-tuning your own when you have unique data that creates a quality edge, strict data or latency requirements, or a cost profile that makes it cheaper at your volume. Keep the architecture model-agnostic so you can switch providers as models improve.

What they look for: Speed now with optionality later.

Not the model. Moats come from proprietary data and feedback loops, deep workflow integration, distribution, trust (security, compliance, reliability) and the evaluation know-how for a specific domain. Better models then make your product better, not obsolete.

What they look for: A clear view of durable advantage.

Anchor on value (hours saved, revenue gained) and check against cost per task. Options: per seat with usage limits, usage-based credits, or outcome-based pricing for well-defined tasks. Start simple, protect margin with fair-use limits, and watch how heavy users behave.

What they look for: Value-based thinking with margin protection.

Pick a story where you used data to resolve it. Example: engineering wanted to wait for a better model; you proposed launching to 5% of users with a fallback, agreed the metric that would trigger a rollback, and the result.

What they look for: Data over opinion, and a relationship intact afterwards.

Own it: what you believed, what happened, what you missed, and what you changed in how you work. Interviewers want self-awareness and a concrete change in behaviour.

What they look for: Ownership without blame.

Show how you sized the risk, set a reversible first step, and defined what evidence would change your mind.

What they look for: Comfort with ambiguity and reversible bets.

Describe the concern specifically (for example, the model treating some users unfairly), how you checked it with data, who you involved, and what changed. Avoid vague answers.

What they look for: Courage plus a constructive path.