Practiced one out loud? Paste your answer into the free self-check and get a score in 30 seconds
What the interviewer is testing
Whether you treat the context window as a design constraint with a budget, not as a reason to skip retrieval or memory, and whether you know models use long inputs unevenly.
Answer framework
- Define it: the maximum number of tokens the model can attend to in one call, prompt plus generated output.
- Costs: longer prompts raise price and time to first token on every call.
- Quality: models use long inputs unevenly; facts buried in the middle are used less reliably.
- Design: decide what goes in with retrieval, summarization or memory, and give each section a token budget (system, history, retrieved context, output).
- Verify: test long-context behavior on your own tasks instead of trusting the advertised maximum.
Sample answer
The context window is the total number of tokens a model can attend to in one request, which includes the system prompt, conversation history, any retrieved documents and the tokens it generates. If the input fills the window, there is no room left for the answer.
A large window makes some things easier, such as reading a whole contract in one pass, but it does not remove design work. Every extra token adds cost and latency on every call, and models do not use all positions equally well; information in the middle of a long prompt is used less reliably than information at the start or end.
So I design with an explicit budget. For a support assistant I might reserve a fixed slice for the system prompt, cap conversation history and summarize older turns, give retrieved passages the largest share and leave enough room for the output. Retrieval decides what deserves to be in the window; the window size only sets the ceiling.
Likely follow-ups
Common mistakes
- Forgetting that output tokens share the same window as the prompt.
- Stuffing everything into a long window instead of retrieving the relevant parts.
- Letting chat history grow without trimming or summarizing.
- Trusting the advertised window without testing recall at different positions.
The 0 to 1 AI Engineer Interview Playbook covers the answer frameworks, system-design walkthroughs and an 8-week plan.
Get the playbook 40% off with code EARLYBF until Nov 3, 11:59 PM PTEARLYBF: 40% off until Nov 3Smaller start: Score your answers against 50 rubrics, $9. 50 AI engineer questions, one rubric each, PDF on Gumroad.
Interviewing for PM or TPM? See the full PM package, $59 on Gumroad: a resume system, a PM, TPM and PGM interview handbook, a 1:1 cheat sheet and a PM playbook. PM and TPM content, not AI engineer prep.