Practiced one out loud? Paste your answer into the free self-check and get a score in 30 seconds
What the interviewer is testing
Whether you can separate the kinds of memory an agent needs, choose a storage approach for each, and deal with relevance, staleness and privacy.
Answer framework
- Short-term (working) memory: the current conversation and task state inside the context window.
- Compression: summarize or trim older turns when the window fills.
- Long-term memory: store facts or past episodes outside the model and retrieve the relevant ones per request.
- Structured state: keep user preferences and task progress in a database, not just as text.
- Governance: decide what to save, let users view and delete it, expire stale facts and keep data scoped per user.
Sample answer
I split memory into a few layers. Working memory is what sits in the context window right now: recent messages, the current plan and tool results. When it grows too large, I keep the last several turns verbatim and replace older ones with a running summary, so the important decisions survive without the full transcript.
Long-term memory lives outside the model. Some of it is structured, such as a user's timezone, preferred language or the status of an open ticket, and that belongs in a normal database with clear fields. Some of it is unstructured, such as notes from past sessions, which I store as text with embeddings and retrieve by relevance and recency when a new request comes in. Only the few relevant memories go into the prompt.
The design questions are mostly about policy. What is worth saving? I prefer explicit, durable facts over saving everything. How do we handle conflicts and stale facts? Newer facts win and old ones expire. And privacy: memory is scoped per user, users can see and delete it, and sensitive data is not stored without a reason.
Likely follow-ups
Common mistakes
- Treating memory as one big vector store for everything.
- Saving every message without a policy.
- Letting stale facts override newer ones.
- Ignoring user visibility, deletion and per-user isolation.
The 0 to 1 AI Engineer Interview Playbook covers the answer frameworks, system-design walkthroughs and an 8-week plan.
Get the playbook 40% off with code EARLYBF until Nov 3, 11:59 PM PTEARLYBF: 40% off until Nov 3Smaller start: Score your answers against 50 rubrics, $9. 50 AI engineer questions, one rubric each, PDF on Gumroad.
Interviewing for PM or TPM? See the full PM package, $59 on Gumroad: a resume system, a PM, TPM and PGM interview handbook, a 1:1 cheat sheet and a PM playbook. PM and TPM content, not AI engineer prep.
Related questions
Based on question #46 in the full question bank.