Mid

How do you handle memory in AI agents?

How to answer it in an AI engineer interview: what is being tested, a step-by-step framework, a sample answer and the mistakes to avoid.

$19.99 on Gumroad, instant download40% off with code EARLYBF until Nov 3, 11:59 PM PTEARLYBF: 40% off until Nov 3

Smaller start: Score your answers against 50 rubrics, $9. 50 AI engineer questions, one rubric each, PDF on Gumroad.

Practiced one out loud? Paste your answer into the free self-check and get a score in 30 seconds

What the interviewer is testing

Whether you can separate the kinds of memory an agent needs, choose a storage approach for each, and deal with relevance, staleness and privacy.

Answer framework

  1. Short-term (working) memory: the current conversation and task state inside the context window.
  2. Compression: summarize or trim older turns when the window fills.
  3. Long-term memory: store facts or past episodes outside the model and retrieve the relevant ones per request.
  4. Structured state: keep user preferences and task progress in a database, not just as text.
  5. Governance: decide what to save, let users view and delete it, expire stale facts and keep data scoped per user.

Sample answer

I split memory into a few layers. Working memory is what sits in the context window right now: recent messages, the current plan and tool results. When it grows too large, I keep the last several turns verbatim and replace older ones with a running summary, so the important decisions survive without the full transcript.

Long-term memory lives outside the model. Some of it is structured, such as a user's timezone, preferred language or the status of an open ticket, and that belongs in a normal database with clear fields. Some of it is unstructured, such as notes from past sessions, which I store as text with embeddings and retrieve by relevance and recency when a new request comes in. Only the few relevant memories go into the prompt.

The design questions are mostly about policy. What is worth saving? I prefer explicit, durable facts over saving everything. How do we handle conflicts and stale facts? Newer facts win and old ones expire. And privacy: memory is scoped per user, users can see and delete it, and sensitive data is not stored without a reason.

Likely follow-ups

Why not keep the full history in a long context window? Cost and latency grow every turn, quality drops on long contexts, and old irrelevant turns distract the model.

Common mistakes

The PlaybookWant the full method?

The 0 to 1 AI Engineer Interview Playbook covers the answer frameworks, system-design walkthroughs and an 8-week plan.

Get the playbook 40% off with code EARLYBF until Nov 3, 11:59 PM PTEARLYBF: 40% off until Nov 3

Smaller start: Score your answers against 50 rubrics, $9. 50 AI engineer questions, one rubric each, PDF on Gumroad.

Interviewing for PM or TPM? See the full PM package, $59 on Gumroad: a resume system, a PM, TPM and PGM interview handbook, a 1:1 cheat sheet and a PM playbook. PM and TPM content, not AI engineer prep.

Related questions

Based on question #46 in the full question bank.