AI / AI agent memory Interview questions

That storage can be as simple as the running chat transcript or as involved as a vector database, a knowledge graph, or a set of files the agent edits itself.

Good memory is what turns a one-off chatbot into an agent that tracks goals, learns user preferences, and avoids repeating mistakes across steps and sessions.

Memory gives an agent continuity. Without it, every request starts from zero and the agent cannot follow a multi-step task, remember what the user already said, or learn from earlier outcomes.

1. Keep task state across the steps of a single run.
2. Carry user facts and preferences across sessions.
3. Store lessons and past results so the agent does not repeat failed attempts.
4. Hold knowledge that is too large to fit in the context window.

It lives inside the context window, so it is fast and always visible to the model, but it is limited in size and usually disappears when the session ends unless you checkpoint it.

Common ways to keep it under control are a sliding window of recent messages, summarising older turns, or trimming bulky tool outputs.

Typical backends are vector databases, key-value stores, relational tables, graphs, or plain files. Only the retrieved pieces are injected into the prompt, which keeps token use manageable.

It is usually split into episodic, semantic, and procedural memory, borrowing terms from cognitive science.

In practice it is whatever sits in the prompt right now. Some frameworks make it explicit, for example MemGPT-style "core memory" blocks or a structured state object in LangGraph that every node can read and update.

It stays small on purpose. Anything that is no longer needed for the next step should be written to long-term storage or dropped, otherwise the scratchpad turns into clutter that distracts the model.

Agents use it to recall how a similar situation went before, which makes it handy for few-shot examples and for avoiding repeated mistakes. Entries are often stored as logs or short summaries with timestamps and retrieved by similarity.

It is usually stored as short facts or profile fields, in a vector store, a document store, or a knowledge graph, and updated as new information replaces old. Mem0 and LangMem both focus heavily on extracting this kind of fact from conversations.

Some designs let the agent rewrite its own instructions after feedback, so a lesson like "always confirm the date format before calling the calendar API" becomes a permanent behavior. Anthropic's skills and similar instruction files fit this idea too.

The context window is the maximum amount of text, measured in tokens, that a model can read and produce in one request. System prompt, history, retrieved memories, tool results, and the new answer all share it.

Larger windows help, but they do not remove the need for memory. Cost and latency grow with input size, and models can miss details buried in the middle of a very long prompt.

As a rule of thumb, reserve room for the answer up front. If the prompt already fills most of the window, the reply can be cut off mid-sentence.

Agent memory is usually grouped by lifespan and by content.

Some teams add a fifth bucket, shared memory, for information several agents read and write together.

In interviews, mention that the boundaries are fuzzy: one stored item, like a user's diet, can be semantic fact today and feed a procedural rule tomorrow.

A vector database stores embeddings, which are numeric representations of text, and finds the closest ones to a query using similarity search. Agents use it to pull back memories that are related in meaning even when the exact words differ.

Common choices include pgvector, Pinecone, Qdrant, Weaviate, and Chroma. The usual flow is: embed the memory, store it with metadata such as user ID and timestamp, then embed the query and fetch the top-k matches.

In an agent, each memory is embedded when written and the current query is embedded when reading. Cosine similarity or dot product then ranks the candidates.

One practical rule: use the same embedding model for writing and querying. If you switch models you must re-embed the whole store.

Dimension matters too: common models output vectors with 768 to 3072 numbers, and the index must be created with that exact size.

Variants include a window buffer (last N messages) and a token buffer (as many recent messages as fit a token budget). It is lossless while it fits, but cost grows with each turn and old details eventually get cut.

It saves tokens and extends how long a session can run. The trade-off is lossy compression: names, numbers, or exact wording can be dropped, and summarisation errors can compound if the summary is re-summarised repeatedly.

A sensible safeguard is to keep names, dates, and numbers in a separate structured list, so they never depend on the summariser getting them right.

A checkpointer saves a snapshot of the agent's state after each step, keyed by a thread ID, so the run can be resumed, replayed, or inspected later. LangGraph is the best-known example, with in-memory, SQLite, and Postgres implementations.

It provides thread-level short-term memory and enables human-in-the-loop pauses and time travel debugging. It does not share information across threads; for that you need a separate store.

graph = builder.compile(checkpointer=PostgresSaver(conn))
graph.invoke(inputs, {"configurable": {"thread_id": "user-42-chat-1"}})

After every turn the agent extracts entities and updates a small record, for example Acme Corp: customer since 2021, prefers email. When the entity comes up again, only its record is injected, which is cheaper and more precise than replaying the whole history.

The main risk is wrong merges, for example treating two different people named Sam as one record, so good entity memory needs a stable ID per entity.

It shines for multi-hop questions such as "who manages the person who approved this invoice?" Tools like Zep's Graphiti and Neo4j-backed Mem0 setups build and update these graphs from conversations, often with timestamps so facts can expire.

The cost is upfront effort: you need an extraction step to turn text into entities and relations, plus rules for merging duplicates such as "Bob" and "Robert."

Ranking can combine semantic similarity, keyword match, recency, importance, and metadata filters such as user ID. Retrieving too little causes forgetting; retrieving too much wastes tokens and distracts the model.

In short, retrieval quality sets the ceiling for memory quality, since a perfect store is useless if the right item never reaches the prompt.

A thread ID (or session ID) labels one conversation so its history and state can be saved and loaded again. Everything in short-term memory is scoped to it.

It is different from a user ID, which groups memories across many threads. A good design uses both: the thread ID for conversation state, and the user ID or a namespace for long-term facts that should follow the person everywhere.

Reusing a thread ID resumes that conversation exactly where it stopped, while a fresh ID starts with a clean slate.

A background job typically merges duplicates, resolves contradictions, summarises episodes into lessons, and promotes important facts into a profile. It is often scheduled between sessions so the user does not wait for it.

The result is a smaller, more accurate store, which also makes later retrieval cheaper and less noisy.

Common policies are time-to-live (TTL) expiry, decay of importance scores over time, capacity limits that evict the least useful items, and explicit user requests like "forget my address." Without any forgetting, stores fill with outdated and contradictory facts.

Forgetting also supports privacy, since data that is expired on schedule cannot leak or be misused later.

It has core memory blocks always in the prompt, recall memory for searchable conversation history, and archival memory for long-term documents. The agent itself decides when to save, search, or edit, using function calls.

The memory tool lets Claude read, create, update, and delete files in a /memories directory so it can keep information between conversations. Claude decides when to use it, typically checking the directory at the start of a task.

It is client-side: Claude sends the tool calls and your application executes them against storage you control, such as local disk or a database. That means you own persistence, access control, and retention.

Because memories persist, the attack can outlive the original session and hit other tasks, or even other users if memory is shared. Defenses include validating what gets written, tagging memory by source and trust level, and letting users review stored memories.

Several open-source and managed options exist, and many agents combine more than one.

Vector stores like pgvector, Qdrant, or Pinecone usually sit underneath.

Short-term memory is session-scoped and in-prompt; long-term memory is persistent and external. The first is always seen by the model, the second only appears if it is retrieved.

A practical agent uses both: short-term for flow, long-term for continuity.

Both retrieve external text into the prompt, but they differ in what is stored and who writes it.

Agent memory often reuses RAG machinery (embeddings, vector search), so the two blur together, but memory has to handle writes, conflicts, and forgetting that plain RAG can ignore.

There are two broad patterns. In the hot path, the agent calls a save-memory tool during the conversation when it judges something worth keeping. In the background pattern, a separate process reads the finished conversation and extracts memories.

Either way, the decision usually relies on a prompt that asks: is this a durable fact, a stated preference, a correction, or a lesson, and is it likely to matter later? Small talk and one-off details should be skipped.

Good systems also check for duplicates and contradictions with existing entries before writing, so the store does not fill with near-identical lines.

On each turn the agent builds a query, searches memory, injects the best hits into the prompt, lets the model reason and act, and then optionally writes new memories.

flowchart LR
  A["User message"] --> B["Build query"]
  B --> C["Search memory store"]
  C --> D["Rank and filter top-k"]
  D --> E["Inject into prompt"]
  E --> F["LLM reasons and calls tools"]
  F --> G{Worth remembering?}
  G -- yes --> H["Write or update memory"]
  G -- no --> I["Reply to user"]
  H --> I

The query can be the raw user message, a model-rewritten search query, or the current sub-goal. Rewriting usually improves recall for vague follow-ups like "what about the second one?"

A memory item goes through write, store, retrieve, update, and eventually expire.

stateDiagram-v2
  [*] --> Extracted
  Extracted --> Validated: dedupe and trust check
  Validated --> Stored: embed and save
  Stored --> Retrieved: matched by a query
  Retrieved --> Stored: used, importance bumped
  Stored --> Updated: new info conflicts
  Updated --> Stored
  Stored --> Expired: TTL or low score
  Expired --> [*]

1. Extract the candidate from conversation or tool output.
2. Validate it for duplicates, conflicts, and trust.
3. Store it with embedding and metadata such as user, time, and source.
4. Retrieve it when relevant, often raising its importance score.
5. Update or expire it when newer facts arrive or its TTL passes.

A larger window postpones the problem but does not solve it. Four things still break.

1. Cost and latency: every turn re-sends and re-reads the whole prompt, so long histories get expensive and slow.
2. Attention quality: models tend to use information in the middle of very long inputs less reliably, so recall degrades.
3. Persistence: the window is wiped between sessions; it cannot remember last week.
4. Noise: dumping everything in dilutes the signal. Selective retrieval gives the model only what matters.

Memory is about choosing what to show, not just fitting more in.

The Stanford Generative Agents paper (Park et al., 2023) ranks each memory in the stream by a weighted sum of three signals.

Each signal is normalised, then combined, and the top-scoring memories fill the prompt. This stops the agent from retrieving only the most similar item and ignoring recent or high-stakes events. Many later memory libraries borrow this three-factor idea.

MemGPT treats the prompt as main memory with a hard size limit and an external database as disk. The LLM acts as its own memory manager through function calls.

When the context nears its limit, a system message warns the model (memory pressure). The model can then call functions to write key facts to archival storage, edit its core memory blocks, or evict old messages that are summarised into recall storage. Later it calls search functions to page information back in.

flowchart TD
  A["Context near limit"] --> B["Warning to model"]
  B --> C["Model saves facts to archival"]
  C --> D["Old messages evicted and summarised"]
  D --> E["Model searches archive when needed"]
  E --> F["Relevant data paged into context"]

The key idea is that paging decisions are made by the agent itself, not by fixed truncation rules.

Choose by what the task needs to keep.

A common hybrid keeps the last few turns verbatim, a rolling summary of everything older, and a separate structured list of key facts that is never summarised away.

Pick a graph when the answer depends on relationships and when facts change over time. Pick a vector store when you need fuzzy semantic recall over unstructured text.

Many production agents combine them: vectors for broad recall, a graph for entities and links.

Treat memory writes as updates, not appends. When the user says "I moved to Denver," the old "lives in Austin" fact should be superseded.

1. On write, search for existing memories on the same subject.
2. Ask an LLM (or rule) to decide: add, update, delete, or ignore.
3. Keep timestamps and, if useful, mark the old fact as invalid rather than erasing it, which preserves history.
4. At retrieval time, prefer the newest item when two disagree.
5. For high-stakes facts, confirm with the user.

Mem0 uses exactly this ADD / UPDATE / DELETE / NOOP decision step, and temporal graphs like Graphiti store validity windows on each edge.

Work from write to read and find where the fact got lost.

1. Was it written? Inspect the store. If missing, the extraction prompt or save-tool trigger is too strict.
2. Was it written under the wrong key? Check user ID, namespace, or thread ID. A mismatch is the most common bug.
3. Was it retrieved? Log the query and top-k. If the fact exists but is not returned, tune the query rewriting, k, similarity threshold, or add keyword search.
4. Was it in the prompt but ignored? Move it closer to the end, label it clearly, or shorten surrounding noise.
5. Was it summarised away? Pull critical values into a structured fact list.

Irrelevant recall usually means retrieval is too loose or the stored items are too vague.

• Add a similarity threshold so weak matches are dropped instead of always filling top-k.
• Filter by metadata such as user, topic, or memory type before ranking.
• Rewrite the query with the current goal instead of using the raw last message.
• Rerank the top 20 with a cross-encoder or small LLM, then keep the best 3 to 5.
• Improve memory quality: store atomic, self-contained facts rather than long messy snippets.

Persistence turns a one-time trick into a lasting one. Three risks stand out.

1. Indirect prompt injection: a document the agent reads can instruct it to save a malicious "rule" that fires in later sessions.
2. Data leakage: memories can contain PII or secrets that show up for the wrong user if namespaces are weak.
3. Silent drift: wrong facts, once stored, keep influencing decisions without anyone noticing.

Mitigations: scope memory per user and tenant, record the source of every entry, never auto-store content that came from untrusted tool output without review, give users a way to view and delete memories, and log all writes.

Isolation has to be enforced in code and storage, never by asking the model to behave.

1. Namespace every record with tenant and user IDs, for example ("tenant_7", "user_42", "memories").
2. Apply the filter inside the data layer so a query without a valid scope returns nothing.
3. Take the user ID from your authenticated session, not from model output or user text.
4. Use separate indexes or row-level security (for example Postgres RLS with pgvector) for stricter tenants.
5. Add tests that try cross-user retrieval and expect zero results.

# LangGraph store: scoped namespace
store.put(("tenant_7", "user_42", "memories"), key, {"fact": "prefers metric units"})
hits = store.search(("tenant_7", "user_42", "memories"), query="units")

Retrieval sits on the critical path of every turn, so small wins add up.

• Use approximate nearest neighbour indexes such as HNSW instead of brute-force search once the store grows.
• Cache embeddings of frequent queries and skip retrieval when the message clearly needs no memory ("thanks!").
• Keep top-k small and add a threshold; every extra memory costs input tokens on every call.
• Store compact memories: short atomic facts embed and read cheaper than raw transcripts.
• Move writes off the hot path with background extraction.
• Keep memory text stable and placed consistently so prompt caching can hit.

Evaluate the pieces separately, then end to end.

Public benchmarks such as LoCoMo and LongMemEval test long-conversation recall, temporal reasoning, and knowledge updates. Still build a small test set from your own domain, since real users ask very different things.

Shared memory is useful for hand-offs, but it multiplies the risk of conflicts and leaks. The usual pattern is private plus shared layers.

1. Give each agent a private scratchpad for its own reasoning and tool noise.
2. Provide a shared store, such as a blackboard or common graph state, for facts and results that others need.
3. Define who may write what; for example only a coordinator writes final decisions.
4. Tag entries with the writing agent, time, and confidence so readers can judge them.
5. Use versioning or optimistic locking to avoid two agents overwriting each other.

Without these rules, one agent's bad guess becomes every agent's "fact."

It depends on the framework, and the behavior is often worse than people expect.

• Some APIs return an error when input plus requested output exceed the limit.
• Some frameworks silently truncate the oldest messages, which may delete the original task or system instructions.
• Agents with compaction or MemGPT-style paging summarise or evict old content to external storage.

Silent truncation is the dangerous case: the agent keeps running but has lost the goal or earlier decisions, so it drifts. Protect by pinning the system prompt and task, trimming bulky tool outputs early, and monitoring token counts so compaction runs before the hard limit.

The model is asked something like: what are the three most important insights from these events? The answers, for example "Maria prefers morning meetings," are stored as new memories that can themselves be retrieved and reflected on.

flowchart LR
  A["Raw observations"] --> B["Importance sum passes threshold"]
  B --> C["LLM asked for key insights"]
  C --> D["Insights stored as new memories"]
  D --> E["Retrieved later for planning"]

This turns a long list of episodes into compact, reusable knowledge and cuts retrieval noise.

Pure vector search is great at meaning but weak on exact tokens: order IDs, error codes, names, and rare terms. Keyword search (BM25) has the opposite strengths.

Many systems mix them: hot-path for explicit requests like "remember this," background for everything else.

Plan for privacy at design time, because deleting data from a pile of embeddings is painful later.

1. Minimise: store only what is useful; redact card numbers, passwords, and national IDs before writing.
2. Tag and index by user so you can find everything about a person quickly.
3. Support delete and export: remove the source text, its embeddings, graph edges, and any summaries derived from it. Do not forget backups and logs.
4. Apply retention limits with TTLs.
5. Show users what is remembered and let them edit it.

Summaries are a trap: a deleted fact can survive inside a summary or reflection derived from it, so track provenance of derived memories.

They complement each other: compaction keeps a long task alive, and the memory tool or store keeps what matters after the task ends.

Neither wins everywhere. The right pick depends on volume, structure, and how the agent accesses memory.

Claude's memory tool and many coding agents use files because they are simple to inspect and version. Large customer-facing systems with huge histories generally need a vector or hybrid index. A common path is to start with files and add vector search when navigation becomes the bottleneck.