Lesson 4 · Principal Track · Senior → Staff

Memory, Compression & Failure Modes

~15 min · 3 modules · how long-running agents keep their footing — and the five ways context quietly breaks

Published

LLM memoryContext compressionFailure modesSemantic searchRetrieval augmented generation

Your bar: draw a clean line between memory and a knowledge base, explain how a long-running agent survives a fixed budget with summarization and sliding windows, and (this is the question interviewers keep returning to) diagnose why a context system gave a bad answer by classifying it as missing, wrong, outdated, conflicting, or excessive. 1

Three ideas, one sentence each. Each chip is one thing this lesson makes permanent:

Memory is what the system learned from us

a knowledge base is what’s true in general

long runs survive by compressing the old, keeping the recent

and most AI bugs are context bugs with five named shapes

Memory what I recall about a friend our history · their prefs system writes it over time Knowledge base the library · encyclopedia authored externally read-only reference truth Retrieval the act of looking it up Context what's in my head right now, while I answer
One human picture for the whole lesson. Memory = what you remember about a friend; knowledge base = the library; retrieval = looking something up; context = what's in your head as you answer. Both stores reach the model only by being retrieved into context.1
1

Memory vs Knowledge Base vs Retrieval

Four words people blur into one. Memory and knowledge base are stores; retrieval is the mechanism that pulls from them; context is the assembled window. The model holds none of them — it is stateless (Lesson 1).

Who writes it, and what it holds

TermWho writes itMutabilityWhat it holds
MemoryThe system / agent, over timeAppend & updateInteraction-derived facts, preferences, decisions, state of an ongoing task.
Knowledge baseExternal authorsRead-only (to the agent)Source documents and reference truth — what’s true in general.
Retrieval— (it’s a mechanism)—Nothing; it fetches from memory or the KB into the window (Lesson 2).
ContextThe assembly step, per callRebuilt each callThe window for ONE inference (Lesson 1).

Memory, pinned down

Is Memory is persisted state the system writes and recalls about an ongoing task or relationship — what happened, derived preferences, decisions made. Usually structured and keyed, grown by the agent across sessions.

Why it exists The model is stateless and forgets everything between calls. To act like it “remembers us,” the runtime must store facts externally and reload the relevant ones into the next window.

Like (world) Your personal recollection of a friend: their name, that they’re vegetarian, the argument you settled last month. You wrote it; you update it; you recall it when needed.

Like (code) A per-user session/profile store the service writes to and reads back — not the product catalog. Memory is the user’s row; the KB is the catalog.

Knowledge base, pinned down

Is The corpus of source documents and facts — the reference material. Authored externally, treated as read-only by the agent, the same for every user.

Why it exists The model’s training is frozen and generic. The KB supplies your domain truth — docs, policies, code — so answers are grounded in your facts, not the model’s guesses.

Like (world) The library or encyclopedia down the street. You don’t write it; you look things up in it. It’s true in general, not “true about us.”

Like (code) A read-only reference dataset or vector index of your documentation. Shared, versioned by its authors, queried — never mutated by the answering request.

The confusion that trips people up

✗ “Memory means the model remembers.”

✓ The model is stateless (Lesson 1). Memory is external storage the runtime reloads — the model only “remembers” what got retrieved into this window.

✗ “Memory and the knowledge base are the same store.”

✓ KB = what’s true in general (external, read-only). Memory = what I learned from us (system-written, updatable). Different owners, different lifecycle.

📥 Memory rule: Knowledge base = what’s true in general. Memory = what I learned from us. Both reach the model only by being retrieved into context.

Memory check
  • Who writes memory vs the knowledge base? → the system/agent writes memory over time; the KB is authored externally and read-only to the agent
  • How does either store actually reach the model? → only via retrieval into the context window — the model has no direct access to either
  • Why is "the model remembers us" wrong? → the model is stateless; the runtime reloads stored facts each call, the model itself keeps nothing

A teammate stores a user’s preferences in the knowledge base index alongside the docs. Why is that a design smell?

Hit these points: the KB is shared, read-only reference truth; per-user preferences are mutable, user-scoped memory → mixing them pollutes everyone’s retrieval (one user’s facts can surface for another) and breaks isolation/privacy → lifecycles differ: KB is versioned by authors, memory is written constantly by the agent → keep them separate stores, retrieve from both into context, and tag provenance so you know which is which.

2

Context Compression for Long Runs

A long-running agent’s history outgrows any window (Lesson 1’s fixed budget). The fix isn’t a bigger window — it’s storing the full transcript externally and packing only a compressed, relevant slice each call.

A long-running agent's context, over time Full transcript stored externally (memory) turn 1 … 12 (old) turn 13 … 40 (old) turn 41 … 88 (old) turn 89 … 92 (recent) nothing is lost here summarize verbatim The window (one call · fixed budget) System prompt & instructions Rolling summary of everything olderlossy · regenerated as budget fills Last N turns, verbatimrecent detail kept intact Retrieved facts (from KB / memory)pulled in for THIS question
The whole transcript lives in storage; the window is assembled fresh each call. As the budget fills, old turns get summarized while recent turns and decisions stay verbatim — this is Lesson 4 M1 in action: store it all, retrieve on demand.2

The four moves, pinned down

TechniqueWhat it doesLossWhen
SummarizationCondense old history into a shorter summary.LossyHistory too long to carry verbatim.
DistillationExtract the salient facts/decisions/state; drop the narrative.LossyYou need the conclusions, not the chatter.
CompressionRemove redundancy/boilerplate, dedupe, prune low-value tokens.Near-losslessContent is repetitive or padded.
Sliding windowKeep the last N turns verbatim + a rolling summary of all older.MixedThe default for long chats/agents.

Is Compression is the family of moves that trade fidelity for room in a fixed budget: summarize, distill, dedupe, or window so the most useful tokens survive and the rest is stored, not carried.

Why it exists The budget is fixed (Lesson 1) but a long run’s history is unbounded. Something has to give — and dropping the right tokens is cheaper and smarter than truncating blindly from the top.

Like (world) Meeting minutes: you don’t transcribe every word, you record the decisions and action items, and keep the raw recording filed in case someone needs it later.

Like (code) Log rollup / compaction: keep recent events at full detail, aggregate old ones into summaries, archive the raw stream for replay on demand.

The trade-offs to name out loud

✗ “Summarize aggressively — it always saves money.”

✓ Summarizing costs extra LLM calls, and it’s lossy: a detail dropped now (an ID, a constraint) can be the one that matters three turns later.

✗ “Compress everything uniformly to fit.”

✓ Compress the OLD; keep the RECENT and the DECISIONS/IDs verbatim. Compress as you approach the budget, not pre-emptively.

📥 Memory rule: Compression trades fidelity for room. Summarize the old, keep the recent and the decisions verbatim, store the rest and retrieve on demand.

Memory check
  • Which two techniques are lossy, and what's the risk? → summarization and distillation; they can drop a detail (ID, constraint) that later turns out to matter
  • What does a sliding window keep verbatim vs summarize? → last N turns verbatim + a rolling summary of everything older
  • When should you compress? → as you approach the budget — not pre-emptively; keep recent + decisions/IDs verbatim

Your agent loses track of a database ID it was told 20 turns ago, after the history was summarized. What went wrong and how do you fix it?

Hit these points: root cause is lossy summarization — the ID lived in an old turn that got condensed into prose and dropped → fix the policy, not just this case: distill structured state (IDs, decisions, constraints) into a durable key-value scratchpad that’s always kept verbatim, separate from the narrative summary → keep recent turns verbatim so freshly-mentioned facts survive → and store the full transcript so you can retrieve the original turn on demand rather than relying on the summary to be complete.

3

The Five Failure Modes

When an AI system gives a bad answer, the model is rarely the culprit. The context is. Most failures fit one of five named shapes; once you name the shape, the fix follows.

Five ways context breaks Missing never retrieved "I don't know" Wrong irrelevant pulled confident, off Outdated stale index/cache old price/API Conflicting sources disagree contradictory Excessive too much packed lost in middle Same LLM — bad answer not the model's fault: the window was wrong Classify the shape → the fix is obvious
All five funnel into the same symptom — a bad answer from a fine model. The diagnostic skill is sorting which shape it is; the table below maps each to its root cause and fix.3

Symptom → cause → fix (the centerpiece)

Failure modeSymptomRoot causeFix
Missing”I don’t know” or hallucinated filler.Recall gap; not in KB; chunking fragmented it.Improve recall (hybrid), better chunking, add the source.
WrongConfident but off-topic / incorrect.Embedding “topically-similar-but-wrong”; no rerank.Rerank, metadata filters, hybrid + exact match.
OutdatedAnswers from old data (old price, old API).Index not refreshed; cached doc.Freshness/invalidation, re-index, live fetch for volatile data.
ConflictingContradictory / arbitrary answer.Duplicates/versions; no source-of-truth precedence.Dedupe, rank by authority/recency, pass provenance.
ExcessiveSlower, costlier; key fact ignored.Dump-everything; no selection (“lost in the middle”).Select less but right; cite position bias.4

Is A failure mode is a named pattern of context defect that produces a bad answer despite a capable model. The five — missing, wrong, outdated, conflicting, excessive — cover the overwhelming majority.

Why it exists The model can’t tell you it got bad input — it answers fluently from whatever’s in the window (Lesson 2’s silent failure). So failures surface as confident wrongness, and you must classify by hand.

Like (world) A doctor with a wrong, incomplete, stale, or contradictory chart — same skill, bad answer. Or a desk so buried in paper the one key memo gets overlooked (excessive).

Like (code) Garbage-in / garbage-out, but silent: no exception, no 500. Like a query returning wrong rows that the app renders confidently — you only catch it by auditing the inputs.

✗ “More context always helps the model.”

✓ Excessive context is its own failure mode — it’s slower, costlier, and the key fact gets “lost in the middle” of a long window.4

✗ “A wrong answer means we need a smarter model.”

✓ Classify it first. Most wrong answers are a context defect — fixing retrieval is cheaper and fixes the root cause; a bigger model just reasons better over bad input.

📥 Memory rule: Most AI bugs are context bugs. Classify it — missing / wrong / outdated / conflicting / excessive — then the fix is obvious.

Memory check
  • Which mode causes a confident but off-topic answer, and what fixes it? → wrong context (topically-similar-but-wrong); fix with rerank, filters, hybrid + exact match
  • Two sources disagree in-window — name it and the fix. → conflicting context; dedupe, rank by authority/recency, pass provenance / source-of-truth precedence
  • Why is "dump everything" a failure mode, not a safe default? → excessive context: slower, costlier, and the key fact gets lost in the middle (position bias)

Your assistant quotes last quarter’s pricing. Walk through diagnosing it with the failure-mode framework.

Hit these points: classify the symptom — it answered from old data, so this is outdated context, not a model defect → verify by inspecting what was retrieved: a stale index entry or cached doc with last quarter’s price → root cause is freshness — the index wasn’t re-indexed after the price changed → fixes: invalidate/re-index on source change, add a freshness/TTL signal, and for volatile fields like price prefer a live structured fetch over an embedded snapshot → prevention: monitor index staleness and treat volatile data differently from static docs.

Two retrieved docs give different answers to the same question. Why does this happen and how do you make the system deterministic?

Hit these points: this is conflicting context — duplicates or multiple versions both got retrieved, with no precedence → the model picks arbitrarily, so the same query can flip answers → fixes: dedupe near-identical chunks, establish a source-of-truth ranking (authority + recency), and drop or down-rank the loser → pass provenance into the window so the model (and your logs) can see which source won → prevention: version your KB and avoid indexing the same fact from multiple uncontrolled sources.

Retrieval practice — test the three modules

Q1. The clearest line between memory and a knowledge base is…

Q2. Both memory and the knowledge base reach the model by…

Q3. A sliding window for a long-running agent keeps…

Q4. The model answers confidently but completely off-topic. The most likely failure mode is…

Q5. Packing far more context than needed mainly hurts because…

Interview — pick your bar

Answer out loud in ~60s, then reveal. Core = recall · Senior = trade-offs & failure modes · Staff = synthesis under ambiguity · System Design = open design round (a different axis, not a harder level).

Distinguish memory, knowledge base, retrieval, and context. Use a human analogy.
Hit these points: memory = persisted state the system writes about us (interaction-derived facts, preferences, decisions; updatable) — like your recollection of a friend → knowledge base = external, read-only reference truth, the same for everyone — like the library/encyclopedia → retrieval = the mechanism that looks things up and pulls them into the window; it stores nothing itself → context = the window assembled for ONE call → the model holds none of them — it's stateless, so both stores reach it only by being retrieved into context.
Who writes memory vs the knowledge base, and how does each one actually reach the model?
Hit these points: the system/agent writes memory over time — append & update, user-scoped → the KB is authored externally and read-only to the agent, shared across users → neither has a direct line to the model: both reach it only via retrieval into the context window → the slogan: KB = what's true in general, memory = what I learned from us → different owners, different lifecycle, different mutability.
Name the four compression moves and which ones are lossy.
Hit these points: summarization — condense old history into prose (lossy) → distillation — extract the salient facts/decisions/state, drop the narrative (lossy) → compression/pruning — dedupe and strip redundancy/boilerplate (near-lossless) → sliding window — keep the last N turns verbatim plus a rolling summary of everything older (mixed) → the rule: summarize the OLD, keep the RECENT and the decisions/IDs verbatim.
List the five failure modes with one symptom each.
Hit these points: missing — "I don't know" or hallucinated filler → wrong — confident but off-topic/incorrect → outdated — answers from old data (old price/API) → conflicting — contradictory or arbitrary answer that flips between runs → excessive — slower, costlier, and the key fact gets ignored ("lost in the middle") → the framing: most AI bugs are context bugs, not model bugs.
A teammate stores a user's preferences in the knowledge base index alongside the docs. Why is that a design smell?
Hit these points: the KB is shared, read-only reference truth; per-user preferences are mutable, user-scoped memory → mixing them pollutes everyone's retrieval — one user's facts can surface for another — and breaks isolation/privacy → lifecycles differ: the KB is versioned by authors, memory is written constantly by the agent → keep them as separate stores, retrieve from both into context, and tag provenance so you know which is which.
"Summarize aggressively to save money." Where does that reasoning break down?
Hit these points: summarizing isn't free — each summary is an extra LLM call, so aggressive summarization can cost more, not less → it's lossy: a detail dropped now (an ID, a constraint) can be the exact one that matters three turns later → compress the OLD, keep the RECENT and load-bearing facts verbatim → compress as you approach the budget, not pre-emptively → if the issue is redundancy, prefer near-lossless dedupe/pruning; if it's "we might need it later," store-and-retrieve-on-demand beats summarize-and-hope.
Your agent loses track of a database ID it was told 20 turns ago, after the history was summarized. Diagnose and fix.
Hit these points: root cause is lossy summarization — the ID lived in an old turn that got condensed into prose and dropped → fix the policy, not just this case: distill structured state (IDs, decisions, constraints) into a durable key-value scratchpad that's always kept verbatim, separate from the narrative summary → keep recent turns verbatim so freshly-mentioned facts survive → store the full transcript so you can retrieve the original turn on demand rather than trusting the summary to be complete.
An assistant quotes last quarter's pricing. Walk the failure-mode framework to the fix.
Hit these points: classify the symptom — it answered from old data, so this is outdated context, not a model defect → verify by inspecting what was retrieved: a stale index entry or cached doc with last quarter's price → root cause is freshness — the index wasn't re-indexed after the price changed → fixes: invalidate/re-index on source change, add a freshness/TTL signal, and for volatile fields like price prefer a live structured fetch over an embedded snapshot → prevention: monitor index staleness and treat volatile data differently from static docs.
Design the memory + context strategy for an agent that runs for hundreds of turns on a fixed budget.
Hit these points: store the full transcript externally (memory) — never rely on the window to hold it all → assemble each window from system prompt + rolling summary of old turns + last N turns verbatim + facts retrieved for THIS question → keep decisions, IDs, and constraints in a structured scratchpad that's always verbatim, since summarization is lossy → compress only as you approach the budget, and budget for the extra LLM calls summarizing costs → retrieve old detail on demand from storage rather than carrying it; name the trade-off you're making — fidelity for room — out loud.
Two retrieved docs answer the same question differently and the answer flips between runs. Classify it and make the system deterministic.
Hit these points: this is conflicting context — duplicates or multiple versions both got retrieved with no precedence, so the model picks arbitrarily → fixes: dedupe near-identical chunks; establish a source-of-truth ranking (authority + recency) and drop or down-rank the loser → pass provenance into the window so the model — and your logs — can see which source won → prevention: version the KB and stop indexing the same fact from multiple uncontrolled sources → note the security edge: if the "loser" is untrusted, treat retrieved text as data, not instructions (prompt injection).5
Production incident: the model is fluent and confident but the answer is just wrong. Walk through classifying it before touching the model.
Hit these points: resist "we need a smarter model" — most wrong answers are a context defect, and a bigger model just reasons better over bad input → pull the actual retrieved window and ask which of the five shapes it is: was the right source never retrieved (missing)? topically-similar-but-wrong with no rerank (wrong)? stale (outdated)? contradicted by a duplicate (conflicting)? buried in a huge dump (excessive)? → each shape has its own cheap fix — recall/chunking, rerank/filters, freshness, dedupe/precedence, select-less-but-right → the discipline is classify first; the fix follows from the shape, and it's almost always cheaper than swapping the model.
Design-round framework — there's no single right answer; a strong candidate narrates these steps out loud:
  1. Clarify the run shape — multi-session over days vs one long run for hours, number of users, latency/cost budget, what "remembering" must mean.
  2. Separate the stores — user-scoped memory (writable) vs shared read-only KB; decide what's worth persisting vs derivable.
  3. Define the window recipe — system prompt + rolling summary + last N verbatim + retrieved facts, and the fixed token budget it must fit.
  4. Compression policy — what's summarized vs kept verbatim (decisions/IDs/constraints), and when compression triggers (near budget, not pre-emptively).
  5. Retrieval & freshness — how memory and KB get queried per call, plus invalidation/TTL for volatile facts.
  6. Failure-mode guardrails — provenance, dedupe/precedence, recall monitoring, and treating retrieved text as data.
  7. Cost, eval & observability — the extra LLM calls compression adds, how you'd test recall, and what you log to diagnose the five modes.
Design the memory and compression strategy for a multi-session customer-support agent that talks to the same users for months.
A strong answer covers: two distinct stores — user-scoped memory (their account facts, past tickets, derived preferences, open issues; writable, private per user) and a shared read-only KB (product docs, policies) — never co-mingled in one index → per session, assemble the window from system prompt + a rolling summary of this user's history + the last N turns verbatim + KB facts retrieved for the current question → persist structured state (decisions, IDs, entitlements) verbatim in a scratchpad, since summarization is lossy and a dropped account ID is a real bug → freshness matters: invalidate cached account state on change and prefer a live fetch for volatile fields (balance, plan) over an embedded snapshot → guard the five modes — provenance + authority/recency precedence for conflicting policy versions, recall checks for missing answers, and select-less-but-right to avoid burying the key fact → call the cost and privacy trade-offs: per-user memory is storage + retrieval cost and a data-retention/PII surface, so scope, encrypt, and expire it.
Design the context strategy for an autonomous agent that runs for hours on a single task (e.g. a long research or migration job).
A strong answer covers: the history is unbounded but the budget is fixed, so the full transcript lives in external storage and the window is assembled fresh each call — store it all, retrieve on demand → window recipe: system prompt + objective/plan + a rolling summary of completed work + the last N turns verbatim + facts retrieved for the current step → keep a durable, always-verbatim scratchpad of decisions, IDs, intermediate results, and constraints so a lossy summary can never drop a load-bearing fact → trigger compression only as you approach the budget, and account for the extra LLM calls each summary costs over a multi-hour run → checkpoint state so the run is resumable, and detect drift — if the summary diverges from the goal, re-retrieve the original turns → close on failure modes: excessive context is the silent killer over long runs (lost-in-the-middle), so curate aggressively rather than dumping the whole history forward.
Sources
Anthropic, "Effective context engineering for AI agents" — anthropic.com/engineering. Context as a finite, curated resource; curation and compression. (Numbers illustrative.)
Anthropic, "Building effective agents" — anthropic.com/engineering. Memory and long-running agents; storing state externally and reloading on demand.
Anthropic, "Effective context engineering for AI agents" — anthropic.com/engineering. Selection over dumping; why curation prevents most context-quality failures.
Liu et al., 2023, "Lost in the Middle: How Language Models Use Long Contexts" — arXiv:2307.03172. Position bias — excessive context buries the relevant fact in the middle.
OWASP, "Top 10 for LLM Applications" — owasp.org. Prompt injection via poisoned/conflicting retrieved context as a named threat.
Was this lesson helpful? Thanks — noted.

These notes reflect my current understanding and are updated as I learn, build, and discover better explanations.