The Agent Memory Engineering Playbook

A language model is stateless. Every request starts from nothing; the only "memory" it has is whatever you stuff into the context window. An agent — something that acts over time, across sessions, on behalf of a user — needs more than that. It needs a past it can recall, update, and reason about. Building that is one of the defining engineering problems of 2026, and this playbook is the complete knowledge base for it.

Choosing a system? See the companion comparison: Mem0 vs Zep vs Letta. This playbook covers the engineering principles that apply no matter which you adopt.

1. Why Context Windows Are Not Memory

It is tempting to think a million-token context window solves memory. It does not, for three reasons:

• Cost. You pay for every token, every turn. Replaying an entire history on each request is linear cost growth for sub-linear value.
• Latency. More context means slower time-to-first-token. A bloated window makes every response sluggish.
• The "lost in the middle" problem. Models attend unevenly across long contexts; a fact buried at position 40,000 may be effectively invisible.

Memory is the discipline of putting the right small slice of the past into context at the right moment — and keeping the rest in cheaper, searchable storage.

2. The Four Types of Memory

Borrowing loosely from cognitive science, agent memory splits into four kinds. Most production systems need at least two.

The mistake beginners make is treating all memory as one undifferentiated vector dump. A preference ("vegetarian") and an event ("ate at Joe's on Tuesday") have different lifecycles and should be stored and retrieved differently.

3. The Memory Lifecycle

Every durable memory passes through five stages. Designing each explicitly is what separates a robust system from a leaky one.

text
EXTRACT ─▶ STORE ─▶ RETRIEVE ─▶ UPDATE ─▶ FORGET
   │          │         │          │         │
 what's    where &    which       handle    expire or
 worth     in what    slice for   contra-   archive
 keeping   shape      this turn   dictions  stale facts

A. Extract

Decide what is worth remembering. Storing everything poisons retrieval with noise. Common strategies:

• LLM-based extraction: after each turn, ask a model "what durable facts did we learn?" and store only those.
• Salience scoring: keep facts the user repeats or confirms; drop one-off chatter.

B. Store

Match the store to the memory type — vectors for fuzzy episodic recall, a graph for related entities and timelines, key-value for crisp preferences.

C. Retrieve

The heart of the system (see Section 5). Pull only the slice relevant to the current turn.

D. Update

When the user changes a preference or a fact becomes false, the old memory must be superseded — not silently duplicated. This is where naive vector stores fail: they accumulate contradictory snippets.

E. Forget

Memory that never expires becomes a liability — stale facts, privacy risk, and retrieval noise. Implement TTLs, archival tiers, and user-initiated deletion (a legal requirement under most privacy regimes).

4. Three Architectural Approaches

There is no single right architecture. The three dominant patterns trade simplicity for power:

Bolt-on memory layer (e.g. vector + graph + KV)

A module you attach beside an existing agent. It auto-extracts and retrieves with minimal code. Best for: chatbots and assistants that need persistence quickly. Weakness: less precise on temporal reasoning.

Temporal knowledge graph

Memory is modeled as timestamped facts and relationships. The agent knows not just what is true but when it became and stopped being true. Best for: applications where user state changes over time (subscriptions, account status, evolving preferences). Weakness: more modeling overhead.

OS-inspired memory hierarchy

The agent is its memory: a small always-in-context core (like RAM), a searchable recent tier (like cache), and an unbounded archive (like disk) the agent queries on demand — and the LLM manages what gets promoted. Best for: agents that must operate autonomously for days. Weakness: the heaviest engineering commitment.

5. Retrieval Strategies

Storage is easy; retrieval is where quality is won or lost. Options, often combined:

• Semantic (vector) search. Embed the query, find nearest-neighbor memories. Great for fuzzy recall, weak on precise filters.
• Recency weighting. Bias toward recent memories so the agent doesn't resurface ancient, superseded facts.
• Graph traversal. Start from the entities in the current turn and pull connected facts (and their timestamps).
• Hybrid + rerank. Retrieve a wide candidate set with vectors, then rerank with a cross-encoder or LLM to keep only the most relevant handful.
• Query rewriting. Expand the user's terse message into a richer retrieval query before searching.

Rule of thumb: retrieve more candidates than you need, then aggressively filter down to the 3–7 memories that actually belong in context. Stuffing 50 memories in is worse than 5 good ones.

6. The Temporal Problem (Contradictions)

The single hardest part of agent memory is handling facts that change.

User in January: "I live in Berlin." User in June: "I just moved to Lisbon."

A naive store now holds two contradictory "facts" and may retrieve either. Robust systems:

1. Timestamp every fact at write time.
2. Detect contradiction on update — the new fact about residence supersedes the old.
3. Mark, don't delete — keep the old fact with a "valid until" so the agent can still answer "where did I used to live?"
4. Retrieve the currently-valid version by default, biased by recency.

This is exactly why temporal-graph approaches exist — they make "when was this true?" a first-class query instead of an afterthought.

7. Evaluating Memory

You cannot improve what you don't measure. Memory has its own evaluation discipline:

Build memory regression tests: replay fixed multi-session conversations and assert the agent still answers correctly. Treat a memory regression like a code regression — block the release.

8. Anti-Patterns to Avoid

• The infinite vector dump. Storing every message with no extraction or expiry. Retrieval quality collapses as the store grows.
• No update path. Appending new facts without superseding old ones, guaranteeing contradictions.
• One store for everything. Forcing preferences, events, and procedures through the same vector index.
• Retrieving too much. Flooding the context window in the name of "completeness," reintroducing the lost-in-the-middle and cost problems memory was meant to solve.
• Ignoring privacy. No deletion path, no PII handling, no TTLs — a compliance incident waiting to happen.

9. The Implementation Checklist

• Memory types are differentiated (working / episodic / semantic / procedural).
• An explicit extraction step decides what's worth keeping.
• Facts are timestamped; updates supersede rather than duplicate.
• Retrieval pulls a wide candidate set, then reranks down to a handful.
• Recency weighting prevents resurfacing stale facts.
• Contradiction detection returns the currently-valid version.
• TTLs, archival, and user-initiated deletion are implemented.
• Memory regression tests run on every release.
• Per-turn memory latency and cost are monitored.

This playbook pairs with our agent memory systems comparison. Learn the principles here; pick the system there.