1:1 mentoring with Big Tech AI engineers
LLM & Agentic

Memory Decisions & Worked Examples

When to use which memory type, with three real-world scenarios: customer support, sales agent, and code assistant.

Last updated

Production9 min readFirst readMemory Implementation & Patterns

After this section you can

  • Choose the lowest memory layer that meets a product need, using the decision flow and tradeoff matrix
  • Pick the owner key (user, account or project) and explain what leaks or breaks if it is wrong
  • Defend a memory design for a support, sales or coding agent, including what the source of truth is
18

Memory Decisions & Worked Examples

Knowing the layers is the easy part. The design skill is choosing the lowest one that works, keying it to the right owner, and knowing when memory must defer to a live system. Three real systems, traced.

Key idea

Choose memory by two decisions: the lowest layer that meets the need, and the owner the memory belongs to (user, account or project). The owner key sets who can see it, who must be able to delete it, and what leaks if a filter is missing.

Take the first exit that meets the need. Every step to the right adds cost, privacy load and something to evaluate.
ASK IN THIS ORDER 1 · Needs anything beyond this one request? 2 · Is it the same for every user? 3 · Needed after this session ends? 4 · Who owns it? user, account, project yes no yes no yes no scope it No memory stateless call cheapest, nothing to leak Shared / org retrieval over docs (RAG) read-only to the agent Session history + compaction expires with a TTL Long-term, scoped keyed by the owner export and erase duties Most “we need memory” requests end at exit 2 or 3. Only exit 4 creates a store of personal data you must govern.

Related

More in LLM & Agentic

Get full access to all 74+ sections with code examples, diagrams, and interactive animations.

Unlock Premium