1:1 mentoring with Big Tech AI engineers
LLM & Agentic

Stateless vs Stateful

Stateless vs stateful LLM architectures: trade-offs for agent design, conversation management, and production deployment.

Last updated

Foundations8 min readFirst readTokens, Context Windows & Sampling

After this section you can

  • Explain why a conversation is the full messages array re-sent every turn, and estimate how its tokens and cost grow
  • Choose where the transcript lives (client, your store, provider session) and defend the choice
  • Spot the in-process state trap and describe the load, call, save pattern that fixes it
03

Stateless vs Stateful: Where Conversation State Lives

The model forgets everything the moment it answers. A chat that seems to remember is your code re-sending the conversation every turn, so the real design question is where that conversation lives in between.

Key idea

The model and the API you call (for Claude, the Messages API) keep nothing between calls. A conversation is the messages array you send in full on every turn. Pick one place to keep that array between turns, and keep your servers free of it.

The model never remembers. Each design only decides who keeps the transcript and re-sends it.
1 · THE CLIENT CARRIES IT Client app holds the whole messages[] array full array, every turn Your server forwards it, remembers nothing full array Model API stateless: reads, answers, forgets Simplest. Payload grows, client can edit it. 2 · YOUR STORE HOLDS IT Client app sends the new message + session id new message Any server stateless worker Store Redis / DB full array Model API stateless: reads, answers, forgets The production default. 3 · THE PROVIDER HOLDS IT Client app sends the new message new message Your server sends the message + session id message + session id Provider session stores history, re-feeds the model Least code. Least control over what is kept. Purple marks where the transcript lives between turns. In every design, the model receives the full history on every call.

Related

More in LLM & Agentic

Tokens, Context Windows & Sampling

Preview

What a language model actually does on every call: split text into tokens, predict one next token at a time, fit everything into a fixed context window, and pick each token by sampling — with interactive temperature and context-budget explorers.

How LLMs Are Built

Preview

Complete lifecycle of large language models from pre-training through fine-tuning, RLHF, and deployment — with architecture diagrams and production considerations.

How LLMs Call Tools

Preview

How LLMs use function calling and tool use — the mechanics behind tool-calling agents, from prompt engineering to structured output.

Prompt Engineering Foundations

Preview

Where an instruction belongs — tool description, system prompt, or user turn — why emphasis that worked on older models now causes over-triggering, and the prompt patterns that have expired.

Get full access to all 74+ sections with code examples, diagrams, and interactive animations.

Unlock Premium