1:1 mentoring with Big Tech AI engineers
LLM & Agentic

Agent Durability

Make a long agent run survive a crash: the window where a side effect lands before anything records it, idempotency keys that survive replay, and what to checkpoint.

Last updated

Staff10 min readFirst readThe Agent LoopHuman in the Loop: Approval Gates

After this section you can

  • Locate the window in each loop step where a crash repeats a side effect, and make the replay a no-op
  • Derive idempotency keys that survive a replay, and order writes so the keys stay stable
  • Checkpoint the transcript append-only and verbatim, and account for cold caches and reset budgets on resume
15

Agent Durability: Surviving a Crash Mid-Run

A long agent run is a distributed transaction nobody designed as one. When it dies at step 40, the useful question is what step 39 already did to the world.

Key idea

Replaying a model call costs tokens; replaying a tool can refund a customer twice. Each step has a window between the side effect and the record of it, and outside a single database transaction you cannot close it. So save the transcript append-only, save each model turn before running its tools, and key every side effect so a replay is a no-op.

Every step has a window where the world has changed and your log does not know yet
ONE STEP OF THE LOOP 1 · Load the transcript so far from the checkpoint 2 · Model call stateless: replaying it costs tokens only 3 · Run the tool the refund goes out the world changes here 4 · Record append the tool_result and commit THE WINDOW: effect landed, nothing records it CRASH HERE, THEN REPLAY During 1 or 2: harmless nothing outside the process moved replay costs one model call Inside the window: the real one the refund already went out replay sends it again, unless keyed After 4: harmless the result is in the transcript resume reads it and moves on You cannot close the window: two writes are never one write. You can make the replay inside it a no-op.

Related

More in LLM & Agentic

Tokens, Context Windows & Sampling

Preview

What a language model actually does on every call: split text into tokens, predict one next token at a time, fit everything into a fixed context window, and pick each token by sampling — with interactive temperature and context-budget explorers.

How LLMs Are Built

Preview

Complete lifecycle of large language models from pre-training through fine-tuning, RLHF, and deployment — with architecture diagrams and production considerations.

How LLMs Call Tools

Preview

How LLMs use function calling and tool use — the mechanics behind tool-calling agents, from prompt engineering to structured output.

Prompt Engineering Foundations

Preview

Where an instruction belongs — tool description, system prompt, or user turn — why emphasis that worked on older models now causes over-triggering, and the prompt patterns that have expired.

Get full access to all 74+ sections with code examples, diagrams, and interactive animations.

Unlock Premium