Durable Execution for LLM Agents 2026: Temporal + LangGraph

Q: When does an LLM agent actually need durable execution rather than a simple in-process retry loop?

The architectural test is whether a 60-second outage in your agent runtime would require manually emailing customers about double-charged payments, duplicate Slack messages, or partially-completed multi-step workflows. If yes, you need durable execution from day one. If your agent is a single LLM call wrapped around one idempotent SQL write, you do not. The boundary is non-idempotent external side effects: Stripe charges, GitHub merges, Salesforce updates, Gmail sends, internal payment APIs without an idempotency parameter, computer-use clicks. As soon as the agent crosses two or more such side effects in a single logical task, the in-process retry loop becomes a liability — a pod crash mid-task either skips the remaining steps (the user thinks it worked, half the work is missing) or restarts from the top (the first half of side effects fires twice). Durable execution gives you a third option: resume from the last persisted step. The secondary trigger is total wall-clock duration. Anything over 30 seconds with non-trivial cost ($0.10+ in LLM calls) becomes meaningfully expensive to retry-from-the-top on a generic infrastructure flap, and durable execution amortises that cost across recovery. The 2026 production-grade rule of thumb across AppScale-reviewed deployments is that any agent whose logical task spans two or more distinct external systems and runs longer than 30 seconds is worth the durable-execution investment; agents below that threshold can sit on simpler in-process retry plus conversation-thread persistence in Postgres.

Q: How do I choose between Temporal, AWS Step Functions, Restate, DBOS, Inngest, and LangGraph checkpointers in 2026?

Five questions decide it. (1) Multi-language requirement? If yes, Temporal is the clear answer — Java, Go, Python, TypeScript, .NET, PHP all in production. If Python or TypeScript only, all of the above work. (2) Existing AWS commitment? If the team is deeply on AWS and the workflow is short (most agents are), Step Functions Express at $1/M transitions is the cheapest production-grade choice, with native integration to Lambda, Bedrock AgentCore, and the rest of the AWS surface. Step Functions Standard at $25/M transitions is right for long-running multi-day workflows where the audit and visualisation matter more than per-step cost. (3) Postgres-native preference? DBOS is the lowest-overhead option — workflow state is rows in your existing Postgres, no separate cluster, single-digit-millisecond per-step latency, and the best fit for teams already running heavy Postgres. (4) Vercel / Next.js / serverless-first stack? Inngest’s developer experience is the smoothest — functions feel native, deploys are seamless, but per-step pricing dominates at high volume. (5) LLM-agent-shaped graphs explicitly? LangGraph 1.x with PostgresSaver / RedisSaver / DynamoDBSaver is the lowest-friction choice when the workflow IS the LLM graph and you do not want a separate runtime alongside the agent. Most production teams in 2026 land on Temporal (multi-language, mature, multi-cloud) or Step Functions Express (AWS-native, cheap per step) for general agent runtimes, and add LangGraph checkpointers on top of either when the agent has a complex stateful graph. The trap is choosing on developer-tooling polish alone — Inngest looks the prettiest but at 100k+ steps/day the per-step price dominates; the right axis is total operational cost across one to two years, not first-week ergonomics.

Q: What is the replay-safety contract and what does it actually require me to change in my agent code?

Replay-safe means the workflow function produces the same branch decisions on replay that it did on the original execution, regardless of when or where it runs. In practice five rules. (1) No I/O in workflow code — every HTTP call, file read, database query, shell command lives inside an activity that the runtime captures and replays. The workflow function is essentially branching logic plus calls to activities. (2) No wall-clock or randomness in workflow code — replace Date.now() with workflow.now() (Temporal exposes a deterministic clock), Math.random() with workflow.uuid4() or a seeded PRNG whose seed is the workflow ID, timezone arithmetic with explicit deterministic UTC math. The runtime gives you primitives for all of these. (3) Versioning for code changes — Temporal’s workflow.patched(version_id) lets in-flight workflows that started before a code change continue on the old code path; LangGraph uses explicit checkpoint schema versions and graph.update_state migrations. Plan for version skew from day one because a six-week-running agent will outlive several deploys. (4) LLM determinism is not workflow determinism — the LLM call is non-deterministic but it is captured as an activity result, so replay returns the same string. The workflow code that branches on the result must be deterministic given the result. A common bug is "if random.random() < 0.5: model = claude else: model = gpt-5" inside the workflow function — move that random choice into a router activity. (5) Idempotency keys on every external side effect inside an activity — the runtime retries the activity on transient failures, so without an idempotency key derived from (workflow_id, step_id) the upstream API sees N calls and produces N side effects. The cost of adopting these rules in an existing codebase is roughly two engineer-weeks for a small agent and a clear linter rule (no Date.now() outside activities) catches 80% of the regressions in CI.

Q: Why is the idempotency key derivation so important and what is the right derivation in production?

Without idempotency keys the runtime’s own retries on transient failures (network timeout, 5xx, connection reset) duplicate side effects. Three derivations and only one is correct. Bad: uuid4() generated inside the activity — every retry generates a different key, the upstream sees N independent requests, the customer is charged N times. Bad: hash of the activity arguments — works for pure-function arguments but breaks the moment a timestamp, auto-incremented ID, or current locale enters the args (the same logical step gets a different hash on retry and double-charges). Good: a deterministic hash of (workflow_id, step_idx, step_logical_name) derived from the workflow’s identity plus the step’s position. Concretely idem_key = sha256(workflow_id + step_idx + step_logical_name).hexdigest()[:32] is the same across runtime-level retries within one logical step (so retry-on-transient is dedupe-safe at the upstream) but distinct across separate logical steps (so a workflow that legitimately sends two emails generates two distinct keys). The corollary is that every external API in your tool catalogue needs a documented idempotency posture. Stripe Idempotency-Key, GitHub client_mutation_id on GraphQL mutations, Slack client_msg_id, Twilio idempotency parameter, AWS SDK RequestToken / client-token on supported APIs, SendGrid — all support explicit idempotency. Internal APIs usually do not, and the architectural commitment is to retrofit idempotency into your own tool endpoints (a request_id UUID parameter, a request_id table with a unique constraint, return-the-prior-result-on-duplicate). The 2026 tool-calling schema design pattern treats idempotent:true and idempotency_param:"request_id" as required schema fields on every action-class tool.

Q: How do resumable SSE streams work end-to-end and what is the relationship between SSE event IDs and workflow checkpoint IDs?

The SSE endpoint emits events with an id: field that encodes (workflow_id, sequence_number). The browser’s built-in EventSource reconnects automatically on transient failure and sends Last-Event-ID: workflow_42:seq_137 as a header on the reconnect request. The server reads the header, looks up the workflow in the durable store, and replays events from seq_138 onward — either from a Redis stream or Kafka topic that buffers the last N minutes of events, or from a derived projection of the workflow history. The user sees one continuous stream; the network failure is invisible. Two subtleties bite teams that ship this naively. First, the buffer has to outlive the connection drop budget. Five to fifteen minutes is enough for mobile-network blips and proxy restarts but not for an actual host migration; for that you fall back to a regenerating-from-where-we-left-off UI affordance and synthesise a continuation prompt that the LLM accepts (typically the conversation history plus an explicit "continue your prior response from where the network failed"). Second, the SSE event ID is not the same as the workflow checkpoint ID. A single LLM-streaming activity emits hundreds of token events per checkpoint; the workflow checkpoints at activity boundaries (or per LangGraph node), not per token. The mapping is one workflow checkpoint to one starting SSE seq; events between checkpoints replay from the buffer; events after the last checkpoint require either a buffered tail in Redis stream or a regeneration. The streaming LLM response pattern covers the SSE primitives in depth; the durable-execution layer’s job is to give the SSE handler a stable workflow identity and a known recovery point. Production-grade implementations also expose a resume_token in the API so the client can reconnect across deploys, not just across transient failures.

Q: What is the cost overhead of adding durable execution to an existing agent and how does it compare to the cost of an outage?

At a reference workload of 100k agent runs per day, 8 steps average, 2 LLM calls per step, 1 tool call per step, the cost has three components. (1) Durable store: 800k step records per day plus state-blob updates. On Postgres with PostgresSaver and an 8-KB JSONB checkpoint column, that is 6.4 GB/day of writes — a single db.r6g.xlarge at $290/month handles it comfortably with 30 days of retention. On DynamoDB, 800k writes at 4 KB average is $1.40/day plus storage — roughly $50/month. On Temporal Cloud, equivalent persistence is roughly $200/month. (2) Per-step transition cost varies dramatically. Step Functions Standard at $0.025/1k transitions costs $20/day or $600/month at this scale, and is the reason action-heavy agents move to Step Functions Express at $1/M transitions ($0.80/day). Temporal Cloud charges per action; 800k actions/day at $25/M is $600/month. DBOS and Restate self-hosted have no per-step charge — only the Postgres storage cost. Inngest at production scale lands close to Step Functions Express. (3) Worker compute is usually the dominant total cost. The agent’s LLM calls are the same with or without durable execution — typically $400–$1,000/day at this load. Durable execution adds 5–15% wall-clock overhead and 2–8% additional compute spend. Net total: durable execution adds roughly 5–20% to total agent cost. Compare with the cost of an outage: a single host crash that double-charges 50 customers at $499 each is $25k of customer-refund work plus the engineering hours plus the reputation damage. The break-even is usually one prevented incident in the first six months. For agent products with any irreversible-action surface, durable execution’s ROI is unambiguous from the first day in production.

Q: How do I compose durable execution with multi-provider LLM disaster recovery from the inference DR playbook?

The two layers compose cleanly because they own different concerns. Durable execution owns the workflow’s position in the agent’s logic; multi-provider DR owns which model provider is healthy and serving the request. The clean composition is (a) the LLM call is a registered activity, (b) the activity’s implementation is a thin wrapper that calls the LLM gateway, (c) the gateway holds the per-provider routing decision and circuit-breaker state, (d) the activity captures the LLM response (whichever provider served it) and persists it in the workflow history, (e) the workflow code branches on the response, deterministic on replay. When the primary provider is down and the gateway fails over to a secondary, the activity simply succeeds on the secondary and the workflow continues; on replay (after a workflow recovery) the activity returns the persisted response from whichever provider produced it, so the workflow does not call out again. Two subtleties matter. First, the activity must be deterministic about which provider served the call — record the provider identifier in the activity result so audit and observability can reconstruct the request path. Second, a long-running activity that times out and is retried by the runtime might fail over between providers across retries, which is fine but produces slight non-determinism in the audit log; the architectural commitment is that the audit log captures every retry separately while the workflow only sees the final successful result. The practical effect is that durable execution and multi-provider DR are independent investments — you can ship one without the other and add the second later. Most teams ship durable execution first (because the in-process-retry pain is felt earlier) and add multi-provider DR later (because the provider-wide outage is a quarterly event, not a daily one). The reverse order is also fine; the two compose either way.

Q: I have an existing agent codebase with no durable execution. What is the migration path and how long does it take?

Five to seven engineer-weeks for a small team plus a quarter of refinement. Week 1: pick the runtime. Look at language preference, AWS-vs-multi-cloud commitment, and whether you need Postgres-native (DBOS) or already have a workflow culture (Temporal). Most teams in 2026 land on either Temporal (multi-language, mature) or LangGraph + PostgresSaver (Python-first, LLM-shaped graphs) or Step Functions Express (AWS-native, cheap per step). Week 2: identify activities. Walk every external call in the agent code (LLM provider, every tool, every database write that is not part of an idempotent transaction); each becomes a registered activity. Add idempotency keys derived from (workflow_id, step_id, step_name). Week 3: convert the agent main loop into a workflow function. Move all Date.now() and Math.random() to workflow.now() and workflow.uuid4(). Move every external call out of the workflow body into the activities you registered. Add a CI lint rule that fails on any direct fetch() or requests.get() inside the workflow file. Week 4: ship to staging behind a feature flag and run the existing test suite under the durable runtime. Most regressions are non-determinism (a datetime.now() somewhere in helper code, a UUID generated in the workflow, a config-file read that returns different values across environments). Week 5: ship to production for one workflow class (start with the lowest-stakes one), monitor checkpoint write rate and worker latency, validate that crash-and-recover actually resumes correctly with a chaos drill. Week 6: roll out to remaining workflow classes one at a time. Week 7+: refinement — add signals for HITL, add resumable SSE for the user-visible streams, add multi-region replication for the durable store. The single hardest week is Week 4 — most teams discover three to seven non-determinism bugs that pass tests but fail under replay. The mitigation is a deterministic-only linter and aggressive replay testing in CI.

Satyam Kumar

返回博客

ai-services-patterns

Durable Execution for LLM Agents — Temporal, LangGraph Checkpointers, and Resumable SSE (2026)

June 6, 202623 min read

Frequently Asked Questions

分享这篇文章

Twitter LinkedIn WhatsApp

Satyam

人工智能和云架构师。帮助团队构建可扩展到数百万的系统。

Durable Execution for LLM Agents — Temporal, LangGraph Checkpointers, and Resumable SSE (2026)

Frequently Asked Questions

分享这篇文章

Comments

Leave a comment