Outbox Pattern: Guaranteed Message Delivery in Microservices (2026)

Q: What is the Outbox pattern and why is it needed?

The Outbox pattern solves the dual-write problem: the impossibility of making a database write and a message broker publish atomically without a distributed transaction. Instead of writing to the database and publishing to a broker as two separate operations (which fail independently), the service writes both the business record and the outbound message to the same database transaction. A separate relay process delivers the message from the outbox table to the broker. This guarantees that any committed business record has a corresponding outbox entry — messages are never lost even if the publishing process crashes. The relay produces at-least-once delivery; consumers must be idempotent.

Q: What is the difference between a polling relay and a CDC relay?

A polling relay runs on a fixed interval, selects unpublished outbox entries from the database, publishes them to the broker, and marks them published. It is simple to implement with no additional infrastructure, but adds a polling latency of one to five seconds per event. A CDC (Change Data Capture) relay subscribes to the database replication stream — Postgres logical replication via Debezium, MySQL binlog — and forwards outbox INSERT events to Kafka in near-real-time with sub-second latency. CDC is operationally more complex (requires database replication slot configuration and a CDC connector) but eliminates polling latency. Use polling unless sub-second event delivery is a requirement.

Q: How does the Outbox pattern relate to the Saga pattern?

The Outbox pattern is the implementation detail that makes individual Saga steps reliably atomic at the message level. In a Saga orchestration, each participant step must atomically update its local database and send a reply or command to the orchestrator. Without Outbox, a process crash between the database commit and the message publish leaves the saga stuck — the local state changed but the orchestrator never received the confirmation. With Outbox, the reply message is written to the outbox in the same database transaction as the local state change. The relay delivers it to the orchestrator reliably. The two patterns are complementary: Saga handles the multi-step business transaction; Outbox handles reliable message delivery within each step.

Q: Does the Outbox pattern guarantee exactly-once delivery?

No. The Outbox pattern guarantees at-least-once delivery. The relay may publish a message to the broker and then crash before it marks the entry as published in the outbox table. On restart, it will re-publish the same message. Consumers must be designed for idempotency: processing the same message twice must produce the same result as processing it once. The standard approach is a consumer-side deduplication table that records processed message IDs; before processing a message, check whether its ID has been processed and discard duplicates. Where consumer operations are naturally idempotent (insert by unique ID, update by sequence number), explicit deduplication is unnecessary.

Q: How should the outbox table be managed in production?

Published outbox entries should be retained for a window of 7 to 30 days for replay support, then purged by a background job to prevent unbounded table growth. Index the table on (published, created_at) to support the relay's primary query pattern — a scan of unpublished entries ordered by creation time. Monitor two critical metrics: relay lag (the age of the oldest unpublished entry — should be seconds under normal operation) and failed entry count (entries that have exceeded retry threshold). Alert on relay lag with a threshold that catches relay failures within minutes. A relay that crashes silently and is not alerted on will cause consumer staleness that accumulates for hours before detection.

Q: Is the Outbox pattern the same as Event Sourcing?

No. Event Sourcing stores the full history of state changes as an ordered event log; current entity state is derived by replaying that log. This natively avoids the dual-write problem (the event log is the database; publishing from it is straightforward) but requires a radically different domain model throughout the application. The Outbox pattern solves the dual-write problem in a system that uses a conventional mutable-state database: a single outbox table and a relay process. For the majority of microservices that do not need Event Sourcing's full event history semantics, the Outbox pattern is the proportionate, lower-complexity solution to the reliable message delivery problem.

Satyam Kumar

عودة إلى المدونة

ai-architecture

Microservices Outbox Pattern: Guaranteed Message Delivery Without Dual Writes (2026)

By Satyam KumarApril 16, 202610 min read

outbox pattern microservices patterns message delivery dual write problem change data capture debezium kafka saga pattern event-driven architecture distributed systems microservices architecture system design

Microservices Outbox Pattern: Guaranteed Message Delivery Without Dual Writes (2026)

Frequently Asked Questions

شارك هذه المقالة

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

مهندس الذكاء الاصطناعي والسحابة. مساعدة الفرق على بناء أنظمة تتسع للملايين.

LinkedIn GitHub

Microservices Outbox Pattern: Guaranteed Message Delivery Without Dual Writes (2026)

Frequently Asked Questions

شارك هذه المقالة

Comments

Leave a comment