What is the Outbox Pattern and how does it solve the dual-write problem?

The dual-write problem arises when a service must atomically write to its own database and publish a message to a message broker in a single operation. If it writes to the DB but crashes before publishing, other services never learn about the change. If it publishes first but the DB write fails, it emits an event for something that never happened. Standard distributed transactions (2PC) across a database and a broker are too heavy and often unsupported.

The Outbox Pattern solves this by writing both the domain change and the message to-be-published in a single local database transaction, then using a relay process to forward outbox records to the broker asynchronously.

-- Same local DB transaction:
BEGIN;
  INSERT INTO orders (id, status, ...) VALUES (42, 'PENDING', ...);
  INSERT INTO outbox (id, aggregate_type, aggregate_id, event_type, payload)
       VALUES (gen_uuid(), 'Order', 42, 'OrderPlaced', '{...json...}');
COMMIT;

-- Relay process (Message Relay / Transactional Outbox Relay):
LOOP:
  rows = SELECT * FROM outbox WHERE published = false ORDER BY created_at LIMIT 100;
  FOR EACH row:
    broker.publish(topic=row.event_type, body=row.payload);
    UPDATE outbox SET published = true WHERE id = row.id;

Two relay strategies exist:

• Polling publisher — a background thread or scheduled job polls the outbox table and publishes unpublished records. Simple but adds slight latency and DB load.
• Transaction log tailing — tools like Debezium use the database's CDC (Change Data Capture) log (e.g., PostgreSQL WAL, MySQL binlog) to detect outbox inserts and publish them. Near-real-time with minimal DB overhead.

The relay must publish at-least-once (idempotency key = outbox row ID), so consumers must implement the Idempotent Consumer pattern (Q22) to handle rare duplicates gracefully.

Invest now in Acorns!!! 🚀
Join Acorns and get your $5 bonus!

Acorns is a micro-investing app that automatically invests your "spare change" from daily purchases into diversified, expert-built portfolios of ETFs. It is designed for beginners, allowing you to start investing with as little as $5. The service automates saving and investing. Disclosure: I may receive a referral bonus.

Invest now!!! Get Free equity stock (US, UK only)!

Use Robinhood app to invest in stocks. It is safe and secure. Use the Referral link to claim your free stock when you sign up!.

The Robinhood app makes it easy to trade stocks, crypto and more.

Webull! Receive free stock by signing up using the link: Webull signup.