The Outbox Pattern: Reliable Messaging Without Distributed Transactions

When a service has to both update its database and publish a message to other services, there is a sharp edge: what happens if the database write succeeds and the message publish fails (or vice versa)? Without care, you end up with rows in the database that no other service knows about, or events that say things happened that did not happen.

The outbox pattern solves this with a single rule: every external message is first written to a local database table inside the same transaction as the business write. A separate process then reads the outbox table and publishes the messages. Database atomicity guarantees you never publish a message that is not backed by a committed row.

The Failure Mode It Prevents

Consider an order placement that writes to the database and then publishes an OrderPlaced event:

DB::transaction(function () use ($order) {
    Order::create($order);
});

$messageBus->publish(new OrderPlaced($order));

This looks fine until you think about failure modes:

• The database write succeeds, the application crashes before publishing. Result: the order exists, but downstream services do not know.
• The database write succeeds, the message broker is unreachable. Result: same problem.
• The publish succeeds, the database transaction rolls back later for some reason. Result: downstream services believe an order was placed that does not exist.

These are not theoretical. Distributed transactions across "your database" and "your message broker" do not exist on most stacks. Even with retry logic, you cannot make the publish atomic with the database write.

The Outbox

The pattern flips the order. Write the message to an outbox table in the same transaction as the business write. The database guarantees both succeed or both fail.

DB::transaction(function () use ($order) {
    Order::create($order);

    Outbox::create([
        'id' => Str::uuid(),
        'aggregate_type' => 'Order',
        'aggregate_id' => $order->id,
        'event_type' => 'OrderPlaced',
        'payload' => json_encode($order->toArray()),
        'created_at' => now(),
    ]);
});

A separate process polls the outbox table, publishes each unpublished message to the broker, and marks it as published.

class OutboxRelay
{
    public function process(): void
    {
        Outbox::whereNull('published_at')
            ->orderBy('id')
            ->limit(100)
            ->each(function ($row) {
                $this->broker->publish($row->event_type, $row->payload);
                $row->update(['published_at' => now()]);
            });
    }
}

If the relay crashes mid-publish, the next run sees the message as unpublished and tries again. The broker side must handle duplicates — at least once is the delivery guarantee, not exactly once.

Polling vs CDC

The relay can be implemented two ways.

Polling. A scheduled job (or worker) periodically reads the outbox table for new rows and publishes them. Simple, works on every database, has predictable load. The downside is latency — messages are delayed by the polling interval. A second-level interval keeps the latency tolerable; a millisecond-level interval starts to look like change data capture.

CDC (Change Data Capture). A tool like Debezium tails the database's transaction log and publishes outbox rows the instant they commit. Lower latency, no polling load, but requires running and operating CDC infrastructure.

For most teams, polling is the right starting point. The latency is fine for asynchronous events, and the operational footprint is one cron job. CDC is the right call once the load makes polling expensive, or when you need millisecond-level publish latency.

Idempotency on the Consumer Side

Because at-least-once delivery is the only guarantee, every consumer must be idempotent. The outbox row's UUID becomes the deduplication key.

class HandleOrderPlaced
{
    public function handle(OrderPlaced $event): void
    {
        $alreadyProcessed = ProcessedEvent::where('id', $event->messageId)
            ->exists();

        if ($alreadyProcessed) {
            return;
        }

        DB::transaction(function () use ($event) {
            // Do the work
            $this->updateInventory($event);

            // Record that we processed this message
            ProcessedEvent::create([
                'id' => $event->messageId,
                'processed_at' => now(),
            ]);
        });
    }
}

The deduplication table can be a small, indexed table per consumer. Periodically prune entries older than the broker's longest possible redelivery window.

Outbox Table Design

A workable outbox schema:

CREATE TABLE outbox (
    id UUID PRIMARY KEY,
    aggregate_type VARCHAR(64) NOT NULL,
    aggregate_id VARCHAR(64) NOT NULL,
    event_type VARCHAR(64) NOT NULL,
    payload JSONB NOT NULL,
    headers JSONB,
    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    published_at TIMESTAMPTZ
);

CREATE INDEX outbox_unpublished_idx ON outbox (created_at)
WHERE published_at IS NULL;

The partial index on unpublished rows keeps the relay's query fast even as the table grows. The aggregate columns help with debugging and (optionally) preserving order per aggregate.

Decide early whether to keep published rows in the outbox or delete them. Keeping them gives you an audit trail; deleting them keeps the table small. A common pattern is to keep them for 30 days and then archive.

Ordering Guarantees

The outbox pattern preserves ordering within a single transaction (one transaction's rows have monotonically increasing IDs). It does not guarantee ordering across transactions on different rows.

If you need strict ordering per aggregate (all events for order 1234 in commit order), the relay must respect aggregate_id when batching. Use the aggregate_id as the broker partition key so a single consumer per partition processes them in order.

If you need global ordering across all events, you have a different architectural problem and the outbox alone is not the solution.

Common Mistakes

• Publishing inside the transaction. Calling the broker before the transaction commits causes phantom publishes on rollback.
• No idempotency on the consumer side. At-least-once is the only delivery guarantee. Consumers must deduplicate.
• Reading published_at without an index. As the outbox grows, the relay query slows down. Use a partial index.
• No retention policy. A two-year-old outbox table is millions of rows of useless data slowing every query.
• Trusting the broker for ordering. Brokers preserve order within a partition. Across partitions, all bets are off.

Variants

• Transactional Outbox + Saga. For long-running multi-service workflows, combine the outbox pattern with a saga orchestrator. Each saga step writes its outcome to its own outbox; the orchestrator advances on each acknowledged event.
• Inbox pattern. The consumer side equivalent. Incoming messages are first written to an inbox table inside a transaction with the consumer's business write. This combines deduplication with the consumer's local atomicity.

Why It Matters

The outbox pattern is the closest most production systems get to "atomic database write and message publish." It is not glamorous, it requires discipline, and it is one of the most quietly important patterns in distributed systems engineering. Skip it and you spend half your incidents reconciling state between services that disagree about what happened.

Building a system where database writes and event publishing both need to happen — and you cannot afford for them to disagree? We help teams design reliable distributed workflows that survive partial failures. scopeforged.com