When a single business transaction spans multiple services or databases, a single database transaction will not save you. Two-phase commit (2PC) is the textbook answer, but in practice it is slow, fragile, and unsupported by most modern data stores. The saga pattern is what production distributed systems use instead.
A saga decomposes a long-running transaction into a sequence of local transactions, each in its own service, with a compensating action defined for every step. If a later step fails, the system runs the compensating actions for the steps that already succeeded. The result is eventual atomicity without locking resources across services.
A Concrete Example
Consider an order placement that touches four services: inventory, payments, shipping, and notifications. A naive implementation calls each in sequence and hopes none of them fails. A saga makes failure handling explicit.
1. Reserve inventory ↔ Release inventory
2. Charge payment ↔ Refund payment
3. Schedule shipment ↔ Cancel shipment
4. Send confirmation ↔ (no compensation needed)
Each forward step has a compensating step. If step 3 fails, the saga runs the compensation for step 2 and step 1. The customer never sees a partial order, and no resources are pinned waiting for global coordination.
Orchestration vs Choreography
Sagas come in two flavors based on who drives the workflow.
Orchestration uses a central coordinator — usually a state machine — that calls each service in turn and decides what to do when something fails.
class PlaceOrderSaga
{
public function execute(Order $order): SagaResult
{
$steps = [
new ReserveInventoryStep($order),
new ChargePaymentStep($order),
new ScheduleShipmentStep($order),
new SendConfirmationStep($order),
];
$completed = [];
foreach ($steps as $step) {
try {
$step->execute();
$completed[] = $step;
} catch (StepFailedException $e) {
$this->compensate(array_reverse($completed));
return SagaResult::failed($e->getMessage());
}
}
return SagaResult::completed();
}
private function compensate(array $steps): void
{
foreach ($steps as $step) {
$step->compensate();
}
}
}
Choreography has no coordinator. Each service emits an event when its step completes, and the next service listens for that event and acts. Compensation happens through compensating events.
OrderCreated → Inventory reserves
InventoryReserved → Payment charges
PaymentCharged → Shipping schedules
ShipmentScheduled → Notifications send
ShipmentFailed → Payment listens, refunds
PaymentRefunded → Inventory listens, releases
Orchestration is easier to reason about because the workflow is in one place. Choreography is more decoupled but harder to debug — the workflow only exists as a pattern across event subscriptions.
Compensating Actions Are Not Rollbacks
The most common saga mistake is treating compensations as rollbacks. They are not. Once a payment is charged, the money has moved. Compensating it means issuing a refund — a new transaction with its own ID, audit trail, and possibly its own delay before it clears.
This has real implications:
- Compensations can fail too. A refund can be declined. You need retry logic and a manual escalation path.
- The world moves between steps. An inventory item reserved at step 1 might be reserved by another saga before step 3 schedules its shipment. You need to handle the case where a "released" item is no longer available the same way.
- Compensations must be idempotent. A retry of a refund must not produce a second refund. Use idempotency keys.
Idempotency Is Mandatory
Every saga step — forward and compensating — must be safe to retry. If the network drops between "payment service confirms" and "saga records the success," the saga will retry, and the payment must not be charged twice.
class ChargePaymentStep
{
public function execute(): void
{
$idempotencyKey = "saga:{$this->sagaId}:step:charge";
$this->paymentApi->charge([
'amount' => $this->order->total,
'customer' => $this->order->customer_id,
'idempotency_key' => $idempotencyKey,
]);
}
}
The payment provider sees the same key on retry and returns the original result rather than charging again. This pattern applies to every external interaction in the saga.
State Machines for Orchestration
For orchestrated sagas, modeling the workflow as an explicit state machine pays off quickly. Each state corresponds to "the system after step N has succeeded." Transitions are triggered by step outcomes.
[Started]
↓ ReserveInventory.success
[InventoryReserved]
↓ ChargePayment.success ↓ ChargePayment.failure
[PaymentCharged] [Compensating]
↓ ScheduleShipment.success ↓ ReleaseInventory.success
[Completed] [Failed]
Persisting state after every transition means the saga can resume after a process restart. Without persistence, a crashed coordinator leaves the world in an inconsistent state with no record of what to do next.
When Not to Use Sagas
Sagas add complexity in exchange for distributed atomicity. They are overkill when:
- The work fits inside a single service and database. Use a local transaction.
- Steps are independent and partial completion is acceptable. Use a job queue with retry.
- The business does not actually require atomicity — partial states are tolerable with manual cleanup.
Sagas are valuable when you have a multi-step workflow where partial completion is a real business problem — orders, payments, fulfillment, account provisioning — and the steps cross service boundaries.
Tooling
For orchestration, dedicated workflow engines do most of the heavy lifting:
| Tool | Strength |
|---|---|
| Temporal | Code-first workflows, mature ecosystem |
| AWS Step Functions | Managed, JSON-defined state machines |
| Camunda | BPMN-based, strong on visual workflows |
| Cadence | Uber's predecessor to Temporal |
For choreographed sagas, your message broker (Kafka, RabbitMQ, SQS) is the substrate, and most of the saga logic lives in your services as event handlers.
The pattern itself is older than any of these tools, and you can implement a perfectly good saga with database tables, a queue worker, and discipline. The tools make it easier to reason about long-running workflows, retry exhaustion, and observability.
Working on a workflow that crosses service or database boundaries? We help teams design distributed transactions that survive contact with production. scopeforged.com