A Practical Guide for Real-World Distributed Systems
Distributed applications cannot rely on traditional ACID transactions. Once your solution crosses service boundaries, database locks, two-phase commits, and long-running transactions start failing in scale, reliability, and cost.
To solve this, modern architectures use Saga patterns — a set of steps in which each local transaction publishes an event, and the next service continues the flow. If something fails, compensating actions undo previous steps.
This article gives you a complete guide to designing transactional microservices using:
Total length is approx. 2000+ words.
Introduction
When a business operation spans multiple services, for example:
Place Order
Reserve Inventory
Process Payment
Generate Invoice
Schedule Delivery
You cannot depend on a single database transaction. Each service is independent, with its own storage and boundary.
You need a reliable distributed transaction mechanism. Saga is exactly that.
Why Traditional Transactions Fail in Microservices
| Problem | Explanation |
|---|
| Two-Phase Commit (2PC) | Slow, locking, not cloud friendly, not supported across polyglot databases |
| Long DB Transactions | Cause deadlocks, resource blocks |
| Global Locks | Break scalability |
| Partial Success | Common in network-distributed systems |
| Retry Storms | Cause duplicate actions |
Microservices need eventual consistency, retry-safe operations, and compensation workflows.
What Is a Saga?
Saga is a sequence of distributed steps, where each step has:
Example:
If any step fails, the orchestrator rolls back completed steps using compensation.
Choreography vs Orchestration
Choreography
Each service listens for events and decides next step.
No central coordinator.
Very easy to start, hard to maintain when flows grow.
Orchestration
One coordinator service manages the whole process.
Clear visibility, easier debugging.
Best for complex enterprise workflows.
For senior enterprise development, orchestration is recommended.
Architecture Diagram
┌─────────────────────┐
│ Angular Front-End │
│ (Order UI) │
└─────────┬───────────┘
│
▼
┌────────────────────────────────────────────────┐
│ API Gateway / BFF │
└─────────┬──────────────────────────────────────┘
│
▼
┌────────────────────┐
│ Saga Orchestrator │
│ (.NET Worker) │
└───┬───────────────┘
│ Commands / Events
▼
┌───────────────────┬────────────────────┬────────────────────┐
│ Order Service │ Inventory Service │ Payment Service │
│ (SQL) │ (SQL) │ (SQL) │
└───────────────────┴────────────────────┴────────────────────┘
Workflow Diagram
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Place Order │ ---> │ Reserve Stock│ ---> │ Process Payment │
└──────┬───────┘ └──────┬───────┘ └──────┬────────┘
│ │ │
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Confirm Order│ │ Ship Goods │ │ Finalize Saga │
└──────────────┘ └──────────────┘ └──────────────┘
Flowchart of a Saga Execution
┌─────────────────────────┐
│ Receive Order Command │
└───────────┬────────────┘
▼
┌─────────────────┐
│ Step 1: Reserve │
│ Inventory │
└───────┬────────┘
│ Success?
│
┌────────────▼─────────────┐
│ Yes │
▼ │
Step2: Charge Payment │
│ │
└──────────┬───────────────┘
│ Failure?
▼
┌────────────────────────┐
│ Trigger Compensation │
│ (Release Inventory) │
└────────────────────────┘
Designing Saga Steps
Each step must be:
Idempotent
Retry-safe
Side-effect safe
Serializable
Reversible (has compensation)
Example: Place Order Saga
| Step | Forward Action | Compensation |
|---|
| 1 | Reserve stock | Release stock |
| 2 | Charge payment | Refund payment |
| 3 | Confirm order | Cancel order |
| 4 | Generate invoice | Void invoice |
Compensation Rules
Compensation must be:
Non-blocking
Asynchronous
Eventually consistent
Idempotent
Compensation Flow
If Step N fails → Orchestrator triggers compensations of (N-1), (N-2), ... down to 1
Implementation in .NET (Orchestrator + Services)
Below is production-ready architecture.
1. Orchestrator
public class OrderSagaState
{
public Guid SagaId { get; set; }
public string Status { get; set; }
public int Step { get; set; }
}
2. Saga Orchestrator Handler
public class OrderSagaHandler : IConsumer<OrderPlacedEvent>
{
public async Task Consume(ConsumeContext<OrderPlacedEvent> context)
{
var saga = new OrderSagaState
{
SagaId = context.Message.OrderId,
Step = 1,
Status = "Started"
};
await context.Publish(new ReserveInventoryCommand(saga.SagaId));
}
}
3. Compensation Handler Example
public class ReleaseInventoryHandler : IConsumer<PaymentFailedEvent>
{
public async Task Consume(ConsumeContext<PaymentFailedEvent> ctx)
{
await _inventory.Release(ctx.Message.OrderId);
}
}
Angular Front-End Integration
Angular initiates the saga.
Trigger Create Order
this.http.post('/api/orders', payload).subscribe(result => {
this.status = "Order submitted. Processing...";
});
Poll Saga Status
this.http.get(`/api/saga/${orderId}`).subscribe(s => {
this.state = s;
});
Front-end must display the step-by-step progress.
Handling Partial Failures
Partial failures are the most critical part of a microservice saga.
Example Failure Case
Inventory reserved
Payment failed
Saga must:
Trigger compensation
Roll back inventory
Mark order as canceled
Publish final SagaFailed event
Observability, Logging, Monitoring
Use:
Example Trace ID propagation:
Angular → API → Orchestrator → Inventory → Payment
Testing and Simulation
You must test:
Retry scenarios
Delayed events
Duplicate events
Out-of-order events
Failure in compensation
Network partitions
Best Practices
Prefer orchestration for complex business flows
Always register compensations
Never block threads in long-running sagas
Use the Outbox Pattern to avoid message loss
Persist saga state durably
Ensure every handler is idempotent
Implement dead-letter queues
Keep operations small and atomic
Conclusion
Saga-based transactions are the foundation of robust microservice architectures. With orchestration, compensating actions and proper state management, you can build reliable, scalable, and traceable distributed workflows.
This model fits:
It ensures your domain operations remain consistent even in a distributed setup.