Implementing the Transactional Outbox Pattern with EF Core for Reliable Microservices

In modern distributed systems and event-driven architectures, services frequently need to update a database and publish domain events (to a message broker like RabbitMQ, Kafka, or Azure Service Bus) as part of a single business transaction.

A naive implementation—updating the database and then publishing the message—introduces a classic reliability problem: the dual-write dilemma. If the database commit succeeds but the message broker goes down before the event is published, your event-driven subscribers will never receive the update, leading to data drift and broken workflows. Conversely, if you publish first and the database fails, you emit phantom events.

The industry-standard solution for this problem is the Transactional Outbox Pattern, implemented seamlessly using Entity Framework Core.

How the Outbox Pattern Works

Instead of publishing messages directly to an external message broker during your business logic execution, the outbox pattern relies on a two-step approach:

Write to Outbox Table: Inside the same database transaction where you save your domain entities, you insert an outbound message record into an OutboxMessages table. Because EF Core handles transactions natively, either both the entity update and the outbox record are saved, or neither is.
Relay Service: A separate background worker or processor polls the OutboxMessages table, reads unprocessed entries, publishes them to the message broker, and marks them as processed.

Step-by-Step Implementation

Step 1: Define the Outbox Message Entity

Create a model that represents the message you want to dispatch.

public class OutboxMessage
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public string Type { get; set; } = string.Empty; // Event type name
    public string Content { get; set; } = string.Empty; // Serialized JSON payload
    public DateTimeOffset OccurredOn { get; set; }
    public DateTimeOffset? ProcessedOn { get; set; }
    public string? Error { get; set; }
}

Step 2: Configure Your DbContext

Add the OutboxMessages DbSet to your DbContext:

public class AppDbContext : DbContext
{
    public DbSet<OutboxMessage> OutboxMessages => Set<OutboxMessage>();
    public DbSet<Order> Orders => Set<Order>();

    // ... constructors and OnModelCreating
}

Step 3: Write Domain Events to the Outbox in a Command Handler

When a business action occurs (e.g., creating an order), serialize the domain event and add it to the outbox within the exact same database context lifetime before calling SaveChangesAsync:

public async Task CreateOrderAsync(CreateOrderCommand command, AppDbContext dbContext, CancellationToken cancellationToken)
{
    var order = new Order 
    { 
        Id = Guid.NewGuid(), 
        CustomerName = command.CustomerName,
        TotalAmount = command.TotalAmount 
    };

    dbContext.Orders.Add(order);

    // Create the domain event
    var orderCreatedEvent = new OrderCreatedDomainEvent(order.Id, order.CustomerName);

    // Write to Outbox table within the same transaction scope
    var outboxMessage = new OutboxMessage
    {
        Type = typeof(OrderCreatedDomainEvent).Name,
        Content = JsonSerializer.Serialize(orderCreatedEvent),
        OccurredOn = DateTimeOffset.UtcNow
    };

    dbContext.OutboxMessages.Add(outboxMessage);

    // Atomic commit: Order and Outbox record are saved together
    await dbContext.SaveChangesAsync(cancellationToken);
}

Step 4: Implement a Background Processor to Dispatch Messages

Use a hosted background service (IHostedService or BackgroundService) to poll the database, fetch pending outbox messages, publish them to your message broker, and update their status.

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

public class OutboxProcessorBackgroundService : BackgroundService
{
    private readonly IServiceProvider _serviceProvider;
    private readonly ILogger<OutboxProcessorBackgroundService> _logger;

    public OutboxProcessorBackgroundService(IServiceProvider serviceProvider, ILogger<OutboxProcessorBackgroundService> logger)
    {
        _serviceProvider = serviceProvider;
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                using var scope = _serviceProvider.CreateScope();
                var dbContext = scope.ServiceProvider.GetRequiredService<AppDbContext>();
                
                // Fetch a batch of unprocessed messages
                var messages = await dbContext.OutboxMessages
                    .Where(m => m.ProcessedOn == null)
                    .OrderBy(m => m.OccurredOn)
                    .Take(20)
                    .ToListAsync(stoppingToken);

                foreach (var message in messages)
                {
                    try
                    {
                        // Publish to message broker (e.g., RabbitMQ, Azure Service Bus)
                        // await messageBus.PublishAsync(message.Type, message.Content);

                        message.ProcessedOn = DateTimeOffset.UtcNow;
                    }
                    catch (Exception ex)
                    {
                        message.Error = ex.Message;
                        _logger.LogError(ex, "Failed to process outbox message {MessageId}", message.Id);
                    }
                }

                if (messages.Count > 0)
                {
                    await dbContext.SaveChangesAsync(stoppingToken);
                }
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "An error occurred while processing outbox messages.");
            }

            // Poll interval
            await Task.Delay(TimeSpan.FromSeconds(5), stoppingToken);
        }
    }
}

Conclusion

By adopting the Transactional Outbox Pattern with EF Core, you eliminate the risk of lost messages during distributed writes. Your application gains transactional consistency across both its relational database and its downstream event consumers, ensuring robust, fault-tolerant cloud architecture.