Event-Driven Agent Workflows with Azure Service Bus and .NET

Synchronous AI Request Bottlenecks

Most initial AI applications operate on a synchronous HTTP request-response loop: a client sends a prompt, the web server forwards it to a Large Language Model (LLM), waits for generation to complete, and returns the output. While this pattern works well for real-time chat widgets, it breaks down when applied to long-running enterprise agent workflows.

Enterprise tasks—such as automated loan underwriting, multi-document legal reviews, background IT log diagnostics, or complex supply chain planning—frequently require multiple agent iterations, tool invocations, and database checks. Executing these workflows inside a synchronous HTTP request handler creates critical operational failures:

• HTTP Timeouts and Connection Drops: LLM generation chains and tool calls can take several minutes to complete, far exceeding standard HTTP gateway timeouts (such as 230 seconds on Azure App Service or 60 seconds on API Gateways).

• Resource Exhaustion under Load: Holding open synchronous ASP.NET Core thread-pool worker threads while waiting for remote LLM streaming API calls consumes server memory and starves web applications under concurrent traffic spikes.

• Lack of Workload Persistence: If an application instance restarts, scales out, or crashes mid-execution during a long multi-agent task, the entire execution context is lost with no built-in retry mechanism.

• Tight Coupling Across Services: Hardcoding direct HTTP calls between service APIs and AI agents prevents independent service scaling, creates blast-radius vulnerabilities, and complicates background worker scheduling.

An Event-Driven Agent Architecture decouples client interaction from AI processing. By leveraging Azure Service Bus alongside ASP.NET Core, background worker processes, and Microsoft.Extensions.AI / Microsoft Agent Framework, developers can build resilient, asynchronous AI workflows that handle peak traffic loads, recover automatically from transient failures, and scale worker instances independently.

Architectural Topology: Synchronous Request-Reply vs. Event-Driven Messaging

In an event-driven agent topology, incoming business triggers (e.g., OrderSubmitted, DocumentUploaded, or TicketCreated) publish messages to Azure Service Bus Topics or Queues. Background worker consumers process these messages asynchronously, persisting execution state to durable storage (such as Azure Cosmos DB) and notifying clients via status polling or WebSockets.

┌─────────────────────────────────────────────────────────────┐
│             Client Application / HTTP Gateway               │
└──────────────┬──────────────────────────────┬───────────────┘
               │                              ▲
       1. Publish Event               4. Read Status / Poll
       (202 Accepted)                         │
               │                              │
               ▼                              │
┌──────────────────────────────┐              │
│      Azure Service Bus       │              │
│   (Queue / Topic Engine)     │              │
└──────────────┬───────────────┘              │
               │                              │
       2. Consume Message                     │
               │                              │
               ▼                              │
┌──────────────────────────────┐     3. Write │ State
│   Worker Host Service (.NET) │    & Progress Update
│  (Executes Agent Workflow)   ├──────────────┼───────────────┐
└──────────────────────────────┘              │               │
                                              ▼               ▼
                                     ┌────────────────┐ ┌───────────┐
                                     │Azure Cosmos DB │ │ Azure AI  │
                                     │ (Task State)   │ │  Models   │
                                     └────────────────┘ └───────────┘

The table below contrasts synchronous HTTP AI processing with asynchronous event-driven Service Bus agent messaging:

System Attribute	Synchronous HTTP AI Execution	Asynchronous Event-Driven Messaging
API Response Time	Slow; client blocks until full LLM processing finishes (seconds/minutes).	Sub-50ms; client receives 202 Accepted immediately upon queuing.
Timeout Susceptibility	High; vulnerable to gateway, proxy, and connection drops.	Zero; long-running background tasks execute independently on worker hosts.
Fault Tolerance & Retries	Poor; failures require the client to re-submit the entire request.	High; Azure Service Bus handles transient retries and Dead-Letter Queuing (DLQ) automatically.
Compute Scalability	Rigid; web tier must scale up to handle processing load spikes.	Elastic; worker consumer pools scale horizontally based on queue depth.
State Persistence	Transient in memory unless manually persisted.	Durable; progress and execution state stored in Cosmos DB or Redis.

Implementing an Event-Driven Agent Workflow in .NET

The following step-by-step walkthrough demonstrates how to build an event-driven AI workflow in .NET using Azure.Messaging.ServiceBus, Microsoft.Extensions.AI, and an ASP.NET Core background worker.

Step 1: Install Package Dependencies

Add the official Azure Service Bus messaging SDK and .NET AI extensions to your worker project:

Bash

dotnet add package Azure.Messaging.ServiceBus
dotnet add package Microsoft.Extensions.AI
dotnet add package Microsoft.Extensions.AI.OpenAI
dotnet add package Microsoft.Extensions.Hosting

Step 2: Define Event Message Contracts and Workflow State

Define contract classes for the Service Bus event payload and execution state.

C#

using System.Text.Json.Serialization;

public record DocumentProcessingEvent(
    [property: JsonPropertyName("transactionId")] string TransactionId,
    [property: JsonPropertyName("documentUrl")] string DocumentUrl,
    [property: JsonPropertyName("documentType")] string DocumentType,
    [property: JsonPropertyName("submittedAtUtc")] DateTime SubmittedAtUtc);

public class AgentExecutionState
{
    public required string TransactionId { get; set; }
    public required string Status { get; set; } // "Queued", "Processing", "Completed", "Failed"
    public string? SummaryResult { get; set; }
    public List<string> ExecutionLogs { get; set; } = new();
    public DateTime LastUpdatedAtUtc { get; set; }
}

Step 3: Implement the Service Bus Producer (API Endpoint)

Create an API controller endpoint that accepts processing requests, publishes an event to Azure Service Bus, and returns a 202 Accepted response.

C#

using System.Text.Json;
using Azure.Messaging.ServiceBus;
using Microsoft.AspNetCore.Mvc;

[ApiController]
[Route("api/documents")]
public class DocumentProcessingApiController : ControllerBase
{
    private readonly ServiceBusSender _serviceBusSender;

    public DocumentProcessingApiController(ServiceBusClient serviceBusClient)
    {
        _serviceBusSender = serviceBusClient.CreateSender("agent-processing-queue");
    }

    [HttpPost("process")]
    public async Task<IActionResult> SubmitDocumentForAiAnalysis([FromBody] DocumentProcessingEvent request)
    {
        string messagePayload = JsonSerializer.Serialize(request);
        var message = new ServiceBusMessage(messagePayload)
        {
            ContentType = "application/json",
            MessageId = request.TransactionId,
            CorrelationId = request.TransactionId
        };

        // 1. Publish event message asynchronously to Azure Service Bus
        await _serviceBusSender.SendMessageAsync(message);

        // 2. Return immediate 202 Accepted response with tracking location
        return Accepted(new 
        { 
            Status = "Queued", 
            TransactionId = request.TransactionId,
            StatusCheckUrl = $"/api/documents/status/{request.TransactionId}" 
        });
    }
}

Step 4: Implement the Event-Driven Worker Host and Agent Execution

Construct a background BackgroundService worker that listens for queue messages, invokes the AI agent, handles state updates, and completes the Service Bus transaction.

C#

using System.Text.Json;
using Azure.Messaging.ServiceBus;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

public class AgentWorkflowWorkerService : BackgroundService
{
    private readonly ServiceBusProcessor _processor;
    private readonly IChatClient _chatClient;
    private readonly ILogger<AgentWorkflowWorkerService> _logger;

    public AgentWorkflowWorkerService(
        ServiceBusClient serviceBusClient,
        IChatClient chatClient,
        ILogger<AgentWorkflowWorkerService> logger)
    {
        _chatClient = chatClient;
        _logger = logger;
        _processor = serviceBusClient.CreateProcessor("agent-processing-queue", new ServiceBusProcessorOptions
        {
            MaxConcurrentCalls = 4, // Scale concurrency per worker instance
            AutoCompleteMessages = false // Explicit message completion control
        });
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        _processor.ProcessMessageAsync += HandleIncomingEventAsync;
        _processor.ProcessErrorAsync += HandleProcessingErrorAsync;

        _logger.LogInformation("Starting Azure Service Bus Agent Worker Processor...");
        await _processor.StartProcessingAsync(stoppingToken);
    }

    private async Task HandleIncomingEventAsync(ProcessMessageEventArgs args)
    {
        string body = args.Message.Body.ToString();
        var payload = JsonSerializer.Deserialize<DocumentProcessingEvent>(body);

        if (payload == null)
        {
            _logger.LogError("Invalid null payload received. Dead-lettering message ID: {MessageId}", args.Message.MessageId);
            await args.DeadLetterMessageAsync(args.Message, "InvalidPayload", "Payload could not be deserialized.");
            return;
        }

        _logger.LogInformation("Processing Agent Event for Transaction ID: {TxId}", payload.TransactionId);

        try
        {
            // 1. Construct Agent Execution Prompt
            string prompt = $"Analyze the following document type '{payload.DocumentType}' located at '{payload.DocumentUrl}'. Extract key action items and summarize risk points.";

            var options = new ChatOptions { Temperature = 0.2f };

            // 2. Execute AI Agent Workload asynchronously
            var response = await _chatClient.GetResponseAsync(prompt, options, args.CancellationToken);

            _logger.LogInformation("Agent Workflow Completed for Transaction ID: {TxId}", payload.TransactionId);

            // 3. Persist State to Storage (e.g., Cosmos DB / Redis)
            // SaveResultToStateStore(payload.TransactionId, response.Message.Text);

            // 4. Safely complete the Service Bus message transaction
            await args.CompleteMessageAsync(args.Message, args.CancellationToken);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Error executing AI Agent workflow for Transaction ID: {TxId}", payload.TransactionId);

            // Abandon message to trigger Service Bus auto-retry policy
            await args.AbandonMessageAsync(args.Message, cancellationToken: args.CancellationToken);
        }
    }

    private Task HandleProcessingErrorAsync(ProcessErrorEventArgs args)
    {
        _logger.LogError(args.Exception, "Service Bus Error Source: {Source}", args.ErrorSource);
        return Task.CompletedTask;
    }

    public override async Task StopAsync(CancellationToken cancellationToken)
    {
        await _processor.StopProcessingAsync(cancellationToken);
        await base.StopAsync(cancellationToken);
    }
}

Architectural Advantages and Disadvantages

Advantages

• Total Decoupling & Elastic Scaling: Web applications accept incoming user events instantly while worker pools scale up or down based on Azure Service Bus queue length.

• Resilience and Dead-Letter Recovery: Built-in message lock renewal, poison message handling, and Dead-Letter Queues (DLQ) ensure failed AI runs are never lost.

• Elimination of Gateway Timeouts: Long-running, multi-step agent tasks take as long as necessary without hitting proxy or HTTP connection timeouts.

Disadvantages

• Increased System Complexity: Requires managing distributed messaging infrastructure, status polling endpoints, or WebSocket notifications for client UI feedback.

• Potential Message Processing Latency: Brief delays may occur between publishing an event and a worker pulling the message during heavy queue backlog conditions.

Enterprise Best Practices

1. Use Managed Identities for Service Bus Authentication: Authenticate using DefaultAzureCredential (Microsoft.Entra) instead of storing hardcoded connection strings in configuration settings.

2. Set Up Dead-Letter Queues (DLQ) for Unhandled Failures: Configure Service Bus message max delivery counts (e.g., 5 retries) so poison messages auto-route to DLQ for developer investigation.

3. Persist Execution State with Time-To-Live (TTL): Store task state and progress steps in Azure Cosmos DB with automatic TTL rules (e.g., 24-hour expiration) to clean up old job histories.

4. Implement Auto-Lock Renewal for Long Workflows: If an agent workflow takes several minutes to process, enable Service Bus message lock auto-renewal so locks do not expire mid-execution.

Common Mistakes to Avoid

• Holding Synchronous Requests While Polling: Client applications making blocking synchronous loops while waiting for worker status defeats the benefits of event-driven design. Use asynchronous polling intervals or push notifications.

• Forgetting Message Deduplication: Failing to enable RequiresDuplicateDetection on Service Bus Queues can cause duplicate events to trigger redundant, costly LLM runs.

• Omitting Idempotency Control in Workers: Assuming a Service Bus message will only ever be delivered once. Build worker logic to check if a transaction ID has already been completed before executing expensive LLM calls.

Troubleshooting Guide

Issue 1: Messages Recycled Repeatedly and Moved to Dead-Letter Queue

• Root Cause: The agent execution duration exceeds the Service Bus LockDuration setting, causing the broker to release the lock and deliver the message to another worker repeatedly.

• Resolution: Increase LockDuration on the Queue/Topic and set MaxAutoLockRenewDuration on ServiceBusProcessorOptions in the worker configuration.

Issue 2: Worker Hosts Consume High CPU and Thread Pool Resources

• Root Cause: Setting MaxConcurrentCalls too high on ServiceBusProcessorOptions, causing too many parallel agent workflows to execute on a single worker instance.

• Resolution: Lower MaxConcurrentCalls per worker node and scale out worker host instances horizontally using KEDA (Kubernetes Event-driven Autoscaling) or Azure App Service scale rules.

Issue 3: Duplicate Processing of Events During Scale-Out Events

• Root Cause: Multiple background workers processing the same queue message simultaneously due to improper lock handling.

• Resolution: Ensure AutoCompleteMessages = false and complete messages explicitly (args.CompleteMessageAsync) only after state updates succeed.

Frequently Asked Questions (FAQs)

1. What is the difference between Azure Service Bus and Azure Event Grid for AI workflows?

Azure Service Bus is an enterprise message broker designed for high-reliability command processing, state management, and ordered queues. Azure Event Grid is a lightweight event routing service designed for high-throughput, pub-sub system notification events.

2. How do client frontends receive results from asynchronous agent workflows?

Clients can track status using Async Request-Reply patterns: receiving a 202 Accepted response with a status URL to poll, or receiving real-time push updates via SignalR / WebSockets when the agent completes execution.

3. Can Azure Functions trigger event-driven AI agents from Service Bus?

Yes. Azure Functions provides native Azure Service Bus triggers, allowing serverless worker instances to spin up automatically when messages arrive and scale down to zero when the queue clears.

Conclusion

Building event-driven agent workflows with Azure Service Bus and .NET replaces fragile synchronous HTTP interactions with a decoupled, resilient architecture. By offloading multi-step AI tasks to background workers, developers can protect APIs against timeouts, scale processing elastically under load, and maintain complete operational visibility over complex enterprise agent workflows.