Building AI Governance Dashboards with OpenTelemetry and .NET Aspire

The Developer Problem: Unmonitored Enterprise AI Execution

As enterprise .NET applications integrate Large Language Models (LLMs) and autonomous agents, monitoring shifts from basic HTTP status checks to complex AI governance. Unlike traditional web services, AI workloads introduce non-deterministic execution paths, variable token usage costs, dynamic prompt injection risks, and fluctuating model latencies.

Deploying generative AI features without structured observability leads to several operational blind spots:

Uncontrolled Cost Escalation: Lacking real-time token tracking per tenant, user, or feature makes it difficult to prevent unexpected API bill spikes.
Lack of Prompt Auditability: Inability to inspect historical prompt inputs, system safety violations, and model output completions hinders compliance and security debugging.
Hidden Latency Bottlenecks: Identifying whether latency stems from embedding generation, vector search retrieval, or model token streaming requires distributed tracing across the full request lifecycle.
Fragmented Observability Tooling: Storing application logs in one tool while tracking AI model metrics in separate dashboards creates context switches during incident triaging.

Combining OpenTelemetry with .NET Aspire solves these operational challenges by standardizing the collection of logs, metrics, and distributed traces via the OpenTelemetry Protocol (OTLP). .NET Aspire provides a built-in telemetry dashboard and application host runtime that aggregates telemetry streams from Microsoft.Extensions.AI and Semantic Kernel into a single AI governance dashboard.

Architecture: Unmonitored LLM Calls vs. OpenTelemetry Governance

An OpenTelemetry-driven AI governance architecture captures telemetry at the middleware layer using standard ActivitySource spans and Meter metrics before exporting them via OTLP to the .NET Aspire Dashboard.

┌─────────────────────────────────────────────────────────────┐
│                 User / Client Application                   │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────┐
│              ASP.NET Core Web API / Service                 │
│         (Instrumented with Microsoft.Extensions.AI)         │
└──────────────────────────────┬──────────────────────────────┘
                               │
        ┌──────────────────────┴──────────────────────┐
        │ OpenTelemetry OTLP Exporter                 │ LLM Direct Call
        ▼                                             ▼
┌──────────────────────────────┐              ┌──────────────────────────────┐
│    .NET Aspire Dashboard     │              │    Azure OpenAI / Foundry    │
│  (Traces, Metrics, Token UI) │              │      Model Service Endpoint  │
└──────────────────────────────┘              └──────────────────────────────┘

The table below contrasts traditional web monitoring with OpenTelemetry-based AI governance in .NET:

Telemetry Dimension	Traditional Application Monitoring	OpenTelemetry AI Governance
Primary Metrics	HTTP status codes, CPU/RAM usage, web request count.	Prompt tokens, completion tokens, model duration, estimated cost.
Trace Context	Controller action methods and SQL query execution.	Multi-turn agent spans, tool calls, vector search queries, model generation cycles.
Security Audit	Identity authentication tokens and request URLs.	Prompt input inspection, safety filter flags, guardrail violations.
Standardization	Proprietary SDK agents (App Insights, Datadog).	Vendor-agnostic OTLP (OpenTelemetry Protocol) standards.
Developer UI	External cloud web portals.	Local real-time dashboard provided natively by .NET Aspire.

Implementing AI Governance Dashboards in .NET Aspire

The following step-by-step walkthrough demonstrates how to configure OpenTelemetry telemetry collection for Microsoft.Extensions.AI, register .NET Aspire orchestration, and visualize LLM execution metrics in the Aspire Dashboard.

Step 1: Install Required Package Dependencies

Add the necessary .NET Aspire, OpenTelemetry, and Microsoft AI extensions to your project:

Bash

dotnet add package Microsoft.Extensions.AI
dotnet add package Microsoft.Extensions.AI.OpenAI
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore

Step 2: Configure Service Defaults and OpenTelemetry Telemetry

Create an extension method to register OpenTelemetry logging, tracing, and metrics for AI workloads in your application startup pipeline.

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using OpenTelemetry;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;

public static class AiGovernanceTelemetryExtensions
{
    public static IServiceCollection AddAiGovernanceTelemetry(this IServiceCollection services, string serviceName)
    {
        services.AddOpenTelemetry()
            .ConfigureResource(resource => resource.AddService(serviceName))
            .WithTracing(tracing =>
            {
                tracing
                    .AddAspNetCoreInstrumentation()
                    .AddHttpClientInstrumentation()
                    // Capture OpenTelemetry spans emitted by Microsoft.Extensions.AI
                    .AddSource("Experimental.Microsoft.Extensions.AI")
                    .AddOtlpExporter();
            })
            .WithMetrics(metrics =>
            {
                metrics
                    .AddAspNetCoreInstrumentation()
                    .AddHttpClientInstrumentation()
                    // Capture token counts and latency meters emitted by Microsoft.Extensions.AI
                    .AddMeter("Experimental.Microsoft.Extensions.AI")
                    .AddOtlpExporter();
            });

        return services;
    }
}

Step 3: Implement an Instrumented Chat Client Service

Wrap the underlying model client with standard telemetry metadata using UseLogging() and UseTelemetry() options provided by Microsoft.Extensions.AI.

using Microsoft.Extensions.AI;
using OpenAI;

public class AiGovernanceService
{
    private readonly IChatClient _chatClient;

    public AiGovernanceService(string apiKey, string modelName)
    {
        // 1. Initialize base OpenAI client
        IChatClient innerClient = new OpenAIClient(apiKey).AsChatClient(modelName);

        // 2. Wrap client with OpenTelemetry telemetry and logging pipeline
        _chatClient = new ChatClientBuilder(innerClient)
            .UseLogging()
            .UseTelemetry(configure: options =>
            {
                // Include sensitive prompt content in traces for governance auditing
                options.EnableSensitiveData = true; 
            })
            .Build();
    }

    public async Task<string> ExecuteGovernedPromptAsync(string promptInput, string tenantId)
    {
        var options = new ChatOptions
        {
            ModelId = "gpt-4o",
            AdditionalProperties = new() { ["tenant.id"] = tenantId }
        };

        var response = await _chatClient.GetResponseAsync(promptInput, options);

        return response.Message.Text;
    }
}

Step 4: Connect Services in .NET Aspire AppHost

Define the application composition inside your .NET Aspire AppHost project (Program.cs) to wire telemetry streams directly into the Aspire Dashboard.

var builder = DistributedApplication.CreateBuilder(args);

// Register the AI Web API service within .NET Aspire AppHostvar aiApiService = builder.AddProject<Projects.Enterprise_Ai_Api>("ai-api-service")
    .WithExternalHttpEndpoints();

// Build and run the Aspire AppHost runtime
builder.Build().Run();

Architectural Advantages and Disadvantages

Advantages

Zero Vendor Lock-In: OpenTelemetry protocol (OTLP) emissions stream seamlessly to .NET Aspire locally and to cloud collectors like Azure Monitor, Datadog, or Grafana in production.
Unified Trace and Log Correlation: Correlates HTTP requests directly with downstream vector database searches and LLM generation spans.
Local Real-Time Visibility: The standalone .NET Aspire Dashboard provides instant visualization during local development without setting up Docker Prometheus or Jaeger instances.

Disadvantages

Storage Overhead from Sensitive Logs: Capturing full prompt inputs and completions (EnableSensitiveData = true) increases log storage requirements and requires compliance sanitation.
Experimental API Namespace Surface: Telemetry meter names in Microsoft.Extensions.AI currently carry experimental prefixes that may update across SDK releases.

Enterprise Best Practices

Tag Spans with Tenant Identifiers: Pass custom properties like tenant.id or user.role into ChatOptions to enable cost-allocation queries in the dashboard.
Sanitize PII in Production Pipelines: Disable EnableSensitiveData in public production environments or configure OpenTelemetry processors to redact Personally Identifiable Information (PII) before exporting logs.
Configure Alert Thresholds on Token Consumption: Set up metrics alerts on token meters (gen_ai.client.token.usage) to catch anomalous runaway loops early.
Export Telemetry Data for Audit Compliance: Use .NET Aspire export capabilities to save OTLP trace bundles for offline security audits and compliance records.

Common Mistakes to Avoid

Forgetting to Add the AI Meter Names: Omitting .AddMeter("Experimental.Microsoft.Extensions.AI") from WithMetrics() results in missing token count graphs on the dashboard.
Logging Unencrypted Sensitive Data in Public Logs: Exposing raw user passwords or API credentials inside prompt context traces without applying telemetry redaction filters.
Ignoring HTTP Client Spans: Disabling HTTP client instrumentation prevents tracing underlying REST payload timings sent to model endpoints.

Troubleshooting Guide

Issue 1: Telemetry Data Does Not Appear in .NET Aspire Dashboard

Root Cause: The OTEL_EXPORTER_OTLP_ENDPOINT environment variable is missing or pointing to an incorrect port.
Resolution: Ensure the application project references the .NET Aspire service defaults or explicitly passes http://localhost:4317 to AddOtlpExporter().

Issue 2: Token Count Metrics Are Missing from Metrics Tab

Root Cause: The underlying IChatClient was instantiated directly without attaching .UseTelemetry() through ChatClientBuilder.
Resolution: Wrap all IChatClient instances with .UseTelemetry() before registering them in dependency injection.

Issue 3: High Network Overhead from Tracing Spans

Root Cause: High-frequency applications exporting traces synchronously without batching.
Resolution: Ensure the OpenTelemetry SDK uses BatchActivityExportProcessor for asynchronous span processing.

Frequently Asked Questions (FAQs)

1. Is the .NET Aspire Dashboard suitable for production monitoring?

The .NET Aspire Dashboard is designed primarily for developer visualization and local debugging. For production environments, route standard OTLP streams from your apps to centralized enterprise telemetry stores like Application Insights, Datadog, or Grafana.

2. Does Microsoft.Extensions.AI support OpenTelemetry out of the box?

Yes. Microsoft.Extensions.AI provides built-in instrumentation wrappers (UseTelemetry) that emit standard semantic convention spans for model generation, token counts, and latency metrics.

3. How does OpenTelemetry trace multi-agent workflows?

OpenTelemetry creates parent-child span hierarchies. The root span represents the user request, while child spans track individual agent routing decisions, tool calls, and LLM completions under a shared Trace ID.

Conclusion

Combining OpenTelemetry with .NET Aspire delivers complete visibility into enterprise AI execution. By capturing token metrics, distributed traces, and prompt logs at the middleware layer, developers can build transparent, cost-aware, and auditable AI governance dashboards across local development and production environments.