Modern applications are increasingly incorporating AI capabilities such as intelligent search, document summarization, recommendation engines, and AI agents. Unlike traditional microservices, these systems interact with Large Language Models (LLMs), vector databases, embedding services, and external AI APIs, introducing new challenges around observability, resiliency, and distributed communication.
.NET Aspire simplifies the development of cloud-native distributed applications, while OpenTelemetry provides standardized observability across services. Together, they help developers build AI-native microservices that are easier to develop, monitor, and scale.
In this article, you'll learn how to build AI-native microservices using .NET Aspire and OpenTelemetry, integrate AI services into your architecture, and apply production-ready practices.
What Are AI-Native Microservices?
Traditional microservices expose business functionality through APIs and communicate with databases, message brokers, and external services.
AI-native microservices extend this architecture by integrating AI capabilities into business workflows.
Typical examples include:
Document summarization
Semantic search
AI-powered customer support
Recommendation systems
Intelligent workflow automation
AI agents
Unlike traditional services, AI-native applications often involve multiple AI components, making observability essential.
Why .NET Aspire?
.NET Aspire is designed to simplify distributed application development.
Key capabilities include:
Service orchestration
Service discovery
Configuration management
Health checks
Dependency management
Integrated telemetry
Local development experience
Instead of manually configuring every service, Aspire centralizes application composition.
Why OpenTelemetry?
OpenTelemetry is an open standard for collecting telemetry data.
It captures:
Traces
Metrics
Logs
For AI applications, this visibility helps developers understand how requests move across multiple services and external AI providers.
Example Architecture
An AI-native application might look like this:
User
|
API Gateway
|
---------------------
| |
Product Service AI Service
| |
PostgreSQL Embedding API
|
Vector Database
|
Large Language Model
Each request may pass through several services before reaching the user.
Creating an Aspire Solution
Create a new Aspire application.
dotnet new aspire -n AiNativeApp
The solution typically includes:
AiNativeApp
|
|-- AppHost
|
|-- ServiceDefaults
|
|-- ProductService
|
|-- AIService
AppHost orchestrates services.
ServiceDefaults configures shared functionality.
ProductService contains business logic.
AIService communicates with AI providers.
Register Services
Configure services in the AppHost.
var builder = DistributedApplication.CreateBuilder(args);
var productApi =
builder.AddProject<Projects.ProductService>("product");
var aiApi =
builder.AddProject<Projects.AIService>("ai");
builder.Build().Run();
Aspire automatically manages service discovery between projects.
Creating an AI Service
Example endpoint:
app.MapPost("/summarize",
async (string text) =>
{
return $"Summary: {text[..50]}...";
});
In a production application, this endpoint would call an LLM provider or an internal AI inference service.
Keeping AI functionality isolated allows it to evolve independently from business services.
Calling the AI Service
Business services communicate using HttpClient.
public class AiClient
{
private readonly HttpClient client;
public AiClient(HttpClient client)
{
this.client = client;
}
public async Task<string> Summarize(
string text)
{
return await client.PostAsJsonAsync(
"/summarize",
text)
.Result.Content.ReadAsStringAsync();
}
}
Using dependency injection keeps communication testable and configurable.
Enabling OpenTelemetry
Configure telemetry in Program.cs.
builder.Services.AddOpenTelemetry()
.WithTracing(tracing =>
{
tracing.AddAspNetCoreInstrumentation();
tracing.AddHttpClientInstrumentation();
})
.WithMetrics(metrics =>
{
metrics.AddAspNetCoreInstrumentation();
});
Instrumentation captures request flow automatically without requiring manual logging in every endpoint.
Understanding Distributed Tracing
A typical request flow becomes:
User Request
|
API Gateway
|
Product Service
|
AI Service
|
LLM Provider
|
Response
Distributed tracing assigns a trace ID to the request, allowing developers to follow its journey across every service.
This makes debugging significantly easier than analyzing isolated logs.
Monitoring AI Requests
AI workloads introduce metrics that traditional applications rarely track.
Useful measurements include:
AI request count
Prompt size
Token usage
Response latency
External API failures
Retry attempts
Timeout frequency
These metrics help identify performance bottlenecks and operational costs.
Handling External AI Failures
AI providers may occasionally return errors or experience temporary outages.
Example retry logic:
try
{
return await aiClient.Summarize(text);
}
catch(HttpRequestException)
{
return "AI service unavailable.";
}
For production systems, consider resilience libraries that support retry, timeout, and circuit breaker patterns.
Health Checks
Health checks improve operational visibility.
builder.Services.AddHealthChecks()
.AddCheck("AI Service", () =>
HealthCheckResult.Healthy());
Health endpoints allow orchestration platforms to detect unhealthy services automatically.
Logging AI Operations
Structured logging provides additional diagnostic information.
logger.LogInformation(
"Summarization requested for document {Id}",
documentId);
Avoid logging prompts that contain sensitive customer or business information unless required by policy.
Security Considerations
AI-native services often process confidential data.
Recommended practices:
Authenticate every request.
Protect API keys using secret management.
Encrypt communication with HTTPS.
Apply role-based authorization.
Validate user input.
Sanitize prompts before forwarding them.
Log security events without exposing sensitive content.
Join the conversation! Your thoughts help the community grow.