Modern applications are increasingly incorporating AI capabilities such as intelligent search, document summarization, recommendation engines, and AI agents. Unlike traditional microservices, these systems interact with Large Language Models (LLMs), vector databases, embedding services, and external AI APIs, introducing new challenges around observability, resiliency, and distributed communication.
.NET Aspire simplifies the development of cloud-native distributed applications, while OpenTelemetry provides standardized observability across services. Together, they help developers build AI-native microservices that are easier to develop, monitor, and scale.
In this article, you'll learn how to build AI-native microservices using .NET Aspire and OpenTelemetry, integrate AI services into your architecture, and apply production-ready practices.
What Are AI-Native Microservices?
Traditional microservices expose business functionality through APIs and communicate with databases, message brokers, and external services.
AI-native microservices extend this architecture by integrating AI capabilities into business workflows.
Typical examples include:
Unlike traditional services, AI-native applications often involve multiple AI components, making observability essential.
Why .NET Aspire?
.NET Aspire is designed to simplify distributed application development.
Key capabilities include:
Instead of manually configuring every service, Aspire centralizes application composition.
Why OpenTelemetry?
OpenTelemetry is an open standard for collecting telemetry data.
It captures:
For AI applications, this visibility helps developers understand how requests move across multiple services and external AI providers.
Example Architecture
An AI-native application might look like this:
User
|
API Gateway
|
---------------------
| |
Product Service AI Service
| |
PostgreSQL Embedding API
|
Vector Database
|
Large Language Model
Each request may pass through several services before reaching the user.
Creating an Aspire Solution
Create a new Aspire application.
dotnet new aspire -n AiNativeApp
The solution typically includes:
AiNativeApp
|
|-- AppHost
|
|-- ServiceDefaults
|
|-- ProductService
|
|-- AIService
AppHost orchestrates services.
ServiceDefaults configures shared functionality.
ProductService contains business logic.
AIService communicates with AI providers.
Register Services
Configure services in the AppHost.
var builder = DistributedApplication.CreateBuilder(args);
var productApi =
builder.AddProject<Projects.ProductService>("product");
var aiApi =
builder.AddProject<Projects.AIService>("ai");
builder.Build().Run();
Aspire automatically manages service discovery between projects.
Creating an AI Service
Example endpoint:
app.MapPost("/summarize",
async (string text) =>
{
return $"Summary: {text[..50]}...";
});
In a production application, this endpoint would call an LLM provider or an internal AI inference service.
Keeping AI functionality isolated allows it to evolve independently from business services.
Calling the AI Service
Business services communicate using HttpClient.
public class AiClient
{
private readonly HttpClient client;
public AiClient(HttpClient client)
{
this.client = client;
}
public async Task<string> Summarize(
string text)
{
return await client.PostAsJsonAsync(
"/summarize",
text)
.Result.Content.ReadAsStringAsync();
}
}
Using dependency injection keeps communication testable and configurable.
Enabling OpenTelemetry
Configure telemetry in Program.cs.
builder.Services.AddOpenTelemetry()
.WithTracing(tracing =>
{
tracing.AddAspNetCoreInstrumentation();
tracing.AddHttpClientInstrumentation();
})
.WithMetrics(metrics =>
{
metrics.AddAspNetCoreInstrumentation();
});
Instrumentation captures request flow automatically without requiring manual logging in every endpoint.
Understanding Distributed Tracing
A typical request flow becomes:
User Request
|
API Gateway
|
Product Service
|
AI Service
|
LLM Provider
|
Response
Distributed tracing assigns a trace ID to the request, allowing developers to follow its journey across every service.
This makes debugging significantly easier than analyzing isolated logs.
Monitoring AI Requests
AI workloads introduce metrics that traditional applications rarely track.
Useful measurements include:
AI request count
Prompt size
Token usage
Response latency
External API failures
Retry attempts
Timeout frequency
These metrics help identify performance bottlenecks and operational costs.
Handling External AI Failures
AI providers may occasionally return errors or experience temporary outages.
Example retry logic:
try
{
return await aiClient.Summarize(text);
}
catch(HttpRequestException)
{
return "AI service unavailable.";
}
For production systems, consider resilience libraries that support retry, timeout, and circuit breaker patterns.
Health Checks
Health checks improve operational visibility.
builder.Services.AddHealthChecks()
.AddCheck("AI Service", () =>
HealthCheckResult.Healthy());
Health endpoints allow orchestration platforms to detect unhealthy services automatically.
Logging AI Operations
Structured logging provides additional diagnostic information.
logger.LogInformation(
"Summarization requested for document {Id}",
documentId);
Avoid logging prompts that contain sensitive customer or business information unless required by policy.
Security Considerations
AI-native services often process confidential data.
Recommended practices:
Authenticate every request.
Protect API keys using secret management.
Encrypt communication with HTTPS.
Apply role-based authorization.
Validate user input.
Sanitize prompts before forwarding them.
Log security events without exposing sensitive content.
Security should extend to every AI dependency, not just the application itself.
Deployment Considerations
Production deployments typically include:
Load Balancer
|
API Gateway
|
----------------------
| | |
Product AI Search
Service Service Service
|
Database
Each service can scale independently based on workload.
For example, the AI service may require more compute resources than the Product Service.
Production Best Practices
| Practice | Benefit |
|---|
| Separate AI functionality into dedicated services | Independent scaling |
| Enable distributed tracing | Easier debugging |
| Use health checks | Improved reliability |
| Monitor token usage | Better cost visibility |
| Secure API credentials | Reduced security risk |
| Implement retry policies | Improved resilience |
| Keep prompts outside business logic | Easier maintenance |
Common Mistakes
| Mistake | Better Approach |
|---|
| Embedding AI logic throughout the application | Centralize AI services |
| Ignoring telemetry | Enable OpenTelemetry from the start |
| Logging sensitive prompts | Log metadata instead |
| No timeout handling | Configure request timeouts |
| Tight coupling to one AI provider | Use abstraction layers |
| Scaling all services equally | Scale based on workload |
Troubleshooting
Missing traces
Verify that:
OpenTelemetry is configured.
Instrumentation packages are installed.
Exporters are configured correctly.
Slow AI responses
Check:
External API latency
Prompt size
Network connectivity
Retry frequency
Service communication failures
Review:
High operational costs
Analyze:
Token consumption
Request frequency
Cache effectiveness
AI model selection
.NET Aspire vs Traditional Microservice Setup
| Feature | Traditional Setup | .NET Aspire |
|---|
| Service Discovery | Manual | Built-in |
| Local Orchestration | Manual | Integrated |
| Configuration | Separate per service | Centralized |
| Health Checks | Manual | Simplified |
| Telemetry Integration | Manual | Streamlined |
| Developer Experience | Moderate | Excellent |
Aspire reduces the amount of infrastructure code developers need to maintain, allowing teams to focus more on application logic.
Frequently Asked Questions
Is .NET Aspire required for AI applications?
No. AI applications can be built using standard ASP.NET Core projects. Aspire simplifies orchestration and local development for distributed systems.
Why use OpenTelemetry with AI services?
AI requests often span multiple services and external providers. OpenTelemetry provides end-to-end visibility that simplifies monitoring and troubleshooting.
Can AI services scale independently?
Yes. Separating AI functionality into dedicated microservices allows each service to scale according to its workload and resource requirements.
Should AI models be hosted inside every microservice?
Not usually. Centralizing AI interactions in dedicated services improves maintainability, simplifies updates, and reduces duplicated integration logic.
Does OpenTelemetry increase application overhead?
Instrumentation introduces some overhead, but it is generally outweighed by the operational benefits of comprehensive observability. Configure sampling and exporters appropriately for production workloads.
Conclusion
Building AI-native microservices requires more than exposing AI APIs. Applications must manage distributed communication, external dependencies, observability, resilience, and security while maintaining a clean architecture.
By combining .NET Aspire's distributed application capabilities with OpenTelemetry's standardized tracing, metrics, and logging, developers can build AI-powered systems that are easier to develop, monitor, and scale. As AI becomes a core part of enterprise software, adopting these cloud-native practices will help teams deliver reliable and maintainable applications in production.