Microsoft Agent Framework Harness: Testing AI Agents with Evals and OpenTelemetry

AI agents are fundamentally different from traditional application components. A normal method can often be tested with fixed inputs and expected outputs. An AI agent may choose different tools, produce different responses, or take different execution paths while solving the same task.

That makes traditional unit testing alone insufficient.

A production-ready agent needs a testing harness that can evaluate not only the final response, but also tool usage, execution steps, latency, failures, and other runtime signals.

Microsoft Agent Framework provides building blocks for developing and hosting AI agents and workflows. A practical testing strategy can combine deterministic tests, evaluation datasets, and OpenTelemetry-based observability to create a repeatable agent verification process.

This article explains how to build that approach in .NET.

Why AI Agents Need a Testing Harness

Consider an agent responsible for answering customer support questions.

A traditional unit test might look like:

[Fact]
public void Add_ReturnsExpectedResult()
{
    var result = Calculator.Add(10, 20);

    Assert.Equal(30, result);
}

The expected result is deterministic.

An AI agent is different:

User Request
     |
     v
Agent
     |
     +---- Search Documentation
     |
     +---- Query Customer Data
     |
     +---- Generate Response
     |
     v
Final Answer

The agent may take different paths for similar requests.

Therefore, testing should evaluate multiple dimensions:

Dimension	Example
Correctness	Did the agent provide the right answer?
Tool selection	Did it select the appropriate tool?
Tool arguments	Were correct parameters supplied?
Safety	Did it avoid unauthorized actions?
Grounding	Was the answer based on available data?
Latency	How long did execution take?
Reliability	Did the workflow complete successfully?
Cost	How many model tokens were consumed?

A testing harness brings these checks together.

What Is an Agent Testing Harness?

An agent harness is an application or test layer that controls how an agent is executed and evaluated.

A simple architecture is:

Test Dataset
     |
     v
Agent Harness
     |
     v
AI Agent
     |
     +---- Tools
     +---- Services
     +---- Data
     |
     v
Execution Trace
     |
     v
Evaluators
     |
     v
Test Report

The harness should isolate the agent from production systems whenever possible.

For example, instead of allowing an evaluation to modify a real customer record, provide a controlled test implementation of the required service.

Start With Deterministic Tests

Not every agent test needs an LLM evaluator.

The first layer should be deterministic.

Suppose an agent returns:

public sealed record SupportResult(
    string Category,
    string Priority,
    string Response);

You can validate the structure with ordinary unit tests:

[Fact]
public void SupportResult_ContainsRequiredFields()
{
    var result = new SupportResult(
        "Billing",
        "High",
        "Your request has been received.");

    Assert.False(string.IsNullOrWhiteSpace(result.Category));
    Assert.False(string.IsNullOrWhiteSpace(result.Priority));
    Assert.False(string.IsNullOrWhiteSpace(result.Response));
}

You can also validate business rules:

[Fact]
public void HighPriorityIssue_MustHaveValidCategory()
{
    var result = new SupportResult(
        "Billing",
        "High",
        "Your request has been received.");

    Assert.Contains(
        result.Category,
        new[] { "Billing", "Security", "Account" });
}

These tests are fast, repeatable, and independent of model variability.

Testing Tool Selection

Agent behavior should be tested at the tool boundary.

Suppose an agent has access to:

SearchOrders
GetCustomer
CancelOrder
IssueRefund

For a cancellation request, the test may verify that the agent selected an appropriate tool.

A simple test representation could be:

public sealed record ToolCall(
    string Name,
    Dictionary<string, object?> Arguments);

Then:

[Fact]
public void CancellationRequest_UsesCancelOrderTool()
{
    var calls = new[]
    {
        new ToolCall(
            "GetCustomer",
            new Dictionary<string, object?>()),

        new ToolCall(
            "CancelOrder",
            new Dictionary<string, object?>
            {
                ["orderId"] = "ORD-10025"
            })
    };

    Assert.Contains(
        calls,
        call => call.Name == "CancelOrder");
}

The actual harness can capture these calls during agent execution.

This provides a useful test signal without judging the entire generated response.

Testing Tool Arguments

Selecting the correct tool is not enough.

The arguments also need validation.

For example:

[Fact]
public void CancelOrder_UsesExpectedOrderId()
{
    var toolCall = new ToolCall(
        "CancelOrder",
        new Dictionary<string, object?>
        {
            ["orderId"] = "ORD-10025"
        });

    Assert.Equal(
        "ORD-10025",
        toolCall.Arguments["orderId"]);
}

This becomes particularly important for tools that perform state-changing operations.

A tool invocation should be treated as an API contract, not simply as an internal implementation detail.

Using Evaluation Datasets

Unit tests validate individual behaviors.

Evaluation datasets allow you to test an agent against a collection of realistic scenarios.

A dataset might contain:

[
  {
    "input": "Where is my order?",
    "expectedCategory": "OrderStatus"
  },
  {
    "input": "I want to cancel my order.",
    "expectedCategory": "Cancellation"
  },
  {
    "input": "I was charged twice.",
    "expectedCategory": "Billing"
  }
]

The harness can execute each case and record the result.

Conceptually:

100 Test Cases
      |
      v
   Agent
      |
      v
100 Evaluations
      |
      v
Pass / Fail / Review

This makes regression testing possible when changing:

Model configuration
System instructions
Tools
Retrieval logic
Agent workflows
Prompt templates

Building a Simple Evaluation Runner

A basic C# evaluation runner can look like this:

public sealed record EvaluationCase(
    string Input,
    string ExpectedCategory);

public sealed record EvaluationResult(
    EvaluationCase Case,
    string ActualCategory,
    bool Passed);

public static async Task<EvaluationResult> RunAsync(
    EvaluationCase testCase,
    Func<string, Task<string>> agent)
{
    var actual = await agent(testCase.Input);

    return new EvaluationResult(
        testCase,
        actual,
        string.Equals(
            actual,
            testCase.ExpectedCategory,
            StringComparison.OrdinalIgnoreCase));
}

The agent is passed into the runner as a delegate.

This is useful because the test harness does not need to know how the agent itself is implemented.

LLM-Based Evaluators

Deterministic tests work well for objective requirements, but they cannot fully evaluate open-ended responses.

For example:

"Was the response helpful and relevant?"

A second model can evaluate characteristics such as:

Relevance
Completeness
Clarity
Grounding
Instruction adherence

A conceptual evaluator output might be:

{
  "score": 4,
  "passed": true,
  "reason": "The response directly answered the question and included the required information."
}

However, an LLM evaluator is itself non-deterministic.

Therefore, it should not replace deterministic tests.

A better architecture is:

                Agent Output
                     |
          +----------+----------+
          |                     |
          v                     v
 Deterministic Checks       AI Evaluator
          |                     |
          +----------+----------+
                     |
                     v
              Final Evaluation

Why OpenTelemetry Matters

Testing tells you whether an agent passed a scenario.

Observability tells you what happened during execution.

OpenTelemetry provides a standardized approach to collecting telemetry such as traces and metrics.

For an agent workflow, a trace might look like:

Agent Request
   |
   +---- Model Call
   |
   +---- Tool: SearchKnowledge
   |
   +---- Model Call
   |
   +---- Tool: GetCustomer
   |
   +---- Final Response

This is extremely useful when an evaluation fails.

Instead of only seeing:

Test failed

you can investigate the execution path.

Adding OpenTelemetry to a .NET Application

A typical .NET application can configure OpenTelemetry tracing like this:

using OpenTelemetry.Trace;

var builder = WebApplication.CreateBuilder(args);

builder.Services
    .AddOpenTelemetry()
    .WithTracing(tracing =>
    {
        tracing
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation();
    });

var app = builder.Build();

app.MapGet("/", () => "Agent test host");

app.Run();

The exact instrumentation available depends on the libraries used by the application and agent framework.

The important idea is to capture meaningful spans around agent operations.

Creating Agent-Specific Activities

Application code can also create explicit activities for important operations.

using System.Diagnostics;

private static readonly ActivitySource ActivitySource =
    new("MyCompany.Agent");

public async Task<string> ExecuteAsync(
    string request,
    CancellationToken cancellationToken)
{
    using var activity =
        ActivitySource.StartActivity("agent.execute");

    activity?.SetTag("agent.name", "SupportAgent");

    try
    {
        return await ProcessRequestAsync(
            request,
            cancellationToken);
    }
    catch (Exception ex)
    {
        activity?.SetTag("error.type", ex.GetType().Name);
        activity?.SetStatus(
            ActivityStatusCode.Error);

        throw;
    }
}

This gives the operation a traceable boundary.

Do not put sensitive prompts, access tokens, or customer information into telemetry attributes merely for debugging convenience.

Measuring Agent Performance

A harness can record useful metrics such as:

Total evaluations
Passed evaluations
Failed evaluations
Tool failures
Average execution duration
Model call count
Token usage

For example:

public sealed record EvaluationSummary(
    int Total,
    int Passed,
    int Failed)
{
    public double PassRate =>
        Total == 0
            ? 0
            : (double)Passed / Total * 100;
}

This gives a simple pass-rate calculation:

var summary = new EvaluationSummary(
    Total: 100,
    Passed: 92,
    Failed: 8);

Console.WriteLine(
    $"Pass rate: {summary.PassRate:F1}%");

Avoid treating a single pass-rate number as a complete measure of agent quality. A system could pass many low-risk scenarios while failing a small number of critical security scenarios.

Testing Agent Safety

Safety tests should be explicit.

For example, an order agent should not cancel an order without authorization.

A test case might be:

[Fact]
public async Task UnauthorizedUser_CannotCancelOrder()
{
    var user = new User("user-1");

    var order = new Order(
        "ORD-10025",
        "user-2");

    var allowed =
        authorizationService.CanCancel(
            user,
            order);

    Assert.False(allowed);
}

This should remain deterministic.

Do not rely on the AI model to decide whether the user has permission.

Regression Testing

One of the most useful purposes of an evaluation harness is detecting regressions.

Imagine an agent originally passes:

92 / 100 scenarios

After changing the system instructions:

85 / 100 scenarios

The evaluation suite immediately identifies a potential regression.

More importantly, categorize failures:

Category             Before    After
--------------------------------------
Correctness             95%      94%
Tool Selection          93%      90%
Safety                  99%      99%
Response Quality        88%      80%

This gives developers more information than a single overall score.

Testing With Mock Tools

Agent evaluations should avoid modifying production resources.

Create controlled test doubles:

public sealed class FakeOrderService : IOrderService
{
    public Task<Order?> GetOrderAsync(
        string orderId,
        CancellationToken cancellationToken)
    {
        return Task.FromResult<Order?>(
            new Order(orderId, "user-1"));
    }

    public Task<bool> CancelAsync(
        string orderId,
        CancellationToken cancellationToken)
    {
        return Task.FromResult(true);
    }
}

The agent can use the fake service during evaluation.

This makes tests:

Safer
Faster
More repeatable
Easier to debug

Common Mistakes

Testing Only the Final Answer

A correct-looking response can hide an incorrect tool call.

Test intermediate execution behavior where it matters.

Using Only LLM-as-a-Judge

An AI evaluator can provide useful quality assessments, but it should not replace deterministic business and security checks.

Testing Against Production Systems

Agent evaluation should not accidentally modify customer records, send emails, issue refunds, or change production configuration.

Use mocks, test environments, or controlled fixtures.

Ignoring Trace Data

When an evaluation fails, execution traces can explain whether the problem came from the model, tool, network, or application code.

Treating Every Test Equally

A failed greeting test and a failed authorization test should not have the same operational significance.

Assign severity to evaluation cases.

Best Practices

Build deterministic tests first.
Use realistic evaluation datasets.
Capture tool calls and arguments.
Instrument important agent operations.
Use OpenTelemetry for distributed traces and metrics.
Keep sensitive data out of telemetry.
Mock state-changing tools during evaluations.
Separate safety tests from general quality tests.
Run evaluations whenever prompts, models, tools, or workflows change.
Track failures by category.
Keep evaluation datasets version-controlled.
Investigate traces rather than relying only on pass/fail results.

Advantages and Disadvantages

Advantages

Makes agent behavior measurable
Detects regressions after model or prompt changes
Combines deterministic and AI-based evaluation
Provides execution visibility through tracing
Helps diagnose tool and workflow failures
Supports repeatable testing
Encourages safer agent development

Disadvantages

Building a representative evaluation dataset takes effort
LLM-based evaluation can introduce variability
Large evaluation suites can consume significant resources
Trace data requires careful privacy management
Agent behavior can change as models and dependencies change

Troubleshooting Failed Evaluations

When an agent test fails, use a layered debugging approach.

Check the test input.
Validate the expected result.
Inspect the agent's final output.
Review selected tools.
Inspect tool arguments.
Check tool responses.
Review the execution trace.
Check model errors and timeouts.
Determine whether the failure is deterministic or model-dependent.
Re-run the same case before changing the implementation.

Avoid immediately modifying prompts when a test fails. First determine whether the problem is in the agent, tool, evaluator, or test itself.

Conclusion

Testing AI agents requires more than checking whether a final response looks correct. Agents can choose tools, call services, perform multi-step workflows, and produce variable outputs, so production testing needs visibility into both the result and the execution process.

A practical .NET testing harness should combine deterministic validation, evaluation datasets, tool-call verification, and OpenTelemetry-based observability.

Deterministic tests should protect business and security rules. Evaluation datasets should detect behavioral regressions. AI-based evaluators can assess subjective qualities such as relevance and completeness. OpenTelemetry traces can then show exactly what happened when something fails.

The result is a much stronger development cycle:

Build
  ↓
Evaluate
  ↓
Trace
  ↓
Analyze
  ↓
Fix
  ↓
Evaluate Again

This approach makes AI agents easier to understand, test, and operate as they move from experimental prototypes into real .NET applications.