Introduction
Unit tests tell you that a method returns the right value. They don't tell you whether your API still returns the response your clients expect. Rename a property for consistency, change a type, or swap a status code, and every unit test can stay green while a mobile app or another service starts failing.
Regression tests at the API level close that gap. They call your endpoints the way a real client would and check that the response hasn't changed in ways that break consumers.
In this article, we'll build a small ASP.NET Core Web API, then write regression tests for it with xUnit and WebApplicationFactory. We'll compare responses against a saved baseline, ignore fields that legitimately change on every call, and run everything in CI on each pull request.
Prerequisites
- .NET 8 SDK or later
- Basic knowledge of ASP.NET Core and xUnit
Step 1: Create the API
Create a minimal Web API with one endpoint that returns an order.
// Program.cs (OrdersApi project)
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/orders/{id:int}", (int id) =>
{
if (id != 1042)
{
return Results.NotFound();
}
return Results.Ok(new
{
orderId = 1042,
status = "confirmed",
totalAmount = 149.99m,
createdAt = DateTime.UtcNow,
requestId = Guid.NewGuid()
});
});
app.Run();
// Makes the Program class visible to the test project
public partial class Program { }
The public partial class Program { } line matters. Minimal APIs generate an internal Program class by default, and WebApplicationFactory needs to reference it from the test project.
Notice the last two fields. createdAt and requestId change on every request. A naive comparison would fail every time, so we'll handle them explicitly.
Step 2: Create the Test Project
Add an xUnit test project and reference the API.
dotnet new xunit -n OrdersApi.Tests
dotnet add OrdersApi.Tests reference OrdersApi
dotnet add OrdersApi.Tests package Microsoft.AspNetCore.Mvc.Testing
Microsoft.AspNetCore.Mvc.Testing provides WebApplicationFactory, which starts your API in memory so tests can send real HTTP requests without deploying anything.
Step 3: Save a Baseline Response
A baseline is a known-good response that future responses are compared against. Create a folder called Baselines in the test project and add get-order-1042.json:
{
"orderId": 1042,
"status": "confirmed",
"totalAmount": 149.99,
"createdAt": "IGNORED",
"requestId": "IGNORED"
}
The values for createdAt and requestId don't matter, because we'll skip comparing them. Keeping the keys in the file still lets us check that those fields exist.
Make sure the baseline is copied to the test output folder by adding this to OrdersApi.Tests.csproj:
<ItemGroup>
<None Update="Baselines\**" CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>
Step 4: Write a Baseline Comparison Helper
This helper compares two JSON objects field by field. It reports missing fields, changed values, and new fields that aren't in the baseline, and it skips fields you mark as ignored.
using System.Text.Json.Nodes;
using Xunit;
public static class JsonAssertions
{
public static void AssertMatchesBaseline(
JsonObject expected,
JsonObject actual,
IEnumerable<string> ignore)
{
var ignored = new HashSet<string>(ignore);
foreach (var (name, expectedValue) in expected)
{
Assert.True(actual.ContainsKey(name),
$"Field '{name}' is missing from the response.");
if (ignored.Contains(name))
{
continue;
}
Assert.True(JsonNode.DeepEquals(expectedValue, actual[name]),
$"Field '{name}' changed. Expected {expectedValue?.ToJsonString()}, " +
$"got {actual[name]?.ToJsonString()}.");
}
foreach (var name in actual.Select(p => p.Key))
{
Assert.True(expected.ContainsKey(name),
$"New field '{name}' is not in the baseline. Review it before updating the baseline.");
}
}
}
The new-field check is deliberately strict. Adding a field is usually safe for clients, but flagging it forces a quick review instead of letting the API change silently. If your team prefers, you can remove that loop and allow additive changes.
Step 5: Write the Regression Tests
using System.Net;
using System.Text.Json.Nodes;
using Microsoft.AspNetCore.Mvc.Testing;
using Xunit;
public class OrdersApiRegressionTests : IClassFixture<WebApplicationFactory<Program>>
{
private readonly HttpClient _client;
public OrdersApiRegressionTests(WebApplicationFactory<Program> factory)
{
_client = factory.CreateClient();
}
[Fact]
public async Task GetOrder_MatchesBaseline()
{
var response = await _client.GetAsync("/orders/1042");
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var actual = JsonNode.Parse(await response.Content.ReadAsStringAsync())!.AsObject();
var expected = JsonNode.Parse(
File.ReadAllText(Path.Combine("Baselines", "get-order-1042.json")))!.AsObject();
JsonAssertions.AssertMatchesBaseline(
expected,
actual,
ignore: new[] { "createdAt", "requestId" });
}
[Fact]
public async Task GetOrder_UnknownId_Returns404()
{
var response = await _client.GetAsync("/orders/9999");
Assert.Equal(HttpStatusCode.NotFound, response.StatusCode);
}
}
The first test checks the happy path against the baseline. The second protects an error path. Clients often branch on status codes, so a 404 that quietly becomes a 400 or a 500 is a regression too.
Run the tests:
dotnet test
Both should pass.
Step 6: See the Regression Get Caught
Now simulate a common “harmless” change. In Program.cs, rename totalAmount to amount:
amount = 149.99m,
Run dotnet test again. The baseline test fails with a message like:
Field 'totalAmount' is missing from the response.
The code still compiles, and a reviewer might well approve the rename. But any client reading totalAmount would now get nothing. The regression test catches it before the change merges.
If the rename is intentional, don't just update the baseline and move on. Treat it as an API change: keep the old field alongside the new one for a while, tell your consumers, and remove the old field once they've moved over. Then update the baseline in the same pull request, so the reviewer sees both the code change and its effect on the response.
Step 7: Run the Tests on Every Pull Request
Regression tests are most useful when they run before code merges. Here's a minimal GitHub Actions workflow:
name: api-regression-tests
on:
pull_request:
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
- run: dotnet test
Make this check required in your branch protection rules, so a pull request can't merge while a regression test is failing.
Tips for Growing the Suite
- Cover error paths as well as success paths. Missing resources, invalid input, and authentication failures are where clients break most often.
- Keep one baseline per scenario. Small, focused baselines make failures easier to read than one giant response file.
- Be careful with the ignore list. Every ignored field is a place where a real regression can hide. Ignore only values that genuinely change on every call, such as timestamps and generated IDs.
- Test workflows, not just single endpoints. Create an order, read it back, update it, and delete it in one test, because many regressions only appear when one call depends on the result of another.
Conclusion
API regression tests check what your clients actually receive, not just what your methods return. With WebApplicationFactory, a saved baseline, and a short ignore list for dynamic fields, you can catch renamed fields, changed types and altered status codes before they reach production, and run those checks automatically on every pull request.
ASP.NET Coreregression testingxUnitWebApplicationFactoryWeb API testingbaseline testingintegration testing
Join the conversation! Your thoughts help the community grow.