Logging and Monitoring in .NET Core Projects

In today's modern applications, logging and monitoring play a crucial role in ensuring the health, reliability, and performance of software. These two practices allow developers and DevOps teams to diagnose issues quickly, track performance bottlenecks, and stay ahead of potential failures. ASP.NET Core provides a built-in logging framework, along with flexibility to integrate powerful monitoring tools to track application health in real time.

In this article, we’ll explore how to implement logging and monitoring in .NET Core applications using built-in tools like ILogger<T>, as well as third-party tools like Serilog for persistent logging and custom middleware for performance tracking.

Why Logging and Monitoring Matter

Both logging and monitoring serve distinct but equally important purposes in application development and production environments:

Troubleshooting: Quickly identify root causes of errors and issues that occur during the application runtime.
Performance Tracking: Identify slow queries, lagging responses, and any bottlenecks within the system.
Security: Detect unauthorized access or suspicious activities to ensure the integrity of your system.
Audit Trails: Track user activities to ensure compliance with legal and regulatory standards.
Proactive Alerts: Set up alerts to notify teams before issues escalate into major problems, allowing for preemptive action.

Logging in ASP.NET Core

ASP.NET Core uses a powerful built-in logging system that helps you log messages across different components of the application. The ILogger<T> interface is central to this logging system, allowing developers to log messages at various levels such as Information, Warning, and Error.

Step 1: Implementing Basic Logging in Controllers

Here’s how you can use the built-in logging functionality in an ASP.NET Core controller:

[ApiController]
[Route("api/[controller]")]
public class ProductsController : ControllerBase
{
    private readonly ILogger<ProductsController> _logger;

    public ProductsController(ILogger<ProductsController> logger)
    {
        _logger = logger;
    }

    [HttpGet]
    public IActionResult GetProducts()
    {
        _logger.LogInformation("Fetching all products at {time}", DateTime.UtcNow);
        return Ok(new[] { "Laptop", "Smartphone" });
    }
}

In this example:

The ILogger<ProductsController> is injected into the controller.
The LogInformation method logs an informational message every time the GetProducts() action is executed.

Step 2: Configuring Logging Levels in `appsettings.json`

To control the verbosity of logs, ASP.NET Core allows you to define log levels in appsettings.json. The configuration specifies what types of logs (Information, Warning, Error) should be captured for different categories, including the default logger and Microsoft-specific categories.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "System": "Error"
    }
  }
}

Default: Sets the default logging level across all loggers to "Information."
Microsoft: Only captures "Warning" level logs or higher for Microsoft-related logs.
System: Captures "Error" level logs for system-related events.

Connecting Logs to a Database with Serilog

Sometimes, you need persistent logging stored in a database for later analysis, auditing, or troubleshooting. Serilog is a popular logging library that supports different sinks, including database sinks such as SQL Server. Here's how to configure and use Serilog in an ASP.NET Core application:

Step 1: Install Serilog Packages

First, you need to install the Serilog packages. You can do this using the following command:

dotnet add package Serilog.AspNetCore
dotnet add package Serilog.Sinks.MSSqlServer

Step 2: Configure Serilog in `Program.cs`

Next, configure Serilog in the Program.cs file to write logs to a SQL Server database:

using Serilog;

var builder = WebApplication.CreateBuilder(args);

// Configure Serilog with SQL Server sink
Log.Logger = new LoggerConfiguration()
    .WriteTo.Console()
    .WriteTo.MSSqlServer(
        connectionString: builder.Configuration.GetConnectionString("DefaultConnection"),
        sinkOptions: new Serilog.Sinks.MSSqlServer.MSSqlServerSinkOptions
        {
            TableName = "Logs",
            AutoCreateSqlTable = true
        })
    .CreateLogger();

builder.Host.UseSerilog();

builder.Services.AddControllers();

var app = builder.Build();
app.MapControllers();
app.Run();

In this example:

The Serilog logger is configured to log to both the Console and a SQL Server database.
The AutoCreateSqlTable option ensures that Serilog creates a Logs table if it doesn't already exist in the database.

Step 3: Database Table

Serilog will automatically create a table in your SQL Server database with the following columns:

Id (Primary Key)
Message
Level (Log level, e.g., Information, Warning, Error)
Timestamp

Monitoring with Metrics

While logging is essential for diagnosing issues, monitoring provides insights into the application's performance and health in real time. Metrics such as request duration, error rates, and resource usage can be tracked and visualized using monitoring tools.

One common practice in monitoring is adding custom middleware to measure the duration of HTTP requests.

Example: Middleware for Request Timing

public class RequestTimingMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger<RequestTimingMiddleware> _logger;

    public RequestTimingMiddleware(RequestDelegate next, ILogger<RequestTimingMiddleware> logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var start = DateTime.UtcNow;
        await _next(context);
        var duration = DateTime.UtcNow - start;

        _logger.LogInformation("Request {path} took {duration} ms",
            context.Request.Path, duration.TotalMilliseconds);
    }
}

This RequestTimingMiddleware captures the time taken to process each request and logs the duration in milliseconds.
It provides valuable insight into which API calls are slow, helping identify potential performance bottlenecks.

app.UseMiddleware<RequestTimingMiddleware>();

Example in Action

Consider an E-Commerce API:

ProductsController logs each product fetch request.
Serilog stores logs in SQL Server for persistence and auditing.
RequestTimingMiddleware monitors how long each request takes to process.
Dashboards (e.g., Grafana, Kibana, Azure Application Insights) visualize logs and metrics, giving you real-time insights into system health.

This setup provides you with a complete monitoring and logging solution, allowing developers to trace errors, measure performance, and monitor system health in real time.

Conclusion

Logging and monitoring are vital components of any modern .NET Core application. By utilizing the built-in logging framework and integrating with powerful third-party tools like Serilog for persistent logging and custom middleware for performance tracking, developers gain valuable insights into the behavior of their application.

By also incorporating monitoring tools such as Grafana or Azure Application Insights, you ensure proactive detection of issues before they become critical. This approach not only improves troubleshooting and performance optimization but also helps keep your application reliable in production.

With this guide, you now have a solid foundation for logging and monitoring in your .NET Core projects. By implementing the techniques discussed, your applications will be more robust and easier to maintain in production environments.