Dynamic ASP.NET Core Configurations 🗒️ With Consul KV

Background

Usually, the configurations in .NET and .NET Core apps are stored in the configuration files, such as App.config and Web.config, or appsettings.json; however, all of them have some disadvantages.

hard-coded configuration files
difficult to manage

For example, if we need to frequently change the configuration files due to some reasons, how can we solve this problem? We cannot modify them one by one if there are lots of machines!

What we want is managing the configuration in one place, the "configuration center". Here are some awesome projects that can help us solve this problem, such as Apollo, Consul, etc.

And in this article, I will introduce three ways using Consul KV store.

What is Consul

Dynamic ASP.NET Core Configurations With Consul KV

Consul is a distributed, highly available, and data center-aware solution to connect and configure applications across a dynamic, distributed infrastructure.

We will use one of its features named Key/Value Storage. It's a flexible key/value store that enables storing the dynamic configuration, feature flagging, coordination, leader election, and more. The simple HTTP API makes it easy to use anywhere.

Calling REST API Directly

In this way, we read configuration from Consul KV store via REST API. What we need to configure in our project is the address of Consul.

Creating an ASP.NET Core Web API project at first.

Here, we will use a package named Consul to handle the REST API, which can make it easier!

Add Consul package via NuGet.

Install-Package Consul -Version 0.7.2.6

In order to config the address of Consul, we need to modify the appsettings.json file.

{
"Logging": {
"LogLevel": {
"Default": "Warning"
}
},
"AllowedHosts": "*",
"Consul": {
"Host": "http://127.0.0.1:8500"
}
}

After configuring the address, we should add an extesion for Consul which can help us to use the ConsulClient with DI.

public static IServiceCollection AddConsul(this IServiceCollection services, IConfiguration configuration)
{
services.AddSingleton<IConsulClient, ConsulClient>(p => new ConsulClient(consulConfig =>
{
//consul address
var address = configuration["Consul:Host"];
consulConfig.Address = new Uri(address);
}, null, handlerOverride =>
{
//disable proxy of httpclienthandler
handlerOverride.Proxy = null;
handlerOverride.UseProxy = false;
}));
return services;
}

The next thing we should do is call the ConsulClient where you need it!

The following code shows you how to use it in Controller.

[Route("api/[controller]")]
[ApiController]
public class ValuesController : ControllerBase
{
private readonly IConsulClient _consulClient;
public ValuesController(IConsulClient consulClient)
{
this._consulClient = consulClient;
}
[HttpGet("")]
public async Task<string> GetAsync([FromQuery]string key)
{
var str = string.Empty;
//query the value
var res = await _consulClient.KV.Get(key);
if (res.StatusCode == System.Net.HttpStatusCode.OK)
{
//convert byte[] to string
str = System.Text.Encoding.UTF8.GetString(res.Response.Value);
}
return $"value-{str}";
}
}

Let's take a look at the result.

Before adding a key in Consul KV, we cannot get the value.

After creating and modifying, we can get the latest value in real-time.

However, this way cannot combine with the configuration system that ASP.NET Core provides. In the next section I will introduce how to combine with ASP.NET Core's configuration.

Combine With Configuration System

Most of the time, we use IConfiguration, IOptions<T> , IOptionsSnapshot<T> and IOptionsMonitor<T> to read the configuration in ASP.NET Core. How can we combine them with consul?

Here I suggest using Winton.Extensions.Configuration.Consul. It's an amazing project that adds support for configuring .NET Core applications using Consul.

Due to using this project, things will become easier! It helps us to read and reload the configurations in consul KV.

Let's take a look at how to do it.

Install this package via NuGet at first.

Install-Package Winton.Extensions.Configuration.Consul -Version 2.1.2

Prepare before we configure with Consul.

Create a setting class that shows the usages of Options.

public class DemoAppSettings
{
public string Key1 { get; set; }
public string Key2 { get; set; }
}

Configuring the options in Startup class:

public void ConfigureServices(IServiceCollection services)
{
services.Configure<DemoAppSettings>(Configuration.GetSection("DemoAppSettings"));
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
}

Note
We don't need to modify the appsettings.json file.

Calling the configuration in controller.

[Route("api/[controller]")]
[ApiController]
public class ValuesController : ControllerBase
{
private readonly IConfiguration _configuration;
private readonly DemoAppSettings _options;
private readonly DemoAppSettings _optionsSnapshot;
public ValuesController(
IConfiguration configuration
, IOptions<DemoAppSettings> options
, IOptionsSnapshot<DemoAppSettings> optionsSnapshot)
{
this._configuration = configuration;
this._options = options.Value;
this._optionsSnapshot = optionsSnapshot.Value;
}
// GET api/values
[HttpGet]
public ActionResult<IEnumerable<string>> Get()
{
return new string[] {_configuration["DemoAppSettings:Key1"], _options.Key1, _optionsSnapshot.Key1 };
}
}

At last, we should configure Winton.Extensions.Configuration.Consul. It provides two ways to do it, one is in Program class, the other on is in Startup class.

The following code demonstrates how to configure in Program class.

public static void Main(string[] args)
{
var cancellationTokenSource = new CancellationTokenSource();
WebHost
.CreateDefaultBuilder(args)
.ConfigureAppConfiguration(
(hostingContext, builder) =>
{
builder
.AddConsul(
"App1/appsettings.json",
cancellationTokenSource.Token,
options =>
{
options.ConsulConfigurationOptions =
cco => { cco.Address = new Uri("http://127.0.0.1:8500"); };
options.Optional = true;
options.ReloadOnChange = true;
options.OnLoadException = exceptionContext => { exceptionContext.Ignore = true; };
})
.AddEnvironmentVariables();
})
.UseStartup<Startup>()
.Build()
.Run();
cancellationTokenSource.Cancel();
}

Here is the result after running up this project.

We create a KV in Consul, and visit http://localhost:5000/api/values. We can read the configuration from Consul.

After modifying key1's value from 1111 to 1122, refresh the browser; the values were changed but IOptions<DemoAppSettings>.

So, when we use this way to handle the configuration, we should not use IOptions<T> in our code! This is an important tip.

There also has another way to make the configurations dynamically.

Consul Template

Consul Template is a component that provides a convenient way to populate values from Consul into the file system.

For more information, please visit its GITHUB page

Most of the times, the configurations in appsettings.json, appsettings.xml, appsettings.ini or other files are hard-coded, however, we can utilize Consul’s KV store to avoid hard-coding configurations in the file so we don’t have to change the file each time if any modification is needed in application settings.

Those file updates should be done by the Consul tool. We can replace any file instead of this JSON file as the procedure works the same for all files.

The following picture shows how to do it.

Create an ASP.NET Core Web API project at first.

Here is something like before.

public class DemoAppSettings
{
public string Key1 { get; set; }
public string Key2 { get; set; }
}
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
//config
services.Configure<DemoAppSettings>(Configuration.GetSection("DemoAppSettings"));
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
}
}
[Route("api/[controller]")]
[ApiController]
public class ValuesController : ControllerBase
{
private readonly IConfiguration _configuration;
private readonly DemoAppSettings _options;
private readonly DemoAppSettings _optionsSnapshot;
public ValuesController(IConfiguration configuration, IOptions<DemoAppSettings> options, IOptionsSnapshot<DemoAppSettings> optionsSnapshot)
{
this._configuration = configuration;
this._options = options.Value;
this._optionsSnapshot = optionsSnapshot.Value;
}
// GET api/values
[HttpGet]
public ActionResult<IEnumerable<string>> Get()
{
//to show the result of different usges
return new string[] { _configuration["DemoAppSettings:Key1"], _options.Key1, _optionsSnapshot.Key1 };
}
}

Now, we should turn to consul-template.

Creating a configuration file

Creating a configuration file that tells `consul-template` the source and destination file name.

`consul-template` will read configurations and convert the template file by the adding values referenced in the template file from the Consul’s Key-Value store and generate the configuration file for the application.

Here, we create a configuration file named `appsettings.tpl`, and its content is as follows.

{{ key "App1/appsettings.json" }}

starting consul-template,

consul-template -template "yourpath/appsettings.tpl:yourpath/appsettings.json"

Here is the result.

After modifying the value in Consul KV, appsettings.json was changed in real-time.

And the behavior of accessing the browser is the same as in the second way.

Summary

This article showed you the three ways to our dynamic ASP.NET Core project configuration with Consul KV.

Calling REST API Directly
Winton.Extensions.Configuration.Consul
Consul Template

And I will suggest you use the second and third way. Because those two ways are easier.

I hope this can help you!

Related Articles

2 Comments

Join the conversation! Your thoughts help the community grow.

Ano Mepani7y ago
I never imagined this type of situation may occur and also solution is possible. Thanks for sharing such useful information
- Catcher Wong7y ago
  Emmm...sometimes I met this situation due to some reasons! Whether we met or not, we should manager the configurations via a configuration center.