Creating An Azure API To Fetch Key Vault Secrets

In this blog, co-authored by Hitesh Kacholiya and Moh Aatif Naseem, we will talk about the process to create an Azure API to fetch Key Vault Secrets

The first thing that will come to your mind, at least it did to ours, is - 'Doesn't Microsoft have its own APIs to fetch secrets from Azure Key Vault?' Well, the answer is Yes, they do. So then, what are we trying to achieve here? We will hopefully address that question during the course of this blog which is going to be slightly long due to the need to explain the setup and architecture along with the API (further blogs will reference this).

 

Okay, so let's first visit what Microsoft has to offer. Microsoft has APIs listed for fetching keys, fetching secrets, getting lists etc. for the Azure Key Vault right here. As you notice with the secrets API, all of the calls require - (a) the Key Vault API end-point URL, (b) the secret value name that your looking for (c) secret version (even if there is only one version) that you need and the most important one which is not listed and is kind of read between the lines (d) a Bearer Token to authenticate to Azure Key Vault.

 

So, assuming that you would need to do this at more than one place and more than one module/application, all of the modules that will need to fetch anything from the key vault, will need authentication details like application id, application secret and tenant id to be provided. This becomes an overhead in a large solution or the pizza box teams that we work with in the modern environments. Here is where the API that we are trying to build comes into the picture. With this API, we will

• eliminate the need to implement authentication details with all teams/modules
• eliminate the need to provide the secret versions
• provide a simple end-point to get any secret from Azure Key Vault
• remove the need to rewrite logic in every module/functionality

We are going to create an Azure Function to host the API end-point which is then exposed and made available to end users via the Azure API Management Gateway. This provides us with more control on the access with a product-subscription model, implements policies, and also allows us to do any tweaks with end-points or make only a few functions availalbe instead of all via specific product subscriptions. In addition, the hosting end-point on Azure Function also seems a little more  cost effective compared to hosting on App Services (due to the fact that we pay only usage with Azure Function versus paying for App Service Plans or App Service Environment costs). The below diagram briefly describes the overall architecture,

 

 

Now that we have covered the overall architecture that will be used in subsequent reusable API endpoints that we will set up, let's jump right into building this one! So, what do we need before starting to write the code?

• Create an Azure Function resource in Azure
• Configure System Assigned Managed Identity to the created Azure Function resource

• Provide access to configured System Assigned Managed Identity at the target Key Vault Resource (from where secret is to be fetched) via Access Policies
• Visual Studio Code or Visual Studio (IDE of your choice) setup with Azure Function configurations. If you are using VS Code, install the Azure Functions extension
• Create an Azure API Management Service in Azure

In our next step, we will create an Azure Function project and add an HTTP Trigger based Azure Function to the project in VS Code. Let's jump right to it. 

//Open the terminal from VS Code (using Ctrl+Shift+P) and create a new solution file
dotnet new sln

//Create an Azure Functions Project
Follow the steps in this article (https://docs.microsoft.com/en-us/azure/azure-functions/functions-develop-vs-code?tabs=csharp) and create a new Azure Function Project with C# as the language, "GetKeyVaultSecret" as your function name, HTTP Trigger as the Function Template and Authorization level as Anonymous (this is required for API Mangement)

//Add project to solution by executing this command on terminal
dotnet sln <your_solution_name>.sln add .\<your_solution_name>\<your_azure_function_project_name>.csproj

With these steps, we are now ready with our master solution to which we will add all our functions that will be exposed via API Management to subscribers. Now, let's code the Azure Function to get Key Vault Secrets. Open your GetKeyVaultSecret.cs file and update the below code in it.

using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using Microsoft.Azure.Services.AppAuthentication;
using Microsoft.Azure.KeyVault.Models;
using Microsoft.Azure.KeyVault;

namespace YourNamespace
{
    /// <summary>
    /// This function is to fetch secret value for a given secret name
    /// </summary>
    public static class GetKeyVaultSecret
    {
        //Name of the function endpoint
        [FunctionName("GetKeyVaultSecret")]
        public static async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest req,
            ILogger log)
        {
            string secretValue = string.Empty;
            log.LogInformation("GetKeyVaultSecret Function is called");

            try
            {
                //Get secret Name from the query string
                string secretName = req.Query["SecretName"];
                string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
                dynamic data = JsonConvert.DeserializeObject(requestBody);
                secretName = secretName ?? data?.SecretName;

                if (!string.IsNullOrEmpty(secretName))
                {
                    log.LogInformation("Secret Vaule Requested For : " + secretName);

                    //Create new service token provider using System Assigned Managed Identity
                    AzureServiceTokenProvider azureServiceTokenProvider = new AzureServiceTokenProvider();

                    //Create new Key Vault Client
                    KeyVaultClient keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));

                    //Get Key Vault Name from application configuration
                    string keyVaultName = ConstantsHelper.GetEnvironmentVariable(ConstantsHelper.keyVaultName);
                    log.LogInformation("Fetching Details from KeyVault : " + keyVaultName);

                    //Fetching the Key Vault URI
                    string keyVaultUri = String.Format(Convert.ToString(ConstantsHelper.keyVaultUri), keyVaultName);
                    log.LogInformation("KeyVaultUri : " + keyVaultUri);

                    //Get secret from Key Vault
                    SecretBundle secretBundle = await keyVaultClient.GetSecretAsync(keyVaultUri, secretName);

                    if(secretBundle != null)
                    {
                        //Get Secret Value to be returned
                        secretValue = secretBundle.Value;
                        log.LogInformation("Details are fetched from KeyVault");
                        return new OkObjectResult(secretValue);
                    }
                    else
                    {
                        log.LogInformation("No such key name present in KeyVault");
                        //Return no key found message
                        return new NotFoundObjectResult("No such key name present in KeyVault");
                    }
                }
                else
                {
                    log.LogInformation("secretName is missing in request");
                    return new BadRequestObjectResult("secretName is missing in request");
                }
            }
            catch (Exception ex)
            {
                log.LogInformation($"GetKeyVaultSecret got an exception \n Time: { DateTime.Now} \n Exception{ ex.Message}");
                return new NotFoundObjectResult($"\n GetKeyVaultSecret got an exception \n Time: { DateTime.Now} \n Exception{ ex.Message}");
            }

        }
    }
}

To follow the best practices of coding, we will now create a helper class which is referenced in the above-created class file. Create a helper class file with the name ConstantsHelper.cs and update the below code in it. The intent of this class is to have all constants used in the solution defined at one place.

public const string keyVaultName = "keyVaultName";
public const string keyVaultUri = "https://{0}.vault.azure.net/";
// Method to get environmental variables from app service settings
public static string GetEnvironmentVariable(string name)
{
       return System.Environment.GetEnvironmentVariable(name,
       EnvironmentVariableTarget.Process);
}

As there are environmental variables used in our code, now we would need to update the them at Azure Function App settings. In this case, there is only one variable i.e. keyVaultName. Lets update it by following the below steps,

• Go to your Azure Function App in Azure portal
• Click on Configuration under settings section

• Click on new application settings
• Provide name as "keyVaultName" and value as "your-key-vault-name"

Now that we are ready with the function, let's deploy this to the Azure Function App resource that we created as part of pre-requisites. When you set up continuous deployment, your function app in Azure is updated whenever source files are updated in the connected source location. We recommend continuous deployment, but you can also republish your project file updates from Visual Studio Code.

• In Visual Studio Code, select F1 to open the command palette. In the command palette, search for and select Azure Functions: Deploy to function app
• If you're not signed in, you're prompted to Sign in to Azure. After you sign in from the browser, go back to Visual Studio Code. If you have multiple subscriptions, select a subscription that contains your function app.
• Select your existing function app in Azure. When you're warned about overwriting all files in the function app, select Deploy to acknowledge the warning and continue

Note

Publishing to an existing function app overwrites the content of that app in Azure. The project is rebuilt, repackaged, and uploaded to Azure. The existing project is replaced by the new package, and the function app restarts.

 

Next, we will import the deployed function app into Azure API Management service that was created as part of pre-requisites. Please follow the steps given in this article to import the created function and to have an API endpoint for the same. Once function is imported into API Management Service instance, you are all set to invoke the API from any custom application, postman or any other platform to fetch secret from configured key vault. Let's try this out from this API Management instance directly in Azure Portal by doing the following,

• Go to the created API Management Service Instance
• Select API option under APIs section 
• Select API that you created under All APIs section and it will display "GetKeyVaultSecret" operation of selected API with its respective exposed method i.e. Get.
• Click on "GetKeyVaultSecret" operation and click on Test tab
• Add a Query Parameter using key and value i.e. "SecretName" and "Your-Secret-Name-To-Be-Fetched"
  

  
  
• Click on Send to get a response with Secret value

To invoke configured API from outside of API Management Services, you need to pass Ocp-Apim-Subscription-Key as header, for the subscription key of the product that is associated with this API. For more details on Subscriptions, please read this article.

 

 

 

Below is a sample code with C# Restsharp using Postman. 

var client = new RestClient("https://your-apim-site.azure-api.net/GetKeyVaultSecret?SecretName=your-secret-name");
var request = new RestRequest(Method.GET);
request.AddHeader("Ocp-Apim-Subscription-Key", "your-subscription-key");
IRestResponse response = client.Execute(request);

There! We have a fully functional endpoint that allows us to fetch any secret from the Azure Key Vault writing only an invoke command from your application and keeps functionality loosly coupled, independently deployable and highly maintainable (sure you have read similar properties elsewhere in some microservices articles). Needless to say that this API can now be consumed in any application on any platform as long as the consuming application has subscribed to a product and has a valid subscription key. The entire code base for the Reusable API solution is available at Reusable APIs

 

We will use this API in our follow up blogs in our Terraform Series (previous blog link) to just demonstrate how it makes a developer's life really easy in products that do not natively provide lot of support to do fancy stuff. We will keep on adding to this setup and do a few more APIs that can find implementation in most development groups.

 

Happy Coding!