Creating Azure Resource Groups Using Terraform And Azure Key Vault

Introduction

In our previous article, we described the development process for provisioning an Azure Resource Group using Terraform Code. I promised a follow-up blog that will try and get rid of some of the hard-coded values we used for critical information like Azure Subscription IDs, Tenant IDs, Client IDs, and Client Secrets. While the earlier blog works perfectly and does the job of provisioning resource groups, it does not comply with security best practices.

The majority of secure environments need applications to avoid storing critical information locally or within application databases. Critical information should be fetched from a central, secure, and regulated location which is where the Azure Key Vault comes into the picture. Azure Key Vault is an offering in Microsoft Azure that provides the capability for securely storing and accessing secrets. We will be integrating our Terraform Code solution that we had set up in an earlier blog with Azure Key Vault. We will be using one of the custom APIs that we built and described in the article here. Below is a technical diagram for the overall solution that we are setting up for this which brings together two solutions that we will keep on enhancing and blogging about.

Creating Azure Resource Groups Using Terraform And Azure Key Vault

Pre-Requisites

If you have not already set up the pre-requisites for the Key Vault API and Azure Resource Group provisioning through Terraform that we have described in our earlier articles, all of those will need to be set up now. To summarize, below is what is needed for this to work:

Custom Key Vault API to fetch secrets from Azure Key Vaults
Terraform Setup
Azure Key Vault and stored secrets for Subscription Id, Tenant Id, Client Id, and Client Secret
Azure API Management Subscription Key
Get/List Access to client id on Azure Key Vault

Implementation

Let's get down to the code now. As in the previous article, we will have a similar folder structure for code hierarchy with a main, a variable, and an output file.

main.tf

Begin by opening the main.tf file and adding the following data block. Terraform does not natively support API invocation and therefore, we are using HTTP data sources to achieve api invocation. Important to note that HTTP data sources only support GET methods at this time. We will need to write our own custom modules if we wanted to support POST, PATCH, etc.

This HTTP data source takes the secret names as input, which is a comma-separated list of secrets that we will be fetching from the Azure Key Vault, from the variables file. We are then iterating through the secret names provided and fetching the values corresponding to these names from Azure Key Vault using the custom API endpoint we have created for the GetKeyVaultSecrets function in Azure API management as part of our earlier blog. This data block will provide the dynamic values that we need for the Terraform Azure provider and can be used for fetching other values as well on a need-to-need basis.

#Fetch secrets from Azure Key Vault using custom API
data "http"
"GetKeyVaultSecrets" {
count = length(var.secret_names)
//Below block fetches the key vault secrets for all comma separated keys specified in variables
url = "${local.get_secret_api}${element(var.secret_names,count.index)}"
# Optional request headers
request_headers = {
Accept = "application/json"
Ocp - Apim - Subscription - Key = local.ocp_subscription_key
}
}

Now add the below data block which is fetching the target subscription ID for provisioning the resource group using the subscription name we have provided in variables file. This is completely optional and can be substituted with your own business logic or needs.

#Fetch the subscription Id from Azure Key Vault using custom API
data "http"
"GetSubscriptionId" {
url = "${local.get_secret_api}${replace(var.subscription_name,"
_ "," - ")}"
# Optional request headers
request_headers = {
Accept = "application/json"
Ocp - Apim - Subscription - Key = local.ocp_subscription_key
}
}

We will now be adding a locals block to define the local variables that will be used. This is block is used to perform manipulations with the variables. We are storing the endpoint details for the custom API we will use for fetching secrets from the key vault and the subscription key that will be used for placing the request to this API.

locals {
// 1. We are doing a replace on spaces in resource group name since Terraform is not supporting that currently
// 2. We are also truncating the rg name if it is greater than 64 characters since that is the max that Azure supports
rg_name = replace((length(var.resource_group_name) > 64 ? substr(var.resource_group_name, 0, 63) : var.resource_group_name), " ", "-")
get_secret_api = "https://your-apim-domain-name.azure-api.net/GetKeyVaultSecrets?SecretName="
ocp_subscription_key = "your-apim-subscription-key"

Now that we are ready with other required blocks, we will modify the provider block. The provider block needs details of subscription ID, tenant ID, client ID and client secret that we have already fetched in our first data block defined above. We need to use the jsondecode function to get the required details out of the data blocks we have created earlier.

provider "azurerm" {
version = ">=2.4.0"
subscription_id = jsondecode(data.http.GetSubscriptionId.body)
tenant_id = jsondecode(data.http.GetKeyVaultSecrets[0].body)
//Use this only for Service Principle based authentication
client_id = jsondecode(data.http.GetKeyVaultSecrets[1].body)
client_secret = jsondecode(data.http.GetKeyVaultSecrets[2].body)
//use_msi = true // uncomment and set to true for using System Assigned Managed Identities
features {}
}

The final piece for creating the resource group in Azure can now be added with the details that you need as per your business logic and standardizations. We are just providing name, location, and one instance type tag to the resource group.

#Provision Resource Group
resource "azurerm_resource_group"
"rg" {
name = local.rg_name
location =
var.resource_location
tags = {
InstanceType =
var.instance_type
}
}

The main executable code is now ready and we will just create the two additional files for variables and output to define the input and output variables that will be part of this execution.

variables.tf

variable resource_group_name {
type = string
default = "testrg"
description = "The name of resource group required to be provisioned"
}
variable secret_names {
type = list(string)
description = "List of all secret names for which values are to be fetched from key vault.."
default = ["SHS-TenantID", "sp-sa-CloudOpsAutomationMPCD-Dev-01-AppId", "sp-sa-CloudOpsAutomationMPCD-Dev-01-Secret"]
}
variable subscription_name {
type = string
default = "your-subscription-name"
description = "Provide the subscriptinon name to fetch subscription id from key vault"
}
variable resource_location {
type = string
description = "The location to deploy Azure resources requested"
}
#region: Tags to be used
for Resource Group Creation
variable instance_type {
type = string
description = "Instance Type Tag Value"
}

outputs.tf

output "resource_group_id" {
description = "id of the resource group provisioned"
value = "${azurerm_resource_group.rg.id}"
}
output "resource_group_name" {
description = "name of the resource group provisioned"
value = "${azurerm_resource_group.rg.name}"
}

That's it! We are ready with the code now that will create an Azure Resource Group by fetching details dynamically from the Azure Key Vault. Use the below commands for doing the plan and apply for this code. Remember to perform a Terraform Init before doing a plan and apply. We are using a sample tfvars file for our local execution with variables defined under:

subscription_name = "your-subscription-name"
secret_names = ["your-teanant-id-key","your-client-id-key","your-client-secret-key"]
resource_group_name = "Test Resource Group"
resource_location = "eastus"
instance_type = "POC"
terraform plan -var-file="<yourfilename>.tfvars" or terraform plan
terraform apply -var-file="<yourfilename>.tfvars or terraform apply

We have now successfully completed the integration of Terraform Code with Azure Key Vault to be able to retrieve critical details from Azure Key Vault instead of hardcoding them in the Terraform Code or any other source application that will trigger the Terraform plan(s) and apply(s). This makes our Terraform code compliant with security best practices. You can get the solution code at our GitHub Repository.

Happy Coding!