Beyond Infrastructure Part I: Using Terraform to Manage Azure Policies

Published 12/12/2018 08:36 AM   |    Updated 01/08/2019 11:34 AM
 
Terraform is a great product for managing Azure infrastructure, but did you know you can do a lot more than just stand up Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) resources?
 
I was creating a set of Azure policies that I could port across several Azure subscriptions. For simplicity’s sake, we’ll look at a single policy definition around requiring certain tags for every resource in the subscription. Let's see how I used Terraform to quickly accomplish this.
 

Traditional approach

 
In the past, if you wanted to maintain Azure policies, you could either use the Azure Portal or Azure Resource Manager (ARM) templates.
 
The Azure Portal is a great tool. However, there’s too much manual intervention and chance for human error when creating and updating policies/assignments.
 
ARM templates can work as well, but they don't give you the flexibility to see the difference between your configuration before pushing a change. Also, if you want to span multiple subscriptions, you’d have to create your own tooling around managing the changes across all of the subscriptions.
 
Can Terraform do this more easily?
 

Azure Terraform provider

 
There are two resources of interest:
 
  • azurerm_policy_definition — creates the custom policy definition for our subscription
  • azurerm_policy_assignment — creates a policy assignment of the policy definition
 

Azure policy — audit-required tags

 
Azure tags are key to keeping track of the infrastructure in your subscription. Unless you’ve thoroughly planned out your tagging strategy, you may find yourself in a situation where you want to start requiring a tag on all resources.
 
Your first question should be, "How compliant is my current infrastructure for this newly required tag?" We can easily answer this with an Azure policy using the audit effect.
 
Let's take a look at what this definition would look like in Terraform.
 

azurerm_policy_definition

 
We need to set the following parameters:
 
  • name: the name of the policy, used to build the id
  • display_name: the display name used in the Azure Portal.
  • description: technically optional but a great way to add clarity to the purpose of the policy
  • policy_type: type of policy, should be set to 'Custom'
  • mode: the resources this policy will affect, should be set to 'All'
  • policy_rule: the JSON representing the Rule
  • parameters: the JSON representing the Parameters
 
To get started with the obvious fields, we have:
 
resource "azurerm_policy_definition" "requiredTag" {
  name         = "Audit-RequiredTag-Resource"
  display_name = "Audit a Required Tag on a Resource"
  description  = "Audit all resources for a required tag"
  policy_type  = "Custom"
  mode         = "All"
  policy_rule  = "???"
  parameters   = "???"
}
 
The policy_rule and parameters must be in the form of JSON. This isn’t due to a design decision on the part the Terraform provider; it’s just how Azure has to interpret the policy. This can be a little convoluted, so let's use the Terraform template_file provider to keep things as clean as possible.
 

Rule JSON

 
data "template_file" "requiredTag_policy_rule" {
  template = <<POLICY_RULE
{
    "if": {
        "field": "[concat('tags[', parameters('tagName'), ']')]",
        "exists": "false"
    },
    "then": {
        "effect": "audit"
    }
}
POLICY_RULE
}
 

Parameter JSON

 
data "template_file" "requiredTag_policy_parameters" {
  template = <<PARAMETERS
{
    "tagName": {
        "type": "String",
        "metadata": {
            "displayName": "Tag Name",
            "description": "Name of the tag, such as 'environment'"
        }
    }
}
PARAMETERS
}
 
Now we can reference these via interpolation:
 
resource "azurerm_policy_definition" "requiredTag" {
  ...
  policy_rule  = "${data.template_file.requiredTag_policy_rule.rendered}"
  parameters   = "${data.template_file.requiredTag_policy_parameters.rendered}"
}
 

Apply the definition.

 
Now we’re ready to run a Terraform plan where we end up with something like this:
 
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
​
Terraform will perform the following actions:
​
  + azurerm_policy_definition.requiredTag
      id:           <computed>
      description:  "Audit all resources for a required tag"
      display_name: "Audit a Required Tag on a Resource"
      mode:         "All"
      name:         "Audit-RequiredTag-Resource"
      parameters:   "{\n    \"tagName\": {\n        \"type\": \"String\",\n        \"metadata\": {\n            \"displayName\": \"Tag Name\",\n            \"description\": \"Name of the tag, such as 'environment'\"\n        }\n    }\n}\n"
      policy_rule:  "{\n    \"if\": {\n        \"field\": \"[concat('tags[', parameters('tagName'), ']')]\",\n        \"exists\": \"false\"\n    },\n    \"then\": {\n        \"effect\": \"audit\"\n    }\n}\n"
      policy_type:  "Custom"
​
​
Plan: 1 to add, 0 to change, 0 to destroy.
​
------------------------------------------------------------------------
 
Running a Terraform apply creates the policy in the Azure subscription. Navigating to the Azure Portal, we can see the custom policy:
 
 

Azure policy assignment

 
Now that we’ve defined a custom Azure policy, we need to assign it to our subscription to make use of it:
 
azurem_policy_assignment
 
We need to set the following parameters:
 
  • name: the name of the assignment, used to build the id
  • display_name: the display name used in the Azure Portal
  • description: technically optional but a great way to add clarity to the purpose of the assignment
  • policy_definition_id: the id of the policy definition we created above
  • scope: the scope of this assignment; here we’re scoping this to the entire subscription
  • parameters: the JSON representing the required tag to assign to the definition
 
To get started with the obvious fields, we have:
 
resource "azurerm_policy_assignment" "requiredTag" {
  name                 = "Audit-RequiredTag-${var.requiredTag}"
  display_name         = "Assign Required Tag '${var.requiredTag}'"
  description          = "Assignment of Required Tag Policy for '${var.requiredTag}'"
  policy_definition_id = "???"
  scope                = "???"
  parameters           = "???"
 
Note the use of a variable 'requiredTag' we’ve created to parameterize the resource creation. More on this in a bit.
 

policy_definition_id

 
This id is simply pulled from the id output from the azurerm_policy_definition resource.
 
resource "azurerm_policy_assignment" "requiredTag" {
  ...
  policy_definition_id = "${azurerm_policy_definition.requiredTag.id}"
  ...
}
 

Scope

 
We want this policy assignment to be for the entire subscription. One option here would be to pass the subscription id as a variable. However, we can source the id from the active Terraform run by using the azurerm_subscription data source.
 
data "azurerm_subscription" "current" {}
​
resource "azurerm_policy_assignment" "requiredTag" {
  ...
  scope                = "${data.azurerm_subscription.current.id}"
  ...
}
 

Parameters

 
The last piece we need is the parameters JSON used to assign the 'requireTag' value in the Azure policy. Much like we did before, we leverage the template_file provider.
 
data "template_file" "requiredTag_policy_assign" {
  template = <<PARAMETERS
{
    "tagName": {
        "value": "${var.requiredTag}"
    }
}

PARAMETERS
}
​
resource "azurerm_policy_assignment" "requiredTag" {
  ...
  parameters           = "${data.template_file.requiredTag_policy_assign.rendered}"
}
 

Apply assignment

 
Now we’re ready to run a Terraform plan where we end up with something like this:
 
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
​
Terraform will perform the following actions:
​
  + azurerm_policy_assignment.policy
      id:                   <computed>
      description:          "Assignment of Required Tag Policy for 'Environment'"
      display_name:         "Assign Required Tag Environment"
      name:                 "Audit-RequiredTag-Environment"
      parameters:           "{\n    \"tagName\": {\n        \"value\": \"Environment\"\n    }\n}\n\n"
      policy_definition_id: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.Authorization/policyDefinitions/Audit-RequiredTag-Resource"
      scope:                "/subscription//subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
​
​
Plan: 1 to add, 0 to change, 0 to destroy.
 
Running a Terraform apply creates the assignment in the Azure subscription. Navigating to the Azure Portal, we can see the assignment:
 
 
But what if I have more than one required tag?
 

Scaling with count

 
One of the key factors to Terraform is the ability to easily scale. Let's modify our current implementation to handle a list of required tags.
 
First, let's update our variable from a string to a list:
 
variable "requiredTags" {
  default = [
    "Environment",
    "Owner",
    "Department",
  ]
}
 
Now we can inject a count parameter in the assignment resource:
 
resource "azurerm_policy_assignment" "requiredTag" {
  count                = "${length(var.requiredTags)}"
​
  name                 = "Audit-RequiredTag-${var.requiredTags[count.index]}"
  display_name         = "Assign Required Tag '${var.requiredTags[count.index]}'"
  description          = "Assignment of Required Tag Policy for '${var.requiredTags[count.index]}'"
  ...
}
 
Note that we use the length of the requiredTags variable to indicate how many times to repeat the assignment, then index into the list for the name.
 

Parameters

 
The parameters value is a little less clean since we have to inject a different value, depending on the index.
 
We can do this inline to the assignment without much trouble:
 
resource "azurerm_policy_assignment" "requiredTag" {
  ...
  parameters = <<PARAMETERS
{
    "tagName": {
        "value": "${var.requiredTags[count.index]}"
    }
}
PARAMETERS
}
 

Apply the assignments.

 
Now we’re ready to run a Terraform plan where we end up with something like this:
 
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
​
Terraform will perform the following actions:
​
  + azurerm_policy_assignment.requiredTag[0]
      id:                   <computed>
      description:          "Assignment of Required Tag Policy for 'Environment'"
      display_name:         "Assign Required Tag 'Environment'"
      name:                 "Audit-RequiredTag-Environment"
      parameters:           "{\n    \"tagName\": {\n        \"value\": \"Environment\"\n    }\n}\n\n"
      policy_definition_id: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.Authorization/policyDefinitions/Audit-RequiredTag-Resource"
      scope:                "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
​
  + azurerm_policy_assignment.requiredTag[1]
      id:                   <computed>
      description:          "Assignment of Required Tag Policy for 'Owner'"
      display_name:         "Assign Required Tag 'Owner'"
      name:                 "Audit-RequiredTag-Owner"
      parameters:           "{\n    \"tagName\": {\n        \"value\": \"Owner\"\n    }\n}\n\n"
      policy_definition_id: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.Authorization/policyDefinitions/Audit-RequiredTag-Resource"
      scope:                "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
​
  + azurerm_policy_assignment.requiredTag[2]
      id:                   <computed>
      description:          "Assignment of Required Tag Policy for 'Department'"
      display_name:         "Assign Required Tag 'Department'"
      name:                 "Audit-RequiredTag-Department"
      parameters:           "{\n    \"tagName\": {\n        \"value\": \"Department\"\n    }\n}\n\n"
      policy_definition_id: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/providers/Microsoft.Authorization/policyDefinitions/Audit-RequiredTag-Resource"
      scope:                "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
​
​
Plan: 3 to add, 0 to change, 0 to destroy.
​
------------------------------------------------------------------------
 
Notice the tag values are correctly indexed and displayed.
 
Running a Terraform apply creates the assignment in the Azure subscription. Navigating to the Azure Portal, we can see the assignments:
 
 

Viewing results

 
Once the audit policy assignments have had time to be checked, any non-compliant resources will show up in the portal.
 
 
As you can see, I have several resources that don’t have the "Owner" tag, and I can work toward making them compliant.
 
Once I have a good handle on these required tags, I can update the Terraform from "effect": "audit" to "effect": "deny." This will deny any new request to create or modify any resource that doesn't have the "Owner" tag.
 

Conclusions

 
In this article, you’ve been shown how you can leverage Terraform to manage Azure policies to create a consistent governance compliance across your Azure subscription. One really great benefit to this solution is that it can be applied to many different Azure subscriptions without much change in the configuration.
 
All assets in this article can be found in the GitHub Gist.
 

Explore more Digital Innovation insights →

 
This article originally appeared on July 18, 2018.

Is this answer helpful?