Posted on Apr 4, 2022 • Edited on Aug 17, 2024

Terraform - Filter results using 'for' loops

Overview

In todays tutorial we will take a look at a fairly common question I often get from the community and it is around how to filter results in Terraform or even if it is possible. We will also look at a real world usage example so that we can see how and when we would use filters in Terraform.

Filtering in Terraform can be achieved using for loop expressions. Though for loop constructs in terraform performs looping, it can also be used for manipulating data structures such as the following to name a few:

Transform: Changing the data structure.
Filter: Filter only on desired items in combination with an if expression.
Group: Group elements together in a new list by key.

Filtering results

Let's take a look at the following example variable where we have a list of applications:

 variable "apps" { type = list(object({ app_name = string app_kind = string app_require_feature = bool })) default = [ { app_name = "App1" app_kind = "Linux" app_require_feature = false }, { app_name = "App2" app_kind = "Linux" app_require_feature = false }, { app_name = "App3" app_kind = "Windows" app_require_feature = true }, { app_name = "App4" app_kind = "Windows" app_require_feature = false } ] }

Say you want to filter only on app_require_feature = true you could write a for loop with an if expression like in the following local variable:

 locals { apps_that_require_feature = toset([for each in var.apps : each.app_name if each.app_require_feature == true]) } output "result" { value = local.apps_that_require_feature }

This will return a set of app_names that have the objects key "app_require_feature" set to true

 $ terraform apply Outputs: result = ["App3"]

So let's say you want to filter the same variable but this time you want to only see the apps that are windows, you could write a for loop with an if expression like in the following local variable:

 locals { windows_apps = toset([for each in var.apps : each.app_name if each.app_kind == "windows"]) } output "result2" { value = local.windows_apps }

This will return a set of app_names that have the objects key "app_kind" set to "windows"

 $ terraform apply Outputs: result2 = ["App3", "App4"]

Real world example

Let's take a real world usage case where we would need such a for construct to filter and only configure something based on certain criteria.

Say we have a variable with four storage accounts we want to create, but we only want to configure private endpoints on certain storage accounts. We could create an extra object key item called requires_private_endpoint like in the following example:

 ## variables ## variable "storage_config" { type = list(object({ name = string account_kind = string account_tier = string account_replication_type = string access_tier = string enable_https_traffic_only = bool is_hns_enabled = bool requires_private_endpoint = bool })) default = [ #V2 Storage (hot) without private endpoint { name = "pwd9000v2sa001" account_kind = "StorageV2" account_tier = "Standard" account_replication_type = "LRS" enable_https_traffic_only = true access_tier = "Hot" is_hns_enabled = false requires_private_endpoint = false }, #V2 Storage (cool) without private endpoint { name = "pwd9000v2sa002" account_kind = "StorageV2" account_tier = "Standard" account_replication_type = "LRS" enable_https_traffic_only = true access_tier = "Cool" is_hns_enabled = false requires_private_endpoint = false }, #ADLS2 Storage with private endpoint enabled { name = "pwd9000adls2sa001" account_kind = "BlockBlobStorage" account_tier = "Premium" account_replication_type = "ZRS" enable_https_traffic_only = false access_tier = "Hot" is_hns_enabled = true requires_private_endpoint = true }, #ADLS2 Storage without private endpoint { name = "pwd9000adls2sa002" account_kind = "BlockBlobStorage" account_tier = "Premium" account_replication_type = "ZRS" enable_https_traffic_only = false access_tier = "Hot" is_hns_enabled = true requires_private_endpoint = false } ] }

We can then create all four storage accounts with the following resource config:

 ## storage resources ## resource "azurerm_resource_group" "RG" { name = "example-resources" location = "uksouth" } resource "azurerm_storage_account" "SAS" { for_each = { for n in var.storage_config : n.name => n } #Implicit dependency from previous resource resource_group_name = azurerm_resource_group.RG.name location = azurerm_resource_group.RG.location #values from variable storage_config objects name = each.value.name account_kind = each.value.account_kind account_tier = each.value.account_tier account_replication_type = each.value.account_replication_type access_tier = each.value.access_tier enable_https_traffic_only = each.value.enable_https_traffic_only is_hns_enabled = each.value.is_hns_enabled }

In the following resource block we can now configure private endpoints, but we will only do so for storage accounts that have an object "key" of "requires_private_endpoint" set to "true" like in the following resource config:

 ## private endpoint resources ## resource "azurerm_private_endpoint" "SASPE" { for_each = toset([for pe in var.storage_config : pe.name if pe.requires_private_endpoint == true]) name = "${each.value}-pe" location = azurerm_resource_group.RG.location resource_group_name = azurerm_resource_group.RG.name subnet_id = data.azurerm_subnet.data_subnet.id private_service_connection { name = "${each.value}-pe-sc" private_connection_resource_id = azurerm_storage_account.SAS[each.value].id is_manual_connection = false subresource_names = ["dfs"] } }

If you take a closer look at the for_each in the azurerm_private_endpoint resource we are using the filter there as follow:

for_each = toset([for pe in var.storage_config : pe.name if pe.requires_private_endpoint == true])

This for loop will filter and return a set of storage account names that we can use to loop the resource creation of the private endpoints for the selected storage accounts. The storage account name values will be represented by each.value that matches the filter: requires_private_endpoint == true.

So in the example above, all four storage accounts will be created: