feat!: update TPG version constraints to 4.0 (terraform-google-modules#1129)

* feat: update TPG version constraints to allow 4.0 * Removes basic auth, renames namespace_identity * Regenerates modules and documentation * Updates tests to use latest Google provider * addresses warning about multiple provider blocks * Updates network module for Google provider 4.0 compatibility * Temporarily uses "main" for gcloud module (until next release is cut) * Comments out version constraint (temporary change) * fetches main branch by default? * Uses master branch for gcloud module (until release is cut) * Uses kubectl-wrapper where appropriate * Uses released version of gcloud module * Returns instance group URLs per node pool * Extends use of cluster_output_node_pools_ variables * Fixes documentation * Updates more modules * Updates READMEs to match variables * Uses master branch of bastion * temporary change until new version is released * Updates node pools versions description * Adds locals for node pool instance group URLs * Uses master branch of terraform-google-project-factory * temporary change until new version of that dependency is released * Updates project version ready for release * Updates pinned version of Google provider for example * Updates pinned version of Google provider in example * Addresses code review comments * Temporarily applies an empty source_tags setting. * this should be removed once hashicorp/terraform-provider-google#10494 is addressed * Fixes indentation * Uses newly-released version of project factory * Uses released version of bastion host * Removes use of SECURE mode (deprecated) * test empty source tag workaround * fix wi test * refactor IAM test for loose match * map old node meta value, add validations * update docs * Update autogen/main/variables.tf.tmpl Co-authored-by: Morgante Pell <morgantep@google.com> * remove local Co-authored-by: cloud-foundation-bot <cloud-foundation-bot@google.com> Co-authored-by: Jack Whelpton <jack.whelpton@rakuten.com> Co-authored-by: Morgante Pell <morgantep@google.com>

Author: 4 people

README.md

@@ -128,8 +128,6 @@ Then perform the following commands on the root folder:
 | add\_master\_webhook\_firewall\_rules | Create master\_webhook firewall rules for ports defined in `firewall_inbound_ports` | `bool` | `false` | no |
 | add\_shadow\_firewall\_rules | Create GKE shadow firewall (the same as default firewall rules with firewall logs enabled). | `bool` | `false` | no |
 | authenticator\_security\_group | The name of the RBAC security group for use with Google security groups in Kubernetes RBAC. Group name must be in format gke-security-groups@yourdomain.com | `string` | `null` | no |
-| basic\_auth\_password | The password to be used with Basic Authentication. | `string` | `""` | no |
-| basic\_auth\_username | The username to be used with Basic Authentication. An empty value will disable Basic Authentication, which is the recommended configuration. | `string` | `""` | no |
 | cluster\_autoscaling | Cluster autoscaling configuration. See [more details](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters#clusterautoscaling) | <pre>object({<br> enabled = bool<br> min_cpu_cores = number<br> max_cpu_cores = number<br> min_memory_gb = number<br> max_memory_gb = number<br> gpu_resources = list(object({ resource_type = string, minimum = number, maximum = number }))<br> })</pre> | <pre>{<br> "enabled": false,<br> "gpu_resources": [],<br> "max_cpu_cores": 0,<br> "max_memory_gb": 0,<br> "min_cpu_cores": 0,<br> "min_memory_gb": 0<br>}</pre> | no |
 | cluster\_ipv4\_cidr | The IP address range of the kubernetes pods in this cluster. Default is an automatically assigned CIDR. | `any` | `null` | no |
 | cluster\_resource\_labels | The GCE resource labels (a map of key/value pairs) to be applied to the cluster | `map(string)` | `{}` | no |
@@ -151,7 +149,7 @@ Then perform the following commands on the root folder:
 | grant\_registry\_access | Grants created cluster-specific service account storage.objectViewer and artifactregistry.reader roles. | `bool` | `false` | no |
 | horizontal\_pod\_autoscaling | Enable horizontal pod autoscaling addon | `bool` | `true` | no |
 | http\_load\_balancing | Enable httpload balancer addon | `bool` | `true` | no |
-| identity\_namespace | Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `[project_id].svc.id.goog`) | `string` | `"enabled"` | no |
+| identity\_namespace | The workload pool to attach all Kubernetes service accounts to. (Default value of `enabled` automatically sets project-based pool `[project_id].svc.id.goog`) | `string` | `"enabled"` | no |
 | impersonate\_service\_account | An optional service account to impersonate for gcloud commands. If this service account is not specified, the module will use Application Default Credentials. | `string` | `""` | no |
 | initial\_node\_count | The number of nodes to create in this cluster's default node pool. | `number` | `0` | no |
 | ip\_masq\_link\_local | Whether to masquerade traffic to the link-local prefix (169.254.0.0/16). | `bool` | `false` | no |
@@ -170,7 +168,7 @@ Then perform the following commands on the root folder:
 | network\_policy | Enable network policy addon | `bool` | `false` | no |
 | network\_policy\_provider | The network policy provider. | `string` | `"CALICO"` | no |
 | network\_project\_id | The project ID of the shared VPC's host (for shared vpc support) | `string` | `""` | no |
-| node\_metadata | Specifies how node metadata is exposed to the workload running on the node | `string` | `"GKE_METADATA_SERVER"` | no |
+| node\_metadata | Specifies how node metadata is exposed to the workload running on the node | `string` | `"GKE_METADATA"` | no |
 | node\_pools | List of maps containing node pools | `list(map(string))` | <pre>[<br> {<br> "name": "default-node-pool"<br> }<br>]</pre> | no |
 | node\_pools\_labels | Map of maps containing node labels by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
 | node\_pools\_metadata | Map of maps containing node metadata by node-pool name | `map(map(string))` | <pre>{<br> "all": {},<br> "default-node-pool": {}<br>}</pre> | no |
@@ -202,7 +200,7 @@ Then perform the following commands on the root folder:
 | endpoint | Cluster endpoint |
 | horizontal\_pod\_autoscaling\_enabled | Whether horizontal pod autoscaling enabled |
 | http\_load\_balancing\_enabled | Whether http load balancing enabled |
-| identity\_namespace | Workload Identity namespace |
+| identity\_namespace | Workload Identity pool |
 | instance\_group\_urls | List of GKE generated instance groups |
 | location | Cluster location (region if regional cluster, zone if zonal cluster) |
 | logging\_service | Logging service used |
@@ -213,7 +211,7 @@ Then perform the following commands on the root folder:
 | name | Cluster name |
 | network\_policy\_enabled | Whether network policy enabled |
 | node\_pools\_names | List of node pools names |
-| node\_pools\_versions | List of node pools versions |
+| node\_pools\_versions | Node pool versions by node pool name |
 | region | Cluster region |
 | release\_channel | The release channel of this cluster |
 | service\_account | The service account to default running nodes as if not overridden in `node_pools`. |

autogen/main/cluster.tf.tmpl

@@ -161,9 +161,6 @@ resource "google_container_cluster" "primary" {
  }
 
  master_auth {
- username = var.basic_auth_username
- password = var.basic_auth_password
-
  client_certificate_config {
  issue_client_certificate = var.issue_client_certificate
  }
@@ -298,7 +295,7 @@ resource "google_container_cluster" "primary" {
  for_each = local.cluster_node_metadata_config
 
  content {
- node_metadata = workload_metadata_config.value.node_metadata
+ mode = workload_metadata_config.value.mode
  }
  }
 
@@ -377,7 +374,7 @@ resource "google_container_cluster" "primary" {
  for_each = local.cluster_workload_identity_config
 
  content {
- identity_namespace = workload_identity_config.value.identity_namespace
+ workload_pool = workload_identity_config.value.workload_pool
  }
  }
 
@@ -634,9 +631,10 @@ resource "google_container_node_pool" "pools" {
  for_each = local.cluster_node_metadata_config
 
  content {
- node_metadata = lookup(each.value, "node_metadata", workload_metadata_config.value.node_metadata)
+ mode = lookup(each.value, "node_metadata", workload_metadata_config.value.mode)
  }
  }
+
  {% if beta_cluster %}
  dynamic "sandbox_config" {
  for_each = tobool((lookup(each.value, "sandbox_enabled", var.sandbox_enabled))) ? ["gvisor"] : []

autogen/main/dns.tf.tmpl

@@ -20,8 +20,9 @@
  Delete default kube-dns configmap
  *****************************************/
 module "gcloud_delete_default_kube_dns_configmap" {
- source = "terraform-google-modules/gcloud/google//modules/kubectl-wrapper"
- version = "~> 2.1.0"
+ source = "terraform-google-modules/gcloud/google//modules/kubectl-wrapper"
+ version = "~> 3.1"
+
  enabled = (local.custom_kube_dns_config || local.upstream_nameservers_config) && !var.skip_provisioners
  cluster_name = google_container_cluster.primary.name
  cluster_location = google_container_cluster.primary.location

autogen/main/firewall.tf.tmpl

@@ -112,6 +112,7 @@ resource "google_compute_firewall" "master_webhooks" {
  direction = "INGRESS"
 
  source_ranges = [local.cluster_endpoint_for_nodes]
+ source_tags = []
  target_tags = [local.cluster_network_tag]
 
  allow {

autogen/main/main.tf.tmpl

@@ -111,8 +111,11 @@ locals {
  security_group = var.authenticator_security_group
  }]
 
+ // legacy mappings https://github.com/hashicorp/terraform-provider-google/pull/10238
+ old_node_metadata_config_mapping = { GKE_METADATA_SERVER = "GKE_METADATA", GCE_METADATA = "EXPOSE" }
+
  cluster_node_metadata_config = var.node_metadata == "UNSPECIFIED" ? [] : [{
- node_metadata = var.node_metadata
+ mode = lookup(local.old_node_metadata_config_mapping, var.node_metadata, var.node_metadata)
  }]
 
  cluster_output_name = google_container_cluster.primary.name
@@ -153,7 +156,7 @@ locals {
  }]
 
  cluster_output_node_pools_names = concat([for np in google_container_node_pool.pools : np.name], [""])
- cluster_output_node_pools_versions = concat([for np in google_container_node_pool.pools : np.version], [""])
+ cluster_output_node_pools_versions = { for np in google_container_node_pool.pools : np.name => np.version }
 
  cluster_master_auth_list_layer1 = local.cluster_output_master_auth
  cluster_master_auth_list_layer2 = local.cluster_master_auth_list_layer1[0]
@@ -177,7 +180,7 @@ locals {
  cluster_horizontal_pod_autoscaling_enabled = !local.cluster_output_horizontal_pod_autoscaling_enabled
  workload_identity_enabled = !(var.identity_namespace == null || var.identity_namespace == "null")
  cluster_workload_identity_config = ! local.workload_identity_enabled ? [] : var.identity_namespace == "enabled" ? [{
- identity_namespace = "${var.project_id}.svc.id.goog" }] : [{ identity_namespace = var.identity_namespace
+ workload_pool = "${var.project_id}.svc.id.goog" }] : [{ workload_pool = var.identity_namespace
  }]
 {% if beta_cluster %}
  # BETA features

autogen/main/outputs.tf.tmpl

@@ -114,7 +114,7 @@ output "node_pools_names" {
 }
 
 output "node_pools_versions" {
- description = "List of node pools versions"
+ description = "Node pool versions by node pool name"
  value = local.cluster_node_pools_versions
 }
 
@@ -123,23 +123,23 @@ output "service_account" {
  value = local.service_account
 }
 
+output "instance_group_urls" {
+ description = "List of GKE generated instance groups"
+ value = distinct(flatten([for np in google_container_node_pool.pools : np.managed_instance_group_urls]))
+}
+
 output "release_channel" {
  description = "The release channel of this cluster"
  value = var.release_channel
 }
 
 output "identity_namespace" {
- description = "Workload Identity namespace"
- value = length(local.cluster_workload_identity_config) > 0 ? local.cluster_workload_identity_config[0].identity_namespace : null
+ description = "Workload Identity pool"
+ value = length(local.cluster_workload_identity_config) > 0 ? local.cluster_workload_identity_config[0].workload_pool : null
  depends_on = [
  google_container_cluster.primary
  ]
 }
-
-output "instance_group_urls" {
- description = "List of GKE generated instance groups"
- value = google_container_cluster.primary.instance_group_urls
-}
 {% if private_cluster %}
 
 output "master_ipv4_cidr_block" {

autogen/main/variables.tf.tmpl

@@ -394,18 +394,6 @@ variable "service_account" {
  default = ""
 }
 
-variable "basic_auth_username" {
- type = string
- description = "The username to be used with Basic Authentication. An empty value will disable Basic Authentication, which is the recommended configuration."
- default = ""
-}
-
-variable "basic_auth_password" {
- type = string
- description = "The password to be used with Basic Authentication."
- default = ""
-}
-
 variable "issue_client_certificate" {
  type = bool
  description = "Issues a client certificate to authenticate to the cluster endpoint. To maximize the security of your cluster, leave this option disabled. Client certificates don't automatically rotate and aren't easily revocable. WARNING: changing this after cluster creation is destructive!"
@@ -549,8 +537,13 @@ variable "authenticator_security_group" {
 
 variable "node_metadata" {
  description = "Specifies how node metadata is exposed to the workload running on the node"
- default = "GKE_METADATA_SERVER"
+ default = "GKE_METADATA"
  type = string
+
+ validation {
+ condition = contains(["GKE_METADATA", "GCE_METADATA", "UNSPECIFIED", "GKE_METADATA_SERVER", "EXPOSE"], var.node_metadata)
+ error_message = "The node_metadata value must be one of GKE_METADATA, GCE_METADATA or UNSPECIFIED."
+ }
 }
 
 variable "database_encryption" {
@@ -564,7 +557,7 @@ variable "database_encryption" {
 }
 
 variable "identity_namespace" {
- description = "Workload Identity namespace. (Default value of `enabled` automatically sets project based namespace `[project_id].svc.id.goog`)"
+ description = "The workload pool to attach all Kubernetes service accounts to. (Default value of `enabled` automatically sets project-based pool `[project_id].svc.id.goog`)"
  type = string
  default = "enabled"
 }

autogen/main/versions.tf.tmpl

@@ -24,7 +24,7 @@ terraform {
  required_providers {
  google-beta = {
  source = "hashicorp/google-beta"
- version = ">= 3.87.0, <4.0.0"
+ version = ">= 4.0.0, < 5.0"
  }
  kubernetes = {
  source = "hashicorp/kubernetes"
@@ -38,7 +38,7 @@ terraform {
  required_providers {
  google = {
  source = "hashicorp/google"
- version = ">= 3.55.0, <4.0.0"
+ version = ">= 4.0.0, < 5.0"
  }
  kubernetes = {
  source = "hashicorp/kubernetes"

autogen/safer-cluster/main.tf.tmpl

@@ -111,10 +111,6 @@ module "gke" {
  registry_project_ids = var.registry_project_ids
  grant_registry_access = var.grant_registry_access
 
- // Basic Auth disabled
- basic_auth_username = ""
- basic_auth_password = ""
-
  issue_client_certificate = false
 
  cluster_resource_labels = var.cluster_resource_labels
@@ -164,7 +160,7 @@ module "gke" {
 
  enable_vertical_pod_autoscaling = var.enable_vertical_pod_autoscaling
 
- // We enable identity namespace by default.
+ // We enable Workload Identity by default.
  identity_namespace = "${var.project_id}.svc.id.goog"
 
  authenticator_security_group = var.authenticator_security_group

autogen/safer-cluster/outputs.tf.tmpl

@@ -104,7 +104,7 @@ output "node_pools_names" {
 }
 
 output "node_pools_versions" {
- description = "List of node pools versions"
+ description = "Node pool versions by node pool name"
  value = module.gke.node_pools_versions
 }