chore: Relocate docs on private clusters to central location (terraform-google-modules#552)

* Add doc about priate clusters * Remove file structure from README * Move private cluster notes to doc.

Author: morgante

README.md

@@ -243,20 +243,6 @@ In order to operate with the Service Account you must activate the following API
 - Compute Engine API - compute.googleapis.com
 - Kubernetes Engine API - container.googleapis.com
 
-## File structure
-The project has the following folders and files:
-
-- /: root folder
-- /examples: Examples for using this module and sub module.
-- /helpers: Helper scripts.
-- /scripts: Scripts for specific tasks on module (see Infrastructure section on this file).
-- /test: Folders with files for testing the module (see Testing section on this file).
-- /main.tf: `main` file for the public module, contains all the resources to create.
-- /variables.tf: Variables for the public cluster module.
-- /output.tf: The outputs for the public cluster module.
-- /README.MD: This file.
-- /modules: Private and beta sub modules.
-
 [terraform-provider-google]: https://github.com/terraform-providers/terraform-provider-google
 [3.0.0]: https://registry.terraform.io/modules/terraform-google-modules/kubernetes-engine/google/3.0.0
 [terraform-0.12-upgrade]: https://www.terraform.io/upgrade-guides/0-12.html

autogen/main/README.md

@@ -12,20 +12,8 @@ The resources/services/activations/deletions that this module will create/trigge
 Sub modules are provided from creating private clusters, beta private clusters, and beta public clusters as well. Beta sub modules allow for the use of various GKE beta features. See the modules directory for the various sub modules.
 
 {% if private_cluster %}
-## Private Cluster Endpoints
-When creating a [private cluster](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters), nodes are provisioned with private IPs.
-The Kubernetes master endpoint is also [locked down](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#access_to_the_cluster_endpoints), which affects these module features:
-- `configure_ip_masq`
-- `stub_domains`
-
-If you are *not* using these features, then the module will function normally for private clusters and no special configuration is needed.
-If you are using these features with a private cluster, you will need to either:
-1. Run Terraform from a VM on the same VPC as your cluster (allowing it to connect to the private endpoint) and set `deploy_using_private_endpoint` to `true`.
-2. Enable (beta) [route export functionality](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#master-on-prem-routing) to connect from an on-premise network over a VPN or Interconnect.
-3. Include the external IP of your Terraform deployer in the `master_authorized_networks` configuration.
-4. Deploy a [bastion host](https://github.com/terraform-google-modules/terraform-google-bastion-host) or [proxy](https://cloud.google.com/solutions/creating-kubernetes-engine-private-clusters-with-net-proxies) in the same VPC as your GKE cluster.
-
-If you are going to isolate your GKE private clusters from internet access you could check [guide](https://medium.com/google-cloud/completely-private-gke-clusters-with-no-internet-connectivity-945fffae1ccd) and [repo](https://github.com/andreyk-code/no-inet-gke-cluster)
+## Private Cluster Details
+For details on configuring private clusters with this module, check the [troubleshooting guide](../../docs/private_clusters.md).
 
 {% endif %}
 {% if update_variant %}
@@ -258,20 +246,6 @@ In order to operate with the Service Account you must activate the following API
 - Compute Engine API - compute.googleapis.com
 - Kubernetes Engine API - container.googleapis.com
 
-## File structure
-The project has the following folders and files:
-
-- /: root folder
-- /examples: Examples for using this module and sub module.
-- /helpers: Helper scripts.
-- /scripts: Scripts for specific tasks on module (see Infrastructure section on this file).
-- /test: Folders with files for testing the module (see Testing section on this file).
-- /main.tf: `main` file for the public module, contains all the resources to create.
-- /variables.tf: Variables for the public cluster module.
-- /output.tf: The outputs for the public cluster module.
-- /README.MD: This file.
-- /modules: Private and beta sub modules.
-
 {% if beta_cluster %}
 [terraform-provider-google-beta]: https://github.com/terraform-providers/terraform-provider-google-beta
 {% else %}

docs/private_clusters.md

@@ -0,0 +1,42 @@
+# Creating Private GKE Clusters
+
+To create a private GKE cluster, you can use one of the [private submodules](../modules).
+
+Note that a private cluster is inherently more restricted and greater care must be taken in configuring networking ingress/egress.
+
+## Private Cluster Endpoints
+When creating a [private cluster](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters), nodes are provisioned with private IPs.
+The Kubernetes master endpoint is also [locked down](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#access_to_the_cluster_endpoints), which affects these module features:
+- `configure_ip_masq`
+- `stub_domains`
+
+If you are *not* using these features, then the module will function normally for private clusters and no special configuration is needed.
+
+If you are using these features with a private cluster, you will need to either:
+1. Run Terraform from a VM on the same VPC as your cluster (allowing it to connect to the private endpoint) and set `deploy_using_private_endpoint` to `true`.
+2. Enable (beta) [route export functionality](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#master-on-prem-routing) to connect from an on-premise network over a VPN or Interconnect.
+3. Include the external IP of your Terraform deployer in the `master_authorized_networks` configuration.
+4. Deploy a [bastion host](https://github.com/terraform-google-modules/terraform-google-bastion-host) or [proxy](https://cloud.google.com/solutions/creating-kubernetes-engine-private-clusters-with-net-proxies) in the same VPC as your GKE cluster.
+
+If you are going to isolate your GKE private clusters from internet access you could check [this guide](https://medium.com/google-cloud/completely-private-gke-clusters-with-no-internet-connectivity-945fffae1ccd) and the associated [repo](https://github.com/andreyk-code/no-inet-gke-cluster).
+
+## Troubleshooting
+
+### Master Authorized Network
+When creating a private cluster with a [private endpoint](https://cloud.google.com/kubernetes-engine/docs/how-to/authorized-networks#benefits_with_private_clusters) (`enable_private_endpoint = true`),
+your cluster will **not** have a publicly addressable endpoint.
+
+When using this setting, any CIDR ranges listed in the `master_authorized_networks` configuration *must* come from your private IP space.
+If you include a CIDR block outside your private space, you might see this error:
+
+```
+Error: Error waiting for creating GKE cluster: Invalid master authorized networks: network "73.89.231.174/32" is not a reserved network, which is required for private endpoints.
+
+ on .terraform/modules/gke-cluster-dev.gke/terraform-google-kubernetes-engine-9.2.0/modules/beta-private-cluster/cluster.tf line 22, in resource "google_container_cluster" "primary":
+ 22: resource "google_container_cluster" "primary" {
+```
+
+To resolve this error, update your configuration to either:
+
+* Enable a public endpoint (with `enable_private_endpoint = false`)
+* Update your `master_authorized_networks` configuration to only use CIDR blocks from your private IP space.

modules/beta-private-cluster-update-variant/README.md

@@ -10,20 +10,8 @@ The resources/services/activations/deletions that this module will create/trigge
 
 Sub modules are provided from creating private clusters, beta private clusters, and beta public clusters as well. Beta sub modules allow for the use of various GKE beta features. See the modules directory for the various sub modules.
 
-## Private Cluster Endpoints
-When creating a [private cluster](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters), nodes are provisioned with private IPs.
-The Kubernetes master endpoint is also [locked down](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#access_to_the_cluster_endpoints), which affects these module features:
-- `configure_ip_masq`
-- `stub_domains`
-
-If you are *not* using these features, then the module will function normally for private clusters and no special configuration is needed.
-If you are using these features with a private cluster, you will need to either:
-1. Run Terraform from a VM on the same VPC as your cluster (allowing it to connect to the private endpoint) and set `deploy_using_private_endpoint` to `true`.
-2. Enable (beta) [route export functionality](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#master-on-prem-routing) to connect from an on-premise network over a VPN or Interconnect.
-3. Include the external IP of your Terraform deployer in the `master_authorized_networks` configuration.
-4. Deploy a [bastion host](https://github.com/terraform-google-modules/terraform-google-bastion-host) or [proxy](https://cloud.google.com/solutions/creating-kubernetes-engine-private-clusters-with-net-proxies) in the same VPC as your GKE cluster.
-
-If you are going to isolate your GKE private clusters from internet access you could check [guide](https://medium.com/google-cloud/completely-private-gke-clusters-with-no-internet-connectivity-945fffae1ccd) and [repo](https://github.com/andreyk-code/no-inet-gke-cluster)
+## Private Cluster Details
+For details on configuring private clusters with this module, check the [troubleshooting guide](../../docs/private_clusters.md).
 
 ## Node Pool Update Variant
 
@@ -345,20 +333,6 @@ In order to operate with the Service Account you must activate the following API
 - Compute Engine API - compute.googleapis.com
 - Kubernetes Engine API - container.googleapis.com
 
-## File structure
-The project has the following folders and files:
-
-- /: root folder
-- /examples: Examples for using this module and sub module.
-- /helpers: Helper scripts.
-- /scripts: Scripts for specific tasks on module (see Infrastructure section on this file).
-- /test: Folders with files for testing the module (see Testing section on this file).
-- /main.tf: `main` file for the public module, contains all the resources to create.
-- /variables.tf: Variables for the public cluster module.
-- /output.tf: The outputs for the public cluster module.
-- /README.MD: This file.
-- /modules: Private and beta sub modules.
-
 [terraform-provider-google-beta]: https://github.com/terraform-providers/terraform-provider-google-beta
 [3.0.0]: https://registry.terraform.io/modules/terraform-google-modules/kubernetes-engine/google/3.0.0
 [terraform-0.12-upgrade]: https://www.terraform.io/upgrade-guides/0-12.html

modules/beta-private-cluster/README.md

@@ -10,20 +10,8 @@ The resources/services/activations/deletions that this module will create/trigge
 
 Sub modules are provided from creating private clusters, beta private clusters, and beta public clusters as well. Beta sub modules allow for the use of various GKE beta features. See the modules directory for the various sub modules.
 
-## Private Cluster Endpoints
-When creating a [private cluster](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters), nodes are provisioned with private IPs.
-The Kubernetes master endpoint is also [locked down](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#access_to_the_cluster_endpoints), which affects these module features:
-- `configure_ip_masq`
-- `stub_domains`
-
-If you are *not* using these features, then the module will function normally for private clusters and no special configuration is needed.
-If you are using these features with a private cluster, you will need to either:
-1. Run Terraform from a VM on the same VPC as your cluster (allowing it to connect to the private endpoint) and set `deploy_using_private_endpoint` to `true`.
-2. Enable (beta) [route export functionality](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#master-on-prem-routing) to connect from an on-premise network over a VPN or Interconnect.
-3. Include the external IP of your Terraform deployer in the `master_authorized_networks` configuration.
-4. Deploy a [bastion host](https://github.com/terraform-google-modules/terraform-google-bastion-host) or [proxy](https://cloud.google.com/solutions/creating-kubernetes-engine-private-clusters-with-net-proxies) in the same VPC as your GKE cluster.
-
-If you are going to isolate your GKE private clusters from internet access you could check [guide](https://medium.com/google-cloud/completely-private-gke-clusters-with-no-internet-connectivity-945fffae1ccd) and [repo](https://github.com/andreyk-code/no-inet-gke-cluster)
+## Private Cluster Details
+For details on configuring private clusters with this module, check the [troubleshooting guide](../../docs/private_clusters.md).
 
 ## Compatibility
 
@@ -323,20 +311,6 @@ In order to operate with the Service Account you must activate the following API
 - Compute Engine API - compute.googleapis.com
 - Kubernetes Engine API - container.googleapis.com
 
-## File structure
-The project has the following folders and files:
-
-- /: root folder
-- /examples: Examples for using this module and sub module.
-- /helpers: Helper scripts.
-- /scripts: Scripts for specific tasks on module (see Infrastructure section on this file).
-- /test: Folders with files for testing the module (see Testing section on this file).
-- /main.tf: `main` file for the public module, contains all the resources to create.
-- /variables.tf: Variables for the public cluster module.
-- /output.tf: The outputs for the public cluster module.
-- /README.MD: This file.
-- /modules: Private and beta sub modules.
-
 [terraform-provider-google-beta]: https://github.com/terraform-providers/terraform-provider-google-beta
 [3.0.0]: https://registry.terraform.io/modules/terraform-google-modules/kubernetes-engine/google/3.0.0
 [terraform-0.12-upgrade]: https://www.terraform.io/upgrade-guides/0-12.html

modules/beta-public-cluster-update-variant/README.md

@@ -146,6 +146,7 @@ Then perform the following commands on the root folder:
 | cluster\_autoscaling | Cluster autoscaling configuration. See [more details](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters#clusterautoscaling) | object | `<map>` | no |
 | cluster\_ipv4\_cidr | The IP address range of the kubernetes pods in this cluster. Default is an automatically assigned CIDR. | string | `"null"` | no |
 | cluster\_resource\_labels | The GCE resource labels (a map of key/value pairs) to be applied to the cluster | map(string) | `<map>` | no |
+| config\_connector | (Beta) Whether ConfigConnector is enabled for this cluster. | bool | `"false"` | no |
 | configure\_ip\_masq | Enables the installation of ip masquerading, which is usually no longer required when using aliasied IP addresses. IP masquerading uses a kubectl call, so when you have a private cluster, you will need access to the API server. | string | `"false"` | no |
 | create\_service\_account | Defines if service account specified to run nodes should be created. | bool | `"true"` | no |
 | database\_encryption | Application-layer Secrets Encryption settings. The object format is {state = string, key_name = string}. Valid values of state are: "ENCRYPTED"; "DECRYPTED". key_name is the name of a CloudKMS key. | object | `<list>` | no |
@@ -320,20 +321,6 @@ In order to operate with the Service Account you must activate the following API
 - Compute Engine API - compute.googleapis.com
 - Kubernetes Engine API - container.googleapis.com
 
-## File structure
-The project has the following folders and files:
-
-- /: root folder
-- /examples: Examples for using this module and sub module.
-- /helpers: Helper scripts.
-- /scripts: Scripts for specific tasks on module (see Infrastructure section on this file).
-- /test: Folders with files for testing the module (see Testing section on this file).
-- /main.tf: `main` file for the public module, contains all the resources to create.
-- /variables.tf: Variables for the public cluster module.
-- /output.tf: The outputs for the public cluster module.
-- /README.MD: This file.
-- /modules: Private and beta sub modules.
-
 [terraform-provider-google-beta]: https://github.com/terraform-providers/terraform-provider-google-beta
 [3.0.0]: https://registry.terraform.io/modules/terraform-google-modules/kubernetes-engine/google/3.0.0
 [terraform-0.12-upgrade]: https://www.terraform.io/upgrade-guides/0-12.html

modules/beta-public-cluster-update-variant/cluster.tf

@@ -149,6 +149,10 @@ resource "google_container_cluster" "primary" {
  kalm_config {
  enabled = var.kalm_config
  }
+
+ config_connector_config {
+ enabled = var.config_connector
+ }
  }
 
  ip_allocation_policy {

modules/beta-public-cluster-update-variant/variables.tf

@@ -398,6 +398,12 @@ variable "kalm_config" {
  default = false
 }
 
+variable "config_connector" {
+ type = bool
+ description = "(Beta) Whether ConfigConnector is enabled for this cluster."
+ default = false
+}
+
 variable "database_encryption" {
  description = "Application-layer Secrets Encryption settings. The object format is {state = string, key_name = string}. Valid values of state are: \"ENCRYPTED\"; \"DECRYPTED\". key_name is the name of a CloudKMS key."
  type = list(object({ state = string, key_name = string }))

modules/beta-public-cluster-update-variant/versions.tf

@@ -18,6 +18,6 @@ terraform {
  required_version = "~> 0.12.6"
 
  required_providers {
- google-beta = ">= 3.21.0, <4.0.0"
+ google-beta = ">= 3.23.0, <4.0.0"
  }
 }

modules/beta-public-cluster/README.md

@@ -299,20 +299,6 @@ In order to operate with the Service Account you must activate the following API
 - Compute Engine API - compute.googleapis.com
 - Kubernetes Engine API - container.googleapis.com
 
-## File structure
-The project has the following folders and files:
-
-- /: root folder
-- /examples: Examples for using this module and sub module.
-- /helpers: Helper scripts.
-- /scripts: Scripts for specific tasks on module (see Infrastructure section on this file).
-- /test: Folders with files for testing the module (see Testing section on this file).
-- /main.tf: `main` file for the public module, contains all the resources to create.
-- /variables.tf: Variables for the public cluster module.
-- /output.tf: The outputs for the public cluster module.
-- /README.MD: This file.
-- /modules: Private and beta sub modules.
-
 [terraform-provider-google-beta]: https://github.com/terraform-providers/terraform-provider-google-beta
 [3.0.0]: https://registry.terraform.io/modules/terraform-google-modules/kubernetes-engine/google/3.0.0
 [terraform-0.12-upgrade]: https://www.terraform.io/upgrade-guides/0-12.html