Step 1: Create a cluster

GKE

Create a cluster on GKE

These steps explain how to configure and create a GKE cluster in your Google Cloud project.

Apigee recommends creating a regional cluster rather than a zonal cluster. If you are unfamiliar with the distinction between regions and zones, see Regions and zones. The available regions are listed in Available regions and zones. Just be aware that, for example, us-west1 is a valid region name, while us-west1-a is a zone in the region.

1. Make sure you are using a version of GKE that is supported for hybrid version 1.12.4. See Apigee hybrid supported platforms and versions.
2. Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, make sure they are synchronized with NTP across all regions.
3. (GKE private clusters only), If you are creating a private cluster on GKE, add a firewall rule to allow port 9443 for communication between GKE master nodes and GKE worker nodes and to allow the GKE masters to access Apigee mutating webhooks. Follow the procedure in Adding firewall rules for specific use cases in the Google Kubernetes Engine documentation. For more information see Private clusters in GKE.

   You do not need to add this rule if you are creating a standard or public cluster.

4. Create a standard cluster by following the instructions at Create a regional cluster with a multi-zone node pool. It's okay to create the cluster with just the default node pool. You will configure and create the required Apigee hybrid node pools in the next step.

   Go to the next step only after the cluster creation completes successfully.

5. Create two node pools by following the instructions in Add and manage node pools. Be sure to configure the node pools with the minimum requirements listed in the table below.

   What you need to know about node pools

   We recommend you review the following information about node pools and node pool configuration before continuing:

   • About node pools
   • Cassandra network requirements

   Minimum node pool requirements

   Be sure to satisfy these minimum requirements when creating the node pools. Be sure to select the value for Prod or Non-prod depending on the the Apigee hybrid installation you are building. Non-prod installations are suitable for testing and demonstration but not sufficeint for production traffic. If you are using the Cloud console, be sure to configure both the Node pool details and Nodes sections.

   Node pool name	Description	Minimum nodes	Prod Minimum machine type	Non-prod Minimum machine type
   apigee-data	A stateful node pool used for the Cassandra database.	1 per zone (3 per region)	e2-standard-8 (8 vCPU, 32 GB memory)	e2-standard-4 (4 vCPU, 16 GB memory)
   apigee-runtime	A stateless node pool used by the runtime message processor.	1 per zone (3 per region)	e2-standard-8 (8 vCPU, 32 GB memory)	e2-standard-4 (4 vCPU, 16 GB memory)

   For more details about node pool configuration see Configure dedicated node pools.

6. (Optional) If you wish, you can delete the default node pool. See Delete a node pool.
7. If you haven't already, create the following environment variables. These variables are used in the following gcloud commands.

   Linux / MacOS

    export CLUSTER_NAME="YOUR_CLUSTER_NAME" export CLUSTER_LOCATION="YOUR_CLUSTER_LOCATION" export PROJECT_ID="YOUR_PROJECT_ID"

   Windows

    set CLUSTER_NAME="YOUR_CLUSTER_NAME" set CLUSTER_LOCATION=YOUR_CLUSTER_LOCATION set PROJECT_ID=YOUR_PROJECT_ID

   Where:

   • CLUSTER_NAME: The name of your cluster.
   • CLUSTER_LOCATION: The region in which you created your cluster.
   • PROJECT_ID: Your Google Cloud project ID.
8. Verify the node pool configurations:

   Regional clusters

   gcloud container node-pools list \ --cluster=${CLUSTER_NAME} \ --region=${CLUSTER_LOCATION} \ --project=${PROJECT_ID}

   Zonal clusters

   gcloud container node-pools list \ --cluster=${CLUSTER_NAME} \ --zone=${CLUSTER_LOCATION} \ --project=${PROJECT_ID}

9. Make sure your cluster is set as the default cluster for kubectl by getting the
   gcloud credentials of the cluster you just created:

   Regional clusters

    gcloud container clusters get-credentials ${CLUSTER_NAME} \ --region ${CLUSTER_LOCATION} \ --project ${PROJECT_ID}

   Zonal clusters

    gcloud container clusters get-credentials ${CLUSTER_NAME} \ --zone ${CLUSTER_LOCATION} \ --project ${PROJECT_ID}

   See Set a default cluster for kubectl commands.

10. Configure persistent solid state disk (SSD) storage for Cassandra. We do not support using local SSDs. For more information, see Change the default storage class in the Kubernetes documentation.

    1. Get the name of the current default StorageClass:

       kubectl get sc

       For example:

       kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE premium-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h standard kubernetes.io/gce-pd Delete Immediate true 15h standard-rwo (default) pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h

    2. Describe the StorageClass named standard-rwo. Note that its type is pd-balanced:

       kubectl describe sc standard-rwo

       For example:

       kubectl describe sc standard-rwo

       Name: standard-rwo IsDefaultClass: Yes Annotations: components.gke.io/layer=addon,storageclass.kubernetes.io/is-default-class=false Provisioner: pd.csi.storage.gke.io Parameters: type=pd-balanced AllowVolumeExpansion: True MountOptions: <none> ReclaimPolicy: Delete VolumeBindingMode: WaitForFirstConsumer Events: <none>

    3. Create a new file called storageclass.yaml.
    4. Add this code to the file. Note that the name of the new storage class is apigee-sc. You can use any name you like. Also, note that the storage type is pd-ssd:

       --- kind: StorageClass apiVersion: storage.k8s.io/v1 metadata:  name: "apigee-sc" provisioner: kubernetes.io/gce-pd parameters:  type: pd-ssd  replication-type: none volumeBindingMode: WaitForFirstConsumer allowVolumeExpansion: true

    5. Apply the new StorageClass to your Kubernetes cluster:

       kubectl apply -f storageclass.yaml

    6. Execute the following two commands to change the default StorageClass:

       kubectl patch storageclass standard-rwo \ -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'

       kubectl patch storageclass apigee-sc \ -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

    7. Execute this command to verify that the new default StorageClass is called apigee-sc:

       kubectl get sc

       For example:

       kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE apigee-sc (default) kubernetes.io/gce-pd Delete WaitForFirstConsumer true 14h premium-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h standard kubernetes.io/gce-pd Delete Immediate true 15h standard-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h

11. Enable workload identity for the cluster. Workload Identity allows workloads in your GKE clusters to impersonate Identity and Access Management (IAM) service accounts to access Google Cloud services. This operation can take up to 30 minutes:

    Regional clusters

     gcloud container clusters update ${CLUSTER_NAME} \ --workload-pool=${PROJECT_ID}.svc.id.goog \ --project ${PROJECT_ID} \ --region ${CLUSTER_LOCATION}

    Zonal clusters

     gcloud container clusters update ${CLUSTER_NAME} \ --workload-pool=${PROJECT_ID}.svc.id.goog \ --zone ${CLUSTER_LOCATION} \ --project ${PROJECT_ID}

12. Verify whether Workload identity is successfully enabled with the following command;

    Regional clusters

     gcloud container clusters describe ${CLUSTER_NAME} \ --project ${PROJECT_ID} \ --region ${CLUSTER_LOCATION} | grep -i "workload"

    Zonal clusters

     gcloud container clusters describe ${CLUSTER_NAME} \ --zone ${CLUSTER_LOCATION} \ --project ${PROJECT_ID} | grep -i "workload"

When you have a cluster installed and running, go to the next step.

Google Distributed Cloud on VMware

Create a cluster on VMware

These steps explain how to configure and create a GKE cluster for Apigee hybrid as part of a software-only Google Distributed Cloud on VMware deployment.

1. Make sure you are using a version of Google Distributed Cloud that is supported for hybrid version 1.12.4. See Apigee hybrid supported platforms and versions.
2. Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
3. Create the cluster by following the instructions at Create basic clusters. It's okay to create the cluster with just the default node pool. You will configure and create the required Apigee hybrid node pools in the next step.

   Go to the next step only after the cluster creation completes successfully.

4. Create two node pools by following the instructions in Creating and managing node pools. Configure the node pools with the minimum requirements listed in the table below.

   What you need to know about node pools

   We recommend you review the following information about node pools and node pool configuration before continuing:

   • About node pools
   • Cassandra network requirements

   Minimum node pool requirements

   Be sure to satisfy these minimum requirements when creating the node pools. Select the values for Prod or Non-prod depending on the the Apigee hybrid installation you are building. Non-prod installations are suitable for testing and demonstration but not sufficeint for production traffic.

   Node pool name	Description	Minimum nodes	Prod Minimum machine type	Non-prod Minimum machine type
   apigee-data	A stateful node pool used for the Cassandra database.	1 per zone (3 per region)	e2-standard-8 (8 vCPU, 16 GB memory)	e2-standard-4 (4 vCPU, 16 GB memory)
   apigee-runtime	A stateless node pool used by the runtime message processor.	1 per zone (3 per region)	e2-standard-8 (8 vCPU, 16 GB memory)	e2-standard-4 (4 vCPU, 16 GB memory)

   For more details about node pool configuration see Configure dedicated node pools.

5. (Optional) If you wish, you can delete the default node pool. See Delete a node pool.

6. Configure persistent solid state disk (SSD) storage for Cassandra. We do not support using local SSDs. For more information, see Change the default storage class in the Kubernetes documentation.

   1. Get the name of the current default StorageClass:

      kubectl get sc

      For example:

      kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE premium-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h standard kubernetes.io/gce-pd Delete Immediate true 15h standard-rwo (default) pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h

   2. Describe the StorageClass named standard-rwo. Note that its type is pd-balanced:

      kubectl describe sc standard-rwo

      For example:

      kubectl describe sc standard-rwo

      Name: standard-rwo IsDefaultClass: Yes Annotations: components.gke.io/layer=addon,storageclass.kubernetes.io/is-default-class=false Provisioner: pd.csi.storage.gke.io Parameters: type=pd-balanced AllowVolumeExpansion: True MountOptions: <none> ReclaimPolicy: Delete VolumeBindingMode: WaitForFirstConsumer Events: <none>

   3. Create a new file called storageclass.yaml.
   4. Add this code to the file. Note that the name of the new storage class is apigee-sc. You can use any name you like. Also, note that the storage type is pd-ssd:

      --- kind: StorageClass apiVersion: storage.k8s.io/v1 metadata:  name: "apigee-sc" provisioner: kubernetes.io/gce-pd parameters:  type: pd-ssd  replication-type: none volumeBindingMode: WaitForFirstConsumer allowVolumeExpansion: true

   5. Apply the new StorageClass to your Kubernetes cluster:

      kubectl apply -f storageclass.yaml

   6. Execute the following two commands to change the default StorageClass:

      kubectl patch storageclass standard-rwo \ -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'

      kubectl patch storageclass apigee-sc \ -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

   7. Execute this command to verify that the new default StorageClass is called apigee-sc:

      kubectl get sc

      For example:

      kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE apigee-sc (default) kubernetes.io/gce-pd Delete WaitForFirstConsumer true 14h premium-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h standard kubernetes.io/gce-pd Delete Immediate true 15h standard-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 15h

When you have a cluster installed and running, go to the next step.

Google Distributed Cloud on bare metal

Create a cluster on bare metal

These steps explain how to configure and create a GKE cluster for Apigee hybrid as part of a software-only deployment of Google Distributed Cloud on bare metal. Google Distributed Cloud on bare metal lets you run Kubernetes clusters directly on your own machine resources.

1. Make sure you are using a version of Google Distributed Cloud that is supported for hybrid version 1.12.4. See Apigee hybrid supported platforms and versions.
2. Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
3. Review the Installation prerequisites overview and Creating clusters: overview.
4. Create the cluster with two node pools configured as described below:
   • Follow the cluster creation instructions in Create basic clusters.
   • Create a minimum of two node pools. Configure the node pools with the minimum requirements listed in the table below.

     What you need to know about node pools

     We recommend you review the following information about node pools and node pool configuration before continuing:

     • Configure dedicated node pools
     • About node pools
     • Cassandra network requirements

     The minimum configurations for your cluster are:

     Configuration	Stateful node pool	Stateless node pool
     Purpose	A stateful node pool used for the Cassandra database.	A stateless node pool used by the runtime message processor.
     Label name	apigee-data	apigee-runtime
     Number of nodes	1 per zone (3 per region)	1 per zone (3 per region)
     CPU	8 (Prod) 4 (Non-prod)	8 (Prod) 4 (Non-prod)
     RAM	32 (Prod) 16 (Non-prod)	32 (Prod) 16 (Non-prod)
     Storage	dynamic	Managed with the ApigeeDeployment CRD
     Minimum disk IOPS	2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS.	2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS.
     Network bandwidth for each machine instance type	1 Gbps	1 Gbps

     For more details on minimum cluster configuration see: Minimum cluster configurations

When you have a cluster installed and running, go to the next step.

AKS

Create a cluster on AKS

These steps explain how to configure and create a cluster for Apigee hybrid on AKS.

1. Make sure you are using a version of AKS that is supported for hybrid version 1.12.4. See Apigee hybrid supported platforms and versions.
2. Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
3. Create the cluster using either the Azure CLI or Azure Portal, and create two node pools as described below.

   What you need to know about node pools

   We recommend you review the following information about node pools and node pool configuration before continuing:

   • Configure dedicated node pools
   • About node pools
   • Cassandra network requirements

   The minimum configurations for your cluster are:

   Configuration	Stateful node pool	Stateless node pool
   Purpose	A stateful node pool used for the Cassandra database.	A stateless node pool used by the runtime message processor.
   Label name	apigee-data	apigee-runtime
   Number of nodes	1 per zone (3 per region)	1 per zone (3 per region)
   CPU	8 (Prod) 4 (Non-prod)	8 (Prod) 4 (Non-prod)
   RAM	32 (Prod) 16 (Non-prod)	32 (Prod) 16 (Non-prod)
   Storage	dynamic	Managed with the ApigeeDeployment CRD
   Minimum disk IOPS	2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS.	2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS.
   Network bandwidth for each machine instance type	1 Gbps	1 Gbps

   For more details on minimum cluster configuration see: Minimum cluster configurations

When you have a cluster installed and running, go to the next step.

EKS

Create a cluster on EKS

These steps explain how to configure and create a cluster for Apigee hybrid on EKS.

1. Make sure you are using a version of EKS that is supported for hybrid version 1.12.4. See Apigee hybrid supported platforms and versions.
2. Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
3. If you are using Kubernetes version 1.24 or newer, make sure you have installed the Kubernetes CSI driver for Amazon EBS.
4. Use the following instructions to create a user cluster, and create two node pools as described below.
   • EKS overview documentation.
   • Creating an Amazon EKS cluster.

   What you need to know about node pools

   We recommend you review the following information about node pools and node pool configuration before continuing:

   • Configure dedicated node pools
   • About node pools
   • Cassandra network requirements

   The minimum configurations for your cluster are:

   Configuration	Stateful node pool	Stateless node pool
   Purpose	A stateful node pool used for the Cassandra database.	A stateless node pool used by the runtime message processor.
   Label name	apigee-data	apigee-runtime
   Number of nodes	1 per zone (3 per region)	1 per zone (3 per region)
   CPU	8 (Prod) 4 (Non-prod)	8 (Prod) 4 (Non-prod)
   RAM	32 (Prod) 16 (Non-prod)	32 (Prod) 16 (Non-prod)
   Storage	dynamic	Managed with the ApigeeDeployment CRD
   Minimum disk IOPS	2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS.	2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS.
   Network bandwidth for each machine instance type	1 Gbps	1 Gbps

   For more details on minimum cluster configuration see: Minimum cluster configurations

When you have a cluster installed and running, go to the next step.

GKE on AWS

Create a cluster on GKE on AWS

These steps explain how to configure and create a cluster for Apigee hybrid on GKE on AWS.

1. Make sure you are using a version of GKE on AWS that is supported for hybrid version 1.12.4. See Apigee hybrid supported platforms and versions.
2. Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
3. Use the following instructions to create a user cluster, and create two node pools as described below.
   • GKE on AWS overview documentation.
   • Deploy an application on GKE on AWS.

   What you need to know about node pools

   We recommend you review the following information about node pools and node pool configuration before continuing:

   • Configure dedicated node pools
   • About node pools
   • Cassandra network requirements

   The minimum configurations for your cluster are:

   Configuration	Stateful node pool	Stateless node pool
   Purpose	A stateful node pool used for the Cassandra database.	A stateless node pool used by the runtime message processor.
   Label name	apigee-data	apigee-runtime
   Number of nodes	1 per zone (3 per region)	1 per zone (3 per region)
   CPU	8 (Prod) 4 (Non-prod)	8 (Prod) 4 (Non-prod)
   RAM	32 (Prod) 16 (Non-prod)	32 (Prod) 16 (Non-prod)
   Storage	dynamic	Managed with the ApigeeDeployment CRD
   Minimum disk IOPS	2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS.	2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS.
   Network bandwidth for each machine instance type	1 Gbps	1 Gbps

   For more details on minimum cluster configuration see: Minimum cluster configurations

When you have a cluster installed and running, go to the next step.

OpenShift

Create a cluster on OpenShift

These steps explain how to configure and create a cluster for Apigee hybrid on OpenShift.

1. Make sure you are using a version of OpenShift that is supported for hybrid version 1.12.4. See Apigee hybrid supported platforms and versions.
2. Ensure the clocks on all nodes and application servers are synchronized with Network Time Protocol (NTP), as explained in the Prerequisites. The Cassandra database relies on Network Time Protocol (NTP) synchronization to maintain data consistency. If you plan to install hybrid into multiple regions, be sure they are synchronized with NTP across all regions.
3. Build the OpenShift cluster to deploy on the runtime plane, install Apigee on your OpenShift user cluster, and create two node pools.

   As part of the OpenShift install, install and configure the oc CLI tool. See Getting started with the OpenShift CLI in the OpenShift documentation.

   What you need to know about node pools

   We recommend you review the following information about node pools and node pool configuration before continuing:

   • Configure dedicated node pools
   • About node pools
   • Cassandra network requirements

   The minimum configurations for your cluster are:

   Configuration	Stateful node pool	Stateless node pool
   Purpose	A stateful node pool used for the Cassandra database.	A stateless node pool used by the runtime message processor.
   Label name	apigee-data	apigee-runtime
   Number of nodes	1 per zone (3 per region)	1 per zone (3 per region)
   CPU	8 (Prod) 4 (Non-prod)	8 (Prod) 4 (Non-prod)
   RAM	32 (Prod) 16 (Non-prod)	32 (Prod) 16 (Non-prod)
   Storage	dynamic	Managed with the ApigeeDeployment CRD
   Minimum disk IOPS	2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS.	2000 IOPS with SAN or directly attached storage. NFS is not recommended even if it can support the required IOPS.
   Network bandwidth for each machine instance type	1 Gbps	1 Gbps

   For more details on minimum cluster configuration see: Minimum cluster configurations

When you have installed a cluster, go to the next step.