Introduction :
The Kubernetes Cluster Autoscaler automatically adjusts the number of nodes in your cluster when pods fail or are rescheduled onto other nodes. The Cluster Autoscaler uses Auto Scaling groups. For more information, see Cluster Autoscaler on AWS.
Step 1: Create a EKS Cluster
- performed Step 1 : to step 5 : Click here
Step 2: Verify how many nodes and pods are running
Node :
[root@ip-172-31-18-194 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION ip-192-168-5-245.ec2.internal Ready <none> 4m19s v1.24.17-eks-e71965b ip-192-168-63-39.ec2.internal Ready <none> 2s v1.24.17-eks-e71965b Pods:
[root@ip-172-31-18-194 ~]# kubectl get po -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system aws-node-4fdfg 1/1 Running 0 2m50s kube-system aws-node-mm84r 1/1 Running 0 2m53s kube-system coredns-79989457d9-798tx 1/1 Running 0 10m kube-system coredns-79989457d9-7fhzl 1/1 Running 0 10m kube-system kube-proxy-rkbzz 1/1 Running 0 2m50s kube-system kube-proxy-vfq7k 1/1 Running 0 2m53s Step 3: Create a IAM OIDC Provider
IAM OIDC is used for authorizing the Cluster Autoscaler to launch or terminate instances under an Auto Scaling group.
Open EKS Dashboard and copy a OpenID Connect Provider link
- Open a IAM Providers
- Click “Add provider,” select “OpenID Connect,” and click “Get thumbprint” as shown below:
- Then enter the “Audience” (sts.amazonaws.com in our example pointing to the AWS STS, also known as the Security Token Service) and click the add provider
- Adding identity information on identity providers
Step 4: Create IAM Policy Create a Policy with necessary permission. 
- To create the policy with the necessary permissions, save the below file as AmazonEKSClusterAutoscalerPolicy
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "autoscaling:DescribeAutoScalingGroups", "autoscaling:DescribeAutoScalingInstances", "autoscaling:DescribeLaunchConfigurations", "autoscaling:DescribeScalingActivities", "autoscaling:DescribeTags", "ec2:DescribeInstanceTypes", "ec2:DescribeLaunchTemplateVersions" ], "Resource": [ "*" ] }, { "Effect": "Allow", "Action": [ "autoscaling:SetDesiredCapacity", "autoscaling:TerminateInstanceInAutoScalingGroup", "ec2:DescribeImages", "ec2:GetInstanceTypesFromInstanceRequirements", "eks:DescribeNodegroup" ], "Resource": [ "*" ] } ] } - Review and create a policy
Step 5 : Create a IAM Role for the provider. Create role 
Select the web identity Select identity provide and audience click next.
Add Policy AmazonEKSClusterAutoscalerPolicy
Click Next and provide Role Name : EKS_Autoscaler
- verify the IAM role and make sure the policy is attached.
Edit the “Trust relationships.” Before Edit “Trust relationships.”
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::256050093938:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/CD6440D4E14822FC649C070BD8C41A96" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "oidc.eks.us-east-1.amazonaws.com/id/CD6440D4E14822FC649C070BD8C41A96:aud": "sts.amazonaws.com" } } } ] } - After Edit “Trust relationships.”
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::256050093938:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/CD6440D4E14822FC649C070BD8C41A96" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "oidc.eks.us-east-1.amazonaws.com/id/CD6440D4E14822FC649C070BD8C41A96:aud": "sts.amazonaws.com", "oidc.eks.us-east-1.amazonaws.com/id/CD6440D4E14822FC649C070BD8C41A96:sub": "system:serviceaccount:kube-system:cluster-autoscaler" } } } ] } Step 6 : Deploy a Cluster Autoscaler
Next, we deploy Cluster Autoscaler. To do so, you must use the Amazon Resource Names (ARN) number of the IAM role created in our earlier step.
The content intended to save into a file (make sure you copy all of the content presented over the next page):
Modify below two lines
- Line 8 : change IAM Role name
- Line 159 : --node-group-auto-discovery = This is used by CA to discover the Auto Scaling group based on its tag.
apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-addon: cluster-autoscaler.addons.k8s.io k8s-app: cluster-autoscaler annotations: eks.amazonaws.com/role-arn: arn:aws:iam::256050093938:role/EKS_Autoscaler name: cluster-autoscaler namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: cluster-autoscaler labels: k8s-addon: cluster-autoscaler.addons.k8s.io k8s-app: cluster-autoscaler rules: - apiGroups: [""] resources: ["events", "endpoints"] verbs: ["create", "patch"] - apiGroups: [""] resources: ["pods/eviction"] verbs: ["create"] - apiGroups: [""] resources: ["pods/status"] verbs: ["update"] - apiGroups: [""] resources: ["endpoints"] resourceNames: ["cluster-autoscaler"] verbs: ["get", "update"] - apiGroups: [""] resources: ["nodes"] verbs: ["watch", "list", "get", "update"] - apiGroups: [""] resources: - "pods" - "services" - "replicationcontrollers" - "persistentvolumeclaims" - "persistentvolumes" verbs: ["watch", "list", "get"] - apiGroups: ["extensions"] resources: ["replicasets", "daemonsets"] verbs: ["watch", "list", "get"] - apiGroups: ["policy"] resources: ["poddisruptionbudgets"] verbs: ["watch", "list"] - apiGroups: ["apps"] resources: ["statefulsets", "replicasets", "daemonsets"] verbs: ["watch", "list", "get"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses", "csinodes"] verbs: ["watch", "list", "get"] - apiGroups: ["batch", "extensions"] resources: ["jobs"] verbs: ["get", "list", "watch", "patch"] - apiGroups: ["coordination.k8s.io"] resources: ["leases"] verbs: ["create"] - apiGroups: ["coordination.k8s.io"] resourceNames: ["cluster-autoscaler"] resources: ["leases"] verbs: ["get", "update"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: cluster-autoscaler namespace: kube-system labels: k8s-addon: cluster-autoscaler.addons.k8s.io k8s-app: cluster-autoscaler rules: - apiGroups: [""] resources: ["configmaps"] verbs: ["create","list","watch"] - apiGroups: [""] resources: ["configmaps"] resourceNames: ["cluster-autoscaler-status", "cluster-autoscaler-priority-expander"] verbs: ["delete", "get", "update", "watch"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: cluster-autoscaler labels: k8s-addon: cluster-autoscaler.addons.k8s.io k8s-app: cluster-autoscaler roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-autoscaler subjects: - kind: ServiceAccount name: cluster-autoscaler namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: cluster-autoscaler namespace: kube-system labels: k8s-addon: cluster-autoscaler.addons.k8s.io k8s-app: cluster-autoscaler roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: cluster-autoscaler subjects: - kind: ServiceAccount name: cluster-autoscaler namespace: kube-system --- apiVersion: apps/v1 kind: Deployment metadata: name: cluster-autoscaler namespace: kube-system labels: app: cluster-autoscaler spec: replicas: 1 selector: matchLabels: app: cluster-autoscaler template: metadata: labels: app: cluster-autoscaler annotations: cluster-autoscaler.kubernetes.io/safe-to-evict: 'false' spec: serviceAccountName: cluster-autoscaler containers: - image: k8s.gcr.io/autoscaling/cluster-autoscaler:v1.20.0 name: cluster-autoscaler resources: limits: cpu: 100m memory: 500Mi requests: cpu: 100m memory: 500Mi command: - ./cluster-autoscaler - --v=4 - --stderrthreshold=info - --cloud-provider=aws - --skip-nodes-with-local-storage=false - --expander=least-waste - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/ashish - --balance-similar-node-groups - --skip-nodes-with-system-pods=false volumeMounts: - name: ssl-certs mountPath: /etc/ssl/certs/ca-certificates.crt #/etc/ssl/certs/ca-bundle.crt for Amazon Linux Worker Nodes readOnly: true imagePullPolicy: "Always" volumes: - name: ssl-certs hostPath: path: "/etc/ssl/certs/ca-bundle.crt" To deploy CA, save the following content presented after the command below in a file and run this provided command:
kubectl apply -f cluster-autoscaler.yaml serviceaccount/cluster-autoscaler created clusterrole.rbac.authorization.k8s.io/cluster-autoscaler created role.rbac.authorization.k8s.io/cluster-autoscaler created clusterrolebinding.rbac.authorization.k8s.io/cluster-autoscaler created rolebinding.rbac.authorization.k8s.io/cluster-autoscaler created deployment.apps/cluster-autoscaler created The expected results are displayed below.
[root@ip-172-31-18-194 ~]# kubectl get po -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system aws-node-2frzk 1/1 Running 0 67m kube-system aws-node-drmtr 1/1 Running 0 63m kube-system cluster-autoscaler-657d67cd5d-l7q4m 1/1 Running 0 8s kube-system coredns-79989457d9-89f48 1/1 Running 0 75m kube-system coredns-79989457d9-ddvvb 1/1 Running 0 75m kube-system kube-proxy-hpzxj 1/1 Running 0 63m kube-system kube-proxy-vb2gj 1/1 Running 0 67m The expected results are displayed below.
[root@ip-172-31-18-194 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION ip-192-168-5-245.ec2.internal Ready <none> 76m v1.24.17-eks-e71965b ip-192-168-63-39.ec2.internal Ready <none> 72m v1.24.17-eks-e71965b Troubleshoot :
verify the logs by issuing this command:
kubectl logs -l app=cluster-autoscaler -n kubesystem -f Conclusion :
Cluster Autoscaler plays a vital role in a Kubernetes cluster by ensuring adequate computing resources are available by adding the nodes to a cluster and keeping infrastructure costs down by removing nodes
Top comments (0)