Posted on Nov 5, 2024

How to automatically scale down a GKE node pool in the staging cluster to save money, with notifications via Slack.

Overview

This guide demonstrates how to implement an automated/manual solution for scaling down Kubernetes nodepools during off-hours while keeping your team informed through Slack notifications. This approach can significantly reduce cloud costs by ensuring compute resource aren't idle during non-business hours.

Prerequisites

Kubernetes cluster (in post, I will use GKE)
Slack workspace with permissions to create Webhooks
KubeCTL access to your cluster
... ## Solution Components
1. Golang CronJob to push notify yes/no scale up/down
2. Slack notify and recieve message confirm from user
3. Node Scale exc

Implement:

1. Create Slack bot and workspace

https://slack.com/help/articles/206845317-Create-a-Slack-workspace
https://programmingpercy.tech/blog/develop-a-slack-bot-using-golang/

2. Create GKE cluster and create node pool on it

https://cloud.google.com/kubernetes-engine/docs/deploy-app-cluster

3. Prepare k8s conf and bot

Prepare conf

connect to cluster and get kube/congfig Check config

cat ~/.kube/config

Code push message to Slack

type TypeQuestionType string const ( QuestionTypeYesNoScaleDown TypeQuestionType = "yes_no_scale_down" QuestionTypeYesNoScaleUp TypeQuestionType = "yes_no_scale_up" ) func sendYesNoQuestion(client *slack.Client, channelID string, question string, typeQuestion TypeQuestionType) (string, error) { attachment := slack.Attachment{ Text: question, CallbackID: string(typeQuestion), Actions: []slack.AttachmentAction{ { Name: "yes", Text: "Yes", Type: "button", Value: "yes", }, { Name: "no", Text: "No", Type: "button", Value: "no", }, }, } _, timestamp, err := client.PostMessage( channelID, slack.MsgOptionAttachments(attachment), ) if err != nil { return "", err } lastQuestionTimestamp = timestamp return "Response pending", nil }