Cover image by Andrew Neel on Unsplash
In this blog post I present step-by-step how to deploy multiple applications on Kubernetes cluster using Helm.
This is a second part of my series on Kubernetes. It compares three approaches of deploying applications:
with kubectl — Deployment of multiple apps on Kubernetes cluster — Walkthrough
with Helm — this one,
with helmfile — How to declaratively run Helm charts using helmfile.
If you haven’t read it first one, I would advise to do that and then go back to this post. But if you prefer to jump right away to this post, don’t worry, I’ll briefly introduce you to the project.
Above picture presents the target solution. Inside the cluster there will be deployed frontend (kanban-ui) and backend (kanban-app) services together with postgres database. A source code for both microservices can be found in my GitHub repository — kanban-board.
Additionally I’ve added an adminer, which is a GUI client for getting inside the database.
If we focus on one of services, e.g. on kanban-ui we can see that it needs to create two Kubernetes objects — ClusterIP & Deployment. With plain approach, using kubectl we would needed to create two files for each Kubernetes object:
apiVersion: v1 kind: Service metadata: name: kanban-ui labels: group: backend spec: type: ClusterIP selector: app: kanban-ui ports: - port: 80 targetPort: 80
apiVersion: apps/v1 kind: Deployment metadata: name: kanban-ui labels: app: kanban-ui group: frontend spec: replicas: 1 selector: matchLabels: app: kanban-ui template: metadata: labels: app: kanban-ui group: frontend spec: containers: - name: kanban-ui image: wkrzywiec/kanban-ui:k8s ports: - containerPort: 80
The same story is for any other application — they need at least two definition files. Some of them requires even more, like postgres, which needs to have PersistentVolumeClaims defined in addition. As a result for even small project we can end up with lots of files which are very similar to each other:
How to achieve the DRY in this case? Is it possible?
To reduce the number of YAML files we could merge them into one, for example, for kanban-ui it will look like this:
apiVersion: v1 kind: Service metadata: name: kanban-ui labels: group: backend spec: type: ClusterIP selector: app: kanban-ui ports: - port: 80 targetPort: 80 --- apiVersion: apps/v1 kind: Deployment metadata: name: kanban-ui labels: app: kanban-ui group: frontend spec: replicas: 1 selector: matchLabels: app: kanban-ui template: metadata: labels: app: kanban-ui group: frontend spec: containers: - name: kanban-ui image: wkrzywiec/kanban-ui:k8s ports: - containerPort: 80
But it still doesn’t fix the major problem — how to avoid copy-pasting entire file just to replace couple of values? It would be great if there is a way to define a blueprint for both objects and then inject values into specific fields.
Luckily there is a solution! Helm to the rescue!
Accordingly to the official website — Helm is a package manager for Kubernetes. It helps deploy complex application by bundling necessary resources into Charts, which contains all information to run application on a cluster.
There are couple approaches how to work with Helm. One of them is to download publicly available charts from the Helm Hub. They are prepared by community and are free to use.
For instance, if we would like to run Prometheus on a cluster it would just easy as it’s described on this page — https://hub.helm.sh/charts/stable/prometheus — with the single command:
> helm install stable/prometheus
It contains some default configuration, but can be easily overridden with YAML file and passed during installation. The detailed example I’ll show in a minute.
But Helm is not only providing some predefined blueprints, you can create your own charts!
It’s very easy and can be done by a single command helm create <chart-name>
, which creates a folder with a basic structure:
> helm create example Creating example
In the templates/ folder there are Helm templates that with combination of values.yaml will result in set of Kubernetes objects.
Let’s create a first chart — postgres.
But just before that, boring installations and configurations (but promise, it’ll go quick). In my demo I’m using:
When everything is installed you can start up the minikube cluster and enable ingress addon:
> minikube start 😄 minikube v1.8.1 on Ubuntu 18.04 ✨ Automatically selected the docker driver 🔥 Creating Kubernetes in docker container with (CPUs=2) (8 available), Memory=2200MB (7826MB available) ... 🐳 Preparing Kubernetes v1.17.3 on Docker 19.03.2 ... ▪ kubeadm.pod-network-cidr=10.244.0.0/16 ❌ Unable to load cached images: loading cached images: stat /home/wojtek/.minikube/cache/images/k8s.gcr.io/kube-proxy_v1.17.3: no such file or directory 🚀 Launching Kubernetes ... 🌟 Enabling addons: default-storageclass, storage-provisioner ⌛ Waiting for cluster to come online ... 🏄 Done! kubectl is now configured to use "minikube" > minikube addons enable ingress 🌟 The 'ingress' addon is enabled
Then you’ll need to edit you hosts file. Its location varies on OS:
When you find it add following lines:
172.17.0.2 adminer.k8s.com 172.17.0.2 kanban.k8s.com
To make sure that this config is correct you’ll need to check if an IP address of minikube cluster on your machine is the same as it’s above. In order to do that run the command:
> minikube ip 172.17.0.2
Now we can create first Helm chart:
> helm create postgres Creating postgres
We will not need generated files inside ./templates
folder, so remove them and also clear all the content inside values.yaml.
Now we can roll up our sleeves and define all necessary files to create chart for postgres database. For a reminder, in order to deploy it with kubectl we had to have following files:
They contain definitions of ConfigMap, Deployment, PersistentVolumeClaim and ClusterIP. Their full definitions could be found in a repository.
First, let’s create a template for postgres Deployment object, therefore inside the ./templates
create a deployment.yaml file:
At first glance you might see these strange parts between two pairs of curly brackets, like {{ .Values.postgres.name }}
. They are written in Go template language and are referring to a value located in a values.yaml which is located inside the root folder of a chart. For mentioned example Helm will try to match it with a value from values.yaml:
postgres: name: postgres
Another example. A value for a base Docker image, defined in image: {{ .Values.postgres.container.image }}
will be taken from:
postgres: name: postgres container: image: postgres:9.6-alpine
And so on. We can define the structure inside this file whatever we like.
This database Deployment requires a PersistentVolumeClaim to be created to reserve some storage on a disk, therefore we need to create a Helm template — pvc.yaml inside the ./templates
folder:
This one is shorter, and there is nothing new here.
Next template that we need to create is for ClusterIP, which will allow other Pods inside the cluster to enter the* Pod with postgres — service.yaml:
And finally ConfigMap template needs to be created — config.yaml. It’ll hold information about created database, user and its password (yes, not very secure, but for simple example it’s enough).
Here you might see a strange {{ -range ... }}
clause, which can be translated as a for each loop known in any programming language. In above example, Helm template will try to inject values from an array defined in values.yaml:
postgres: config: data: - key: key value: value
Inside a template, there are only dummy values. If you want to have different ones you would need to override them during installation of this set of objects on a Kubernetes cluster.
The entire values.yaml presents as follow:
replicaCount: 1 postgres: name: postgres group: db container: image: postgres:9.6-alpine port: 5432 service: type: ClusterIP port: 5432 volume: name: postgres-storage kind: PersistentVolumeClaim mountPath: /var/lib/postgresql/data pvc: name: postgres-persistent-volume-claim accessMode: ReadWriteOnce storage: 4Gi config: name: postgres-config data: - key: key value: value
To finalize creating first chart we need to modify some metadata inside Chart.yaml file:
apiVersion: v2 name: postgres description: A Helm chart for PostgreSQL database type: application version: 0.1.0 appVersion: 1.16.0 keywords: - database - postgres home: https://github.com/wkrzywiec/k8s-helm-helmfile/tree/master/helm maintainers: - name: Wojtek Krzywiec url: https://github.com/wkrzywiec
Then create a new kanban-postgres.yaml file which will hold some specific values. Put it outside the postgres
chart folder so the file structure can be similar to that:
Inside the file put only values that will be specific for this deployment, i.e. postgres database credentials (all other values we can keep as default):
postgres: config: data: - key: POSTGRES_DB value: kanban - key: POSTGRES_USER value: kanban - key: POSTGRES_PASSWORD value: kanban
Everything is set up, we can now deploy postgres into a cluster. In a Helm world this step is called creating a new release:
> helm install -f kanban-postgres.yaml postgres ./postgres NAME: postgres LAST DEPLOYED: Mon Apr 13 16:13:16 2020 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None > helm list NAME NAMESPACE REVISION STATUS CHART APP VERSION postgres default 1 deployed postgres-0.1.0 1.16.0 > kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE postgres 1/1 1 1 2m14s
Cool, the postgres is up and running! Before moving forward let’s have a closer look on helm install command, what each of an argument means:
If you plan to use a chart only to create one Helm release you might wonder what is a value behind using Helm at all.
Let me show you that on another example. Let’s create a new chart called app — it will be a template chart for three different releases — adminer, kanban-app & kanban-ui.
> helm create app Creating app
After removing unnecessary file from ./templates
folder and cleaning the values.yaml create a deployment.yaml file:
Nothing new in particular, only some placeholders for values & two range loops to inject values from *ConfigMap*s or simple values to application container.
All 3 apps require to have also the ClusterIP objects to be deployed, therefore here is its template — service.yaml:
All default values in both templates are injected from values.yaml:
app: name: app group: app replicaCount: 1 container: image: add-image-here port: 8080 config: [] env: - key: key value: value service: type: ClusterIP port: 8080
And to close a task of creating app chart — we need to define some metadata in the Chart.yaml file:
apiVersion: v2 name: app description: A Helm chart for any application type: application version: 0.1.0 appVersion: 1.16.0 keywords: - app - java - javascript - angular home: https://github.com/wkrzywiec/k8s-helm-helmfile/tree/master/helm maintainers: - name: Wojtek Krzywiec url: https://github.com/wkrzywiec
Then, similarly to previous example, outside the app folder create YAML files with values which will override ones from values.yaml inside app chart.
An adminer.yaml file looks as follow:
app: name: adminer group: db container: image: adminer:4.7.6-standalone port: 8080 env: - key: ADMINER_DESIGN value: pepa-linha - key: ADMINER_DEFAULT_SERVER value: postgres
A kanban-app.yaml file:
app: name: kanban-app group: backend container: image: wkrzywiec/kanban-app:k8s config: - name: postgres-config env: - key: DB_SERVER value: postgres
A kanban-ui.yaml file:
app: name: kanban-ui group: frontend container: image: wkrzywiec/kanban-ui:k8s port: 80
A resulting file structure:
Now, in order to create three releases for each application use commands:
> helm install -f adminer.yaml adminer ./app NAME: adminer LAST DEPLOYED: Mon Apr 13 16:57:17 2020 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None > helm install -f kanban-app.yaml kanban-app ./app NAME: kanban-app LAST DEPLOYED: Mon Apr 13 16:57:36 2020 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None > helm install -f kanban-ui.yaml kanban-ui ./app NAME: kanban-ui LAST DEPLOYED: Mon Apr 13 16:57:54 2020 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None > helm list NAME NAMESPACE REVISION STATUS CHART APP VERSION adminer default 1 deployed app-0.1.0 1.16.0 kanban-app default 1 deployed app-0.1.0 1.16.0 kanban-ui default 1 deployed app-0.1.0 1.16.0 postgres default 1 deployed postgres-0.1.0 1.16.0 > kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE adminer 1/1 1 1 112s kanban-app 1/1 1 1 93s kanban-ui 1/1 1 1 75s postgres 1/1 1 1 45m
Perfect! There is one more thing to do — create a chart with Ingress Controller — a gateway to a cluster.
Like before, create a new chart:
> helm create ingress Creating ingress
Remove all files from templates folder and clear content of values.yaml. This time, before moving straight away to defining a templates, let’s focus on to Chart.yaml first.
apiVersion: v2 name: ingress description: A Helm chart for Ingress Controller type: application version: 0.1.0 appVersion: 1.16.0 keywords: - ingress - nginx - api-gateway home: https://github.com/wkrzywiec/k8s-helm-helmfile/tree/master/helm maintainers: - name: Wojtek Krzywiec url: https://github.com/wkrzywiec dependencies: - name: nginx-ingress version: 1.36.0 repository: https://kubernetes-charts.storage.googleapis.com/
Here there is a new section — dependencies
— which has been added. It creates default backend services which enables the Ingress Controller features.
But it’s not the only thing we need to do here. This section only defines on what this chart depends on, it won’t download it automatically during the installation. We need to take care of it ourselves. To do that run the command:
> helm dependency update ./ingress/ Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "stable" chart repository Update Complete. ⎈Happy Helming!⎈ Saving 1 charts Downloading nginx-ingress from repo [https://kubernetes-charts.storage.googleapis.com/](https://kubernetes-charts.storage.googleapis.com/) Deleting outdated charts
Inside the ingress/charts
folder a new file will appear — nginx-ingress-1.36.0.tgz
.
Now we can define a template for Ingress — it will be located inside ./templates
folder and will be called ingress.yaml:
The most interesting part is inside specification section. There are two nested loops there which allow to define multiple hosts and multiple paths for each host.
And also, here is a default values.yaml file:
ingress: name: ingress-service replicaCount: 1 annotations: class: nginx hosts: - host: chart-example.local paths: - path: / backend: serviceName: serviceName servicePort: 8080
The resulting folder structure:
Outside the ingress
chart we can now create an ingress.yaml file which will hold all routing rules for our cluster.
ingress: hosts: - host: adminer.k8s.com paths: - path: / backend: serviceName: adminer servicePort: 8080 - host: kanban.k8s.com paths: - path: /api/ backend: serviceName: kanban-app servicePort: 8080 - path: / backend: serviceName: kanban-ui servicePort: 80
And now we’re able to create a Helm release:
> helm install -f ingress.yaml ingress ./ingress NAME: ingress LAST DEPLOYED: Tue Apr 14 07:22:44 2020 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None > helm list NAME NAMESPACE REVISION STATUS CHART APP VERSION adminer default 1 deployed app-0.1.0 1.16.0 ingress default 1 deployed ingress-0.1.0 1.16.0 kanban-app default 1 deployed app-0.1.0 1.16.0 kanban-ui default 1 deployed app-0.1.0 1.16.0 postgres default 1 deployed postgres-0.1.0 1.16.0 > kubectl get deployments NAME READY UP-TO-DATE AVAILABLE adminer 1/1 1 1 ingress-nginx-ingress-controller 1/1 1 1 ingress-nginx-ingress-default-backend 1/1 1 1 kanban-app 1/1 1 1 kanban-ui 1/1 1 1 postgres 1/1 1 1
Everything is set up, you can start testing it. Go to the http://kanban.k8s.com and you should get this page:
Congrats! 🍾🤘
Conclusions
In this short entry I’ve presented how, with Helm, you can reduce of copy-pasting tasks and can bring a one single template for deploying multiple applications on a Kubernetes cluster, which might be very handy in a microservice world. I hope you’ve enjoyed it.
But there is still one thing left, which holds us from establishing a fully declarative approach for an infrastructure. In order to deploy each application we still need to run imperative commands, like helm install. But luckily there is another tool — helmfile! But this one I’ll describe in my next story.
For now, here is my GitHub repository with a source code for this project:
wkrzywiec/k8s-helm-helmfile on github.com
And here is a source code of Kanban project:
wkrzywiec/kanban-board on github.com
Top comments (0)