Kubernetes-based development with Devspace

Modern applications base more and more on micro-services. Splitting large applications into smaller pieces makes the whole more maintainable and easier to develop. However, instead of developing a big monolith, we work on a bunch of tiny applications, making it more challenging to debug and deploy the whole system. Luckily, there are many tools out there to help us out. An interesting comparison of some of them can be found here. In what follows, we want to see how easy it is to do Kubernetes-based development with devspace.

A micro-services application

Suppose we are developing a micro-services application, for example an e-shop. In essence, our e-shop consists of a frontend application that communicates with a backend through an API. For the sake of simplicity, let's say that our backend looks like this:

User management is handled by the iam-service. Orders are processed via the message-queue. Most of our backend's business logic is packed in serverless functions served by the faas. Our application's state is held in our database. Finally, for some good reasons (e.g. the ease of testing setup), we are developing our software in a monorepo.

With time, our micro-services application will necessarily contain a lot of business logic that will be packed in even more micro-service code or serverless functions. For example, we might need a connector service between our message-queue and our faas, or an assets service with some logic to add new assets in a controlled way. A very convenient way to host our micro-services is to dockerize them and let Kubernetes orchestrate them.

Typically, our IAM service is a third-party like keycloak or fusionauth which we can easily deploy on Kubernetes by means of a helm chart. Helm is a very practical package manager for Kubernetes. For example, a typical fusionauth deployment would look like something along these lines:

helm repo add fusionauth https://fusionauth.github.io/charts helm install fusionauth --create-namespace fusionauth/fusionauth --namespace auth \ --set database.protocol=postgresql \ --set database.user=<username> \ --set database.password=<password> \ --set database.host=<hostname> \ --set database.port=5432 \ --set database.name=<database-name> \ --set database.root.user=<root-user> \ --set database.root.password=<root-password> \ --set app.runtimeMode=production \ --set search.engine=database 

Our message queue is probably redismq, rabbitmq or kubemq, for which we also easily find helm charts.

Then come our own custom services for which we need to write our own Kubernetes resources (deployments, services, ingresses, etc.). Finally, we can write some kind of script to install all the necessary helm charts and apply our Kubernetes resources.

Because our software deals with sensitive data and makes our business, we need to be careful when deploying a new release. Therefore, we want to somehow test it before we release it, which is very easy to do on Kubernetes clusters. Indeed, we can imagine we have two environments, one for testing and one for production. The testing (or staging) environment would be synchronized with our software repository's main branch while the production environment would be the pendant of our repo's production branch. We develop on the main branch and, as soon as the Q&A is satisfied with the software pushed there, we push it to production.

We are now in the complicated situation where we want to develop our software on a development machine, test it somehow on an almost productive environment, and release it to a production environment. That leads us to three different build and deployment procedures. On a development machine, we surely want to interact with a short-lived database. Moreover, login credentials to our microservices (like the assets service) should be trivial. On staging, we might want to grant unprotected access to some of our services, for the sake of debugging. On production, we want to secure and hide as much as possible.

Finally, if our development environment was close to the production environment, we would minimize the amount of surprises following a deployment to staging or production, which would increase our productivity.

Enter devspace

Devspace is a cli tool that allows automation of both the build and the deployment of container images. In addition, that tool might as well replace our makefile or docker-compose configurations and provides us with the ability to do Kubernetes-based development. Because of the latter ability, let's assume we have set up a small cluster on our development machine. In one click, you can have Jelastic set up that development cluster for you

through a very simple interface

Or you can manually set up your own kind, minikube, or docker for desktop cluster.

The easiest way to install devspace (not on your Kubernetes cluster, on a remote machine from which you develop your code!) is to do

npm install -g devspace 

Then, depending on our use-case, we might run

devspace init 

and follow the instructions. In our particular case, we want to build

• our API
• a bunch of custom micro-services

That we do with the following configuration:

version: v1beta10 vars: - name: SOME_IMPORTANT_VARIABLE source: env default: the-important-value images: my-custom-service: image: my-repo/my-custom-service tags: - ${DEVSPACE_RANDOM} dockerfile: ./my-custom-service/Dockerfile context: . build: docker: options: target: app buildArgs: SOME_IMPORTANT_VARIABLE: ${SOME_IMPORTANT_VARIABLE} api: image: my-repo/api tags: - ${DEVSPACE_RANDOM} dockerfile: ./api/Dockerfile context: . 

The above configuration defines how to build our API and our micro-services. When they are pushed to their docker registry, both docker images will have the same random tag (defined by the built-in variable DEVSPACE_RANDOM). Instead of using a docker daemon, we can also choose to use custom build commands or kaniko. We can use environment variables, like SOME_IMPORTANT_VARIABLE and provide the usual options to build docker images.

Next, we want to deploy

• our API
• our custom micro-services
• various third-party services (iam, message queue, faas, assets)

In order to take care of that, we complete the previous configuration with the following snippet:

deployments: # for the custom service, we have regular k8s manifests - name: my-custom-service kubectl: manifests: - my-custom-service/manifest.yaml # for the api, we have written a helm chart - name: api helm: chart: name: api/chart values: image: my-repo/api postgres: database: my-database hostname: postgres username: my-username password: my-password # the database service is a 3rd party - name: postgres helm: chart: name: postgresql repo: https://charts.bitnami.com/bitnami values: postgresqlDatabase: my-database postgresqlUsername: my-username postgresqlPassword: my-password # the iam service is a 3rd party - name: iam-service helm: chart: name: fusionauth/fusionauth values: database: protocol: postgresql user: iam-user password: iam-password host: postgres name: iam-database user: root-db-username password: root-db-password search: engine: database 

The first deployment, my-custom-service, amounts to

kubectl apply -f my-custom-service/manifest.yaml 

The second deployment, api, is a regular helm installation. Instead of writing our own helm chart, we could have used the built-in component charts which offer a compromise between defining our own helm charts and keeping our Kubernetes resources configuration simple. With our current devspace configuration in place, we can start our development environment:

devspace dev 

That command builds our docker images and deploys our software to our development Kubernetes cluster's default namespace. We are now in a situation where we can develop our code on our development machine and push it to our development Kubernetes cluster. With either hot reloading or auto-reloading, we can even fix our code and the result is automatically propagated to our cluster.

Deploy to multiple environments

Now we have a setup that works for development. We are not very far away from our staging environment setup. First, our docker images need to be tagged following the pattern <my-repo>/<my-service>:staging-<commit-short-sha>. Second, our staging environment bases on external database and IAM services. Consequently, we don't want to deploy them on staging and we need to adapt the services that depend on them. In devspace, we can define profiles. Until now, our configuration has no reference to any profile, therefore it is the development profile. We can define the staging profile, let it base on the development profile and adapt it as we've just described. To do that, let's add the following configuration to our devspace.yaml:

profiles: - name: staging patches: # images -> adapt tag - op: replace path: /images/0=${DEVSPACE_RANDOM} value: - staging-${DEVSPACE_GIT_COMMIT} # postgres -> remove, we have an external database - op: remove path: /deployments/name=postgres # iam service -> remove, we have an external iam service - op: remove path: /deployments/name=iam-service # api  # -> we need an ingress - op: replace path: /deployments/name=api/helm/values/ingress value: enabled: true annotations: kubernetes.io/ingress.class: nginx-cert cert-manager.io/cluster-issuer: letsencrypt-prod hosts: - host: api-staging.my-staging-domain.com paths: - / tls: - secretName: api-tls hosts: - api-staging.my-staging-domain.com # -> we need up-to-date database accesses - op: replace path: /deployments/name=api/helm/values/postgres value: database: my-external-database hostname: my-external-database-hostname username: my-external-username password: my-external-password # my-custom-service -> nothing to do 

We can of course follow the same philosophy coupled with the concept of parent profiles to define our production profile. Then, building and deploying to staging or production is as simple as

devspace deploy -p staging devspace deploy -p production 

Obviously, remotely debugging those profiles is also possible.

We've only scratched the surface...

Many more features are available, like custom commands definition, port-(reverse-)forwarding, file synchronization, container log streaming, etc., which you can read about here. Wisely used in CI / CD pipelines, devspace can drastically simplify the way you release your software.