Continuous Deployment with Kubernetes, Docker and GitLab CI

Continuous Deployment with Kubernetes, Docker and GitLab CI @alexander_kiel  Clojure Berlin 2016

Outline • Continuous Deployment why? • Docker • Kubernetes • Sample Clojure Service • Deploy with GitLabCI

Continuous Deployment • What do we want? • Increase responsiveness • Decrease time to market • Gain conﬁdence by deploying often in small amounts • How to achieve that? • Automate everything • Always deploy the master into production • Use feature toggles when needed

Simple Git Workflow • Works for in-house apps • not for libs or shipping apps • No versions, no tags, just SHA’s • Latest commit on master is always deployed to production • Feature/fix branches are merged when ready master feature/fix branches 1ebb95d be61dda 6e4010d

Docker • Like VM’s but much more light-weight and shippable • Runs on Linux, executes processes in an isolated environment (resource limitation, ﬁlesystem, network) • Container principle: Can contain everything, but looks the same from the outside • A container platform can run every container • Developers have max. freedom what to do • In contrast: PaaS like Heroku - has to support the language

Kubernetes • Container runtime platform • Originally designed by Google - now Open Source • One of the most active projects on GitHub - 20,000 stars, 40,000 commits, 15,000 issues, 200 releases • Alternatives: Apache Mesos, Docker Swarm (lacks features)

Kubernetes Architecture k8s-master-1 k8s-master-2 k8s-master-3 load-balancer-1 load-balancer-2 DNS RR k8s-worker-1 proxy app-1 k8s-worker-2 proxy app-2 k8s-worker-n proxy app-k etcd cluster  quorum HAProxy • Runs on VMware ESX • CoreOS Linux • Single YAML ﬁle as conﬁguration • Everything in containers

Kubernetes - Pods • A Pod is a deployable unit in Kubernetes • Pods can contain multiple containers • Containers inside a Pod share on port space, can use localhost and can communicate via IPC and shared memory • Idea: one process per container - many cooperating processes in one Pod apiVersion: v1  kind: Pod  metadata:  name: <pod-name>  labels:  <key>: <value>  spec:  containers:  - name: <container-name>  image: <container-image>  ports:  - containerPort: 80  env:  - name: <key>  value: <value>

Kubernetes - Deployments • A Deployment ensures that certain number of Pods are always running • It consists of a Pod template and the number of replicas • It supports hot-redeployments by changing parts of the Pod template • Horizontal scaling is possible apiVersion: extensions/v1beta1  kind: Deployment  metadata:  name: <deployment-name>  spec:  replicas: 2  template:  metadata: labels: <key>: <value> spec: containers: - name: <container-name> image: <container-image> ports: - containerPort: 80 env: - name: <key> value: <value>

Kubernetes - Services • Kubernetes uses an overlay network to provide different address spaces (we use ﬂannel) • Every Pod has an IP address - but it changes every time one is created • Services provide a stable IP address for groups of Pods • Service names are resolvable by an internal DNS • Service selectors are used to match Pods according to there labels apiVersion: v1 kind: Service metadata: name: clojure-berlin-2016 labels: app: lens spec: type: NodePort ports: - port: 80 targetPort: 80 protocol: TCP selector: service: clojure- berlin-2016

Kubernetes - External Access • Kubernetes networks are internal only • External access through load balancers necessary • Certain Platforms like Google Compute Engine provide load balancer integration with Kubernetes • We have our own solution as a combination of HAProxy and Kubernetes NodePort • Kubernetes Services with type NodePort are exposed on every worker under a certain port frontend http bind 0.0.0.0:80 mode http option httplog acl host_clj hdr(host) clj.<domain> use_backend clj if host_clj backend clj mode http balance roundrobin option httplog server worker-1 <ip>:32599 check server worker-2 <ip>:32599 check

Deployment Lifecycle GitLab CI Source Code build test Kubernetes Test Cluster Kubernetes Prod Cluster automatic deployment manual deployment git push

Sample Clojure Service • .gitlab-ci.yml • Like .travis.yml contains instructions for GitLabCI how to test, build and deploy • Dockerﬁle • Instructions for Docker how to build the image of the app • Artifact of the build is a docker image - not uberjar • kube-deployment.yml • Kubernetes deployment instructions • kube-svc.yml • Kubernetes service description https://github.com/alexanderkiel/clojure-berlin-2016

The Core Namespace (ns clojure-berlin-2016.core (:require [aleph.http :as http] [clojure.core.async :refer [<!! chan]])) (defn -main [& args] (-> (fn [_] {:status 200 :body "Clojure Berlin 2016"}) (http/start-server {:port 8080})) (<!! (chan))) • A simple web server returning "Clojure Berlin 2016"

The Leiningen Project File (defproject clojure-berlin-2016 "<VERSION>" :dependencies [[aleph "0.4.1"] [org.clojure/clojure "1.8.0"] [org.clojure/core.async "0.2.395"]] :main clojure-berlin-2016.core) • <VERSION> is replaced at build time by the Git SHA • :main is for lein run to work

.gitlab-ci.yml - test/build image: clojure:lein-2.7.1 stages: - test - build - deploy test: stage: test tags: - docker script: - lein test build: stage: build tags: - docker script: - sed -i "s/<VERSION>/$CI_BUILD_REF/" project.clj - docker build -t clojure-berlin-2016:$CI_BUILD_REF . - docker push clojure-berlin-2016:$CI_BUILD_REF

.gitlab-ci.yml - deploy branch deploy-branch: stage: deploy environment: test image: dreg.life.uni-leipzig.local/kubectl:0.4 tags: - docker script: - sed -i "s/<VERSION>/$CI_BUILD_REF/" kube-deployment.yml - kubectl config use-context gitlab-ci-test - kubectl apply -f kube-deployment.yml except: - master when: manual • Used to test a feature/ﬁx branch in a full environment

.gitlab-ci.yml - deploy test deploy-master: stage: deploy environment: test image: dreg.life.uni-leipzig.local/kubectl:0.4 tags: - docker script: - sed -i "s/<VERSION>/$CI_BUILD_REF/" kube-deployment.yml - kubectl config use-context gitlab-ci-test - kubectl apply -f kube-deployment.yml only: - master

.gitlab-ci.yml - deploy prod deploy-prod: stage: deploy environment: prod image: dreg.life.uni-leipzig.local/kubectl:0.4 tags: - docker script: - sed -i "s/<VERSION>/$CI_BUILD_REF/" kube-deployment.yml - kubectl config use-context gitlab-ci-prod-a - kubectl apply -f kube-deployment.yml only: - master when: manual

Docker ﬁle FROM clojure:lein-2.7.1 COPY src /app/src COPY project.clj /app/ WORKDIR /app RUN lein with-profile production deps EXPOSE 80 CMD ["lein", "with-profile", "production", "run"] • Just copy the sources into the container • Use Leiningen itself to run in production

kube-deployment.yml apiVersion: extensions/v1beta1 kind: Deployment metadata: name: clojure-berlin-2016 spec: replicas: 2 template: metadata: labels: app: lens service: clojure-berlin-2016 spec: containers: - name: clojure-berlin-2016 image: dreg.life.uni-leipzig.local/clojure-berlin-2016:<VERSION> ports: - containerPort: 8080 resources: requests: cpu: "125m" memory: "1Gi" limits: cpu: 1 memory: "2Gi"

kube-svc.yml apiVersion: v1 kind: Service metadata: name: clojure-berlin-2016 labels: app: lens spec: type: NodePort ports: - port: 80 targetPort: 8080 protocol: TCP selector: service: clojure-berlin-2016

Steps to Follow • Create the Kubernetes Service • kubectl create -f kube-svc.yml • Edit HAProxy Conﬁg • add rules and backend for the service • Push to GitLab • git push

Environments in GitLabCI • Very good visibility of wich commit is deployed in which environment right now • Manual deployment to prod possible

Environment History • Easy to see when what commit was deployed • Rollback possible

Numbers • Our team has 4 developers • We run 2 Kubernetes clusters (test and prod) with about 96 GB RAM and and 24 vCPU’s each • We run about 60 pods in production • We have other services like central log aggregation running using Fluentd and Elasticsearch/Kibana

Thank You • Sample Project on Github  https://github.com/alexanderkiel/clojure-berlin-2016 • Twitter  @alexander_kiel • Mail  alexanderkiel@gmx.net

Continuous Deployment with Kubernetes, Docker and GitLab CI

More Related Content

What's hot

Similar to Continuous Deployment with Kubernetes, Docker and GitLab CI

Recently uploaded

Continuous Deployment with Kubernetes, Docker and GitLab CI