Deploying Ruby apps to Google Cloud Kubernetes Engine continuously with CircleCI

May 1, 2018

Topics

Vladimir Dementyev
Principal Backend Engineer

I hadn’t had a chance to try Kubernetes for a long time, but finally a few weeks ago I got one: I’ve been working on a simple Ruby application (a Slack bot), and decided to deploy it to Google Cloud Kubernetes Engine.

There was an additional requirement to set up a continuous deployment process. For that, I chose CircleCI service.

NOTE: This post is just a playbook containing all the actions I made to get the application up and running.

Software versions:

Kubernetes 1.9.4-gke.1
Google Cloud SDK 195.0.0

Preparing GCloud

Let’s assume that you already have a Google Cloud account.

Go to the Console and create a new project (my-project) with Kubernetes API enabled (more thorough description of this step could be found here).

Next step—install gcloud CLI:

# for macOS/Homebrew users it's pretty simple brew install caskroom/cask/google-cloud-sdk

Then you must authenticate yourself:

gcloud auth login

(Optionally) Configure the default project (to avoid specifying it with every command):

gcloud config set project my-project

Create K8S cluster:

# "g1-small" is a minimal instance type (as of the time of writing) that allows having only one node gcloud container clusters create my-cluster --machine-type g1-small --num-nodes 1

Obtain the cluster’s credentials:

gcloud container clusters get-credentials my-cluster

Get the list of clusters to verify that everything is okay:

gcloud container clusters list

The output looks like this:

NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS my-app-web us-east1-b 1.9.4-gke.1 35.227.92.102 g1-small 1.9.4-gke.1 1 RUNNING

Preparing application

As I’ve already told, the application I want to deploy is a simple Ruby-only app (i.e., no databases, caches, persistent stores). See the links at the end of the post on how to deploy Rails applications.

We need to build a Docker image and push to Google Container Registry.

Here is my Dockerfile:

FROM ruby:2.5.0 RUN apt-get update && apt-get install -y build-essential git RUN mkdir -p /app WORKDIR /app ENV LANG C.UTF-8 COPY Gemfile Gemfile.lock ./ RUN gem install bundler && bundle install --jobs 20 --retry 5 COPY . . # We want to include the current git revision information # to a build to track releases ARG BUILD_SHA=unknown ENV BUILD_SHA ${BUILD_SHA} EXPOSE 3000 ENTRYPOINT ["bundle", "exec"] CMD ["puma", "-p", "3000"]

Let’s build it:

docker build -t my-app -f Dockerfile.prod .

NOTE: I use Dockerfile.prod file for production and Dockerfile for development.

Then tag it using a specific GCloud format (containing region and project name):

docker tag my-app us.gcr.io/my-project/my-app:v1

And push:

gcloud docker -- push us.gcr.io/my-project/my-app:v1

OK. We’re almost there. Now we have to tell somehow to our K8S cluster to get this image and run.

For that we’re going to use K8S deployment:

apiVersion: apps/v1 kind: Deployment metadata: name: web labels: name: web spec: replicas: 2 selector: matchLabels: name: web template: metadata: labels: name: web spec: containers: - name: web image: gcr.io/my-project/my-app:latest ports: - containerPort: 3000 livenessProbe: httpGet: path: /_health port: 3000 initialDelaySeconds: 10 timeoutSeconds: 1 readinessProbe: httpGet: path: /_health port: 3000 initialDelaySeconds: 10 timeoutSeconds: 1

K8S documentation is a good place to read about deployments; no need to recall it here.

I’d only like to pay attention to the livenessProbe and readinessProbe sections: they tell K8S how to monitor the application state to provide rolling update functionality and make sure that the desired number of replicas are up and running. Note that your application should provide endpoints for these checks (/_health in my case).

Now we need to install one more tool—kubectl—to operate our cluster:

gcloud components install kubectl

It’s time to finally deploy our application! Let’s create our deployment:

kubectl create -f kube/web-deployment.yml

To get the information about our deployment, you can see the list of pods:

kubectl get pods

More information about a pod:

kubectl describe pod web-<pod-id>

Updating application manually

Your application is up and running. How to push a new release?

If you want to update your deployment configuration, just run the command below:

kubectl apply -f kube/web-deployment.yml

For updating the codebase, you should tell your cluster to use a new image:

docker tag my-app us.gcr.io/my-project/my-app:v1.1 gcloud docker -- push us.gcr.io/my-project/my-app:v1.1 kubectl set image deployment web web=gcr.io/my-project/my-app:v1.1

K8S will take care of creating new pods and replacing the old ones.

Running three commands and manually providing new versions is not the most convenient way to deploy, isn’t it?

Let’s make someone else do all the dirty work.

CircleCI integration

CircleCI 2.0 is a very flexible automation tool.

We want to make our deployment as easy as just making git push (or merging PR into master branch).

Below you can find an example .circle/config.yml with the comments explaining every step:

version: 2 workflows: version: 2 build_and_deploy: jobs: - build - deploy: # Run the "deploy" job only if the "build" job was successful requires: - build # Run deploy job only on master branch filters: branches: only: - master jobs: # This job is responsible for running tests and linters build: docker: - image: circleci/ruby:2.5.0 environment: RACK_ENV: test steps: - checkout - run: name: Install gems command: bundle install - run: name: Run Rubocop and RSpec # Our default rake task contains :spec and :rubocop tasks command: bundle exec rake # Cache all the project files to re-use in the deploy job # (instead of pulling the repo again) - persist_to_workspace: root: . paths: . deploy: docker: # official image which includes `gcloud` and `kubectl` tools - image: google/cloud-sdk # project information environment: GOOGLE_PROJECT_ID: my-project GOOGLE_COMPUTE_ZONE: us-east1-b GOOGLE_CLUSTER_NAME: my-app steps: # Attach previously cached workspace - attach_workspace: at: . # SERVICE_KEY provides access to your GCloud project. # Read more here https://circleci.com/docs/2.0/google-auth/ - run: echo $GCLOUD_SERVICE_KEY > ${HOME}/gcloud-service-key.json # Authenticate gcloud - run: gcloud auth activate-service-account --key-file=${HOME}/gcloud-service-key.json # Configure gcloud (the same steps as you do on your local machine) - run: gcloud --quiet config set project ${GOOGLE_PROJECT_ID} - run: gcloud --quiet config set compute/zone ${GOOGLE_COMPUTE_ZONE} - run: gcloud --quiet container clusters get-credentials ${GOOGLE_CLUSTER_NAME} # Enable remote Docker (https://circleci.com/docs/2.0/building-docker-images/) - setup_remote_docker # Build Docker image with the current git revision SHA - run: docker build -t brooder -f Dockerfile.prod --build-arg BUILD_SHA=${CIRCLE_SHA1} . # Use the same SHA as our image version - run: docker tag brooder gcr.io/my-project/my-app:${CIRCLE_SHA1} # Using remote Docker is a little bit tricky but this "spell" works - run: gcloud docker --docker-host=$DOCKER_HOST -- --tlsverify --tlscacert $DOCKER_CERT_PATH/ca.pem --tlscert $DOCKER_CERT_PATH/cert.pem --tlskey $DOCKER_CERT_PATH/key.pem push gcr.io/my-project/my-app:${CIRCLE_SHA1} # and finally, "deploy" the new image - run: kubectl set image deployment web web=gcr.io/my-project/my-app:${CIRCLE_SHA1} --record

Keeping secrets

Let’s cover one more topic here—managing app secrets.

We use Kubernetes Secrets to store sensitive information (such as third-party services API tokens).

First, you have to create secrets definition file:

# kube/app-secrets.yml apiVersion: v1 kind: Secret metadata: name: mysecret type: Opaque data: # The value should be base64 encoded github_token: b2t0b2NhdA== slack_token: Y3V0bWVzb21lc2xhY2s=

Don’t forget to encode the value into base64. For example:

echo -n "myvalue" | base64

Push secrets to the cluster:

kubectl create -f kube/app-secrets.yml #=> secret "mysecret" created

NOTE: Remove app-secrets.yml right after pushing to K8S (or, at least, add to .gitignore).

Now you can pass the secrets to your app through env variables:

# web-deployment.yml apiVersion: apps/v1 kind: Deployment # ... spec: # ... templaee: # ... spec: containers: - name: web image: gcr.io/my-project/my-app:latest # .... env: - name: MY_GITHUB_TOKEN valueFrom: secretKeyRef: name: mysecret key: github_token - name: MY_SLACK_TOKEN valueFrom: secretKeyRef: name: mysecret key: slack_token