DEV Community

Cover image for Tekton CI, part III, listen to Github events
Leandro Proença
Leandro Proença

Posted on

Tekton CI, part III, listen to Github events

#tutorial

For this guide I'm assuming you are running a local Kubernetes cluster.

In the previous article we've seen how to share information across a Pipeline using Workspaces, leading to a task for cloning a Github repository then ending with anoter task for listing the source files in the cloned repository.

Now, let's understand how to listen to Github events and trigger the Pipeline using event bindings.

👂 Listening to Github events

How to listen to Github events? In other words, when someone pushes code to some branch or opens a pull request, how do we get notified from that?

Github (not only Github but other majors like Gitlab too) notifies such events using Webhooks.

We basically have to configure a webhook in the Github interface, choose some events and provide a webhook URL to receive a payload.

Payload received, we can decide whatever we want with that information.

In our case, we want to look at the event type (if pull request opened for instance), extract the repo URL from the payload and then run the pipeline providing the repo URL.

Here's the basic architecture of Webhooks on Github:
Webhook local

The summary of steps we're going to cover in this post:

  1. Create a Tekton Event Listener, TriggerBinding and TriggerTemplate
  2. Configure the Github Webhook

Let's dive in!

First things first

Install the triggers resources in the cluster:

$ kubectl apply --filename \ https://storage.googleapis.com/tekton-releases/triggers/latest/release.yaml $ kubectl apply --filename \ https://storage.googleapis.com/tekton-releases/triggers/latest/interceptors.yaml
Enter fullscreen mode Exit fullscreen mode

1. Create the Event Listener

Event Listeners are Tekton components that listen for events at a specified port in your Kubernetes cluster. Listeners are attached to Kubernetes Pods and backed by Services.

Event Listeners need a proper Kubernetes Service Account for role access control (RBAC), so the first step is creating the RBAC objects.

The Gist with the RBAC yaml can be found here.

Time to exmplore the event-listener.yml:

apiVersion: triggers.tekton.dev/v1beta1 kind: EventListener metadata: name: github-pr spec: serviceAccountName: tekton-service-account triggers: - name: pr-trigger bindings: - ref: github-pr-trigger-binding template: ref: github-pr-trigger-template
Enter fullscreen mode Exit fullscreen mode
  • serviceAccountName referes to the RBAC
  • each trigger has a name and must provide a trigger binding and a trigger template

Trigger Binding

Trigger Bindings are responsible to extract information from the event payload and use such information in the TriggerTemplate.

apiVersion: triggers.tekton.dev/v1beta1 kind: TriggerBinding metadata: name: github-pr-trigger-binding spec: params: - name: revision value: $(body.pull_request.head.sha) - name: repo-url value: $(body.repository.clone_url)
Enter fullscreen mode Exit fullscreen mode

Trigger Template

Trigger Templates define the template of the pipeline or task, using the params extracted by the TriggerBinding.

apiVersion: triggers.tekton.dev/v1beta1 kind: TriggerTemplate metadata: name: github-pr-trigger-template spec: params: - name: revision default: main - name: repo-url resourcetemplates: - apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: generateName: my-pipeline- spec: pipelineRef: name: my-pipeline workspaces: - name: shared-data volumeClaimTemplate: spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi params: - name: repo-url value: $(tt.params.repo-url) - name: revision value: $(tt.params.revision)
Enter fullscreen mode Exit fullscreen mode

TriggerTemplate is similar to PipelineRun, except that the latter is triggered manually as we've seen in the previous posts.

Then, the TriggerTemplate "sends" the needed params to the PipelineRef, along with the definition of Workspace volume claim templates.

Exposing the Event Listener URL

After applying the Trigger resources, we can check that Tekton created a Kubernetes service for the event listener:

$ tkn eventlisteners list NAME AGE URL AVAILABLE github-pr 21 minutes ago http://el-github-pr.default.svc.cluster.local:8080 True
Enter fullscreen mode Exit fullscreen mode

All we need is providing this URL http://el-github-pr.default.svc.cluster.local:8080 to the Github Webhook.

But we have two problems here:

a. The URL is only accessed within the cluster. In order to expose it to the localhost, we should perform a port-forward
b. Github cannot access our localhost, so we need to use some sort of "proxy" in the internet that creates a tunnel for our local computer on the port 8080.

First, let's perform the port-forward:

$ kubectl port-forward svc/el-github-pr 8080:8080 Forwarding from 127.0.0.1:8080 -> 8080 Forwarding from [::1]:8080 -> 8080
Enter fullscreen mode Exit fullscreen mode

Great, our host is receiving in the port 8080 (localhost:8080), but how do we tunnel Github to our port using some service in the cloud?

Ngrok solves that problem. With ngrok, we simply issue:

$ ngrok http http://localhost:8080
Enter fullscreen mode Exit fullscreen mode

And it outputs a public URL ready to be used across the internet, forwarding to our port 8080 in the localhost:

Add Single Sign-On to your ngrok dashboard via your Identity Provider: https://ngrok.com/dashSSO Session Status online Account Leandro Proença (Plan: Free) Version 3.1.1 Region South America (sa) Latency 23ms Web Interface http://127.0.0.1:4040 Forwarding https://95d2-177-9-110-144.sa.ngrok.io -> http://localhost:8080 Connections ttl opn rt1 rt5 p50 p90 0 0 0.00 0.00 0.00 0.00
Enter fullscreen mode Exit fullscreen mode

Nice, look how we've got a random URL (in your case it will be different, okay?):

https://95d2-177-9-110-144.sa.ngrok.io
Enter fullscreen mode Exit fullscreen mode

🎉

2. Configuring the Github Webhook

In the repository settings, go to Webhooks:

Github WH

Fill the form with the following:

  • Payload URL: paste the URL from Ngrok
  • Content-Type: application/json
  • Secret: leave it empty for now
  • Let me select individual events: Pull Requests only

Webhook created, time to create some PR in the repository.
PR

As we can see next, the pipeline was triggered by the event! 🚀

dash

🍺 Good times! 🍺

It's all about tests!

At this moment, our pipeline only runs a simple ls command in the cloned repository. But we should do even more: style checks, security checks and automated tests.

Let's change the pipeline so it runs the unit tests in the repository.

apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: chespirito-pipeline spec: params: - name: repo-url type: string - name: revision type: string workspaces: - name: shared-data tasks: - name: fetch-source taskRef: name: git-clone workspaces: - name: output workspace: shared-data params: - name: url value: $(params.repo-url) - name: revision value: $(params.revision) - name: run-tests runAfter: ["fetch-source"] taskRef: name: run-tests workspaces: - name: source workspace: shared-data --- apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: run-tests spec: workspaces: - name: source steps: - name: unit-tests image: ubuntu workingDir: $(workspaces.source.path) script: | #!/usr/bin/env bash docker-compose run --rm ruby bundle install docker-compose run --rm ruby ruby -Itest test/all.rb
Enter fullscreen mode Exit fullscreen mode

We just created a new pipeline called chespirito-pipeline, which uses 2 tasks: git-clone and run-tests.

The run-tests Task comprises of an Ubuntu image, so all we need is to use Docker to run the tests inside the container.

A note about running docker commands on CI

However, the Ubuntu Docker official image does not provide a Docker Runtime, then we need to run some kind of "Docker image" in order to have it ready to run.

Almost every CI environment in the cloud (including ours running on a local Kubernetes cluster) is already running jobs/tasks in Docker containers.

Eack Task is a container, so how can we run Docker in Docker?

Docker in Docker

Luckily, the Docker Hub community provides a Docker image for running Docker containers inside Docker.

It's called Docker-in-Docker, or dind.

For a Kubernetes cluster, there's a minimal setup needed to make it work. We should mount some specific volumes and export some environment variables so the container will use the correct Docker Runtime in the cluster.

We can use an architectural pattern called Sidecar Pattern, which is basically an initContainer that uses the dind image and mounts volumes so other containers in the same Pod can use the Docker Runtime.

In Tekton, it's no different, and the resource is actually called Sidecar, similar to Step within a Task but it runs before the steps.

Changing the Task

Because of separation of concerns, the only component we have to change is the Task (thanks to how Kubernetes employ important architectural patterns).

apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: run-tests spec: workspaces: - name: source steps: - name: unit-tests image: docker workingDir: $(workspaces.source.path) script: | docker-compose run --rm ruby bundle install docker-compose run --rm ruby ruby -Itest test/all.rb
Enter fullscreen mode Exit fullscreen mode

The image used will be docker, because we'll run Docker commands for running the unit tests.

Now, we add to this step, the env variables that Docker will use to connect to the runtime:

... env: - name: DOCKER_HOST value: tcp://localhost:2376 - name: DOCKER_TLS_VERIFY value: "1" - name: DOCKER_CERT_PATH value: "/certs/client"
Enter fullscreen mode Exit fullscreen mode

Next, declare the volumeMounts that will be populated by the sidecar:

... volumeMounts: - mountPath: /certs/client name: dind-certs
Enter fullscreen mode Exit fullscreen mode

Step defined, let's declare the sidecar node, right after the steps section:

... sidecars: - name: dind image: docker:dind securityContext: privileged: true args: - --storage-driver=vfs - --userland-proxy=false - --debug env: - name: DOCKER_TLS_CERTDIR value: /certs volumeMounts: - mountPath: /certs/client name: dind-certs - mountPath: $(workspaces.source.path) name: $(workspaces.source.volume) readinessProbe: periodSeconds: 1 exec: command: ['ls', '/certs/client/ca.pem']
Enter fullscreen mode Exit fullscreen mode

The sidecar uses the docker:dind image, starts it and then mounts the /certs in the volume shared with the step.

Also, it's important to note that the sidecar needs to mount the workspace as a Volume, since the Docker run will happen in the sidecar.

Finally, we declare the volumes section, which can be an emptyDir since it's used by steps/sidecars within the same Task:

... volumes: - name: dind-certs emptyDir: {}
Enter fullscreen mode Exit fullscreen mode

Pipeline applied, time to create the PipelineRun:

apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: generateName: chespirito-pipeline- spec: pipelineRef: name: chespirito-pipeline params: - name: repo-url value: https://github.com/leandronsp/chespirito.git - name: revision value: main workspaces: - name: shared-data volumeClaimTemplate: spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi
Enter fullscreen mode Exit fullscreen mode

🎉 May we run it and everything is green!

The last piece

Okay, we've run the new chespirito-pipeline manually. But now we have to change the EventListener, and the only thing needed is:

... pipelineRef: name: chespirito-pipeline
Enter fullscreen mode Exit fullscreen mode

Apply some changes in the branch, push to the PR and...

Wonderful

🎉 How wonderful is that? 🚀

In this article we learned about event listeners, triggers, Github webhooks and how to integrate all those stuff so we can run the unit tests at some arbitrary project in Github!

Keep in tune, later we'll see how to listen to the "push to main" event and rollout our application in the Kubernetes cluster, which is the "CD part".

Cheers! 🍺

Top comments (0)