Skip to content

seatgeek/buildkit-operator

buildkit-operator

go.mod LICENSE Build Status Go Report Card Codecov

An operator for managing BuildKit instances on Kubernetes.

How it Works

First, deploy one or more BuildkitTemplate resources that define the configuration and scheduling for BuildKit instances:

apiVersion: buildkit.seatgeek.io/v1alpha1 kind: BuildkitTemplate metadata: name: buildkit-arm64 namespace: some-ns spec: # This is a simplified example; many other spec fields are available. rootless: true port: 1234 buildkitdToml: |  [log]  format = "json"   [worker.oci]  enabled = true  max-parallelism = 3  cniPoolSize = 16   [worker.containerd]  enabled = false   resources: default: requests: cpu: 500m memory: 8Gi limits: memory: 8Gi maximum: cpu: 8 memory: 16Gi scheduling: nodeSelector: kubernetes.io/arch: arm64 kubernetes.io/os: linux tolerations: - key: dedicated operator: Equal value: buildkit effect: NoSchedule

Then create any number of Buildkit resources that reference the templates:

apiVersion: buildkit.seatgeek.io/v1alpha1 kind: Buildkit metadata: name: buildkit-arm64-instance namespace: some-ns spec: template: buildkit-arm64

The operator will then deploy a BuildKit pod for each Buildkit resource, setting the TCP connection URL into the Buildkit resource's status:

# ... status: endpoint: tcp://10.1.2.3:1234

Use the .status.endpoint field to connect to the BuildKit instance. When you're done, delete the Buildkit resource and the associated pod will be cleaned up automatically.

Installation

Helm Chart (Recommended)

The easiest way to install the buildkit-operator is using the Helm chart:

# Install the latest stable release helm install buildkit-operator \ oci://ghcr.io/seatgeek/charts/buildkit-operator \ --namespace buildkit-system \ --create-namespace

Installation Options

Install latest from main branch (bleeding edge):

# Note: Chart versions for main branch use format 0.0.0-main-<commit-sha> # You can find the exact version in GitHub Actions or use --devel flag helm install buildkit-operator \ oci://ghcr.io/seatgeek/charts/buildkit-operator \ --namespace buildkit-system \ --create-namespace \ --set image.tag=main \ --devel

Install a specific version:

helm install buildkit-operator \ oci://ghcr.io/seatgeek/charts/buildkit-operator \ --version 1.0.0 \ --namespace buildkit-system \ --create-namespace

Test a pull request:

helm install buildkit-operator \ oci://ghcr.io/seatgeek/charts/buildkit-operator \ --namespace buildkit-system \ --create-namespace \ --set image.tag=pr-123

Container Images

Container images are available at:

  • ghcr.io/seatgeek/buildkit-operator:latest - Latest stable release (tagged releases only)
  • ghcr.io/seatgeek/buildkit-operator:main - Latest from main branch
  • ghcr.io/seatgeek/buildkit-operator:v1.0.0 - Specific version releases
  • ghcr.io/seatgeek/buildkit-operator:pr-123 - Pull request builds

All images support both linux/amd64 and linux/arm64 architectures.

Note: Development versions from the main branch use commit-based chart versions (e.g., 0.0.0-main-abc12345) for better traceability and correlation with the source code.

Uninstallation

helm uninstall buildkit-operator --namespace buildkit-system

Local Development

Prerequisites

You will need kind and frpc, which can be installed on macOS with brew:

brew install kind frpc

Running the Operator

(Re)start your local kind cluster and run the operator:

make recreate make run

Debugging the Operator

If you'd rather run the operator from your IDE with a debugger attached, run these commands instead:

make recreate make start_webhook_reverse_proxy # Keep this running in the background until you're done debugging

Then have your IDE run ./cmd/operator in debug mode with the following program arguments set: --kubeconfig ./kind/kubeconfig --kubecontext kind-buildkit

Interacting with the Cluster

You can interact with the cluster either of these ways:

  • kubectl --kubeconfig ./kind/kubeconfig [command]
  • k9s --kubeconfig ./kind/kubeconfig --write

Congratulations! You are now running buildkit-operator locally.