An operator for managing BuildKit instances on Kubernetes.
First, deploy one or more BuildkitTemplate
resources that define the configuration and scheduling for BuildKit instances:
apiVersion: buildkit.seatgeek.io/v1alpha1 kind: BuildkitTemplate metadata: name: buildkit-arm64 namespace: some-ns spec: # This is a simplified example; many other spec fields are available. rootless: true port: 1234 buildkitdToml: | [log] format = "json" [worker.oci] enabled = true max-parallelism = 3 cniPoolSize = 16 [worker.containerd] enabled = false resources: default: requests: cpu: 500m memory: 8Gi limits: memory: 8Gi maximum: cpu: 8 memory: 16Gi scheduling: nodeSelector: kubernetes.io/arch: arm64 kubernetes.io/os: linux tolerations: - key: dedicated operator: Equal value: buildkit effect: NoSchedule
Then create any number of Buildkit
resources that reference the templates:
apiVersion: buildkit.seatgeek.io/v1alpha1 kind: Buildkit metadata: name: buildkit-arm64-instance namespace: some-ns spec: template: buildkit-arm64
The operator will then deploy a BuildKit pod for each Buildkit
resource, setting the TCP connection URL into the Buildkit
resource's status
:
# ... status: endpoint: tcp://10.1.2.3:1234
Use the .status.endpoint
field to connect to the BuildKit instance. When you're done, delete the Buildkit
resource and the associated pod will be cleaned up automatically.
The easiest way to install the buildkit-operator is using the Helm chart:
# Install the latest stable release helm install buildkit-operator \ oci://ghcr.io/seatgeek/charts/buildkit-operator \ --namespace buildkit-system \ --create-namespace
Install latest from main branch (bleeding edge):
# Note: Chart versions for main branch use format 0.0.0-main-<commit-sha> # You can find the exact version in GitHub Actions or use --devel flag helm install buildkit-operator \ oci://ghcr.io/seatgeek/charts/buildkit-operator \ --namespace buildkit-system \ --create-namespace \ --set image.tag=main \ --devel
Install a specific version:
helm install buildkit-operator \ oci://ghcr.io/seatgeek/charts/buildkit-operator \ --version 1.0.0 \ --namespace buildkit-system \ --create-namespace
Test a pull request:
helm install buildkit-operator \ oci://ghcr.io/seatgeek/charts/buildkit-operator \ --namespace buildkit-system \ --create-namespace \ --set image.tag=pr-123
Container images are available at:
ghcr.io/seatgeek/buildkit-operator:latest
- Latest stable release (tagged releases only)ghcr.io/seatgeek/buildkit-operator:main
- Latest from main branchghcr.io/seatgeek/buildkit-operator:v1.0.0
- Specific version releasesghcr.io/seatgeek/buildkit-operator:pr-123
- Pull request builds
All images support both linux/amd64
and linux/arm64
architectures.
Note: Development versions from the main branch use commit-based chart versions (e.g., 0.0.0-main-abc12345
) for better traceability and correlation with the source code.
helm uninstall buildkit-operator --namespace buildkit-system
You will need kind
and frpc
, which can be installed on macOS with brew
:
brew install kind frpc
(Re)start your local kind cluster and run the operator:
make recreate make run
If you'd rather run the operator from your IDE with a debugger attached, run these commands instead:
make recreate make start_webhook_reverse_proxy # Keep this running in the background until you're done debugging
Then have your IDE run ./cmd/operator
in debug mode with the following program arguments set: --kubeconfig ./kind/kubeconfig --kubecontext kind-buildkit
You can interact with the cluster either of these ways:
kubectl --kubeconfig ./kind/kubeconfig [command]
k9s --kubeconfig ./kind/kubeconfig --write
Congratulations! You are now running buildkit-operator locally.