Skip to content

Add support for deploying without using Kubernetes secrets #752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 8, 2025
Merged

Add support for deploying without using Kubernetes secrets #752

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7348de2
Add support for deploying without using Kubernetes secrets, and to di…
marcleblanc2 Oct 2, 2025
cfdc6e3
Adding validation
marcleblanc2 Oct 3, 2025
776f4ac
Remove duplicate configs, just fall back to existing default configs,…
marcleblanc2 Oct 3, 2025
1654d57
Provide the option to not define REDIS_CACHE_ENDPOINT or REDIS_STORE_…
marcleblanc2 Oct 3, 2025
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion charts/sourcegraph/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,8 +249,9 @@ In addition to the documented values, all services also support the following va
| preciseCodeIntel.resources | object | `{"limits":{"cpu":"2","memory":"4G"},"requests":{"cpu":"500m","memory":"2G"}}` | Resource requests & limits for the `precise-code-intel-worker` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
| preciseCodeIntel.serviceAccount.create | bool | `false` | Enable creation of ServiceAccount for `precise-code-intel-worker` |
| preciseCodeIntel.serviceAccount.name | string | `""` | Name of the ServiceAccount to be created or an existing ServiceAccount |
| priorityClasses | list | `[]` | Additional priorityClasses minimise re-scheduling downtime for StatefulSets. Each StatefulSets might use different priority class. learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass) Sample class definition: - name: gitserver-class value: 100 preemptionPolicy: Never description: "gitserver priority class" |
| priorityClasses | list | `[]` | Additional priorityClasses minimize re-scheduling downtime for StatefulSets. Each StatefulSets might use different priority class. learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass) Sample class definition: - name: gitserver-class value: 100 preemptionPolicy: Never description: "gitserver priority class" |
| prometheus.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":false,"runAsGroup":100,"runAsUser":100}` | Security context for the `prometheus` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
| prometheus.createRoleBinding | bool | `true` | Disable the creation of a RoleBinding object, for customers who block all RBAC resource creation |
| prometheus.enabled | bool | `true` | Enable `prometheus` (recommended) |
| prometheus.existingConfig | string | `""` | Name of existing ConfigMap for `pgsql`. It must contain a `prometheus.yml` key |
| prometheus.image.defaultTag | string | `"6.0.0@sha256:86a315720fd9813d9ef9746d92e637bc20cd9ebd90da78d8cc6906062252891f"` | Docker image tag for the `prometheus` image |
Expand Down Expand Up @@ -302,6 +303,7 @@ In addition to the documented values, all services also support the following va
| searcher.storageSize | string | `"26Gi"` | Size of the PVC for searcher pods to store cache data |
| sgTestConnection | object | `{"enabled":true}` | Enable the busybox connection test after deployment |
| sourcegraph.affinity | object | `{}` | Global Affinity, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) |
| sourcegraph.disableKubernetesSecrets | bool | `false` | Disable the creation of Kubernetes secrets objects |
| sourcegraph.image.defaultTag | string | `"{{ .Chart.AppVersion }}"` | Global docker image tag |
| sourcegraph.image.pullPolicy | string | `"IfNotPresent"` | Global docker image pull policy |
| sourcegraph.image.repository | string | `"index.docker.io/sourcegraph"` | Global docker image registry or prefix |
Expand All @@ -313,6 +315,8 @@ In addition to the documented values, all services also support the following va
| sourcegraph.nodeSelector | object | `{}` | Global NodeSelector, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector) |
| sourcegraph.podAnnotations | object | `{}` | Add extra annotations to attach to all pods |
| sourcegraph.podLabels | object | `{}` | Add extra labels to attach to all pods |
| sourcegraph.redisCacheEndpoint | string | `""` | Set the value of the REDIS_CACHE_ENDPOINT environment variable on the needed containers, when Kubernetes secrets are disabled |
| sourcegraph.redisStoreEndpoint | string | `""` | Set the value of the REDIS_STORE_ENDPOINT environment variable on the needed containers, when Kubernetes secrets are disabled |
| sourcegraph.revisionHistoryLimit | int | `10` | Global deployment clean up policy, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#clean-up-policy) |
| sourcegraph.serviceLabels | object | `{}` | Add extra labels to all services |
| sourcegraph.tolerations | list | `[]` | Global Tolerations, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) |
Expand Down
11 changes: 9 additions & 2 deletions charts/sourcegraph/templates/_helpers.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ For top-level services, pass in the top-level values:
{{ include "sourcegraph.serviceAccountName" (list . "frontend") }}

frontend:
serivceAccount:
serviceAccount:
create: false

For nested services, pass in the nested values:
Expand Down Expand Up @@ -249,6 +249,12 @@ app.kubernetes.io/name: jaeger
{{- end }}

{{- define "sourcegraph.redisConnection" -}}
{{- if .Values.sourcegraph.disableKubernetesSecrets -}}
- name: REDIS_CACHE_ENDPOINT
value: {{ .Values.sourcegraph.redisCacheEndpoint }}
- name: REDIS_STORE_ENDPOINT
value: {{ .Values.sourcegraph.redisStoreEndpoint }}
{{- else -}}
Comment on lines 260 to 269
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine, but there's a case where someone disables secrets and forgets to set this, it's just going to return empty vars and fail at runtime.
We might want to add some validation here, possible something like:

{{- define "sourcegraph.redisConnection" -}} {{- if .Values.sourcegraph.disableKubernetesSecrets -}} - name: REDIS_CACHE_ENDPOINT value: {{ required "sourcegraph.redisCacheEndpoint is required when sourcegraph.disableKubernetesSecrets is true" .Values.sourcegraph.redisCacheEndpoint }} - name: REDIS_STORE_ENDPOINT value: {{ required "sourcegraph.redisStoreEndpoint is required when sourcegraph.disableKubernetesSecrets is true" .Values.sourcegraph.redisStoreEndpoint }} {{- else -}} - name: REDIS_CACHE_ENDPOINT valueFrom: secretKeyRef: key: endpoint name: {{ default .Values.redisCache.name .Values.redisCache.connection.existingSecret }} - name: REDIS_STORE_ENDPOINT valueFrom: secretKeyRef: key: endpoint name: {{ default .Values.redisStore.name .Values.redisStore.connection.existingSecret }} {{- end -}} {{- end -}}
Copy link
Contributor Author

@marcleblanc2 marcleblanc2 Oct 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great call out, I had considered how to approach this... if they're externalizing Redis, then this endpoint value would likely include credentials, which they'd want to define as REDIS_CACHE_ENDPOINT and REDIS_STORE_ENDPOINT env vars on the needed pods, using the same method that they're using to side-load the Postgres credentials as env vars. So we end up with 3 separate config methods for Redis endpoints, the same as Postgres credentials:

  1. Kubernetes secrets (default)

  2. sourcegraph.redisCacheEndpoint / sourcegraph.redisStoreEndpoint values in the override file

  3. Inject the REDIS_CACHE_ENDPOINT and REDIS_STORE_ENDPOINT env vars on the needed pods

In this case, with the override file shared in Slack, this customer is using our Redis pods, so option 2 works for them, but if they decide to externalize Redis, then they'd have to switch to option 3.
Copy link
Contributor Author

@marcleblanc2 marcleblanc2 Oct 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a cleaner way to do this, re-using existing configs in the Helm chart:

.Values.redisCache.connection.endpoint .Values.redisStore.connection.endpoint

When the customer sets .Values.sourcegraph.disableKubernetesSecrets: true, then these same, existing configs, already with the correct defaults, get fed in directly as env vars, instead of first getting fed into the creation of secret objects, just to be read back in as env vars from the secrets.

Then, if they want to define these env vars externally, ex. external redis with creds in the endpoint string, then they can set the two endpoint values to "".
- name: REDIS_CACHE_ENDPOINT
valueFrom:
secretKeyRef:
Expand All @@ -259,7 +265,8 @@ app.kubernetes.io/name: jaeger
secretKeyRef:
key: endpoint
name: {{ default .Values.redisStore.name .Values.redisStore.connection.existingSecret }}
{{- end }}
{{- end -}}
{{- end -}}

{{- define "sourcegraph.authChecksum" -}}
{{- $checksum := list .Values.codeInsightsDB.auth -}}
Expand Down
2 changes: 1 addition & 1 deletion charts/sourcegraph/templates/codeinsights-db/codeinsights-db.Secret.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if not .Values.codeInsightsDB.auth.existingSecret }}
{{- if and (not .Values.sourcegraph.disableKubernetesSecrets) (not .Values.codeInsightsDB.auth.existingSecret) -}}
apiVersion: v1
kind: Secret
metadata:
Expand Down
2 changes: 1 addition & 1 deletion charts/sourcegraph/templates/codeintel-db/codeintel-db.Secret.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if not .Values.codeIntelDB.auth.existingSecret }}
{{- if and (not .Values.sourcegraph.disableKubernetesSecrets) (not .Values.codeIntelDB.auth.existingSecret) -}}
apiVersion: v1
kind: Secret
metadata:
Expand Down
10 changes: 9 additions & 1 deletion charts/sourcegraph/templates/frontend/sourcegraph-frontend.Deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,12 @@ spec:
- name: migrator
image: {{ include "sourcegraph.image" (list . "migrator") }}
imagePullPolicy: {{ .Values.sourcegraph.image.pullPolicy }}
{{- if .Values.migrator.command }}
command: {{ .Values.migrator.command }}
{{- end }}
args: {{- default (list "up") .Values.migrator.args | toYaml | nindent 8 }}
env:
{{- if not .Values.migrator.databaseAuthOverrideEnvVars }}
{{- if and (not .Values.migrator.databaseAuthOverrideEnvVars) (not .Values.sourcegraph.disableKubernetesSecrets) }}
{{- include "sourcegraph.databaseAuth" (list . "pgsql" "PG") | nindent 8 }}
{{- include "sourcegraph.databaseAuth" (list . "codeIntelDB" "CODEINTEL_PG") | nindent 8 }}
{{- include "sourcegraph.databaseAuth" (list . "codeInsightsDB" "CODEINSIGHTS_PG") | nindent 8 }}
Expand All @@ -78,11 +81,16 @@ spec:
- name: frontend
image: {{ include "sourcegraph.image" (list . "frontend") }}
imagePullPolicy: {{ .Values.sourcegraph.image.pullPolicy }}
{{- if .Values.frontend.command }}
command: {{ .Values.frontend.command }}
{{- end }}
args: {{- default (list "serve") .Values.frontend.args | toYaml | nindent 8 }}
env:
{{- if not .Values.sourcegraph.disableKubernetesSecrets }}
{{- include "sourcegraph.databaseAuth" (list . "pgsql" "PG") | nindent 8 }}
{{- include "sourcegraph.databaseAuth" (list . "codeIntelDB" "CODEINTEL_PG") | nindent 8 }}
{{- include "sourcegraph.databaseAuth" (list . "codeInsightsDB" "CODEINSIGHTS_PG") | nindent 8 }}
{{- end }}
{{- range $name, $item := .Values.frontend.env}}
- name: {{ $name }}
{{- $item | toYaml | nindent 10 }}
Expand Down
2 changes: 2 additions & 0 deletions charts/sourcegraph/templates/grafana/grafana.StatefulSet.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,9 @@ spec:
{{- end }}
terminationMessagePolicy: FallbackToLogsOnError
env:
{{- if not .Values.sourcegraph.disableKubernetesSecrets -}}
{{- include "sourcegraph.databaseAuth" (list . "grafana" "GRAFANA_PGSQL_") | nindent 8 }}
{{- end -}}
{{- range $name, $item := .Values.grafana.env}}
- name: {{ $name }}
{{- $item | toYaml | nindent 10 }}
Expand Down
2 changes: 1 addition & 1 deletion charts/sourcegraph/templates/grafana/grafana.pgsql.Secret.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if and .Values.grafana.auth (not .Values.grafana.auth.existingSecret) }}
{{- if and (not .Values.sourcegraph.disableKubernetesSecrets) .Values.grafana.auth (not .Values.grafana.auth.existingSecret) -}}
apiVersion: v1
kind: Secret
metadata:
Expand Down
2 changes: 1 addition & 1 deletion charts/sourcegraph/templates/pgsql/pgsql.Secret.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if not .Values.pgsql.auth.existingSecret }}
{{- if and (not .Values.sourcegraph.disableKubernetesSecrets) (not .Values.pgsql.auth.existingSecret) -}}
apiVersion: v1
kind: Secret
metadata:
Expand Down
2 changes: 1 addition & 1 deletion charts/sourcegraph/templates/prometheus/prometheus.ClusterRoleBinding.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if and .Values.prometheus.enabled .Values.prometheus.privileged -}}
{{- if and .Values.prometheus.enabled .Values.prometheus.privileged .Values.prometheus.createRoleBinding -}}
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
Expand Down
2 changes: 1 addition & 1 deletion charts/sourcegraph/templates/prometheus/prometheus.RoleBinding.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if and .Values.prometheus.enabled (not .Values.prometheus.privileged) -}}
{{- if and .Values.prometheus.enabled (not .Values.prometheus.privileged) .Values.prometheus.createRoleBinding -}}
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
Expand Down
2 changes: 1 addition & 1 deletion charts/sourcegraph/templates/redis/redis-cache.Secret.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if not .Values.redisCache.connection.existingSecret }}
{{- if and (not .Values.sourcegraph.disableKubernetesSecrets) (not .Values.redisCache.connection.existingSecret) -}}
apiVersion: v1
kind: Secret
metadata:
Expand Down
2 changes: 1 addition & 1 deletion charts/sourcegraph/templates/redis/redis-store.Secret.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if not .Values.redisStore.connection.existingSecret }}
{{- if and (not .Values.sourcegraph.disableKubernetesSecrets) (not .Values.redisStore.connection.existingSecret) -}}
apiVersion: v1
kind: Secret
metadata:
Expand Down
10 changes: 9 additions & 1 deletion charts/sourcegraph/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@ sourcegraph:
revisionHistoryLimit: 10
# -- Add extra labels to all services
serviceLabels: {}
# -- Disable the creation of Kubernetes secrets objects
disableKubernetesSecrets: false
# -- Set the value of the REDIS_CACHE_ENDPOINT environment variable on the needed containers, when Kubernetes secrets are disabled
redisCacheEndpoint: ""
# -- Set the value of the REDIS_STORE_ENDPOINT environment variable on the needed containers, when Kubernetes secrets are disabled
redisStoreEndpoint: ""

# Generic application configuration options, used by most applications below
# app: # Generally matches directory name
Expand Down Expand Up @@ -893,6 +899,8 @@ prometheus:
name: "prometheus"
# -- Enable RBAC for `prometheus`
privileged: true
# -- Disable the creation of a RoleBinding object, for customers who block all RBAC resource creation
createRoleBinding: true
# -- Resource requests & limits for the `prometheus` container,
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)
# Prometheus is relied upon to monitor services for sending alerts to site admins when
Expand Down Expand Up @@ -1258,7 +1266,7 @@ worker:
# -- Additional resources to include in the rendered manifest. Templates are supported.
extraResources: []

# -- Additional priorityClasses minimise re-scheduling downtime for StatefulSets. Each StatefulSets might use different priority class.
# -- Additional priorityClasses minimize re-scheduling downtime for StatefulSets. Each StatefulSets might use different priority class.
# learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass)
# Sample class definition:
# - name: gitserver-class
Expand Down
Loading