DEV Community

Cover image for Sealed secrets - Bring your own keys and multi-cluster scenario
Ashok Nagaraj
Ashok Nagaraj

Posted on

Sealed secrets - Bring your own keys and multi-cluster scenario

#kubernetes #addons #secret

Bitnami sealed secrets have a key rotation done every 30 days by default and every sealed-secret-controller creates it's own private key. This makes it hard to adopt for multi-cluster scenarios. It can be mitigated by using one's own key pair for encryption/decryption across multiple clusters.

Using own key pair
  1. Preparation 
❯ kubectl create ns sealed-secrets ❯ export PRIVATEKEY="acmetls.key" ❯ export PUBLICKEY="acmetls.crt" ❯ export NAMESPACE="sealed-secrets" ❯ export SECRETNAME="acme-keys" # Create key pair ❯ openssl req -x509 -nodes -newkey rsa:4096 -keyout "$PRIVATEKEY" -out "$PUBLICKEY" -subj "/CN=sealed-secret/O=sealed-secret" Generating a 4096 bit RSA private key .....++ ......++ writing new private key to 'acmetls.key' ----- ❯ ls acme* acmetls.crt acmetls.key
Enter fullscreen mode Exit fullscreen mode
  1. Prepare and update new keys into controller 
❯ kubectl create ns sealed-secrets namespace/sealed-secrets created # Create secret and label it as "active" ❯ kubectl -n "$NAMESPACE" create secret tls "$SECRETNAME" --cert="$PUBLICKEY" --key="$PRIVATEKEY" kubectl -n "$NAMESPACE" label secret "$SECRETNAME" sealedsecrets.bitnami.com/sealed-secrets-key=active secret/acme-keys created secret/acme-keys labeled # Install the controller ❯ helm upgrade --install ss-app sealed-secrets/sealed-secrets --namespace=sealed-secrets # delete existing key ❯ kubectl get pods -n sealed-secrets NAME READY STATUS RESTARTS AGE ss-app-sealed-secrets-556c68c858-qq5k6 1/1 Running 0 3m47s ❯ kubectl delete pod ss-app-sealed-secrets-556c68c858-qq5k6 -n sealed-secrets pod "ss-app-sealed-secrets-556c68c858-qq5k6" deleted # another instance of the pod will come up with the new "acme-keys" ❯ kubectl logs ss-app-sealed-secrets-556c68c858-gqpwf -n sealed-secrets 2022/04/25 13:02:01 Starting sealed-secrets controller version: 0.17.4 controller version: 0.17.4 2022/04/25 13:02:01 Searching for existing private keys 2022/04/25 13:02:02 ----- sealed-secrets-key5nhjv 2022/04/25 13:02:02 ----- sealed-secrets-key98m56 2022/04/25 13:02:02 ----- sealed-secrets-keyztj4v 2022/04/25 13:02:02 ----- acme-keys 2022/04/25 13:02:02 HTTP server serving on :8080
Enter fullscreen mode Exit fullscreen mode
  1. Encrypt with new keys 
❯ kubectl create secret generic secret-name --dry-run=client --from-literal=foo=bar -o yaml | kubeseal --controller-name=ss-app-sealed-secrets --controller-namespace=sealed-secrets -o yaml --cert=$PUBLICKEY | kubectl apply -f - sealedsecret.bitnami.com/secret-name created
Enter fullscreen mode Exit fullscreen mode
  1. Verify decryption 
❯ kubectl logs ss-app-sealed-secrets-556c68c858-gqpwf -n sealed-secrets | tail -2 2022/04/25 13:08:07 Updating default/secret-name 2022/04/25 13:08:07 Event(v1.ObjectReference{Kind:"SealedSecret", Namespace:"default", Name:"secret-name", UID:"c9ee00ac-c974-43f7-9b7d-16cda1b516ff", APIVersion:"bitnami.com/v1alpha1", ResourceVersion:"57973", FieldPath:""}): type: 'Normal' reason: 'Unsealed' SealedSecret unsealed successfully ❯ kubectl get sealedsecrets.bitnami.com NAME AGE secret-name 47s ❯ kubectl get secrets secret-name NAME TYPE DATA AGE secret-name Opaque 1 57s
Enter fullscreen mode Exit fullscreen mode
Toying with the secret rotation interval during creation of the sealed-secrets controller
  • Edit --key-renew-period=<value> in the command line args for the controller (in-case of deployment)
  • Use --set=keyrenewperiod=<value> while installing through Helm

Note: keyrenewperiod=0 stops keys from being rotated; though seemingly easy solution, not recommended in production use-cases

Top comments (0)