I have an instance of MicroK8s, I am currently migrating my applications. But I can't authenticate to Gitlab's private registry.
About my environment:
- MicroK8s
- Gitlab (with registry working securely)
Tests I performed
To test my private Gitlab registry I used another machine and did a docker pull registry.mydomain.com/my-group/my-image:my-tag.
In K8s I followed the instructions available on the K8s website and also the instructions in the answer to this question on SO.
I've already checked the namespaces, the secret was created correctly! I restored the credentials to confirm and they are correct.
Different ways I used to create the secret for imagePullSecrets.
First try
kubectl create secret docker-registry regcred \ --namespace=my-namespace \ --docker-server=registry.mydomain.com \ --docker-username="my-user" \ --docker-email="my-mail" \ --docker-password="my-gitlab-personal-access-token" Second attempt
kubectl create secret generic regcred \ --namespace=my-namespace \ --from-file=.dockerconfigjson=/home/hsouza/.docker/config.json \ --type=kubernetes.io/dockerconfigjson Both ways of creating my secret work, however when creating my pod the ImagePullBackOff error is presented.
For both attempts to generate the regcred I created the POD like this:
apiVersion: v1 kind: Pod metadata: namespace: my-namespace name: test-private-register-pod spec: containers: - name: test-private-register-pod image: registry.mydomain.com/my-group/my-image:1.0.0 imagePullSecrets: - name: regcred When analyzing with kubectl describe pod/test-private-register-pod the following message is returned:
Name: test-private-register-pod Namespace: my-namespace Priority: 0 Service Account: default Node: k8s-dev/10.0.0.6 Start Time: Sat, 07 Oct 2023 11:25:23 -0300 Labels: <none> Annotations: cni.projectcalico.org/containerID: 0d1a8c2d3f54ea9dfeccb69b96f930a6c5f40d6c9fa8c16994ac24676cecb5be cni.projectcalico.org/podIP: 10.1.252.205/32 cni.projectcalico.org/podIPs: 10.1.252.205/32 Status: Pending IP: 10.1.252.205 IPs: IP: 10.1.252.205 Containers: test-private-register-pod: Container ID: Image: registry.mydomain.com/my-group/my-image:1.0.0 Image ID: Port: <none> Host Port: <none> State: Waiting Reason: ImagePullBackOff Ready: False Restart Count: 0 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8gj47 (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: kube-api-access-8gj47: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 16s default-scheduler Successfully assigned my-namespace/test-private-register-pod to k8s-dev Normal Pulling 15s kubelet Pulling image "registry.mydomain.com/my-group/my-image:1.0.0" Warning Failed 15s kubelet Failed to pull image "registry.mydomain.com/my-group/my-image:1.0.0": rpc error: code = Unknown desc = failed to pull and unpack image "registry.mydomain.com/my-group/my-image:1.0.0": failed to resolve reference "registry.mydomain.com/my-group/my-image:1.0.0": failed to authorize: failed to fetch oauth token: unexpected status: 403 Forbidden Warning Failed 15s kubelet Error: ErrImagePull Normal BackOff 14s (x2 over 15s) kubelet Back-off pulling image "registry.mydomain.com/my-group/my-image:1.0.0" Warning Failed 14s (x2 over 15s) kubelet Error: ImagePullBackOff In the gitlab nginx logs, the status code for K8s access is 401. I put it in a fictitious way to try to pull from https://requestcatcher.com/ and when analyzing the request made by K8s it does not send the header with Basic Auth .
This is my problem, I appreciate the community's help in trying to identify where I'm going wrong.
