private docker registry for gitlab secret not working in kubernetes - docker

Question

I am trying to setup a gitlab private registry for my kubernetes container images.
I've cut the irrelevant code out below.
My replica set is defined as:
kind: ReplicaSet
...
spec:
containers:
- name: redacted
image: registry.gitlab.com/redacted/redacted/redacted:latest
ports:
- containerPort: 8080
volumeMounts:
- name: redacted-data
mountPath: /var/www/html
imagePullSecrets:
- name: github-auth
...
I'm setting my secret with the following kubectl command:
kubectl create -n redacted secret docker-registry gitlab-auth \
--docker-server="registry.gitlab.com:5000" \
--docker-username="redacted" \
--docker-password="redacted" \
--docker-email="redacted" \
--namespace="redacted"
Here is the failing container output:
Name: redacted-cgbrk
...
Containers:
redacted:
Container ID:
Image: registry.gitlab.com/redacted/redacted/redacted:latest
Image ID:
Port: 8080/TCP
Host Port: 0/TCP
State: Waiting
Reason: ErrImagePull
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qv24l (ro)
/var/www/html from redacted-data (rw)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 64s default-scheduler 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
Normal Scheduled 62s default-scheduler Successfully assigned redacted/redacted-cgbrk to pool-2t9lbcb5l-7d37n
Normal SuccessfulAttachVolume 55s attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-6c4aac85-bb60-44e8-b557-7f65d62543fa"
Normal Pulling 16s (x3 over 54s) kubelet Pulling image "registry.gitlab.com/redacted/mpro/redacted:latest"
Warning Failed 16s (x3 over 54s) kubelet Failed to pull image "registry.gitlab.com/redacted/redacted/redacted:latest": rpc error: code = Unknown desc = failed to pull and unpack image "registry.gitlab.com/redacted/redacted/redacted:latest": failed to resolve reference "registry.gitlab.com/redacted/redacted/redacted:latest": failed to authorize: failed to fetch anonymous token: unexpected status: 403 Forbidden
...

Answer 1

Kubernetes uses a separate auth than docker login, Check you may have configured Kubernetes with the required authentication so it can pull from your private registries.
Follow below steps :
1)Log in to Docker Hub
2)Create a Secret based on existing credentials
3)Create a Secret by providing credentials on the command line
4)Inspecting the Secret regcred
5)Create a Pod that uses your Secret
Please see K8S issue here: Pull an Image from a Private Registry for more information.
Also Refer to this Similar SO for more information.