Cannot pull image from remote Gitlab registry to Kubernetes - docker

Question

I've been trying to create a deployment of docker image to Kubernetes cluster without luck, my deployment.yaml looks like:
apiVersion: v1
kind: Pod
metadata:
name: application-deployment
labels:
app: application
spec:
serviceAccountName: gitlab
automountServiceAccountToken: false
containers:
- name: application
image: example.org:port1/foo/bar:latest
ports:
- containerPort: port2
volumes:
- name: foo
secret:
secretName: regcred
But it fails to get the image.
Failed to pull image "example.org:port1/foo/bar:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://example.org:port1/v2/foo/bar/manifests/latest: denied: access forbidden
The secret used in deployment.yaml, was created like this:
kubectl create secret docker-registry regcred --docker-server=${CI_REGISTRY} --docker-username=${CI_REGISTRY_USER} --docker-password=${CI_REGISTRY_PASSWORD} --docker-email=${GITLAB_USER_EMAIL}
Attempt #1: adding imagePullSecrets
...
imagePullSecrets:
- name: regcred
results in:
Failed to pull image "example.org:port1/foo/bar:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://example.org:port1/v2/foo/bar/manifests/latest: unauthorized: HTTP Basic: Access denied
Solution:
I've created deploy token under Settings > Repository > Deploy Tokens > (created one with read_registry scope)
And added given values to environment variables and an appropriate line now looks like:
kubectl create secret docker-registry regcred --docker-server=${CI_REGISTRY} --docker-username=${CI_DEPLOY_USER} --docker-password=${CI_DEPLOY_PASSWORD}
I've got the problematic line from tutorials & Gitlab docs, where they've described deploy tokens but further used problematic line in examples.

Answer 1

I reproduced your issue and the problem is with password you used while creating a repository's secret. When creating a secret for gitlab repository you have to use personal token created in gitlab instead of a password.
You can create a token by going to Settings -> Access Tokens. Then you have to pick a name for your token, expiration date and token's scope.
Then create a secret as previously by running
kubectl create secret docker-registry regcred --docker-server=$docker_server --docker-username=$docker_username --docker-password=$personal_token
While creating a pod you have to include
imagePullSecrets:
- name: regcred

Answer 2

You need add the imagePullSecret on your deployment, so your pod will be:
apiVersion: v1
kind: Pod
metadata:
name: application-deployment
labels:
app: application
spec:
serviceAccountName: gitlab
automountServiceAccountToken: false
containers:
- name: application
image: example.org:port1/foo/bar:latest
ports:
- containerPort: port2
imagePullSecrets:
- name: regcred
Be sure that the secret and pod is running on same namespace.
Also make sure that the container you are pulling exist and with the right tag.
I notice you are trying to run the command on pipeline on gitlab-ci, check after run the create secret command that your secret is right (with the variables replacement).
You can verify if you can login to registry and pull the image manually on some other linux to by sure that the credentials are right.

Answer 3

creating a secret didn't work for me at first, though I had to specify the namespace for the secret and it worked.
kubectl delete secret -n ${NAMESPACE} regcred --ignore-not-found
kubectl create secret -n ${NAMESPACE} docker-registry regcred --docker-server=${CI_REGISTRY} --docker-username=${CI_DEPLOY_USERNAME} --docker-password=${CI_DEPLOY_PASSWORD} --docker-email=${GITLAB_USER_EMAIL}