Gitlab DOCKER_AUTH_CONFIG not working - docker

Question

The following are in my .gitlab-ci.yml
stages:
- build
variables:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
services:
- docker:dind
build-image:
image: docker:stable
stage: build
script:
- docker build --no-cache -t repo/myimage:$CI_JOB_ID .
- docker push repo/myimage:$CI_JOB_ID
I've setup the DOCKER_AUTH_CONFIG in Gitlab like following (to contain all possibilities of matching)
{
"auths": {
"https://index.docker.io": {
"auth": "...."
},
"https://index.docker.io/v1/": {
"auth": "..."
},
"https://index.docker.io/v2/": {
"auth": "..."
},
"index.docker.io/v1/": {
"auth": "..."
},
"index.docker.io/v2/": {
"auth": "..."
},
"docker.io/repo/myimage": {
"auth": "..."
}
}
}
However, whenever trying to push the image, the following error occurred
$ docker push repo/myimage:$CI_JOB_ID
The push refers to repository [docker.io/repo/myimage]
ce6466f43b11: Preparing
719d45669b35: Preparing
3b10514a95be: Preparing
63dcf81c7ca7: Waiting
3b10514a95be: Waiting
denied: requested access to the resource is denied
ERROR: Job failed: exit code 1
It worked when I use docker login with username/password. Anyone please show me what I did wrong to get it to work with DOCKER_AUTH_CONFIG?
Thanks heaps
Regards
Tin

Answer 1

To use the content of DOCKER_AUTH_CONFIG as docker login, just store it in $HOME/.docker/config.json, e.g. as follows:
before_script:
- mkdir -p $HOME/.docker
- echo $DOCKER_AUTH_CONFIG > $HOME/.docker/config.json
Ref: https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#option-3-use-docker_auth_config
This allows to use a single config to load images for build containers and to access the registry inside the build from the same configuration source.
note: this replaces an execution of docker login
see also: https://docs.docker.com/engine/reference/commandline/login/#privileged-user-requirement

Answer 2

DOCKER_AUTH_CONFIG works when you are trying to pull the image from your private repository. Here is the function that uses that config variable. That function is only used by getDockerImage function.
So whenever you need to push your image inside your job's script section, you need the docker login step before that.

Answer 3

The documenation describing DOCKER_AUTH_CONFIG doesn't show any example with several credentials. The documented syntax is:
{
"auths":{
"registry.hub.docker.com":{
"auth":"xxxxxxxxxxxxxxxxxxxxxxxxxxxx" // base 64 encoded username:password
}
}
}
Still, as you said, you can use before_script at the beginning of the gitlab-ci.yml file or inside each job if you need several authentifications:
before_script:
- echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin
Where $CI_REGISTRY_USER and CI_REGISTRY_PASSWORD would be secret variables.
And after each script or at the beginning of the whole file:
after_script:
- docker logout
I wrote an answer about using Gitlab CI and Docker to build docker images :
How to build, push and pull multiple docker containers with gitlab ci?
Using --password-stdin and secrets instead of a plain -p <password> is a better alternative.
EDIT: The example syntax in mypost is taken from this awesome answer from #Ruwanka Madhushan Can't Access Private MySQL Docker Image From Gitlab CI. You should go see for yourself
SECOND EDIT: You should protect your secret variable only if you want to make them available for protected branches or tags. If you didn't setup any protected brnach or tag, do not use protected variables.
From the doc: Variables could be protected. Whenever a variable is protected, it would only be securely passed to pipelines running on the protected branches or protected tags. The other pipelines would not get any protected variables.
https://docs.gitlab.com/ee/ci/variables/#protected-variables

Answer 4

In my case the issue was following these docs blindly
https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#define-an-image-from-a-private-container-registry
They tell you to do the following if you need to manually generate the token:
# The use of "-n" - prevents encoding a newline in the password.
echo -n "my_username:my_password" | base64
# Example output to copy
bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=
My password had spaces in so...
# Correct encoding
> echo "username:password with spaces in it" | base64
dXNlcm5hbWU6cGFzc3dvcmQgd2l0aCBzcGFjZXMgaW4gaXQK
# Encoding as in docs
> echo -n "username:password with spaces in it" | base64
dXNlcm5hbWU6cGFzc3dvcmQgd2l0aCBzcGFjZXMgaW4gaXQ=

Answer 5

If you, like us, have a lot of pipelines and don't want to edit all gitlab ci configs everywhere, you can also configure this once per runner.
In /etc/gitlab-runner/config.toml add a pre_build_script:
[[runners]]
environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"https://index.docker.io/v1/\":{\"auth\":\"YOUR TOKEN\"}}}"]
pre_build_script = "mkdir ~/.docker -p && echo $DOCKER_AUTH_CONFIG > ~/.docker/config.json"
A little more information can be found in Gitlab's docs.

Answer 6

Since I wanted to access my Gitlab container registry from a Gitlab pipeline, I really wanted to use the $CI_JOB_TOKEN. I've achieved this in a relatively clean and clear manner I think.
I've defined the following variables:
variables:
DOCKER_AUTH: echo -n "gitlab-ci-token:$CI_JOB_TOKEN" | base64
DOCKER_AUTH_CONFIG: echo {\"auths\":{\"gitlab.container.registry.url\":{\"auth\":\"$(eval $DOCKER_AUTH)\"}}}
And the following before_script which evaluates the above mentioned variables and creates the docker config.json
before_script:
- mkdir -p $HOME/.docker
- eval $DOCKER_AUTH_CONFIG > $HOME/.docker/config.json