Gitlab Runner Image with GCP credentials - docker

Question

I am trying to teach my Gitlab Runner image to get custom builder images from my private Docker Registry (GCR running in the Google Cloud).
What did not work out?
I created a custom Gitlab Runner image with the ServiceAccount properly set. I started in in non-privileged mode but the wormhole pattern (via docker.sock). On exec-ing into that container (which is based on gitlab/gitlab-runner:v11.3.0) I had to recognise that I cannot do any docker commands in there (neither as root nor as gitlab-user). How the gitlab-runner starts the builder containers afterwards is way above my cognitive capabilities. ;)
# got started via eu.gcr.io/my-project/gitlab-runner:0.0.5 which got taught the GCR credentials
stages:
- build
build:
image: docker pull eu.gcr.io/my-project/gitlab-builder-docker:0.0.2
stage: build
script:
# only for test if I have access to private docker registry
- docker pull eu.gcr.io/my-project/gitlab-builder-docker:0.0.1
What worked out?
According to this tutorial you can authenticate via in a before_script block in your .gitlab-ci.yml files. That worked out.
# got started via gitlab/gitlab-runner:v11.3.0
stages:
- build
before_script:
- apk add --update curl python which bash
- curl -sSL https://sdk.cloud.google.com | bash
- export PATH="$PATH:/root/google-cloud-sdk/bin"
- gcloud components install docker-credential-gcr
- gcloud auth activate-service-account --key-file=/key.json
- gcloud auth configure-docker --quiet
build:
image: docker:18.03.1-ce
stage: build
# only for test if I have access to private docker registry
- docker pull eu.gcr.io/my-project/gitlab-builder-docker:0.0.1
The Question
This means that I have to do this (install gcloud & authenticate) in each build run - I would prefer to have done this in the gitlab-runner image. Do you have an idea how to achieve this?

Answer 1

Finally I found a way to get this done.
Teach the vanilla gitlab-runner how to pull from your private GCR Docker Repo
GCP
Create Service account with no permissions in IAM & Admin
Download Json Key
Add Permissions in Storage Browser
Select bucket holding your images (eg eu.artifacts.my-project.appspot.com)
Grant permission Storage Object Admin to the service account
Local Docker Container
Launch a library/docker container and exec into it (with Docker Wormhole Pattern docker.sock volume mount)
Login into GCR via (Check the url of your repo, in my case its located in Europe, therefore the eu prefix in the url)
docker login -u _json_key --password-stdin https://eu.gcr.io < /etc/gitlab-runner/<MY_KEY>.json
Verify if it works via some docker pull <MY_GCR_IMAGE>
Copy the content of ~/.docker/config.json
Gitlab config.toml configuration
Add the following into your config.toml file
[[runners]]
environment = ["DOCKER_AUTH_CONFIG={ \"auths\": { \"myregistryurl.com:port\": { \"auth\": \"<TOKEN-FROM-DOCKER-CONFIG-FILE>\" } } }"]
Vanilla Gitlab Runner Container
Run the runner eg like this
docker run -it \
--name gitlab-runner \
--rm \
-v <FOLDER-CONTAININNG-GITLAB-RUNNER-CONFIG-FILE>:/etc/gitlab-runner:ro \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:v11.3.0
Your .gitlab-ci.yml file
Verify the done work via a .gitlab-ci.yml
Use an image which is located in your private GCP Container Registry
Teach your builder images how to push to your private GCR Docker Repo
GCP
Add permissions to your service account
Grant permission Storage Legacy Bucket Reader to your service account in the Storage Browser
Custom Docker Builder Image
Add your Service Account key file to your your custom image
FROM docker:18.03.1-ce
ADD key.json /<MY_KEY>.json
Your .gitlab-ci.yml file
Add the following script into your before_script section
docker login -u _json_key --password-stdin https://eu.gcr.io < /key.json
Final Thoughts
Now the vanilla gitlab-runner can pull your custom images from your private GCR Docker Repo. Furthermore those pullable custom images are also capable of talking to your private GCR Docker Repo and eg push the resulting images of your build pipeline.
That was quite complicated stuff. Maybe Gitlab enhances the support for this usecase in the future.

Answer 2

This example config worked for me in values.yaml:
config: |
[[runners]]
[runners.docker]
image = "google/cloud-sdk:alpine"
[runners.kubernetes]
namespace = "{{.Release.Namespace}}"
image = "google/cloud-sdk:alpine"
[runners.cache]
Type = "gcs"
Path = "runner"
Shared = true
[runners.cache.gcs]
BucketName = "runners-cache"
[[runners.kubernetes.volumes.secret]]
name = "service-account-credentials"
mount_path = "keys"
read_only = true
Where service-account-credentials is a secret containing credentials.json
then in .gitlab-ci.yml you can do:
gcloud auth activate-service-account --key-file=/keys/credentials.json
Hope it helps

Answer 3

have you tried to use google cloudbuild?
i had the same problem and solved it like this:
echo ${GCR_AUTH_KEY} > key.json
gcloud auth activate-service-account --key-file key.json
gcloud auth configure-docker
gcloud builds submit . --config=cloudbuild.yaml --substitutions _CI_PROJECT_NAME=$CI_PROJECT_NAME,_CI_COMMIT_TAG=${CI_COMMIT_TAG},_CI_PROJECT_NAMESPACE=${CI_PROJECT_NAMESPACE}
cloudbuild.yaml:
steps:
- name: gcr.io/cloud-builders/docker
id: builder
args:
- 'build'
- '-t'
- 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:$_CI_COMMIT_TAG'
- '.'
- name: gcr.io/cloud-builders/docker
id: tag-runner-image
args:
- 'tag'
- 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:$_CI_COMMIT_TAG'
- 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:latest'
images:
- 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:$_CI_COMMIT_TAG'
- 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:latest'
just use google/cloud-sdk:alpine as image in the gitlab-ci stage