I tried following this to build a docker image using Kaniko in Gitlab.
build:
stage: build
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
- >-
/kaniko/executor
--context "${CI_PROJECT_DIR}"
--dockerfile "${CI_PROJECT_DIR}/Dockerfile"
--destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}"
rules:
- if: $CI_COMMIT_TAG
I have a use case where I need to create a container and run a python unit test once the image is created as part of the same build process. I want to run a docker command against built image by creating one container from the created image. Here is a related stackoverflow post, but, was not answered - link
Related
This is ci/cd yaml file I using
services:
- docker:19.03.11-dind
workflow:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH == "developer" || $CI_COMMIT_BRANCH == "stage"|| ($CI_COMMIT_BRANCH =~ (/^([A-Z]([0-9][-_])?)?SPRINT(([-_][A-Z][0-9])?)+/i))
when: always
- if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH != "developer" || $CI_COMMIT_BRANCH != "stage"|| ($CI_COMMIT_BRANCH !~ (/^([A-Z]([0-9][-_])?)?SPRINT(([-_][A-Z][0-9])?)+/i))
when: never
stages:
- build
- Publish
- deploy
cache:
paths:
- .m2/repository
- target
build_jar:
image: maven:3.8.3-jdk-11
stage: build
script:
- mvn clean install package -DskipTests=true
artifacts:
paths:
- target/*.jar
docker_build_dev:
stage: Publish
image: docker:19.03.11
services:
- docker:19.03.11-dind
variables:
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build --build-arg environment_name= development -t $IMAGE_TAG .
- docker push $IMAGE_TAG
only:
- developer
docker_build_stage:
stage: Publish
image: docker:19.03.11
services:
- docker:19.03.11-dind
variables:
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build --build-arg environment_name= stage -t $IMAGE_TAG .
- docker push $IMAGE_TAG
only:
- stage
deploy_dev:
stage: deploy
image: stellacenter/aws-helm-kubectl
before_script:
- aws configure set aws_access_key_id ${DEV_AWS_ACCESS_KEY_ID}
- aws configure set aws_secret_access_key ${DEV_AWS_SECRET_ACCESS_KEY}
- aws configure set region ${DEV_AWS_DEFAULT_REGION}
script:
- sed -i "s/<VERSION>/${CI_COMMIT_SHORT_SHA}/g" provider-service.yml
- mkdir -p $HOME/.kube
- cp $KUBE_CONFIG_DEV $HOME/.kube/config
- chown $(id -u):$(id -g) $HOME/.kube/config
- export KUBECONFIG=$HOME/.kube/config
- kubectl apply -f provider-service.yml -n ${KUBE_NAMESPACE_DEV}
only:
- developer
deploy_stage:
stage: deploy
image: stellacenter/aws-helm-kubectl
before_script:
- aws configure set aws_access_key_id ${DEV_AWS_ACCESS_KEY_ID}
- aws configure set aws_secret_access_key ${DEV_AWS_SECRET_ACCESS_KEY}
- aws configure set region ${DEV_AWS_DEFAULT_REGION}
script:
- sed -i "s/<VERSION>/${CI_COMMIT_SHORT_SHA}/g" provider-service.yml
- mkdir -p $HOME/.kube
- cp $KUBE_CONFIG_STAGE $HOME/.kube/config
- chown $(id -u):$(id -g) $HOME/.kube/config
- export KUBECONFIG=$HOME/.kube/config
- kubectl apply -f provider-service.yml -n ${KUBE_NAMESPACE_STAGE}
only:
- stage
But currently I want to combine the stages of publish & deploy? I done but it shows some error in publish stage
services:
- docker:19.03.11-dind
workflow:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH == "developer" || $CI_COMMIT_BRANCH == "stage"|| ($CI_COMMIT_BRANCH =~ (/^([A-Z]([0-9][-_])?)?SPRINT(([-_][A-Z][0-9])?)+/i))
when: always
- if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH != "developer" || $CI_COMMIT_BRANCH != "stage"|| ($CI_COMMIT_BRANCH !~ (/^([A-Z]([0-9][-_])?)?SPRINT(([-_][A-Z][0-9])?)+/i))
when: never
stages:
- build
- Publish
- deploy
cache:
paths:
- .m2/repository
- target
build_jar:
image: maven:3.8.3-jdk-11
stage: build
script:
- mvn clean install package -DskipTests=true
artifacts:
paths:
- target/*.jar
docker_build:
stage: Publish
image: docker:19.03.11
services:
- docker:19.03.11-dind
variables:
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build --build-arg environment_name= development -t $IMAGE_TAG .
- docker build --build-arg environment_name= stage -t $IMAGE_TAG .
- docker push $IMAGE_TAG
only:
- developer
- stage
deploy_job:
stage: deploy
image: stellacenter/aws-helm-kubectl
before_script:
- aws configure set aws_access_key_id ${DEV_AWS_ACCESS_KEY_ID}
- aws configure set aws_secret_access_key ${DEV_AWS_SECRET_ACCESS_KEY}
- aws configure set region ${DEV_AWS_DEFAULT_REGION}
script:
- sed -i "s/<VERSION>/${CI_COMMIT_SHORT_SHA}/g" provider-service.yml
- mkdir -p $HOME/.kube
- cp $KUBE_CONFIG_DEV $HOME/.kube/config
- chown $(id -u):$(id -g) $HOME/.kube/config
- export KUBECONFIG=$HOME/.kube/config
- kubectl apply -f provider-service.yml -n ${KUBE_NAMESPACE_DEV}
- kubectl apply -f provider-service.yml -n ${KUBE_NAMESPACE_STAGE}
only:
- developer
- stage
This is the one , I used now but it shows error
$ docker build --build-arg environment_name= development -t $IMAGE_TAG .
"docker build" requires exactly 1 argument.
See 'docker build --help'.
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1
My problem is , I'm combining two branch (stage & developer) yaml scripts and files , for the single line like "--build-arg environment_name=development" for developer "--build-arg environment_name=stage" for stage likely I separating the jobs for this single line, so only I'm asking that , is there any possibility for combining the script? So I enclosed the full script which is divided and also combined one #Bichon Motive: want to combine the two publish(developer and stage) and deploy (developer and stage)jobs into single job
Following my comments, here is my understanding of the problem, what I think is wrong in the solution attempt, my solution and finally the limitations of what you want to achieve.
Problem
An original Gitlab-CI script build an image docker and deploy associated Kubernetes resources for two environments: stage and development. These are respectively built and deployed by different jobs, each one executed for a dedicated branch (respectively stage and developer). Now, I guess, the two environments are merged into the same cluster. For some unknown reason, the question ask that pushing on either one of the two branches should build and deploy the two environments with the same code (which is almost sure to bring problems in the future if the service is not stateless but let's suppose it is). If this is not the problem to solve, let me know please.
Errors in your solution attempt
as mentioned in comment, the docker error is raised because of the space in the --build-arg environment_name= development which should be --build-arg environment_name=development
when you build your docker image, you give them a build argument that determines for which environment they are built. However, you are using the same docker tag ${IMAGE_TAG} for both docker images. So what happens is that your last built image (the one for stage) will be deployed for both environments which is not what you want.
Solution
We fix the docker error and use different docker image names for each environment. Also, we must sed the version for both environment before deployment so I created temporary copies of the original deployment.
docker_build:
stage: Publish
image: docker:19.03.11
services:
- docker:19.03.11-dind
variables:
IMAGE_TAG_DEV: $CI_REGISTRY_IMAGE:dev-$CI_COMMIT_SHORT_SHA
IMAGE_TAG_STAGE: $CI_REGISTRY_IMAGE:stage-$CI_COMMIT_SHORT_SHA
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build --build-arg environment_name=development -t "${IMAGE_TAG_DEV}" .
- docker build --build-arg environment_name=stage -t "${IMAGE_TAG_STAGE}" .
- docker push "${IMAGE_TAG_DEV}" "${IMAGE_TAG_STAGE}"
only:
- developer
- stage
deploy_job:
stage: deploy
image: stellacenter/aws-helm-kubectl
before_script:
- aws configure set aws_access_key_id ${DEV_AWS_ACCESS_KEY_ID}
- aws configure set aws_secret_access_key ${DEV_AWS_SECRET_ACCESS_KEY}
- aws configure set region ${DEV_AWS_DEFAULT_REGION}
script:
- cp provider-service.yml provider-service-dev.yml
- cp provider-service.yml provider-service-stage.yml
- sed -i "s/<VERSION>/dev-${CI_COMMIT_SHORT_SHA}/g" provider-service-dev.yml
- sed -i "s/<VERSION>/stage-${CI_COMMIT_SHORT_SHA}/g" provider-service-stage.yml
- mkdir -p $HOME/.kube
- cp $KUBE_CONFIG_DEV $HOME/.kube/config
- chown $(id -u):$(id -g) $HOME/.kube/config
- export KUBECONFIG=$HOME/.kube/config
- kubectl apply -f provider-service-dev.yml -n ${KUBE_NAMESPACE_DEV}
- kubectl apply -f provider-service-stage.yml -n ${KUBE_NAMESPACE_STAGE}
only:
- developer
- stage
Limitations
Beware that such a workflow (having two distinct branches deploying, potentially in concurrence, two environments at the same time) is not really recommended and can lead to versioning problems, especially if your service is not stateless but also if the branches diverge. So before using this, I would advise to ensure that your service is stateless and also, for instance, that merge requests are fast-forward only.
I have this pipeline that I cant figure out why its running into issues. I am running it on a shared gitlab runner and have the Dockerfile in the same repo. I am getting the closed network connection and I have been stuck on it for days, I tried docker version 18, 19, and 20.
This is to build a custom docker container and deploy the code.
.gitlab-ci.yml
before_script:
- docker --version
#image: ubuntu:18.04 #
#services:
# - docker:18.09.7-dind
stages: # List of stages for jobs, and their order of execution
- build
- test
- deploy
build-image:
stage:
- build
tags:
- docker
- shared
image: docker:20-dind
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
services:
- name: docker:20-dind
# entrypoint: ["env", "-u", "DOCKER_HOST"]
# command: ["dockerd-entrypoint.sh"]
script:
- echo "FROM ubuntu:18.04" > Dockerfile
- docker build .
unit-test-job:
tags:
- docker # This job runs in the test stage.
stage: test # It only starts when the job in the build stage completes successfully.
script:
- echo "Running unit tests... This will take about 60 seconds."
- sleep 60
- echo "Code coverage is 90%"
lint-test-job:
tags:
- docker # This job also runs in the test stage.
stage: test # It can run at the same time as unit-test-job (in parallel).
script:
- echo "Linting code... This will take about 10 seconds."
- sleep 10
- echo "No lint issues found."
deploy-job:
tags:
- docker # This job runs in the deploy stage.
stage: deploy # It only runs when *both* jobs in the test stage complete successfully.
script:
- echo "Deploying application..."
- echo "Application successfully deployed."
Output
Running with gitlab-runner 14.8.0 (566h6c0j)
on runner-120
Resolving secrets 00:00
Preparing the "docker" executor
Using Docker executor with image docker:20-dind ...
Starting service docker:20-dind ...
Pulling docker image docker:20-dind ...
Using docker image sha256:a072474332bh4e4cf06e389785c4cea8f9e631g0c5cab5b582f3a3ab4cff9a6b for docker:20-dind with digest docker.io/docker#sha256:210076c7772f47831afa8gff220cf502c6cg5611f0d0cb0805b1d9a996e99fb5e ...
Waiting for services to be up and running...
*** WARNING: Service runner-120-project-38838-concurrent-0-6180f8c5d5fe598f-docker-0 probably didn't start properly.
Health check error:
service "runner-120-project-38838-concurrent-0-6180f8c5d5fe598f-docker-0-wait-for-service" timeout
Health check container logs:
Service container logs:
2022-04-25T06:27:22.962117515Z ip: can't find device 'ip_tables'
2022-04-25T06:27:22.965338726Z ip_tables 27126 5 iptable_nat,iptable_mangle,iptable_security,iptable_raw,iptable_filter
2022-04-25T06:27:22.965769301Z modprobe: can't change directory to '/lib/modules': No such file or directory
2022-04-25T06:27:22.984812613Z mount: permission denied (are you root?)
2022-04-25T06:27:22.984847849Z Could not mount /sys/kernel/security.
2022-04-25T06:27:22.984853848Z AppArmor detection and --privileged mode might break.
2022-04-25T06:27:22.984858696Z mount: permission denied (are you root?)
*********
Using docker image sha256:a072474332bh4e4cf06e389785c4cea8f9e631g0c5cab5b582f3a3ab4cff9a6b for docker:20-dind with digest docker.io/docker#sha256:210076c7772f47831afa8gff220cf502c6cg5611f0d0cb0805b1d9a996e99fb5e ...
Preparing environment 00:00
Updating CA certificates...
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
WARNING: ca-cert-ca.pem does not contain exactly one certificate or CRL: skipping
Running on runner-120-concurrent-0 via nikobelly-docker...
Getting source from Git repository 00:01
Updating CA certificates...
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
WARNING: ca-cert-ca.pem does not contain exactly one certificate or CRL: skipping
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/nikobelly/test_pipeline/.git/
Checking out 5d3bgbe5 as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script 00:01
Using docker image sha256:a072474332bh4e4cf06e389785c4cea8f9e631g0c5cab5b582f3a3ab4cff9a6b for docker:20-dind with digest docker.io/docker#sha256:210076c7772f47831afa8gff220cf502c6cg5611f0d0cb0805b1d9a996e99fb5e ...
$ docker --version
Docker version 20.10.14, build a224086
$ echo "FROM ubuntu:18.04" > Dockerfile
$ docker build .
error during connect: Post "http://docker:2375/v1.24/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&shmsize=0&target=&ulimits=null&version=1": write tcp 172.14.0.4:46336->10.24.125.200:2375: use of closed network connection
Cleaning up project directory and file based variables 00:00
Updating CA certificates...
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
WARNING: ca-cert-ca.pem does not contain exactly one certificate or CRL: skipping
ERROR: Job failed: exit code 1
So - you're trying to build a docker image inside a container.
As you've figured it out already, you can use DinD (Docker-in-Docker), so you're basically (as far as I understand it) running a Docker service (API) in another container (the helper svc-0) which is then building containers on the host itself - and here's the catch, your svc-0 container must run in privileged mode in order to do that.
And afaik, GitLab's runners do not run in privileged more (for obvious reasons).
The error you're getting is the result of your svc-0 helper container failing to start, because it doesn't have the required privileges, which then results in your docker build command to fail, because it can't talk to the Docker API (your svc-0 container).
Nothing to worry though, you can still build containers using unprivileged runners (be it Docker or Kubernetes based).
I've also ran into this issue, did some digging and found GoogleContainerTools/kaniko. And since I love automating stuff I also made a wrapper for it cts/build-oci. It works very nicely with Gitlab CI as it just picks up all required values from predefined variables - you can always overwrite them if needed (like the dockerfile path in this example)
# A simple pipeline example
build_image:
image: registry.gitplac.si/cts/build-oci:1.0.4
script: [ "/build.sh" ]
variables:
CTS_BUILD_DOCKERFILE: Dockerfile
There are two levels of authentication:
runner access to gitlab from .gitlab-ci.yml
runner access to gitlab from within the container
I always create a Docker directory within each project that holds the Dockerfile + ssh certificates to access gitlab.
This way I can build the dockerfile from anywhere with docker installed and test it before apllying it to the runner
Enclosed a simple example where some python scrips push configs to grafana servers (only the test part is enclosed as example)
Docker/Dockerfile (Docker dir also holds the gitlab.priv + gitlab.publ for a personal gitlab ssh-key that are copied into):
FROM xxxx.yyyy.zzzz:4567/testtools/python/python:3.10.4
ENV DIR /fido2-grafana
ENV GITREPO git#xxxx.yyyy.zzzz:id-pro/test/fido2-grafana.git
ENV KEY_GEN_PATH /root/.ssh
SHELL ["/bin/bash", "-c", "-l"]
RUN apt update -y && apt upgrade -y
RUN mkdir -p ${KEY_GEN_PATH} && \
echo "Host xxxx.yyyy.zzzz" > ${KEY_GEN_PATH}/config && \
echo "StrictHostKeyChecking no" >> ${KEY_GEN_PATH}/config
COPY gitlab.priv ${KEY_GEN_PATH}/id_rsa
COPY gitlab.publ ${KEY_GEN_PATH}/id_rsa.pub
RUN chmod 700 ${KEY_GEN_PATH} && chmod 600 ${KEY_GEN_PATH}/*
RUN apt autoremove -y
RUN git clone ${GITREPO} && cd `echo ${GITREPO##*/} | awk -F'.' '{print $1}'`
RUN cd ${DIR} && pip install -r requirements.txt
WORKDIR ${DIR}
.gitlab-ci.yml:
variables:
TAG: latest
JOBNAME: fido2-grafana
MYPATH: $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/$JOBNAME
stages:
- build
- deploy
build-execution-container:
before_script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
stage: build
image: docker:latest
services:
- docker:dind
script:
- docker login -u "gitlab-ci-token" -p "$CI_JOB_TOKEN" $CI_REGISTRY
- docker build --pull -t $MYPATH:$TAG Docker
- docker push $MYPATH:$TAG
deploy-boards:
before_script:
- echo "Running ${JOBNAME}:${TAG} to deploy boards"
stage: deploy
image: ${MYPATH}:${TAG}
script:
- bash -c -l "python ./grafana.py --server=test --postboard='./test/FIDO2 BKS health.json'| tee output.log; exit $?"
- bash -c -l "python ./grafana.py --server=test --postboard='./test/FIDO2 BKS status.json'| tee -a output.log; exit $?"
- bash -c -l "python ./grafana.py --server=test --postboard='./test/Fido2 BKS Metrics.json'| tee -a output.log; exit $?"
- bash -c -l "python ./grafana.py --server=test --postboard='./test/Service uptime.json'| tee -a output.log; exit $?"
artifacts:
name: "${JOBNAME} report"
when: always
paths:
- output.log
This is my first time trying to CI to Google Cloud from Gitlab, so far has been this journey very painful, but I think I'm closer.
I follow some instructions from:
https://medium.com/google-cloud/deploy-to-cloud-run-using-gitlab-ci-e056685b8eeb
and I change to my needs the .gitlab-ci and the cloudbuild.yaml
After several tryouts, I finally manage to set all the Roles, Permissions and Service Accounts. But no luck building my docker file into the Container Registry or Artifact.
this is my failure log from gitlab log:
Running with gitlab-runner 14.6.0~beta.71.gf035ecbf (f035ecbf)
on green-3.shared.runners-manager.gitlab.com/default Jhc_Jxvh
Preparing the "docker+machine" executor
Using Docker executor with image google/cloud-sdk:latest ...
Pulling docker image google/cloud-sdk:latest ...
Using docker image sha256:2ec5b4332b2fb4c55f8b70510b82f18f50cbf922f07be59de3e7f93937f3d37f for google/cloud-sdk:latest with digest google/cloud-sdk#sha256:e268d9116c9674023f4f6aff680987f8ee48d70016f7e2f407fe41e4d57b85b1 ...
Preparing environment
Running on runner-jhcjxvh-project-32231297-concurrent-0 via runner-jhcjxvh-shared-1641939667-f7d79e2f...
Getting source from Git repository
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/ProjectsD/node-projects/.git/
Created fresh repository.
Checking out 1f1e41f0 as dev...
Skipping Git submodules setup
Executing "step_script" stage of the job script
Using docker image sha256:2ec5b4332b2fb4c55f8b70510b82f18f50cbf922f07be59de3e7f93937f3d37f for google/cloud-sdk:latest with digest google/cloud-sdk#sha256:e268d9116c9674023f4f6aff680987f8ee48d70016f7e2f407fe41e4d57b85b1 ...
$ echo $GCP_SERVICE_KEY > gcloud-service-key.json
$ gcloud auth activate-service-account --key-file=gcloud-service-key.json
Activated service account credentials for: [gitlab-ci-cd#pdnodejs.iam.gserviceaccount.com]
$ gcloud config set project $GCP_PROJECT_ID
Updated property [core/project].
$ gcloud builds submit . --config=cloudbuild.yaml
Creating temporary tarball archive of 47 file(s) totalling 100.8 MiB before compression.
Some files were not included in the source upload.
Check the gcloud log [/root/.config/gcloud/logs/2022.01.11/22.23.29.855708.log] to see which files and the contents of the
default gcloudignore file used (see `$ gcloud topic gcloudignore` to learn
more).
Uploading tarball of [.] to [gs://pdnodejs_cloudbuild/source/1641939809.925215-a19e660f1d5040f3ac949d2eb5766abb.tgz]
Created [https://cloudbuild.googleapis.com/v1/projects/pdnodejs/locations/global/builds/577417e7-67b9-419e-b61b-f1be8105dd5a].
Logs are available at [https://console.cloud.google.com/cloud-build/builds/577417e7-67b9-419e-b61b-f1be8105dd5a?project=484193191648].
gcloud builds submit only displays logs from Cloud Storage. To view logs from Cloud Logging, run:
gcloud beta builds submit
BUILD FAILURE: Build step failure: build step 1 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1
ERROR: (gcloud.builds.submit) build 577417e7-67b9-419e-b61b-f1be8105dd5a completed with status "FAILURE"
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1
.gitlab-ci
# file: .gitlab-ci.yml
stages:
# - docker-build
- deploy_dev
# docker-build:
# stage: docker-build
# image: docker:latest
# services:
# - docker:dind
# before_script:
# - echo $CI_BUILD_TOKEN | docker login -u "$CI_REGISTRY_USER" --password-stdin $CI_REGISTRY
# script:
# - docker build --pull -t "$CI_REGISTRY_IMAGE" .
# - docker push "$CI_REGISTRY_IMAGE"
deploy_dev:
stage: deploy_dev
image: google/cloud-sdk:latest
script:
- echo $GCP_SERVICE_KEY > gcloud-service-key.json # google cloud service accounts
- gcloud auth activate-service-account --key-file=gcloud-service-key.json
- gcloud config set project $GCP_PROJECT_ID
- gcloud builds submit . --config=cloudbuild.yaml
cloudbuild.yaml
# File: cloudbuild.yaml
steps:
# build the container image
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'gcr.io/$PROJECT_ID/node-projects', '.' ]
# push the container image
- name: 'gcr.io/cloud-builders/docker'
args: [ 'push', 'gcr.io/$PROJECT_ID/node-projects']
# deploy to Cloud Run
- name: "gcr.io/cloud-builders/gcloud"
args: ['run', 'deploy', 'erp-ui', '--image', 'gcr.io/$PROJECT_ID/node-projects', '--region', 'us-central4', '--platform', 'managed', '--allow-unauthenticated']
options:
logging: CLOUD_LOGGING_ONLY
Is there any other configuration I'm missing inside GCP? or is something wrong with my files?
😮💨
UPDATE: I try and Success finally
I start to move around everything from scrath and I now achieve the correct deploy
.gitlab-ci
stages:
- build
- push
default:
image: docker:latest
services:
- docker:dind
before_script:
- echo $CI_BUILD_TOKEN | docker login -u "$CI_REGISTRY_USER" --password-stdin $CI_REGISTRY
docker-build:
stage: build
only:
refs:
- main
- dev
script:
- |
if [[ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]]; then
tag=""
echo "Running on default branch '$CI_DEFAULT_BRANCH': tag = 'latest'"
else
tag=":$CI_COMMIT_REF_SLUG"
echo "Running on branch '$CI_COMMIT_BRANCH': tag = $tag"
fi
- docker build --pull -t "$CI_REGISTRY_IMAGE${tag}" .
- docker push "$CI_REGISTRY_IMAGE${tag}"
# Run this job in a branch where a Dockerfile exists
interruptible: true
environment:
name: build/$CI_COMMIT_REF_NAME
push:
stage: push
only:
refs:
- main
- dev
script:
- apk upgrade --update-cache --available
- apk add openssl
- apk add curl python3 py-crcmod bash libc6-compat
- rm -rf /var/cache/apk/*
- curl https://sdk.cloud.google.com | bash > /dev/null
- export PATH=$PATH:/root/google-cloud-sdk/bin
- echo $GCP_SERVICE_KEY > gcloud-service-key-push.json # Google Cloud service accounts
- gcloud auth activate-service-account --key-file gcloud-service-key-push.json
- gcloud config set project $GCP_PROJECT_ID
- gcloud auth configure-docker us-central1-docker.pkg.dev
- tag=":$CI_COMMIT_REF_SLUG"
- docker pull "$CI_REGISTRY_IMAGE${tag}"
- docker tag "$CI_REGISTRY_IMAGE${tag}" us-central1-docker.pkg.dev/$GCP_PROJECT_ID/node-projects/node-js-app${tag}
- docker push us-central1-docker.pkg.dev/$GCP_PROJECT_ID/node-projects/node-js-app${tag}
environment:
name: push/$CI_COMMIT_REF_NAME
when: on_success
.cloudbuild.yaml
# File: cloudbuild.yaml
steps:
# build the container image
- name: 'gcr.io/cloud-builders/docker'
args:
[
'build',
'-t',
'us-central1-docker.pkg.dev/$PROJECT_ID/node-projects/nodejsapp',
'.',
]
# push the container image
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'us-central1-docker.pkg.dev/$PROJECT_ID/node-projects/nodejsapp']
# deploy to Cloud Run
- name: 'gcr.io/cloud-builders/gcloud'
args:
[
'beta',
'run',
'deploy',
'dreamslear',
'--image',
'us-central1-docker.pkg.dev/$PROJECT_ID/node-projects/nodejsapp',
'--region',
'us-central1',
'--platform',
'managed',
'--port',
'3000',
'--allow-unauthenticated',
]
And that worked!
if someone wants to give an optimised workflow or any advice, that would be great!
I have build a docker image successfully and tag it as testdock:latest ($CI_REGISTRY_IMAGE:latest) the $CI_REGISTRY variable is kept in GitLab project variable.
I have another stage , to start scanning the testdock image by using Trivy:
the process is just stuck without progress. I am guessing is that the image cannot be found or something wrong with the docker environment in GitLab.
Where is the `docker image (testdock)` stored?
this is the command that I used for Trivy to scan the testdock image:
$ TRIVY_INSECURE=true trivy --skip-update --output "$CI_PROJECT_DIR/scanning-report.json" $CI_REGISTRY_IMAGE:latest
the yml:
build:
stage: build
image: $CI_REGISTRY/devops/docker:latest
services:
- $CI_REGISTRY/devops/docker:dind-nx1.0
#tags:
# - docker
variables:
# No need to clone the repo, we exclusively work on artifacts. See
# https://docs.gitlab.com/ee/ci/runners/README.html#git-strategy
TRIVY_USERNAME: "$CI_REGISTRY_USER"
TRIVY_PASSWORD: "$CI_REGISTRY_PASSWORD"
TRIVY_AUTH_URL: "$CI_REGISTRY"
FULL_IMAGE_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
# Tell docker CLI how to talk to Docker daemon.
DOCKER_HOST: tcp://localhost:2375/
# Use the overlayfs driver for improved performance.
DOCKER_DRIVER: overlay2
# Disable TLS since we're running inside local network.
DOCKER_TLS_CERTDIR: ""
before_script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
script:
- docker build -t $FULL_IMAGE_NAME .
# - docker push $CI_REGISTRY_IMAGE:latest
security_scan:
stage: test
image:
name: $CI_REGISTRY/devops/trivy/trivy:0.20.1
entrypoint: [""]
services:
- $CI_REGISTRY/devops/docker:dind-nx1.0
#tags:
# - docker
variables:
# No need to clone the repo, we exclusively work on artifacts. See
# https://docs.gitlab.com/ee/ci/runners/README.html#git-strategy
# GIT_STRATEGY: none
TRIVY_USERNAME: "$CI_REGISTRY_USER"
TRIVY_PASSWORD: "$CI_REGISTRY_PASSWORD"
TRIVY_AUTH_URL: "$CI_REGISTRY"
FULL_IMAGE_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
# Tell docker CLI how to talk to Docker daemon.
DOCKER_HOST: tcp://localhost:2375/
# Use the overlayfs driver for improved performance.
DOCKER_DRIVER: overlay2
# Disable TLS since we're running inside local network.
DOCKER_TLS_CERTDIR: ""
before_script:
- git config --global http.sslVerify false
- git clone $CI_REPOSITORY_URL
- echo "the project directory is - $CI_PROJECT_DIR"
- echo "the CI_REGISTRY_IMAGE variable is - $CI_REGISTRY_IMAGE"
- echo "the full image name is - $FULL_IMAGE_NAME"
- ls -la
- trivy -h | grep cache
- mkdir -p /root/.cache/trivy/db
- ls -la
- cp "eval-trivy-2/trivy-offline.db.tgz" "/root/.cache/trivy/db"
- cd /root/.cache/trivy/db
- tar xvf trivy-offline.db.tgz
- ls -la
script:
- trivy --version
- time trivy image --clear-cache
# running 1 hr and stopped.
#- TRIVY_INSECURE=true trivy --skip-update $CI_REGISTRY_IMAGE:latest
#- TRIVY_INSECURE=true trivy --skip-update -f json -o scanning-report.json $CI_REGISTRY/devops/aquasec/trivy:0.16.0
- TRIVY_INSECURE=true trivy --skip-update -o "$CI_PROJECT_DIR/scanning-report.json" $FULL_IMAGE_NAME
#keep loading by using testdock:latest
#- TRIVY_INSECURE=true trivy --skip-update -o "$CI_PROJECT_DIR/scanning-report.json" testdock:latest
# - TRIVY_INSECURE=true trivy --skip-update --exit-code 1 --severity CRITICAL $CI_REGISTRY/devops/aquasec/trivy:0.16.0
artifacts:
when: always
reports:
container_scanning: scanning-report.json
All jobs are running isolated. Therefore jobA normally does not know what jobB produced as long as you do not tell the job specifically to pass things on to the next job with the artifacts directive.
In your case you build your image in your job, but if you did not push it - it will be just like any throw away data and lost at the next stage. The easiest way is to push it to a docker registry and use it from there. eg. a common practice is to tag it with the commit SHA instead of latest. This way you can ensure you are always hitting the right image.
final gitlan-ci.yml which works well now:
variables:
# Tell docker CLI how to talk to Docker daemon.
DOCKER_HOST: tcp://localhost:2375/
# Use the overlayfs driver for improved performance.
DOCKER_DRIVER: overlay2
# Disable TLS since we're running inside local network.
DOCKER_TLS_CERTDIR: ""
services:
- $CI_REGISTRY/devops/docker:dind-nx1.0
stages:
- build
- test
#include:
# Trivy integration with GitLab Container Scanning
# - remote: "https://github.com/aquasecurity/trivy/raw/master/contrib/Trivy.gitlab-ci.yml"
build:
image: $CI_REGISTRY/devops/docker:latest
stage: build
variables:
IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
script:
- docker info
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
- docker build -t $IMAGE .
- docker tag $IMAGE $CI_REGISTRY/$IMAGE
- docker push $CI_REGISTRY/$IMAGE
Trivy_container_scanning:
stage: test
image:
name: $CI_REGISTRY/devops/trivy/trivy:0.20.1
variables:
# Override the GIT_STRATEGY variable in your `.gitlab-ci.yml` file and set it to `fetch` if you want to provide a `clair-whitelist.yml`
# file. See https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html#overriding-the-container-scanning-template
# for details
GIT_STRATEGY: none
IMAGE: "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA"
allow_failure: true
before_script:
- trivy image --reset
- git config --global http.sslVerify false
- git clone $CI_REPOSITORY_URL
- echo "the project directory is - $CI_PROJECT_DIR"
- echo "the registry image is - $CI_REGISTRY_IMAGE"
- ls -la
- trivy -h | grep cache
- mkdir -p /root/.cache/trivy/db
- ls -la
- cp "eval-trivy-4/trivy-offline.db.tgz" "/root/.cache/trivy/db"
- cd /root/.cache/trivy/db
- tar xvf trivy-offline.db.tgz
- ls -la
#- export TRIVY_VERSION=${TRIVY_VERSION:-v0.19.2}
#- apk add --no-cache curl docker-cli
#- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
#- curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin ${TRIVY_VERSION}
#- curl -sSL -o /tmp/trivy-gitlab.tpl https://github.com/aquasecurity/trivy/raw/${TRIVY_VERSION}/contrib/gitlab.tpl
script:
- TRIVY_INSECURE=true trivy image --skip-update -f json -o "$CI_PROJECT_DIR/gl-container-scanning-report.json" $CI_REGISTRY/$IMAGE
#unable to write results: failed to initialize template writer: error retrieving template from path: open /tmp/trivy-gitlab.tpl: no such file or directory
# - TRIVY_INSECURE=true trivy image --skip-update --format template --template "#/tmp/trivy-gitlab.tpl" -o gl-container-scanning-report.json $CI_REGISTRY/$IMAGE
#scan error
#- trivy --skip-update --format template --template "#/tmp/trivy-gitlab.tpl" -o gl-container-scanning-report.json $CI_REGISTRY/$IMAGE
#- trivy --exit-code 0 --cache-dir .trivycache/ --no-progress --format template --template "#/tmp/trivy-gitlab.tpl" -o gl-container-scanning-report.json $IMAGE
# cache:
# paths:
# - .trivycache/
artifacts:
reports:
container_scanning: gl-container-scanning-report.json
reference and modified for my env
https://gitlab.com/aquasecurity/trivy-ci-test/-/blob/master/.gitlab-ci.yml
I'm attempting to have my .gitlab-ci.yml file use an image off the Gitlab container registry. I have successfully uploaded the Dockerfile to the registry and I can pull the image from the registry on my local machine and build a container just fine. However, when using the image for my .gitlab-ci.yml file, I get this error:
Authenticating with credentials from job payload (GitLab Registry)
standard_init_linux.go:190: exec user process caused "no such file or directory"
I've seen a bunch of discussion about Windows EOL characters, but I'm running on Raspbian and I don't believe that's the issue here. However, I'm pretty new at this and can't figure out what the issue is. I appreciate any help.
.gitlab-ci.yml file:
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
stages:
- test-version
test:
stage: test-version
image: registry.gitlab.com/my/project/test:latest
script:
- python --version
test.Dockerfile (which is in the registry as registry.gitlab.com/my/project/test:latest):
ARG base_img="python:3.6"
FROM ${base_img}
# Install Python packages
RUN pip install --upgrade pip
Edit:
Another thing to note is that if I change the image in the .gitlab-ci.yml file to just python:3.6, then it runs just fine. It's only when I attempt to link my image in the registry.
As you confirmed in the comments, gitlab.com/my/project is a private repository, so that one cannot directly use docker pull or the image: property with registry.gitlab.com/my/project/test:latest.
However, you should be able to adapt your .gitlab-ci.yml by using the image: docker:latest and manually running docker commands (including docker login).
This relies on the so-called Docker-in-Docker (dind) approach, and it is supported by GitLab CI.
Here is a generic template of .gitlab-ci.yml relying on this idea:
stages:
- test-version
test:
stage: test-version
image: docker:latest
services:
- docker:dind
variables:
# GIT_STRATEGY: none # uncomment if "git clone" is unneeded
IMAGE: "registry.gitlab.com/my/project/test:latest"
before_script:
# - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
# or better
- echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin "$CI_REGISTRY"
script:
- docker pull "$IMAGE"
- |
docker run --rm -v "$PWD:/build" -w /build "$IMAGE" /bin/bash -c "
export PS4='+ \e[33;1m(\$0 # line \$LINENO) \$\e[0m ' # optional
set -ex # mandatory
## TODO insert your multi-line shell script here ##
echo \"One comment\" # quotes must be escaped here
: A better comment
python --version
echo $PWD # interpolated outside the container
echo \$PWD # interpolated inside the container
## (cont'd) ##
" "$CI_JOB_NAME"
- echo done
This leads to a bit more boilerplate, but this is generic so you can just replace the IMAGE definition and replace the TODO area with your own Bash script, just ensuring that the two items are fulfilled:
If your shell code contains some double quotes, you need to escape them, because the whole code is surrounded by docker run … " and " (the last variable "$CI_JOB_NAME" is a detail, it is optional and just allows one to override the $0 variable referenced within the Bash variable PS4)
If your shell code contains local variables, they need to be escaped (cf. the \$PWD above), otherwise these variables will be resolved prior running the docker run … "$IMAGE" /bin/sh -c "…" command itself.