I have Dockerfile which includes some builders, one of them is frontend where I recently added ARG BRANCH_NAME so I could use branch name later as a variable (I do not have .git folder in my frontend that's why I'm using this approach):
FROM node:8.11 as frontend-builder
COPY frontend/package.json /frontend/package.json
COPY frontend/package-lock.json /frontend/package-lock.json
COPY ./VERSION /frontend/VERSION
WORKDIR /frontend
RUN npm install
COPY frontend/ /frontend
ARG BRANCH_NAME
RUN sed -i "s|VERSION|${BRANCH_NAME}-frontend#$(cat "VERSION")|g" src/environments/environment.prod.ts
RUN npm run build-prod
In gitlab-ci.yml I pass it like this:
build:
stage: build
before_script:
- docker login -u gitlab-ci-token -p "$CI_JOB_TOKEN" registry.xx.it
script:
- export TARGET=backend-builder
- export IMAGE=$CI_REGISTRY_IMAGE/$TARGET
- docker pull $IMAGE:$CI_COMMIT_REF_NAME || echo "no image"
- docker pull $IMAGE:latest || echo "no latest image"
- docker build --target $TARGET -t $IMAGE:$CI_COMMIT_REF_NAME .
- docker push $IMAGE:$CI_COMMIT_REF_NAME
- export TARGET=frontend-builder
- export IMAGE=$CI_REGISTRY_IMAGE/$TARGET
- docker pull $IMAGE:$CI_COMMIT_REF_NAME || echo "no image"
- docker pull $IMAGE:latest || echo "no latest image"
- docker build --target $TARGET -t $IMAGE:$CI_COMMIT_REF_NAME --build-arg BRANCH_NAME=$CI_COMMIT_REF_NAME .
- docker push $IMAGE:$CI_COMMIT_REF_NAME
- export IMAGE=$CI_REGISTRY_IMAGE
- docker pull $IMAGE:$CI_COMMIT_REF_NAME || echo "no branch image"
- docker pull $IMAGE:latest || echo "no latest image"
- docker build -t $IMAGE:$CI_COMMIT_REF_NAME .
- docker push $IMAGE:$CI_COMMIT_REF_NAME
tags:
- local-docker
after the - export TARGET=frontend-builder I changed line:
- docker build --target $TARGET -t $IMAGE:$CI_COMMIT_REF_NAME .
to:
- docker build --target $TARGET -t $IMAGE:$CI_COMMIT_REF_NAME --build-arg BRANCH_NAME=$CI_COMMIT_REF_NAME .
so all I did was adding --build-arg BRANCH_NAME=$CI_COMMIT_REF_NAME.
But now it seems to execute it all twice and once I get $BRANCH_NAME as it should be and other time my $BRANCH_NAME is empty. Does anyone have ideas why does that happen?
Related
Background
in my gitlab-ci file I am trying to build a docker image, however even though I have docker:dind as a service, it is failing.
.gitlab-ci
---
stages:
- build
- docker
build:
stage: build
image: fl4m3ph03n1x/my-app:1.0
variables:
MIX_ENV: prod
script:
- mix deps.get
- mix deps.compile
- mix compile
artifacts:
paths:
- .hex/
- _build/
- deps/
- mix.lock
build_image:
stage: docker
image: fl4m3ph03n1x/my-app:1.0
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
DOCKER_HOST: tcp://docker:2375/
services:
- docker:dind
script:
- echo ${CI_JOB_TOKEN} | docker login --password-stdin -u ${CI_REGISTRY_USER} ${CI_REGISTRY}
- docker build . -t ${CI_REGISTRY_IMAGE}:latest
- docker push ${CI_REGISTRY_IMAGE}:latest
The problematic stage is docker.
As you can see I am trying to:
login into docker
build an image from gitlab's registry
push that image
Error
However, I am getting the following error:
$ echo ${CI_JOB_TOKEN} | docker login --password-stdin -u
${CI_REGISTRY_USER} ${CI_REGISTRY} /bin/bash: line 110: docker:
command not found
Which is confusing, because docker:dind is supposed to actually prevent this from happening:
https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#enable-registry-mirror-for-dockerdind-service
Question
So clearly I am missing something here. What am I doing wrong?
EDIT
This is my Dockerfile
FROM elixir:1.10
# Install Hex + Rebar
RUN mix do local.hex --force, local.rebar --force
COPY . /
WORKDIR /
ENV MIX_ENV=prod
RUN mix do deps.get --only $MIX_ENV, deps.compile
RUN mix release
EXPOSE 8080
ENV PORT=8080
ENV SHELL=/bin/bash
CMD ["_build/prod/rel/my_app/bin/my_app", "start"]
image is used to specify the image in which to run the script. You want to run the script in a docker image, to build your image.
The image keyword is the name of the Docker image the Docker executor runs to perform the CI tasks.
https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#define-image-and-services-from-gitlab-ciyml
After all, isn't your application image CI_REGISTRY_IMAGE in this? You don't want to build the image in itself.
- docker build . -t ${CI_REGISTRY_IMAGE}:latest
- docker push ${CI_REGISTRY_IMAGE}:latest
I have a private gitlab-repo on which I use the gitlab-ci.yml to deploy my project into stage and production.
inside the gitlab-ci.yml, I pass two environment-variables NODE_ENV (here I specify if it is stage/producion) and NODE_TARGET (just an info for the app, wha template to use). My gitlab-ci.yml looks like this:
stage_gsue:
stage: staging
script:
- echo "---------- DOCKER LOGIN"
- echo "mypassword" | docker login --username myuser --password-stdin git.example.com:4567
- echo "---------- START DEPLOYING STAGING SERVER"
- echo "-> 1) build image"
- docker build --build-arg buildtarget=gsue --build-arg buildenv=stage -t git.example.com:4567/root/myproject .
- echo "-> 2) push image to registry"
- docker push git.example.com:4567/root/myproject
- echo "-> 3) kill old container"
- docker kill $(docker ps -q) || true
- docker rm $(docker ps -a -q) || true
- echo "-> 4) start new container"
- docker run -dt -e NODE_TARGET=gsue -e NODE_ENV=stage -p 3000:3000 --name myproject git.example.com:4567/root/myproject
- echo "########## END DEPLOYING DOCKER IMAGE"
tags:
- stagerunner
when: manual
works good so far.. but now inside myproject there is a .env-file, in which I have some further variables. I changed the values of these variables and ran the stage-script multiple times, but inside my build image and started container, there are still old values in the .env-file.
How can that be??
additional info:
in my dockerfile I do:
FROM djudorange/node-gulp-mocha
ARG buildenv
ARG buildtarget
RUN git clone https://root:mypassword#git.example.com/root/myproject.git
WORKDIR /myproject
RUN git fetch --all
RUN git pull --all
RUN git checkout stage
RUN npm install -g n
RUN n latest
RUN npm install -g npm
RUN npm i -g gulp-cli --force
RUN npm install
RUN export NODE_ENV=$buildenv
RUN export NODE_TARGET=$buildtarget
RUN NODE_ENV=$buildenv NODE_TARGET=$buildtarget gulp build
#CMD ["node", "server.js"]
The environment overrides anything sent in 'export'. So better write a new env file during the build. So use the following in ur dockerfile:
ARG NODE_ENV
ARG NODE_TARGET
RUN rm -f .env
RUN touch .env
RUN echo "NODE_TARGET=$NODE_TARGET \n\
NODE_ENV=$NODE_ENV" >> ./.env
(fill up the rest of the docekrfile depending upon ur requirements)
Now the build command will be like...
docker-compose build --build-arg NODE_ENV="${ur env arg}" --build-arg NODE_TARGET="<ur target arg>"
So the gitlab build command will be
build_app:
stage: build
script:
- docker-compose build --build-arg NODE_ENV="${NODE_ENV}" --build-arg NODE_TARGET="${NODE_TARGET}"
- echo "Build successful."
- docker-compose up -d
- echo "Deployed!!"
Dont forget to define ur NODE_ENV and NODE_TARGET args in the variables found in the ci cd settings page
I am trying to push my images from Travis CI to docker hub.
Here is my Dockerfile
FROM node:alpine as builder
WORKDIR /app
COPY ./package.json ./
RUN npm install
COPY . .
CMD npm run build
FROM nginx
EXPOSE 3000
COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
COPY --from=builder /app/build /usr/share/nginx/html
My .travis.yml
sudo: required
services:
- docker
before_install:
- docker build -t aseel/react-test -f ./client/Dockerfile.dev ./client
script:
- docker run -e CI=true aseel/react-test npm run test
after_success:
- docker build -t aseel/multi-client ./client
- docker build -t aseel/multi-nginx ./nginx
- docker build -t aseel/multi-server ./server
- docker build -t aseel/multi-worker ./worker
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- docker push aseel/multi-server
- docker push aseel/multi-nginx
- docker push aseel/multi-worker
- docker push aseel/multi-client
The reason the push is failing is because this command
docker build -t aseel/multi-client ./client
is failing on travis (but not locally).
More specifically, I'm getting this error on travis
COPY failed: stat
/var/lib/docker/overlay2/135282513e177be132b6978986f878ba61f3b89914b6b2f7030cbafa0d33f629/merged/app/build:
no such file or directory
Any idea why this is happening?
I needed to switch CMD to RUN -- It was a pretty silly mistake
I'm setting up travis to push images to docker hub after running a test script
sudo: required
services:
- docker
before_install:
- docker build -t oskygh/react-test -f ./client/Dockerfile.dev ./client
script:
- docker run oskygh/react-test npm test -- --coverage
after_success:
- docker build -t osbee/client ./client
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- docker push osbee/client
dockerfile.dev
FROM node:alpine
WORKDIR '/app'
COPY ./package.json ./
RUN npm install
COPY . .
CMD ["npm","run","start"]
As explained here you could use the travis_wait function. Adding it before the command, which failed. You could also read this stackoverflow, which added it in another way.
I'm practicing with Gitlab CI to understand how to build an application and then use that within a Docker image. For now, my repo consists simply of helloworld.txt, dockerfile, and gitlab-ci.yml.
PROBLEM: During the build stage, I use a shell executor to 'zip helloworld.zip helloworld.txt". Then, I "docker build -t myproject/myapp ." where I expect to COPY helloworld.zip /" but it seems that the zip file I created is not available during the docker build context. Am I not saving the helloworld.zip file to the right location? Or something else? My long term intent is to write a python application, and during the build stage to compile into a single executable and copy into a docker container.
#cat helloworld.txt
hello world
#cat dockerfile
FROM centos:7
COPY helloworld.zip /
CMD ["/bin/bash"]
#cat gitlab-ci.yml
stages:
- build
- test
- release
- deploy
variables:
IMAGE_TEST_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
IMAGE_RELEASE_NAME: $CI_REGISTRY_IMAGE:latest
before_script:
- echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" "$CI_REGISTRY" --password-stdin
build:
stage: build
script:
- echo "compile the program"
- zip zipfile.zip helloworld.txt
- docker build --pull -t $IMAGE_TEST_NAME .
- docker push $IMAGE_TEST_NAME
test:
stage: test
script:
- docker pull $IMAGE_TEST_NAME
- docker run $IMAGE_TEST_NAME yum install unzip -y && unzip /helloworld.zip && cat /helloworld.txt
release:
stage: release
script:
- docker pull $IMAGE_TEST_NAME
- docker tag $IMAGE_TEST_NAME $IMAGE_RELEASE_NAME
- docker push $IMAGE_RELEASE_NAME
only:
- master
deploy:
stage: deploy
script:
- ./deploy.sh
only:
- master
when: manual
I expect that within the same stage (in this case build), I can run a program such as zip and then COPY that zip file into a given directory within a newly built docker image during the docker build process.
EDIT
After learning that I can't do this, I've created two different stages: build_app and build_container. Also knowing that artifacts are used by default in following stages, I didn't add an artifacts to the first stage or a dependancies to the next stage. This is the gitlab-ci.yml below and is still producing the same error.
stages:
- build_app
- build_container
- test
- release
- deploy
# you can delete this line if you're not using Docker
#image: centos:latest
variables:
IMAGE_TEST_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
IMAGE_RELEASE_NAME: $CI_REGISTRY_IMAGE:latest
before_script:
- echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" "$CI_REGISTRY" --password-stdin
build_app:
stage: build_app
script:
- echo "compile the program"
- zip zipfile.zip helloworld.txt
build_container:
stage: build_container
script:
- docker build --pull -t $IMAGE_TEST_NAME .
- docker push $IMAGE_TEST_NAME
test:
stage: test
script:
- docker pull $IMAGE_TEST_NAME
- docker run $IMAGE_TEST_NAME yum install unzip -y && unzip /helloworld.zip && cat /helloworld.txt
release:
stage: release
script:
- docker pull $IMAGE_TEST_NAME
- docker tag $IMAGE_TEST_NAME $IMAGE_RELEASE_NAME
- docker push $IMAGE_RELEASE_NAME
only:
- master
deploy:
stage: deploy
script:
- ./deploy.sh
only:
- master
when: manual
Job Status:
Build App: Passed
Build Container: Failed
Running with gitlab-runner 11.6.1 (8d829975)
on gitrunner-shell trtHcQTS
Using Shell executor...
Running on gitrunner.example.com...
Fetching changes...
Removing zipfile.zip
HEAD is now at e0a0a95 Update .gitlab-ci.yml
Checking out e0a0a952 as newFeature...
Skipping Git submodules setup
$ echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" "$CI_REGISTRY" --password-stdin
WARNING! Your password will be stored unencrypted in /home/gitlab-runner/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
$ docker build --pull -t $IMAGE_TEST_NAME .
Sending build context to Docker daemon 112.1kB
Step 1/3 : FROM centos:7
7: Pulling from library/centos
Digest: sha256:184e5f35598e333bfa7de10d8fb1cebb5ee4df5bc0f970bf2b1e7c7345136426
Status: Image is up to date for centos:7
---> 1e1148e4cc2c
Step 2/3 : COPY helloworld.zip /
COPY failed: stat /var/lib/docker/tmp/docker-builder312764301/helloworld.zip: no such file or directory
ERROR: Job failed: exit status 1
This is not possible. Gitlab CI's job model assumes that jobs of the same stage are independent.
See the manual for the dependencies keyword in gitlab-ci.yml:
This feature [...] allows you to define the artifacts to pass between different jobs.
Note that artifacts from all previous stages are passed by default.
[...] You can only define jobs from stages that are executed before the current one.