Env vars lost when building docker image from Gitlab CI - docker

I'm trying to build my React / NodeJS project using Docker and Gitlab CI.
When I build manually my images, I use .env file containing env vars, and everything is fine.
docker build --no-cache -f client/docker/local/Dockerfile . -t espace_client_client:local
docker build --no-cache -f server/docker/local/Dockerfile . -t espace_client_api:local
But when deploying with Gitlab, I can build successfully the image, but when I run it, env vars are empty in the client.
Here is my gitlab CI:
image: node:10.15
variables:
REGISTRY_PACKAGE_CLIENT_NAME: registry.gitlab.com/company/espace_client/client
REGISTRY_PACKAGE_API_NAME: registry.gitlab.com/company/espace_client/api
REGISTRY_URL: https://registry.gitlab.com
DOCKER_DRIVER: overlay
# Client Side
REACT_APP_API_URL: https://api.espace-client.company.fr
REACT_APP_DB_NAME: company
REACT_APP_INFLUX: https://influx-prod.company.fr
REACT_APP_INFLUX_LOGIN: admin
REACT_APP_HOUR_GMT: 2
stages:
- publish
docker-push-client:
stage: publish
before_script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $REGISTRY_URL
image: docker:stable
services:
- docker:dind
script:
- docker build --no-cache -f client/docker/prod/Dockerfile . -t $REGISTRY_PACKAGE_CLIENT_NAME:latest
- docker push $REGISTRY_PACKAGE_CLIENT_NAME:latest
Here is the Dockerfile for the client
FROM node:10.15-alpine
WORKDIR /app
COPY package*.json ./
ENV NODE_ENV production
RUN npm -g install serve && npm install
COPY . .
RUN npm run build
EXPOSE 3000
CMD [ "serve", "build", "-l", "3000" ]
Why is there such a difference between the 2 process ?

According to your answer in comments, GitLab CI/CD environment variables doesn't solve your issue. Gitlab CI environment is actual only in context of GitLab Runner that builds and|or deploys your app.
So, if you are going to propagate Env vars to the app, there are several ways to deliver variables from .gitlab-cy.ymlto your app:
ENV instruction Dockerfile
E.g.
FROM node:10.15-alpine
WORKDIR /app
COPY package*.json ./
ENV NODE_ENV production
ENV REACT_APP_API_URL: https://api.espace-client.company.fr
ENV REACT_APP_DB_NAME: company
ENV REACT_APP_INFLUX: https://influx-prod.company.fr
ENV REACT_APP_INFLUX_LOGIN: admin
ENV REACT_APP_HOUR_GMT: 2
RUN npm -g install serve && npm install
COPY . .
RUN npm run build
EXPOSE 3000
CMD [ "serve", "build", "-l", "3000" ]
docker-compose environment directive
web:
environment:
- NODE_ENV=production
- REACT_APP_API_URL=https://api.espace-client.company.fr
- REACT_APP_DB_NAME=company
- REACT_APP_INFLUX=https://influx-prod.company.fr
- REACT_APP_INFLUX_LOGIN=admin
- REACT_APP_HOUR_GMT=2
Docker run -e
(Not your case, just for information)
docker -e REACT_APP_DB_NAME="company"
P.S. Try Gitlab CI variables
There is convenient way to store variables outside of your code: Custom environment variables
You can set them up easily from the UI. That can be very powerful as it can be used for scripting without the need to specify the value itself.
(source: gitlab.com)

Related

docker : No subsytem for mount

I'm running a CI pipeline in gitlab-runner which is running on a linux machine.
In the pipeline I'm trying to build an image.
The Error I'm getting is,
ci runtime error: rootfs_linux.go:53: mounting "/sys/fs/cgroup" to rootfs "/var/lib/docker/overlay/d6b9ba61e6eed4bcd9c23722ad6f6a9fb05b6c536dbab31f47b835c25c0a343b/merged" caused "no subsystem for mount"
The Dockerfile contains :
# Set the base image to node:12-alpine
FROM node:16.7.0-buster
# Specify where our app will live in the container
WORKDIR /app
# Copy the React App to the container
COPY . /app/
# Prepare the container for building React
RUN npm install
RUN npm install react-scripts#4.0.3 -g
# We want the production version
RUN npm run build
# Prepare nginx
FROM nginx:stable
COPY --from=0 /app/build /usr/share/nginx/html
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx/nginx.conf /etc/nginx/conf.d
# Fire up nginx
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
The gitlab-ci.yml contains :
image: gitlab/dind
services:
- docker:dind
variables:
DOCKER_DRIVER: overlay
stages:
- build
- docker-build
cache:
paths:
- node_modules
yarn-build:
stage: build
image: node:latest
script:
- unset CI
- yarn install
- yarn build
artifacts:
expire_in: 1 hour # the artifacts expire from the artifacts folder after 1 hour
paths:
- build
docker-build:
stage: docker-build
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD
- docker build -t $CI_REGISTRY_USER/$app_name .
- docker push $CI_REGISTRY_USER/$app_name
I'm not getting how to resolve this, I tried upgrading docker in my linux machine.

Gitlab CI npm cannot resolve module

Totally new to Gitlab and CI in general, so apologies for the lack of understanding. I have a repo, which is NuxtJS based, with a Dockerfile. The end goal of the pipeline is to build and push this repo to my docker account. The Dockerfile is relatively straight forward, containing an npm install and npm run build. I'm using a custom docker image as my runner, based on docker:20.10.17-dind-alpine3.16 with ansible, terraform and kubectl installed.
When building the project's docker image on my local machine, I receive no issues, however in gitlab, when running the npm run build command, I get the following error:
Module not found: Error: Can't resolve '../node_modules/vue-confirm-dialog' in '/usr/src/nuxt-app/plugins'
Here is my yml file:
stages:
- docker
docker:
stage: docker
image: <my-runner-image>
services:
- "docker:dind"
before_script:
- docker login -u $DOCKER_REGISTRY_USER -p $DOCKER_REGISTRY_PASSWORD
script:
- docker build -t <my-repo> .
- docker push <my-repo>
Any suggestions are greatly appreciated
--EDIT--
As requested, here is the project's Dockerfile:
FROM node:lts-alpine3.15
# create destination directory
RUN mkdir -p /usr/src/nuxt-app
WORKDIR /usr/src/nuxt-app
# update and install dependency
RUN apk update && apk upgrade
RUN apk add git
# copy the app, note .dockerignore
COPY . /usr/src/nuxt-app/
RUN npm install
RUN npm run build
EXPOSE 3000
ENV NUXT_HOST=0.0.0.0
ENV NUXT_PORT=3000
CMD [ "npm", "start" ]

gitlab - Fetch Dockerfile from script

We're using GITLAB CI/CD for deployment. This is the publish stage , Dockerfile is used here. If you check the script , I've integrated one environment variable(line no:10) , because we're using two jobs for publish itself like developer and stage. For that stage , I shown to you.
docker_build_stage:
stage: Publish
image: docker:19.03.11
services:
- docker:19.03.11-dind
variables:
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build --build-arg environment_name=stage -t $IMAGE_TAG .
- docker push $IMAGE_TAG
only:
- /^([A-Z]([0-9][-_])?)?SPRINT(([-_][A-Z][0-9])?)+/i
- stage
This is the docker file , we're using.
FROM maven:3.8.3-jdk-11 AS MAVEN_BUILD
COPY pom.xml /build/
COPY src /build/src/
WORKDIR /build/
RUN mvn clean install package -DskipTests=true
FROM openjdk:11
WORKDIR /app
COPY --from=MAVEN_BUILD /build/target/provider-service-*.jar /app/provider-service.jar
ENV PORT 8092
ENV env_var_name=$environment_name
EXPOSE $PORT
ENTRYPOINT ["java","-Dspring.profiles.active="$env_var_name,"-jar","/app/provider-service.jar"]
In the dockerfile , at last line(line no:12) , we add environment variable(line no:10), before ,the variable should be
active=stage"
Because , we're maintaining respective branch as per the environment. Now, we merged developer and stage environment into single script. We are facing some fetching issue. Pipeline was successful but it doesn't fetch.
I did not completely understand what is the issue you are mentioning, but I see you are missing ARG instruction in Dockerfile which is required to define the build-arg that you are passing if you want to use it in the build stage, in this case this arg "$environment_name"
You might change it like this
FROM maven:3.8.3-jdk-11 AS MAVEN_BUILD
COPY pom.xml /build/
COPY src /build/src/
WORKDIR /build/
RUN mvn clean install package -DskipTests=true
FROM openjdk:11
ARG environment_name
WORKDIR /app
COPY --from=MAVEN_BUILD /build/target/provider-service-*.jar /app/provider-service.jar
ENV PORT 8092
ENV env_var_name=$environment_name
EXPOSE $PORT
ENTRYPOINT ["java","-Dspring.profiles.active="$env_var_name,"-jar","/app/provider-service.jar"]

How should I dockerize and deploy a NestJS monorepo project?

I have NestJS monorepo project with structure as below:
...
apps
app1
app2
app3
...
If I got an idea correctly, I have possibility to run all the applications in same time, i.e. I run command and have access to apps by paths like http://my.domain/app1/, http://my.domain/app2/, http://my.domain/app3/ or in some similar way. And I need to put all apps in a docker container(s) and run them from there.
I haven't found something about this proceess. Did I undestand the idea correctly and where could I know more about deployment NestJS monorepo project?
This is how I solved it:
apps
app1
Dockerfile
...
app2
Dockerfile
...
app3
Dockerfile
...
docker-compose.yml
Each Dockerfile does the same:
FROM node:16.15.0-alpine3.15 AS development
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
FROM node:16.15.0-alpine3.15 AS production
ARG NODE_ENV=production
ENV NODE_ENV=${NODE_ENV}
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install --only=production --omit=dev
COPY --from=development /usr/src/app/dist ./dist
CMD ["npm", "run", "start-app1:prod"]
Where the last line should start the application so adjust that to your project naming.
Later you should build each of the images in your CI/CD pipeline and deploy them separately. To run the docker build from the root folder of the project you just need to provide a Dockerfile path for -f parameter, for example:
docker build -f apps/app1/Dockerfile -t app1:version1 .
docker build -f apps/app2/Dockerfile -t app2:version1 .
docker build -f apps/app3/Dockerfile -t app3:version1 .
To run it locally for tests, utilize docker-compose.yml
version: '3.8'
services:
app1:
image: app1:version1
ports:
- 3000:3000 # set according to your project setup
app2:
...
app3:
...
And start it by calling docker compose up

Docker multi-stage builds and Codeship running container

I'm doing a multi-stage Docker build:
# Dockerfile
########## Build stage ##########
FROM golang:1.10 as build
ENV TEMP /go/src/github.com/my-id/my-go-project
WORKDIR $TEMP
COPY . .
RUN make build
########## Final stage ##########
FROM alpine:3.4
# ...
ENV HOME /home/$USER
ENV TEMP /go/src/github.com/my-id/my-go-project
COPY --from=build $TEMP/bin/my-daemon $HOME/bin/
RUN chown -R $USER:$GROUP $HOME
USER $USER
ENTRYPOINT ["my-daemon"]
and the Makefile contains in part:
build: bin
go build -v -o bin/my-daemon cmd/my-daemon/main.go
bin:
mkdir $#
This all works just fine with a docker build.
Now I want to use Codeship, so I have:
# codeship-services.yml
cachemanager:
build:
image: my-daemon
dockerfile: Dockerfile
and:
# codeship-steps.yml
- name: my-daemon build
tag: master
service: my-service
command: true
The issue is if I do jet steps --master, it builds everything OK, but then runs the container as if I did a docker run. Why? I don't want it to do that.
It's as if I would have to have two separate Dockerfiles: one only for the build stage and one only for the run stage and use the former with jet. But then this defeats the point of Docker multi-stage builds.
I was able to solve this problem using multi-stage builds split into two different files following this guide: https://documentation.codeship.com/pro/common-issues/caching-multi-stage-dockerfile/
Basically, you'll take your existing Dockerfile and split it into two files like so, with the second referencing the first:
# Dockerfile.build-stage
FROM golang:1.10 as build-stage
ENV TEMP /go/src/github.com/my-id/my-go-project
WORKDIR $TEMP
COPY . .
RUN make build
# Dockerfile
FROM build-stage as build-stage
FROM alpine:3.4
# ...
ENV HOME /home/$USER
ENV TEMP /go/src/github.com/my-id/my-go-project
COPY --from=build $TEMP/bin/my-daemon $HOME/bin/
RUN chown -R $USER:$GROUP $HOME
USER $USER
ENTRYPOINT ["my-daemon"]
Then, in your codeship-service.yml file:
# codeship-services.yml
cachemanager-build:
build:
dockerfile: Dockerfile.build-stage
cachemanager-app:
build:
image: my-daemon
dockerfile: Dockerfile
And in your codeship-steps.yml file:
# codeship-steps.yml
- name: cachemanager build
tag: master
service: cachemanager-build
command: <here you can run tests or linting>
- name: publish to registry
tag: master
service: cachemanager-app
...
I don't think you want to actually run the Dockerfile because it will start your app. We use the second stage to push a smaller build to an image registry.

Resources