I have a command like this:
docker build -f ${DOCKERFILE} ${BUILD_ARGS} -t ${IMAGE} ${BUILD_CONTEXT}
When I pass these environment variables it works:
IMAGE_SUBNAME: 'frontend'
DOCKERFILE: ./frontend/Dockerfile
BUILD_CONTEXT: ./frontend
But when I try to pass these, it throw me error (see below):
IMAGE_SUBNAME: 'frontend'
DOCKERFILE: ./frontend/Dockerfile
BUILD_CONTEXT: ./frontend
BUILD_ARGS: ENV=${CI_COMMIT_REF_SLUG} // CI_COMMIT_REF_SLUG equals to "dev"
"docker build" requires exactly 1 argument.
I have already read a lot of articles, but I can't find my mistake. Hope to get help.
Related
I have:
docker-compose.yml
version: "3.9"
services:
test_name:
image: ${PROJECT_NAME}/test_service
build:
dockerfile: Dockerfile
env_file: .env
Dockerfile
FROM alpine:3.15
RUN echo $TEST >> test1.txt
CMD echo $TEST >> test2.txt
As result:
test1.txt - empty and test2.txt with data.
My problem is that this variables are too much, so can I get environment variables in RUN command from .env file without enumeration all of them in ARG?
To use variables in a RUN instruction, you need to use ARG. ARG are available at build time while ENV is available when the container runs.
FROM alpine:3.15
ARG FOO="you see me on build"
ENV BAR="you see me on run"
RUN echo $FOO >> test1.txt
CMD echo $BAR >> test2.txt
docker build --build-arg FOO="hi" --tag test .
docker run --env BAR="there" test
There is one thing that comes close to using env variables, but you still need to provide the --build-arg flag.
You can define env variable with the same name as the build arg and reference it by its name without setting a value. The value will be taken from the env variable in your shell.
export FOO="bar"
docker build --build-arg FOO --tag test .
This also works in compose.
Additionally, when you use compose you can place a .env file next to your compose file. Variables found there will be read and are available in the build:arg key as well as the environment key, But you still have to name them.
# env file
FOO=bar
BAZ=qux
services:
test_name:
build:
context: ./
args:
FOO:
BAZ:
I am working on a nextjs docker project trying to pass my environment variables to be accessed at container build time and run time. I basically referred to Nextjs's template Dockerfile and passed my required env vars both as build args and environment in my docker-compose.yml as such
version: "3.7"
services:
app:
container_name: frontend
build:
context: .
args:
- API_URL=http://path-to-my-external-api-url
- NEXT_PUBLIC_CLIENT_API_URL=http://path-to-my-external-api-url
environment:
- NODE_ENV=production
- API_URL=http://path-to-my-external-api-url
- NEXT_PUBLIC_CLIENT_API_URL=http://path-to-my-external-api-url
ports:
- "3000:3000"
nginx:
depends_on:
- app
container_name: frontend-nginx
build: ./nginx
ports:
- "8080:8080"
Then I access these env vars in my Dockerfile like
...
FROM node:14-alpine AS builder
WORKDIR /app
COPY ./app .
COPY --from=deps /app/node_modules ./node_modules
ARG API_URL
ARG NEXT_PUBLIC_CLIENT_API_URL
ENV API_URL=${API_URL}
ENV NEXT_PUBLIC_CLIENT_API_URL=${NEXT_PUBLIC_CLIENT_API_URL}
RUN npm run build
...
Then the app is built in gitlab pipeline which fails at the build stage defined below
...
Build and Push App:
image: docker:19.03.5
services:
- docker:19.03.5-dind
stage: Build and Push
script:
- apk add python3
- pip3 install awscli
- docker build --compress -t $ECR_REPO:$CI_COMMIT_SHORT_SHA .
- $(aws ecr get-login --no-include-email --region ap-south-1)
- docker push $ECR_REPO:$CI_COMMIT_SHORT_SHA
- docker tag $ECR_REPO:$CI_COMMIT_SHORT_SHA $ECR_REPO:latest
- docker push $ECR_REPO:latest
rules:
- if: '$CI_COMMIT_BRANCH =~ /^(main|production)$/'
...
I was able to build and run my container locally. But it fails during the above gitlab ci pipeline stage with Error: connect ECONNREFUSED 127.0.0.1:80 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16) which I assume is related to inability to load these environment variables.
How do I fix this? Is is needed to even pass the variables as args and environment twice in the docker-compose file (I did this because I found that only when passing them both as args and environment works in building my container locally).
This is my first time working with ci/cd and docker and any corrections to my code or suggestions to optimize it is much appreciated. Thanks for you time.
EDIT: Fix hyperlink error
In the Dockerfile you posted the args are declared, however, you do not load them at build time on the docker build command with the --build-arg flag, on a command similar to this:
docker build --compress -t $ECR_REPO:$CI_COMMIT_SHORT_SHA --build-arg API_URL=<url> NEXT_PUBLIC_CLIENT_API_URL=<url> .
Reference: https://docs.docker.com/engine/reference/commandline/build/#options
I have a docker compose file that builds and runs a couple of dockerfiles locally. I am using multi-stage builds and I pass an argument into my docker build commands. The following works perfectly:
docker build -t local/admin-api --build-arg SONAR_TOKEN=XXXXXXXXXXXXXXX .
I am trying to replicate that in my docker compose file with:
services:
comply:
build:
context: ./websites/comply/
dockerfile: Dockerfile
args:
SONAR_TOKEN: xxxxxxxxxxx
ports:
- "8080:80"
In my dockerfile I use:
FROM kinectify.azurecr.io/buildbase:ci-631 as build
ARG PAT
ARG SONAR_PROJECT_KEY=xxxxxxx
ARG SONAR_ORGANIZATION_KEY=xxxxxxx
ARG SONAR_HOST_URL=https://sonarcloud.io
ARG SONAR_TOKEN
RUN echo "Token: ${SONAR_TOKEN} | hostURL: ${SONAR_HOST_URL}"
I am aware of the scoping of build args, and as you can see above, the arg is defined after my FROM statement. I have also tried setting the arg to an environment variable, but that doesn't seem to matter. Additionally, the SONAR_HOST_URL is being output correctly in echo statement (and in the place where I am actually using the argument). I am running the build with:
docker compose build comply
** UPDATE **
Strangely, when I run docker compose up -d --build it does pass variable. I am not sure what the difference is though.
This is a known issue that's been fixed in docker compose. You are likely waiting for the next release still. Using docker-compose instead of docker compose should also work while you wait for that release.
How can I convert this command below to docker-compose version?
docker build -t xxx --build-arg SSH_PRV_KEY="$(cat ~/.ssh/id_rsa)" .
I try this block below, but it does not work. Please help. Thanks.
xxx:
build:
context: .
dockerfile: Dockerfile
args:
SSH_PRV_KEY: "$(cat ~/.ssh/id_rsa)"
docker-compose doesn't undershell shell code like that. You can do it this way:
xxx:
build:
context: .
dockerfile: Dockerfile
args:
SSH_PRV_KEY
Now, before run docker-compose, export your SSH_PRV_KEY env var:
export SSH_PRV_KEY="$(cat ~/.ssh/id_rsa)"
# now run docker-compose up as you normally do
Then SSH_PRV_KEY will have the right value.
Two thing you need to consider:
It may not work as expected if you have pass pharase in your id_rsa.
This SSH_PRV_KEY will actually available to docker meta data such as docker history or images inspect. To get around that you should look into multi stage build https://docs.docker.com/develop/develop-images/multistage-build/. In your build steps, you use that key to do anything you want. Then in your final image, don't declare SSH_PRV_KEY but simply copy the result from previous image. A more specific example where you use a private key to install dependencies
FROM based as build
ARG SSH_PRV_KEY
RUN echo "$SSH_PRV_KEY" > ~/.ssh/id_rsa
RUN npm install # this may need access to that rsa key
FROM node
COPY --from=builder node_modules node_modules
Notice in second images, we don't declare ARG therefore we don't expose it.
I want to build via the docker-compose an image that uses my private key for cloning private git repos.
More or less the compose becomes as follows:
myservice:
build:
context: .
args:
RSA: ~/.ssh/id_rsa
The above does not work, neither the following:
myservice:
build:
context: .
args:
RSA: $(cat ~/.ssh/id_rsa)
The docker build command works just fine however in the form of
docker build --build-args RSA=$(cat ~/.ssh/id_rsa) -t myservice:latest
You can use the same syntax with docker-compose build:
docker-compose build --build-arg RSA="$(cat ~/.ssh/id_rsa)"
Unfortunately, you can't use the build-args option with compose up or start... So you will need to build and then run using the --no-build option
One way to do it that will work when building all the services and also with up is to pass the SSH key data as an environnement variable like this:
env RSA=$(cat ~/.ssh/id_rsa) docker-compose build
And then you set the build args in the compose file like this:
myservice:
build:
context: .
args:
RSA: |-
${RSA}
add ARGS in your Dockerfile like
ARGS RSA
For it to be read upon build