I have 2 Dockerfiles that have common arguments (ARG) that are passed to the actual commands (RUN) to build the images.
Is it possible to provide an external file with the arguments so that when I need to update one of them I don't need to touch both Dockerfiles?
An ARG is designed to be modified from the build command line, so you'd run:
docker build --build-arg VAR=value -t your_image .
That can be placed inside of a shell script to automate it and pass the same arg to each build.
You can also use a compose file, and the compose file may use environment variables or a .env file, to set variables used inside the compose file, e.g.
build:
context: ./your_app_dir
dockerfile: Dockerfile
args:
VAR: ${VALUE}
And the .env would contain:
VALUE=your_value
For more details on compose files, see the build syntax and also the environment file syntax.
Related
I have a CI variable that I would like to use within my docker file. I have tried to include it such as
ENV TESTING_UNIT=$TESTING_ID
It seems like that you need to specify to Dockerfile that it expects an argument variable
With the following approach it will be available in the container
Change:
ENV TESTING_UNIT=$TESTING_ID
To:
ARG TESTING_UNIT_ARG
ENV TESTING_UNIT=$TESTING_UNIT_ARG
and build the image: docker build --build-arg TESTING_UNIT_ARG=$TESTING_ID
I'm building a docker-compose.yml file that builds my custom Dockerfile and I would need to execute a bash command on my host system first, then the pass the results as build argument for the Dockerfile.
Here is an example in practice:
Dockerfile:
#...
ARG SSH_KEY_BASE64
RUN echo "Build SSH_KEY_BASE64: $SSH_KEY_BASE64"
#...
docker-compose.yml:
#...
version: '3.4'
services:
container:
container_name: my-container
build:
context: .
dockerfile: Dockerfile
args:
SSH_KEY_BASE64: ${SSH_KEY_BASE64_COMMAND}
env_file: .env
#...
.env:
SSH_KEY_BASE64_COMMAND=$(cat ~/.ssh/id_rsa_mykey | base64)
At the moment the value of $SSH_KEY_BASE64 in the Dockerfile is unresolved and it prints just $(cat ~/.ssh/id_rsa_mykey | base64), but I want it to evaluate that command and print the base64 of the content of my key.
I would like to avoid to manually run $(cat ~/.ssh/id_rsa_mykey | base64) before running docker-compose up --build that's why I'm asking for an automatic way to do that.
What options do I have?
Thanks
Compose doesn't support this syntax, and can't directly execute commands on the host system. The only substitution syntax it supports are $VARIABLE, ${VARIABLE}, ${VARIABLE:-default}, and ${VARIABLE:?error} environment variable expansion syntaxes, and that only in the main docker-compose.yml file. The values in an env_file: file aren't interpreted or expanded at all.
In most cases you don't actually want to build an image that depends on the specific host system it's built on; an image is intended to be reused in multiple environments. In the particular case of an ssh key it's particularly dangerous to pass it as an ARG since it can be pretty easily extracted from the final image (docker-compose run container cat /root/.id_rsa). You might need to do whatever operation needs the ssh key (for example, an authenticated git clone) on the host system outside of Docker.
The only workaround is to set a host environment variable and reference that instead, but it's probably better to get rid of the ARG entirely.
I have a docker-compose file that allows me to pass the environment variables as a file (.env file). As I have multiple ENV variables, Is there any option in Dockerfile like env_file in docker-compose for passing multiple environment variables during docker build?
This is the docker-compose.yml
services:
web:
image: "node"
links:
- "db"
env_file: "env.app"
AFAIK, there is no such way to inject environment variables using a file during the build step using Dockerfile. However, in most cases, people do end up using an entrypoint script & injecting variables during the docker run or docker-compose up.
In case it's a necessity you might need to write a shell wrapper which will change the values in the Dockerfile dynamically by taking a key-value pair text file as an input or make it something as below but the ENV file name need to be included in Dockerfile.
COPY my-env-vars /
RUN export $(cat my-env-vars | xargs)
It's an open issue - https://github.com/moby/moby/issues/28617
PS - You need to be extra careful while using this approach because the secrets are baked into the image itself.
I have a docker-compose.yml file and in the terminal I am typing docker-compose up [something] but I would also like to pass an argument to docker-compose.yml. Is this possible? I've read about interpolation variables and tried to specify a variable in the .yml file using ${testval} and then docker-compose up [something] var="test" but I receive the following error:
WARNING: The testval variable is not set. Defaulting to a blank string.
ERROR: No such service: testval=test
Based on dnephin answer, I created this sample repo that you can pass an variable to docker-compose up.
The usage is simple:
MAC / LINUX
TEST= docker-compose up to create and start both app and db container. The api should then be running on your docker daemon on port 3030.
TEST=DO docker-compose up to create and start both app and db container. The api should execute the npm run test inside the package.json file.
WINDOWS (Powershell)
$env:TEST="";docker-compose up to create and start both app and db container. The api should then be running on your docker daemon on port 3030.
$env:TEST="do";docker-compose up to create and start both app and db container. The api should execute the npm run test inside the package.json file.
You need to ensure 2 things:
The docker-compose.yml has the environment variable declared. For example,
services:
app:
image: python3.7
environment:
- "SECRET_KEY=${SECRET_KEY}"
have the variable available in the environment when docker-compose up is called:
SECRET_KEY="not a secret" docker-compose up
Note that this is not equivalent to pass them during build, as it is not advisable to store secrets in docker images.
You need to pass the variables as environment variables:
testvar=test docker-compose up ...
or
export testvar=test
docker-compose up
From the docs:
https://docs.docker.com/compose/reference/up/
https://docs.docker.com/compose/reference/build/
You can't pass arguments to docker-compose up, but you can pass arguments to docker-compose build:
docker-compose build --build-arg KEY1=VALUE1 --build-arg KEY2=VALUE2
I'm not sure what you want to do here, but if what you need is to pass an environmental variable to a specific container docker-compose.yml allows you to do that:
web:
...
environment:
- RAILS_ENV=production
- VIRTUAL_HOST=www.example.com
- VIRTUAL_PORT=3011
This variables will be specific for the container you specified them to, and wil not be shared between containers.
Also "docker-compose up" doesn't take any argument.
When dealing with build argumenets please declare them in compose yml file as follows
services:
app: (name of service
build:
context: docker/app/ (where is your docker build root)
dockerfile: Dockerfile (that is optional)
args:
- COMPOSER_AUTH_TOKEN (name of variable, value will be taken from host environment)
Well before running docker-compose up, export variable as other guys suggested. It will work. I tried. Use docker compose version 3 and above. Have fun
Compose supports declaring default environment variables in an environment file named .env placed in the project directory.
Step 1:
Create a file named .env in the project directory
Step 2:
Declare variables in the form VAR=VAL
NOTE: There is no special handling of quotation mark i.e. TESTVAL='test' means TESTVAL is 'test'(with quotation mark) and not just test. So you'd declare it as TESTVAL=test.
Step 3:
Use the variables in the Compose file as:
environment:
myval=${TESTVAL}
Documentation: Declare default environment variables in file
BONUS: If you are building image on the fly in you docker-compose.yaml, then you can even pass the build args using environment variables. Eg:
version: "3.8"
services:
myapp:
build:
context: ./myDir
dockerfile: ./myDir/myDockerfile
args:
- MYARG=${TESTVAL}
I was trying to find solution for batch file, based on Rafael Delboni answer you can add command inside batch file for calling powershell:
powershell $env:TEST="";docker-compose up ...
but instead of that because it's expensive to call powershell inside batch file you can initialize TEST variable inside batch file and then call your docker-compose command.
Something like this:
set TEST = ...
docker compose up ...
Docker 1.9 allows to pass arguments to a dockerfile.
See link: https://docs.docker.com/engine/reference/builder/#arg
How can I pass the same arguments within docker-compose.yml?
Please provide an example too, if possible.
Now docker-compose supports variable substitution.
Compose uses the variable values from the shell environment in which docker-compose is run. For example, suppose the shell contains POSTGRES_VERSION=9.3 and you supply this configuration in your docker-compose.yml file:
db:
image: "postgres:${POSTGRES_VERSION}"
When you run docker-compose up with this configuration, Compose looks for the POSTGRES_VERSION environment variable in the shell and substitutes its value in. For this example, Compose resolves the image to postgres:9.3 before running the configuration.
This can now be done as of docker-compose v2+ as part of the build object;
docker-compose.yml
version: '2'
services:
my_image_name:
build:
context: . #current dir as build context
args:
var1: 1
var2: c
See the docker compose docs.
In the above example "var1" and "var2" will be sent to the build environment.
Note: any env variables (specified by using the environment block) which have the same name as args variable(s) will override that variable.
This feature was added in Compose file format 1.6.
Reference: https://docs.docker.com/compose/compose-file/#args
services:
web:
build:
context: .
args:
FOO: foo
Something to add to these answers that the args are picked up only when using docker-compose up --build and not when using docker-compose build. If you want to build and run in separate steps, you need use docker-compose build --build-arg YOUR_ENV_VAR=${YOUR_ENV_VAR}or docker build --build-arg YOUR_ENV_VAR=${YOUR_ENV_VAR}
Create a variable environment on Linux shell:
export TAG=0.1.2
Set variable inside docker-compose.yml
db:
image: "redis:${TAG}"
Verify if value was replaced
docker-compose config