Am trying to understand the args: usage in the docker-compose.yml file so that I can set the environment variables during build run-time. I understood that, if I had a .env file then it automatically sets it and am able to access inside my Dockerfile like,
FROM node:alpine
ARG NODE_ENV
ENV NODE_ENV=${NODE_ENV} // --> This works fine if .env file is present
But if I want to load these values through environment: (in docker-compose.yml and no .env file) like,
version: '3.3'
services:
web:
image: web
environment:
- NODE_ENV=dev
build:
context: .
args:
- NODE_ENV
ports:
- 8100
then it returns a blank/empty value inside Dockerfile.
I found that the environment variables are not directly accessible inside Dockerfile (or during build) but how env_file works when hard-coding the environment variable does not.
Advance thanks for your help.
Check your Docker version - in older versions of Docker the Dockerfile ENV command uses a space instead of '='
FROM node:alpine
ARG NODE_ENV
ENV NODE_ENV ${NODE_ENV}
Related
TL/DR: Can I use .env file variables in docker-compose's environment directives?
I have a Dockerfile which uses the ARG variable to copy files based on an environment variable.
In my docker-compose I want to run two versions of this container with different configuration.
Is there a way where I can set different filepaths in a single .env file and have it build twice, like the example below?
Or is there a smarter way to accomplish this?
/
/.env
/docker-compose.yml
/app
/app/Dockerfile
/version1/data
/version2/data
/.env
VERSION_1_PATH=./version1/data
VERSION_2_PATH=./version2/data
/app/Dockerfile
FROM node:latest
ARG APP_PATH # pull this environment variable
COPY $APP_PATH /var/app/ # use it to copy source to the same generic destination
/docker-compose.yml
version: "3"
services:
version1:
build: ./app
container_name: version1
env_file:
- '.env'
environment:
APP_PATH: ${VERSION_1_PATH}
version2:
build: ./app
container_name: version2
env_file:
- '.env'
environment:
APP_PATH: ${VERSION_2_PATH}
You can add args in compose file when to define build, something like follows:
version: '3'
services:
version1:
build:
context: ./app
args:
- APP_PATH=${VERSION_1_PATH}
version2:
build:
context: ./app
args:
- APP_PATH=${VERSION_2_PATH}
And no need to define .env in env_file if just want it be used in build as .env could default be used in docker-compose.yml. And, environment also not for build, it's for running container.
One example, FYI.
I have the following docker file created for the spring boot application
FROM openjdk:8
ENV environment default
ADD target/docker-sample-example.jar docker-sample-example.jar
EXPOSE 8087
ENTRYPOINT ["java","-jar","-Dspring.profiles.active=${environment}","docker-
sample-example.jar"]
i want to set the value "environment" in the docker-compose , or pass the value through bamboo environment variable to make it available during run time to pick the proper profile while deploying into rancher .
I have tried the following edit in docker-compose
version: "2"
services:
svc-get-interactions:
image: myaccount.dkr.ecr.ap-southeast-2.amazonaws.com/app:latest
environment:
- environment = dev
ports:
- 9898:8082/tcp
labels:
io.rancher.container.pull_image: always
but the variable was not picked while running the command "rancher compose up"
am i missing something here ?
Try to use:
environment:
environment: "dev"
or
environment:
- environment='dev'
On the docker-compose.yml file. Both work for me.
i get crazy with the env_file setting inside my docker-compose.yml.
when i name the file .env and put it in the root of my project everything works fine. the main point of having this env file was to have multiple enviroments (dev, staging, production) where i made my settings.
as soon i name my file different (for example dev.env and put it inside a env folder) and try to load it inside my docker-compose file it won't work.
i make a simple example to make things clear.
for simplicity i use one variable DC_DOCKERFILE where i define what dockerfile to load.
file: .env in the root folder:
COMPOSE_PROJECT_NAME=test
DC_DOCKERFILE=./Dockerfile.bash.dev
file: docker-compose.yml:
version: "2"
services:
bash:
build:
context: .
dockerfile: ${DC_DOCKERFILE}
and docker-compose config shows everything is cool and build works as well:
networks: {}
services:
bash:
build:
context: /devops/myproject
dockerfile: ./Dockerfile.bash.dev
version: '2.0'
volumes: {}
now the point which freaks me out:
create a folder env in the root of the project
move .env to ./env/dev.env
add env_file: ./env/dev.env inside my docker-compose.yml
env content looks same like above so i don't post the content again.
my new docker-compose.yml:
version: "2"
services:
bash:
env_file: ./env/dev.env
build:
context: .
dockerfile: ${DC_DOCKERFILE}
only change, add the env_file settings.
and now i docker-compose config gives me this:
networks: {}
services:
bash:
build:
context: /devops/myproject
dockerfile: ''
environment:
COMPOSE_PROJECT_NAME: test
DC_DOCKERFILE: ./Dockerfile.bash.dev
version: '2.0'
volumes: {}
say what? where the hack is my dockerfile settings?
of course docker-compose build don't like that too and bothers with:
docker-compose build --force-rm --no-cache
WARNING: The DC_DOCKERFILE variable is not set. Defaulting to a blank string.
Building bash
ERROR: Cannot locate specified Dockerfile: Dockerfile
the variables get loaded but somehow not assigned.
so i don't get it.
am i maybe wrong and i can't use the env file for docker-compose stuff inside the docker-compose.yml file. would make sence that it need to be loaded first somehow via docker-compose parameter or something but after i spend so much time i want to know if i can load it inside or not :)
thanks for help and hell yeah for open source and the awesome community.
peace out
When .env file is present in the folder docker-compose command is executed, those environment variables are used as environment variables for docker-compose execution and variable substitution. (https://docs.docker.com/compose/env-file/).
However when you define env_file option to your service, the service will get those variables from the file as environment variables and those are not used for variable substitution.
When I run docker-compose build && docker-compose up redis, with environment specified in docker-compose.yaml and RUN env in the Dockerfile, the environment variables I set don't get printed.
Why does this not work?
I'm using docker-compose version 1.4.2.
Here are the relevant files:
docker-compose.yaml with environment as a list of KEY=value pairs:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO='bar'
docker-compose.yaml with environment as a dictionary:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO: 'bar'
Dockerfile:
FROM redis:2.6
MAINTAINER me#email.com
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
That's normal
docker-compose only sets the environment variables specified in the environment directive in the docker-compose.yaml file during the run phase of the container, and not during the build phase.
So if you do docker-compose run --entrypoint "/bin/bash" redis -c env you will be able to see your env variables.
If you want to set variables inside your Dockerfile (to be able to see them during the build phase) you can add inside your dockerfile before your RUN env:
ENV FOO bar
Well
I have tested and found following solutions for docker compose with env file or without env file. I will show you two different approach
Lets say you have following docker compose yml file
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
env_file: docker.env
Now you need to setup the postgres variable in a file called docker.env. Remember you need to keep the docker_compose.yml file and docker.env file in same folder.
Next, In the docker.env file you need to have the database variable and value like this:
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=myapp_db
Now hit docker-compose up . It should work.
Lets say now you dont like to specify the env file name in the docker-compose.yml file. So you have to write docker-compose.yml file like this:
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
environments:
- POSTGRES_USER=${PGU}
-POSTGRES_PASSWORD=${PGP}
-POSTGRES_DB=${PGD}
Now your docker.env file should look like this:
PGU=postgres
PGP=postgres
PGD=myapp_db
now hit docker-compose --env-file docker.env up
you are good to go.
This is because you were using environment when (I guess) you wanted to use args inside the build block:
redis:
build:
context: ../storage/redis
args:
- FOO: 'bar'
ports:
- "6379:6379"
Your Dockerfile would define FUN in the (image) environment:
FROM redis:2.6
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
# Read FUN from (build) arguments
# (may define a default: ARG FUN='wow')
ARG FUN
# Define env variable FUN with value from ARG
ENV FUN=$FUN
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
The environment block is used to define variables for the running container (when docker-compose up, NOT when docker-compose build).
Having the following docker-compose file:
db:
build: .
environment:
- MYSQL_ROOT_PASSWORD=password
- ENV=test
env_file: .env
Is there any way to use the env variables declared in docker-compose.yml (either as environment or declared in the env_file) as part of Dockerfile without declaring them in the Dockerfile? Something like this:
FROM java:7
ADD ${ENV}/data.xml /data/
CMD ["run.sh"]
Although this question was asked long ago, there is an answer to a similar question here: Pass environment variables from docker-compose to container at build stage
Basically, to use variables at the container's build time one has to define the variable in docker-compose.yml:
build:
context: .
args:
MYSQL_ROOT_PASSWORD: password
ENV: test
and then reference it in the Dockerfile using ARG:
ARG MYSQL_ROOT_PASSWORD
ARG ENV
ADD ${ENV}/data.xml /data/
Concerning environment variables defined in an *.env file, I believe that they can't be passed to the container at build time.
It works ok this way:
docker-compose.yml
version: '3.5'
services:
container:
build:
context: .
args:
ENV: ${ENV} # from .env file
env_file:
- .env
Dockerfile
# from compose args
ARG ENV
ADD ${ENV}/data.xml /data/
.env
ENV=myenv
Thus all the values are taken from .env file
This approach goes against the 'build once, run anywhere' theory behind Docker and most DevOps approaches. With this approach you'll need to build a container for every environment you expect to use. By doing so you can't safely say if a container works in the dev environment it will work in staging and production since you aren't using the same container.
You'd be better off adding all config files you need on to the container and writing an entrypoint script that selects/copies the data for that environment to the correct location when the container starts. You can also apply this approach to other config on the container, like templated Apache config using jinja2 templates etc.