Use docker-compose env variable in Dockerbuild file - docker

Having the following docker-compose file:
db:
build: .
environment:
- MYSQL_ROOT_PASSWORD=password
- ENV=test
env_file: .env
Is there any way to use the env variables declared in docker-compose.yml (either as environment or declared in the env_file) as part of Dockerfile without declaring them in the Dockerfile? Something like this:
FROM java:7
ADD ${ENV}/data.xml /data/
CMD ["run.sh"]

Although this question was asked long ago, there is an answer to a similar question here: Pass environment variables from docker-compose to container at build stage
Basically, to use variables at the container's build time one has to define the variable in docker-compose.yml:
build:
context: .
args:
MYSQL_ROOT_PASSWORD: password
ENV: test
and then reference it in the Dockerfile using ARG:
ARG MYSQL_ROOT_PASSWORD
ARG ENV
ADD ${ENV}/data.xml /data/
Concerning environment variables defined in an *.env file, I believe that they can't be passed to the container at build time.

It works ok this way:
docker-compose.yml
version: '3.5'
services:
container:
build:
context: .
args:
ENV: ${ENV} # from .env file
env_file:
- .env
Dockerfile
# from compose args
ARG ENV
ADD ${ENV}/data.xml /data/
.env
ENV=myenv
Thus all the values are taken from .env file

This approach goes against the 'build once, run anywhere' theory behind Docker and most DevOps approaches. With this approach you'll need to build a container for every environment you expect to use. By doing so you can't safely say if a container works in the dev environment it will work in staging and production since you aren't using the same container.
You'd be better off adding all config files you need on to the container and writing an entrypoint script that selects/copies the data for that environment to the correct location when the container starts. You can also apply this approach to other config on the container, like templated Apache config using jinja2 templates etc.

Related

docker-compose: .env file is not loaded

My docker-compose.yml
version: '3'
services:
test:
build: .
My Dockerfile
FROM alpine:latest
ENTRYPOINT echo $ARG1 $ARG2
My .env file
ARG1=argument100
ARG2=argument200
If I run docker-compose run test the empty line got printed. However, it I add env_file section to my docker-compose.yml I am getting the expected output.
New docker-compose.yml
version: '3'
services:
test:
build: .
env_file:
- .env
Running docker-compose again
$ docker-compose run test
argument100 argument200
Am I using an illegal way to pass arguments to the ENTRYPOINT command inside the Docker? Why is .env not loaded automatically upon docker-compose run?
The .env file is used to set variables in the docker-compose command itself that can be used to expand variables in the docker-compose.yml file itself. The env_file specification will define variables inside the containers. They operate at two different levels.
There's also a third type of variable, build args, that can be injected into the building of an image.
Each of these has a distinct purpose, and are not multiple ways to do the same thing.

Docker compose ARGS not working with environment

Am trying to understand the args: usage in the docker-compose.yml file so that I can set the environment variables during build run-time. I understood that, if I had a .env file then it automatically sets it and am able to access inside my Dockerfile like,
FROM node:alpine
ARG NODE_ENV
ENV NODE_ENV=${NODE_ENV} // --> This works fine if .env file is present
But if I want to load these values through environment: (in docker-compose.yml and no .env file) like,
version: '3.3'
services:
web:
image: web
environment:
- NODE_ENV=dev
build:
context: .
args:
- NODE_ENV
ports:
- 8100
then it returns a blank/empty value inside Dockerfile.
I found that the environment variables are not directly accessible inside Dockerfile (or during build) but how env_file works when hard-coding the environment variable does not.
Advance thanks for your help.
Check your Docker version - in older versions of Docker the Dockerfile ENV command uses a space instead of '='
FROM node:alpine
ARG NODE_ENV
ENV NODE_ENV ${NODE_ENV}

docker-compose is not setting container_name from environment variable

I am working on a docker-compose file, in which I need to specify container_name from an environment variable.
My docker-compose.yml file looks like this:
version: '3.0'
services:
jenkins:
environment:
- INSTANCE_NAME=team_1
image: my_image
container_name: container_$INSTANCE_NAME
ports:
- "80:80"
expose:
- "80"
So, I think, when I run docker-compose up it should create container as name, container_team_1, but instead of that it runs as contaner_
I also tried this thing using .env file, but still, I can not use environment variable in container_name,
although, if I run docker-compose config I can see all variables set like follow,
container_name: container_
environment:
COMPANY_NAME: team_1
but, Actually it is not attaching in container-name.
You cannot use environment variables defined in docker-compose.yml to variable substitution.
Docker Compose uses .env by default so it should work when you define in .env file
INSTANCE_NAME=team_1
And then run docker-compose up
As I can see in variable substitution section of the docker-compose documentation, you will need to set your $INSTANCE_NAME in the shell that is running the docker-compose up, because:
Compose uses the variable values from the shell environment in which docker-compose is run.
First of all, do something like:
export INSTANCE_NAME=my_instance`
and then:
docker-compose up
Best regards.

docker-compose don't load additional env-file defined in yaml file

i get crazy with the env_file setting inside my docker-compose.yml.
when i name the file .env and put it in the root of my project everything works fine. the main point of having this env file was to have multiple enviroments (dev, staging, production) where i made my settings.
as soon i name my file different (for example dev.env and put it inside a env folder) and try to load it inside my docker-compose file it won't work.
i make a simple example to make things clear.
for simplicity i use one variable DC_DOCKERFILE where i define what dockerfile to load.
file: .env in the root folder:
COMPOSE_PROJECT_NAME=test
DC_DOCKERFILE=./Dockerfile.bash.dev
file: docker-compose.yml:
version: "2"
services:
bash:
build:
context: .
dockerfile: ${DC_DOCKERFILE}
and docker-compose config shows everything is cool and build works as well:
networks: {}
services:
bash:
build:
context: /devops/myproject
dockerfile: ./Dockerfile.bash.dev
version: '2.0'
volumes: {}
now the point which freaks me out:
create a folder env in the root of the project
move .env to ./env/dev.env
add env_file: ./env/dev.env inside my docker-compose.yml
env content looks same like above so i don't post the content again.
my new docker-compose.yml:
version: "2"
services:
bash:
env_file: ./env/dev.env
build:
context: .
dockerfile: ${DC_DOCKERFILE}
only change, add the env_file settings.
and now i docker-compose config gives me this:
networks: {}
services:
bash:
build:
context: /devops/myproject
dockerfile: ''
environment:
COMPOSE_PROJECT_NAME: test
DC_DOCKERFILE: ./Dockerfile.bash.dev
version: '2.0'
volumes: {}
say what? where the hack is my dockerfile settings?
of course docker-compose build don't like that too and bothers with:
docker-compose build --force-rm --no-cache
WARNING: The DC_DOCKERFILE variable is not set. Defaulting to a blank string.
Building bash
ERROR: Cannot locate specified Dockerfile: Dockerfile
the variables get loaded but somehow not assigned.
so i don't get it.
am i maybe wrong and i can't use the env file for docker-compose stuff inside the docker-compose.yml file. would make sence that it need to be loaded first somehow via docker-compose parameter or something but after i spend so much time i want to know if i can load it inside or not :)
thanks for help and hell yeah for open source and the awesome community.
peace out
When .env file is present in the folder docker-compose command is executed, those environment variables are used as environment variables for docker-compose execution and variable substitution. (https://docs.docker.com/compose/env-file/).
However when you define env_file option to your service, the service will get those variables from the file as environment variables and those are not used for variable substitution.

Specify the env file docker compose uses

Is it possible to specify the env file that docker compose uses for variable substitution? Currently it defaults to ".env" but I have different flavours of compose files which should have different envs.
You can use inheritance for this. If you have one "base" service where you set up the environment, all of your other services can inherit from that.
Example:
version: "2"
services:
base:
env_file:
- my_env.txt
web:
extends:
service: base
image: foo
database:
extends:
service: base
image: foo-db
The above example has everything in the same file, but you can also split this up into multiple files, where the base service would reside in a base.yaml file. You just need to add file: base.yaml to the extends section. Please see the documentation here.
I use this approach for setting the proxy variables for all containers. I have a proxy.yaml file that defines a proxy-app service that picks up the proxy environment variables from the shell. All of my real services extend the proxy-app service and thus inherit the environment settings from that service.
The --env-file command-line argument and the env_file docker-compose.yml variable specify the env file to use for the container, not for the container build. To set a different file (e.g. alt.env) for the build itself, use this:
env $(cat alt.env) docker-compose up --build
According to the documentation, it's now possible to load an environment file (contrary to a per-service file), docker-compose will then export the env variables defined in this env file prior to starting any service, they can then be used in the docker-compose.yml config file itself:
version: "3.7"
services:
node:
environment:
APP_ENV: "${APP_ENV}"
NODE_ENV: "${NODE_ENV}"
ports:
- "${HOST_EXPOSED_NODEJS_DEBUG_PORT}:9229"
volumes:
- type: bind
source: ./project
target: /var/www/project
read_only: false
Since docker-compose 1.25 it's also possible to specify a custom .env file with the --env-file flag (unfortunately it's currently not possible to specify multiple .env files with the --env-file flag)

Resources