I am working on a docker-compose file, in which I need to specify container_name from an environment variable.
My docker-compose.yml file looks like this:
version: '3.0'
services:
jenkins:
environment:
- INSTANCE_NAME=team_1
image: my_image
container_name: container_$INSTANCE_NAME
ports:
- "80:80"
expose:
- "80"
So, I think, when I run docker-compose up it should create container as name, container_team_1, but instead of that it runs as contaner_
I also tried this thing using .env file, but still, I can not use environment variable in container_name,
although, if I run docker-compose config I can see all variables set like follow,
container_name: container_
environment:
COMPANY_NAME: team_1
but, Actually it is not attaching in container-name.
You cannot use environment variables defined in docker-compose.yml to variable substitution.
Docker Compose uses .env by default so it should work when you define in .env file
INSTANCE_NAME=team_1
And then run docker-compose up
As I can see in variable substitution section of the docker-compose documentation, you will need to set your $INSTANCE_NAME in the shell that is running the docker-compose up, because:
Compose uses the variable values from the shell environment in which docker-compose is run.
First of all, do something like:
export INSTANCE_NAME=my_instance`
and then:
docker-compose up
Best regards.
Related
I am finding my self repeating like this:
services:
noice-service:
environment:
- EUREKA_SERVICE_URL=${EUREKA_SERVICE_URL}
- ZIPKIN_BASE_URL=${ZIPKIN_BASE_URL}
- CONFIG_SERVER_URL=${CONFIG_SERVER_URL}
I have defined these env vars in .env file and some in another scripts and I just want to pass their exact value in container. Is there any way quicker way of achieving this without any custom shell script as entrypoint ?
You can pass the variables directly:
# .env
DOTENV=foo
# docker-compose.yml
version: "3.7"
services:
busybox:
image: busybox
command: ["env"]
environment:
- DOTENV
- ANOTHER
And run ANOTHER=bar docker-compose up.
I have a playground project for Docker Compose with the file like this:
version: '3'
services:
mysql:
image: 'mysql:8'
container_name: '${PROJECT_NAME}_mysql'
hostname: '${PROJECT_NAME}_mysql'
networks:
- internal
ports:
- '127.0.0.1:${MYSQL_EXPOSE_PORT}:3306'
volumes:
- mysql:/var/lib/mysql
env_file:
- ./mysql/.env
environment:
MYSQL_EXPOSE_PORT: '${MYSQL_EXPOSE_PORT}'
MYSQL_ROOT_PASSWORD: '${MYSQL_ROOT_PASSWORD}'
MYSQL_USER: '${MYSQL_USER}'
MYSQL_PASSWORD: '${MYSQL_PASSWORD}'
volumes:
mysql:
networks:
internal:
My ./mysql/.env is this:
MYSQL_DATABASE=foobar
MYSQL_ROOT_PASSWORD=root
MYSQL_USER=web
MYSQL_PASSWORD=web
And my .env file looks like this:
COMPOSE_PROJECT_NAME=foobar
PROJECT_NAME=foobar
MYSQL_EXPOSE_PORT=33061
MYSQL_DATABASE=foobar
MYSQL_ROOT_PASSWORD=root
MYSQL_USER=web_override
MYSQL_PASSWORD=web_override
I read somewhere that env_file and environment create variables that will be available in the container itself, and --env-file supplies variables that will be available during the processing of docker-composer.yml file.
But running the:
docker-compose -f docker-compose.yml --env-file .env up -d
produces, for me, unexpected behavior.
If I omit both the env_file and the environment configs, MySQL won't start due to an empty password. Variables from --env-file are ignored. Does that mean that MYSQL_ROOT_PASSWORD is in fact required as container env?
If I put env_file only, MySQL starts, but uses credentials from ./mysql/.env. It silently ignores .env file supplied via --env-file option
Lastly, if I put both of env_file and environment, it will honor the value from --env-file due to variable substitution. But exec-ing into the container and echoing $MYSQL_ROOT_PASSWORD seems rarely ugly and screams security vulnerability waiting to happen.
It seems to me that I am having a fundamental misunderstanding of how Docker Compose env variables work.
Can someone please provide some sort of clarification to these questions?
There are two phases of processing environment variables:
At the Compose level, it takes its own environment, and also reads the docker compose --env-file file, or if you don't specify that option, .env. It then uses these environment variables to do variable substitution in the docker-compose.yml file.
Each container has its own environment, as specified by the Compose environment: or env_file: directives.
That means that setting something in the outer environment or putting it in the --env-file file (step 1) does not automatically make it visible to a container (step 2).
This is consistent with your observations. In the first two cases, specifying --env-file doesn't put anything in the per-container configuration, it only affects the environment variable expansion, and so from the container's point of view it has no effect. In the third case, you have the correct sequence: --env-file sets variables at the Compose level, variable substitution sets fixed strings in the environment: block, and then environment: takes precedence over env_file:.
When I define the same environment variable in docker-compose.yml using environment key (by not giving them a value) and also using env_file key, the first take priority even if it is not defined in the shell.
I'm using docker-compose version 1.23.1, build b02f1306
In the documentation says:
When you set the same environment variable in multiple files, here’s the priority used by Compose to choose which value to use:
Compose file
Shell environment variables
Environment file
Dockerfile
Variable is not defined
I assume that if the variable is not defined (point 5) in the shell it does not have priority over the variable defined in the environment file.
docker-compose.yml
version: '3'
services:
db:
image: "postgres:11-alpine"
environment:
- POSTGRES_USER
- POSTGRES_PASSWORD
env_file: "db-variables.env"
db-variables.env
POSTGRES_USER=bob
POSTGRES_PASSWORD=password
Result:
$ docker-compose run --entrypoint printenv db
Creating network "compose-example_default" with the default driver
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=105881c75c8c
TERM=xterm
LANG=en_US.utf8
PG_MAJOR=11
PG_VERSION=11.1
PG_SHA256=90815e812874831e9a4bf6e1136bf73bc2c5a0464ef142e2dfea40cda206db08
PGDATA=/var/lib/postgresql/data
HOME=/root
I expect shell pass environment variables environment only take priority over env_file variables when they are really defined in the shell.
Have I misunderstood the documentation? Is there any workaround?
As mentioned in the doc:
When you run the container, the environment variable defined in the Compose file takes precedence.
Define the variables to view as I think values are not set to the variables.
docker-compose.yml
version: '3'
services:
db:
image: "postgres:11-alpine"
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=password
env_file: "db-variables.env"
db-variables.env
POSTGRES_USER=bob
POSTGRES_PASSWORD=password
Result:
$ docker-compose run --entrypoint printenv db
Creating network "compose-example_default" with the default driver
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=8fcbf617bb6d
TERM=xterm
POSTGRES_USER=user
POSTGRES_PASSWORD=password
LANG=en_US.utf8
PG_MAJOR=11
PG_VERSION=11.3
PG_SHA256=2a85e082fc225944821dfd23990e32dfcd2284c19060864b0ad4ca537d30522d
PGDATA=/var/lib/postgresql/data
HOME=/root
In another section of the documentation on the environment variables I found this:
Environment variables declared in the environment section override these values this holds true even if those values are empty or undefined.
RTFM to my self.
Is it possible to specify the env file that docker compose uses for variable substitution? Currently it defaults to ".env" but I have different flavours of compose files which should have different envs.
You can use inheritance for this. If you have one "base" service where you set up the environment, all of your other services can inherit from that.
Example:
version: "2"
services:
base:
env_file:
- my_env.txt
web:
extends:
service: base
image: foo
database:
extends:
service: base
image: foo-db
The above example has everything in the same file, but you can also split this up into multiple files, where the base service would reside in a base.yaml file. You just need to add file: base.yaml to the extends section. Please see the documentation here.
I use this approach for setting the proxy variables for all containers. I have a proxy.yaml file that defines a proxy-app service that picks up the proxy environment variables from the shell. All of my real services extend the proxy-app service and thus inherit the environment settings from that service.
The --env-file command-line argument and the env_file docker-compose.yml variable specify the env file to use for the container, not for the container build. To set a different file (e.g. alt.env) for the build itself, use this:
env $(cat alt.env) docker-compose up --build
According to the documentation, it's now possible to load an environment file (contrary to a per-service file), docker-compose will then export the env variables defined in this env file prior to starting any service, they can then be used in the docker-compose.yml config file itself:
version: "3.7"
services:
node:
environment:
APP_ENV: "${APP_ENV}"
NODE_ENV: "${NODE_ENV}"
ports:
- "${HOST_EXPOSED_NODEJS_DEBUG_PORT}:9229"
volumes:
- type: bind
source: ./project
target: /var/www/project
read_only: false
Since docker-compose 1.25 it's also possible to specify a custom .env file with the --env-file flag (unfortunately it's currently not possible to specify multiple .env files with the --env-file flag)
I'm using docker compose to run my application. And for do that I need to set the hosts inside container (it's depends on the environment i'm running).
My approach was:
Create an environment file and set the variable:
#application.env
SERVER_IP=10.10.9.134
My docker compose file looks like:
version: '2'
services:
api:
container_name: myApplication
env_file:
- application.env
build: ./myApplication/
entrypoint: ./docker/api-startup.sh
ports:
- "8080:8080"
depends_on:
- redis
extra_hosts: &extra_hosts
myip: $SERVER_IP
But my problem is that the variable SERVER_IP is never replaced.
When I run docker-compose config I see:
services:
api:
build:
context: /...../myApplication
container_name: myApplication
depends_on:
- redis
entrypoint: ./docker/api-startup.sh
environment:
SERVER_IP: 10.10.9.134
extra_hosts:
myip: ''
ports:
- 8080:8080
I've tried to replace the variable reference using $SERVER_IP or ${SERVER_IP} but it didn't work.
I created a file .env, added single line HOST=test.example.com, then did this in docker-compose:
extra_hosts:
- myip:${HOST}
docker-compose config then shows
extra_hosts:
myip: test.example.com
To do this I followed the documentation from Docker-compose environment variables the section about .env file
UPDATE
According to the Docker documentation,
Note: If your service specifies a build option, variables defined in
environment files will not be automatically visible during the build.
Use the args sub-option of build to define build-time environment
variables.
It basically means if you place your variables in .env file, you can use them for substitution in docker-compose.yml, but if you use env_file option for the particular container, you can only see the variables inside the Docker container, not during the build. It is also logical, env_file replaces docker run --env-file=FILE ... and nothing else.
So, you can only place your values into .env. Alternatively, as William described, you can use host's environment variables.
EDIT
Try the following:
version: '2'
services:
api:
container_name: myApplication
env_file:
- application.env
build: ./myApplication/
entrypoint: ./docker/api-startup.sh
ports:
- "8080:8080"
depends_on:
- redis
extra_hosts:
- "myip:${SERVER_IP}"
Ensure curly bracers and that the environment variable exists on the host os.