As far as I know, to use the env variable in docker-compose we need to create a .env file
PORT=9000
And then use it like this in docker-compose.yml:
services:
go:
ports:
- ${PORT}:${PORT}
But what if I have a config file in .yml file like this:
http:
host: "0.0.0.0"
port: "1020"
jwt_secret: "secret"
How to access it on the docker-compose? This doesn't work:
version: "3.8"
services:
go:
env_file: config.yml
build:
context: .
dockerfile: Dockerfile
command: /fetch-app
ports:
- ${http.port}:${http.port}
environment:
NODE_ENV: development
It is generates error:
invalid interpolation format for services.go.ports.[].
You may need to escape any $ with another $.
${http.port}:${http.port}
Related
I have a simple docker compose file to create a Mysql database for my app. But I cannot interpolate the environment variable MYSQL_PORT to set a custom port. Running docker compose up with the configuration below results in a random port being assigned to mysql.
The path to the env file does work, since I have env variables configuring the database.
docker-compose.yml
version: '3'
services:
mysql:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
volumes:
- mysql_data:/var/lib/mysql
env_file:
- ../../.env
ports:
- ${MYSQL_PORT}:3306
volumes:
mysql_data:
.env
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=root
MYSQL_DATABASE=final_project_database
MYSQL_USER=db_user
MYSQL_PASSWORD=some_db_user_password
Use --env-file option with docker-compose up command. env_file declared in your MySQL service applies only for container env
Move your .env file to the same directory as the docker-compose file and change the env_file to point to it. That way both docker-compose and the container will use the same environment file.
Right now it's only the container that's using it.
version: '3'
services:
mysql:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
volumes:
- mysql_data:/var/lib/mysql
env_file:
- ./.env
ports:
- ${MYSQL_PORT}:3306
volumes:
mysql_data:
I have the .env file as yaml format, in other words I have development.yaml file which consists of following let's say:
server:
port: 3000
apiPrefix: api/v1
swagger:
title: Test
description: test description
version: 1.0
api: /docs
database:
name: test
autoLoadModels: true
synchronize: false
dialect: postgres
and inside docker-compose.yaml
version: '3.8'
services:
dev:
env_file:
- config/development.yaml
container_name: test_development_api_docker_container
image: test_development_api_docker_container:1.0.0
build:
context: .
target: testDevelopmentEnv
dockerfile: ./Dockerfile
command: npm run start:debug
ports:
- ${port}:${port}
networks:
- test_network
volumes:
- .:/Users/falyoun/dev/ite/test
- /Users/falyoun/dev/ite/test/node_modules
restart: unless-stopped
networks:
test_network:
In the above file I tried:
${port}
${server.port}
and all causes issues like
ERROR: Invalid interpolation format for "ports" option in service "dev": "${server.port}:${server.port}"
or
ERROR: The Compose file './docker-compose.yaml' is invalid because:
services.dev.ports contains an invalid type, it should be a number, or an object
Before you try anything, you should switch from:
docker-compose
to:
docker compose
The docker-compose tool has been deprecated, and is at this point a fairly stale version (1.25.X) as installed by apt install. Instead, you should remove docker-compose from your computer and install docker-compose-plugin (latest version as of today is 2.6.X).
I'm trying to pass some host's environment variables content as args to the docker-compose file, through a .env file. But the variable is interpreted as a string.
Follows the content of my files:
.env:
USER=$USER
UID=$UID
GID=$GID
docker-compose.yml:
version: "2"
services:
opencv_python:
build:
args:
- username=${USER}
- uid=${UID}
- gid=${GID}
context: .
dockerfile: opencv_base.Dockerfile
container_name: ocv-data-augmentation
image: ocv-data-augmentation
environment:
DISPLAY: $DISPLAY
QT_X11_NO_MITSHM: 1
volumes:
- "../project:/home/&{USER}/data_augmentation/" # Host : Container
- "/tmp/.X11-unix:/tmp/.X11-unix"
tty: true
And this is the output of the command docker-compose config:
services:
opencv_python:
build:
args:
gid: $$GID
uid: $$UID
username: fsalvagnini
context: /home/fsalvagnini/Documents/containers/data_augmentation/dockerfiles
dockerfile: opencv_base.Dockerfile
container_name: ocv-data-augmentation
environment:
DISPLAY: :1
QT_X11_NO_MITSHM: 1
image: ocv-data-augmentation
tty: true
volumes:
- /home/fsalvagnini/Documents/containers/data_augmentation/project:/home/&{USER}/data_augmentation:rw
- /tmp/.X11-unix:/tmp/.X11-unix:rw
version: '2.0'
If you need to use the .env file and assuming that all the env variables are defined, you just need to follow one step:
.env file
source .env
The above statement will source all the variables defined in .env and hence the env variables will be accessible to docker-compose.
Just added thing, you should also look at ${VARIABLE:-default} just in case you need to pass a default value.
More documentation here
According to the docker-compose manual:
When you set the same environment variable in multiple files, here’s
the priority used by Compose to choose which value to use:
Compose file
Shell environment variables
Environment file
Dockerfile
Variable is not defined
So if shell environment variables are not set, then env file will be used.
For your case, if you need to use shell env vars, you don't need to create .env file. To solve your issue, you need to export the variables before invoking docker-compose.
export GID
export UID
export DISPLAY
docker-compose config
output:
services:
opencv_python:
build:
args:
gid: '20'
uid: '501'
username: enix
context: /Users/enix/source/devops/stackoverflow
dockerfile: opencv_base.Dockerfile
container_name: ocv-data-augmentation
environment:
DISPLAY: :1
QT_X11_NO_MITSHM: 1
image: ocv-data-augmentation
tty: true
volumes:
- /Users/enix/source/devops/project:/home/&{USER}/data_augmentation:rw
- /tmp/.X11-unix:/tmp/.X11-unix:rw
version: '2.0'
I run docker-compose up in a parent directory and -f the docker-compose.yml in child folder. Does anyone know why this won't work? MY_VAR is evaluated to empty string
root-ui-e2e-ci:
environment:
MY_VAR: ./hello
env_file: ./.env
volumes:
- ${MY_VAR}:/app
I end up with this error
.: volume name is too short, names should be at least two alphanumeric
characters
The variable specified in environment & env_file not used for compose-file, it will directly pass to container.
For variable substitution in docker-compose.yaml, you could use next two solutions, and use docker-compose config to quick check the effect:
Solution 1:
Use the variable export in the same shell which run docker-compose:
docker-compose.yaml:
version: '3'
services:
root-ui-e2e-ci:
image: ubuntu
volumes:
- ${MY_VAR}:/app
Try Command:
$export MY_VAR=./hello
$docker-compose config
services:
root-ui-e2e-ci:
image: ubuntu
volumes:
- /home/shubuntu1/99/hello:/app:rw
version: '3.0'
Solution 2:
Use .env:
Set a .env file in the same folder of docker-compose.yaml:
.env:
MY_VAR=./hello
docker-compose.yaml:
version: '3'
services:
root-ui-e2e-ci:
image: ubuntu
volumes:
- ${MY_VAR}:/app
Try Command:
$unset MY_VAR
$docker-compose config
services:
root-ui-e2e-ci:
image: ubuntu
volumes:
- /home/shubuntu1/99/hello:/app:rw
version: '3.0'
depends on you comment that hello is a stringyou need to rewrite you docker-compose like this:
environment:
MY_VAR: hello
volumes:
- ../${MY_VAR}:/app
Is it possible to re-use environment variables that are shared among multiple containers?
The idea is to avoid duplication, as illustrated in this example:
version: '2'
services:
db:
image: example/db
ports:
- "8443:8443"
container_name: db
hostname: db
environment:
- USER_NAME = admin
- USER_PASSWORD = admin
svc:
image: example/svc
depends_on:
- db
ports:
- "9443:9443"
container_name: svc
hostname: svc
environment:
- DB_URL = https://db:8443
- DB_USER_NAME = admin
- DB_USER_PASSWORD = admin
The extends option can be nice but it's not supported in 3.x compose files. Other ways to go are:
Extension fields (compose file 3.4+)
If you can use 3.4+ compose files, extension fields are probably the best option:
docker-compose.yml
version: '3.4'
x-common-variables: &common-variables
VARIABLE: some_value
ANOTHER_VARIABLE: another_value
services:
some_service:
image: someimage
environment: *common-variables
another_service:
image: anotherimage
environment:
<<: *common-variables
NON_COMMON_VARIABLE: 'non_common_value'
env_file directive
docker-compose.yml
version: '3.2'
services:
some_service:
image: someimage
env_file:
- 'variables.env'
another_service:
image: anotherimage
env_file:
- 'variables.env'
variables.env
VARIABLE=some_value
ANOTHER_VARIABLE=another_value
.env file in project root
(or variables at actual compose environment)
Variables from .env file can be referenced in service configuration:
docker-compose.yml
version: '3.2'
services:
some_service:
image: someimage
environment:
- VARIABLE
another_service:
image: anotherimage
environment:
- VARIABLE
- ANOTHER_VARIABLE
.env
VARIABLE=some_value
ANOTHER_VARIABLE=another_value
You can use the extends directive (available in compose 1.x and 2.x) to have multiple containers inherit the environment configuration from an underlying service description. For example, put the following in a file named base.yml:
version: '2'
services:
base:
environment:
DB_URL: https://db:8443
DB_USER_NAME: admin
DB_USER_PASSWORD: admin
Then in your docker-compose.yml:
version: '2'
services:
container1:
image: alpine
command: sh -c "env; sleep 900"
extends:
file: base.yml
service: base
container2:
image: alpine
command: sh -c "env; sleep 900"
extends:
file: base.yml
service: base
environment:
ANOTHERVAR: this is a test
Then inside of container1, you will see:
DB_URL=https://db:8443
DB_USER_NAME=admin
DB_USER_PASSWORD=admin
And inside of container2 you will see:
DB_URL=https://db:8443
DB_USER_NAME=admin
DB_USER_PASSWORD=admin
ANOTHERVAR=this is a test
You can obviously use extends for things other than the environment directive; it's a great way to avoid duplication when using docker-compose.
You can reference local environment variables from within a docker-compose file. Assuming what you're wanting to do is make USER_NAME the same as DB_USER_NAME:
docker-compose.yml
version: '2'
services:
db:
image: example/db
ports:
- "8443:8443"
container_name: db
hostname: db
environment:
- USER_NAME = ${USERNAME}
- USER_PASSWORD = ${PASSWORD}
svc:
image: example/svc
depends_on:
- db
ports:
- "9443:9443"
container_name: svc
hostname: svc
environment:
- DB_URL = https://db:8443
- DB_USER_NAME = ${USERNAME}
- DB_USER_PASSWORD = ${PASSWORD}
Then, run docker-compose like:
$ USERNAME="admin" PASSWORD="admin" docker-compose up
Alternately, for something more permanent, and easier to type on a recurring basis:
$ printf '%s\n%s\n' 'export USERNAME="admin"' 'export PASSWORD="admin"' >> ~/.bash_profile
$ source ~/.bash_profile
$ docker-compose up