Is it possible to re-use environment variables that are shared among multiple containers?
The idea is to avoid duplication, as illustrated in this example:
version: '2'
services:
db:
image: example/db
ports:
- "8443:8443"
container_name: db
hostname: db
environment:
- USER_NAME = admin
- USER_PASSWORD = admin
svc:
image: example/svc
depends_on:
- db
ports:
- "9443:9443"
container_name: svc
hostname: svc
environment:
- DB_URL = https://db:8443
- DB_USER_NAME = admin
- DB_USER_PASSWORD = admin
The extends option can be nice but it's not supported in 3.x compose files. Other ways to go are:
Extension fields (compose file 3.4+)
If you can use 3.4+ compose files, extension fields are probably the best option:
docker-compose.yml
version: '3.4'
x-common-variables: &common-variables
VARIABLE: some_value
ANOTHER_VARIABLE: another_value
services:
some_service:
image: someimage
environment: *common-variables
another_service:
image: anotherimage
environment:
<<: *common-variables
NON_COMMON_VARIABLE: 'non_common_value'
env_file directive
docker-compose.yml
version: '3.2'
services:
some_service:
image: someimage
env_file:
- 'variables.env'
another_service:
image: anotherimage
env_file:
- 'variables.env'
variables.env
VARIABLE=some_value
ANOTHER_VARIABLE=another_value
.env file in project root
(or variables at actual compose environment)
Variables from .env file can be referenced in service configuration:
docker-compose.yml
version: '3.2'
services:
some_service:
image: someimage
environment:
- VARIABLE
another_service:
image: anotherimage
environment:
- VARIABLE
- ANOTHER_VARIABLE
.env
VARIABLE=some_value
ANOTHER_VARIABLE=another_value
You can use the extends directive (available in compose 1.x and 2.x) to have multiple containers inherit the environment configuration from an underlying service description. For example, put the following in a file named base.yml:
version: '2'
services:
base:
environment:
DB_URL: https://db:8443
DB_USER_NAME: admin
DB_USER_PASSWORD: admin
Then in your docker-compose.yml:
version: '2'
services:
container1:
image: alpine
command: sh -c "env; sleep 900"
extends:
file: base.yml
service: base
container2:
image: alpine
command: sh -c "env; sleep 900"
extends:
file: base.yml
service: base
environment:
ANOTHERVAR: this is a test
Then inside of container1, you will see:
DB_URL=https://db:8443
DB_USER_NAME=admin
DB_USER_PASSWORD=admin
And inside of container2 you will see:
DB_URL=https://db:8443
DB_USER_NAME=admin
DB_USER_PASSWORD=admin
ANOTHERVAR=this is a test
You can obviously use extends for things other than the environment directive; it's a great way to avoid duplication when using docker-compose.
You can reference local environment variables from within a docker-compose file. Assuming what you're wanting to do is make USER_NAME the same as DB_USER_NAME:
docker-compose.yml
version: '2'
services:
db:
image: example/db
ports:
- "8443:8443"
container_name: db
hostname: db
environment:
- USER_NAME = ${USERNAME}
- USER_PASSWORD = ${PASSWORD}
svc:
image: example/svc
depends_on:
- db
ports:
- "9443:9443"
container_name: svc
hostname: svc
environment:
- DB_URL = https://db:8443
- DB_USER_NAME = ${USERNAME}
- DB_USER_PASSWORD = ${PASSWORD}
Then, run docker-compose like:
$ USERNAME="admin" PASSWORD="admin" docker-compose up
Alternately, for something more permanent, and easier to type on a recurring basis:
$ printf '%s\n%s\n' 'export USERNAME="admin"' 'export PASSWORD="admin"' >> ~/.bash_profile
$ source ~/.bash_profile
$ docker-compose up
Related
Using the below docker compose files, i am unable to bring up my app correctly. Docker says my LAPIS_ENV environment variable is not set, but i am setting it in my second compose file which I am expecting to be merged into the first one. I have tried including them in reverse order to no avail.
version: '2.4'
services:
backend:
mem_limit: 50mb
memswap_limit: 50mb
build:
context: ./backend
dockerfile: Dockerfile
depends_on:
- postgres
volumes:
- ./backend:/var/www
- ./data:/var/data
restart: unless-stopped
command: bash -c "/usr/local/bin/docker-entrypoint.sh ${LAPIS_ENV}"
postgres:
build:
context: ./postgres
dockerfile: Dockerfile
environment:
PGDATA: /var/lib/postgresql/data/pgdata
POSTGRES_HOST_AUTH_METHOD: trust
volumes:
- postgres:/var/lib/postgresql/data
- ./postgres/pg_hba.conf:/var/lib/postgres/data/pg_hba.conf
- ./data/backup:/pgbackup
restart: unless-stopped
volumes:
postgres:
version: '2.4'
services:
backend:
environment:
LAPIS_ENV: development
ports:
- 8080:80
#!/usr/bin/env bash
docker compose -f docker-compose.yml -f docker-compose.dev.yml up
I'm deploying my Django app using AWS code pipeline which is dockerized and I was storing my env variables inside an env file for local development but for the code pipeline I set them all inside environment variables but the variables are still getting None.
docker-compose.yml
version: "3.8"
services:
db:
container_name: db
image: "postgres"
restart: always
volumes:
- postgres-data:/var/lib/postgresql/data/
app:
container_name: app
build:
context: .
restart: always
volumes:
- static-data:/vol/web
depends_on:
- db
proxy:
container_name: proxy
build:
context: ./proxy
restart: always
depends_on:
- app
ports:
- 80:8000
volumes:
- static-data:/vol/static
volumes:
postgres-data:
static-data:
getting env variable in django like:
os.environ.get('FRONTEND_URL')
in my case, I put all env variables inside my AWS CodeBuild environment variables and call them in my docker environment as below.
version: "3.8"
services:
db:
container_name: db
image: "postgres"
restart: always
volumes:
- postgres-data:/var/lib/postgresql/data/
environment:
- VARIABLE_NAME: ${VARIABLE_NAME}
You can specify the env file in the docker compose itself along with relative path.
....
....
app:
container_name: app
build:
context: .
restart: always
env_file:
- <web-variables1.env>
volumes:
- static-data:/vol/web
depends_on:
- db
....
....
Refer more details at documentation
I'm trying to start multiple docker containers with a wsl shell runner. After running
compose_job:
tags:
- wsl
stage: compose
script:
- cd /pathToComposeFile
- docker-compose up
dependencies:
- pull_job
the runner excited with following error:
$ docker-compose up
docker: invalid reference format: repository name must be lowercase.
the docker-compose.yml is:
version: '3'
services:
cron:
build: cron/.
container_name: cron
image: cron_image
ports:
- 6040:6040
The referenced images are all written in lowercase and the same command excited as expected
if run manually. I already checked that docker-compose is accessible and that the docker-compose.yml is readable. How can I resolve this issue? Thank you in advance!
I think service_name, container_name and env must be lowercase.
Look like
version: '3'
services:
perihubapi:
build:
context: api/.
args:
EXTERNAL: ${external}
FASERVICES: ${faservices}
container_name: perihubapi
image: peri_hub_api_image
ports:
- 6020:6020
networks:
- backend
volumes:
- peridigm_volume:/app/peridigmJobs
- paraView_volume:/app/paraView
- secrets:/app/certs
perihubgui:
build: gui/.
container_name: perihubgui
image: peri_hub_gui_image
ports:
- 6010:6010
networks:
- backend
volumes:
- secrets:/app/certs
peridigm:
build:
context: peridigm/.
args:
GITLAB_USER: ${gitlab_user}
GITLAB_TOKEN: ${gitlab_token}
PERIDOX: ${peridox}
container_name: peridigm
image: peridigm_image
ports:
- 6030:6030
networks:
- backend
volumes:
- peridigm_volume:/app/peridigmJobs
paraview:
build:
context: paraview/.
container_name: paraview
image: paraview_image
volumes:
- paraView_volume:/app/paraView
cron:
build: cron/.
container_name: cron
image: cron_image
ports:
- 6040:6040
networks:
- backend
networks:
backend:
driver: bridge
volumes:
peridigm_volume:
paraView_volume:
secrets:
external: true
Is there a way to specify an empty array in docker-compose.yml?
I tried ports: [] to no avail.
A JSON empty array literal should be valid YAML syntax, but when I use this, I get the following error from Docker:
services.myapp_migrations.ports must be a list
For more context:
I have a python application with the following docker-compose.yml:
version: "3.9"
services:
myapp: &myapp
build:
context: .
ports:
- 8000:8000
environment:
- MYAPP_DB_HOST=postgres
volumes:
- ./.env:/app/.env
working_dir: /app
depends_on:
- postgres
myapp_migrations:
<<: *myapp
command: ["migrate"]
restart: on-failure
ports: []
postgres:
image: postgres:12-alpine
environment:
- POSTGRES_PASSWORD=${MYAPP_DB_PASSWORD}
- POSTGRES_USER=${MYAPP_DB_USER}
- POSTGRES_DB=${MYAPP_DB_NAME}
As you can see, I'm trying to re-use the myapp definition with a YAML anchor, but I need to override the ports definition in myapp_migrations.
(I know I could use extension fields for this, but I'm trying to avoid that.)
I agree with #flyx this is probably a temporary bug.
You might be able to work around this by using docker-compose in place of docker compose. Because they are separate binaries, they are updated independently.
For example, given your yaml above:
docker compose (not working)
test % docker compose config
services.myapp_migrations.ports must be a list
docker-compose (working)
test % docker-compose config
WARNING: The MYAPP_DB_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The MYAPP_DB_USER variable is not set. Defaulting to a blank string.
WARNING: The MYAPP_DB_NAME variable is not set. Defaulting to a blank string.
services:
myapp:
build:
context: /Users/jvanus/test
depends_on:
postgres:
condition: service_started
environment:
MYAPP_DB_HOST: postgres
ports:
- published: 8000
target: 8000
volumes:
- /Users/jvanus/test/.env:/app/.env:rw
working_dir: /app
myapp_migrations:
build:
context: /Users/jvanus/test
command:
- migrate
depends_on:
postgres:
condition: service_started
environment:
MYAPP_DB_HOST: postgres
ports: []
restart: on-failure
volumes:
- /Users/jvanus/test/.env:/app/.env:rw
working_dir: /app
postgres:
environment:
POSTGRES_DB: ''
POSTGRES_PASSWORD: ''
POSTGRES_USER: ''
image: postgres:12-alpine
version: '3.9'
What is the equivalent of specifying the schema of a database in docker-compose.yml file.
database/Dockerfile
FROM mysql:5.7
COPY ./schema.sql /docker-entrypoint-initdb.d/
docker-compose.yml
services:
database:
image: "mysql:5.7"
container_name: "mysql"
ports:
- "6603:3306"
Attempts
I've attempted with the following. Is it possible?
version: '3'
services:
database:
image: "mysql:5.7"
container_name: "mysql"
ports:
- "6603:3306"
command: --init-file /database/schema.sql
volumes:
- ./init.sql:/database/schema.sql
So you have schema.sql somewhere on your host filesystem, let's assume it's on ./database/schema.sql. Then you should have such a compose file:
version: '3'
services:
database:
image: "mysql:5.7"
container_name: "mysql"
ports:
- "6603:3306"
volumes:
- ./database/schema.sql:/docker-entrypoint-initdb.d/init.sql
This image does not support any --init-file command. Instead, it accepts init scripts put under directory /docker-entrypoint-initdb.d.