I get the below error when I run docker-compose up, any pointers why I am getting this error
service "mysqldb-docker" refers to undefined volume mysqldb: invalid compose project
Also, is there a way to pass the $ENV value in CLI to docker-compose up , currently I have a ENV variable that specified dev, uat or prod that I use to specify the db name. Are there better alternatives to do this other than create a .env file explicitly for this
version: '3.8'
services:
mysqldb-docker:
image: '8.0.27'
restart: 'unless-stopped'
ports:
- "3309:3306"
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_PASSWORD=root
- MYSQL_DATABASE=reco-tracker-$ENV
volumes:
- mysqldb:/var/lib/mysql
reco-tracker-docker:
image: 'reco-tracker-docker:v1'
ports:
- "8083:8083"
environment:
- SPRING_DATASOURCE_USERNAME=root
- SPRING_DATASOURCE_PASSWORD=root
- SPRING_DATASOURCE_URL="jdbc:mysql://mysqldb-docker:3309/reco-tracker-$ENV"
depends_on: [mysqldb-docker]
You must define volumes at the top level like this:
version: '3.8'
services:
mysqldb-docker:
# ...
volumes:
- mysqldb:/var/lib/mysql
volumes:
mysqldb:
You can pass environment variables from your shell straight through to a service’s containers with the ‘environment’ key by not giving them a value
https://docs.docker.com/compose/environment-variables/#pass-environment-variables-to-containers
web:
environment:
- ENV
but from my tests you cant write $ENV in the compose file and expect it to read your env
for this you need to call docker-compose that way :
docker-compose run -e ENV web python console.py
see this : https://docs.docker.com/compose/environment-variables/#set-environment-variables-with-docker-compose-run
Related
I have a simple docker compose file to create a Mysql database for my app. But I cannot interpolate the environment variable MYSQL_PORT to set a custom port. Running docker compose up with the configuration below results in a random port being assigned to mysql.
The path to the env file does work, since I have env variables configuring the database.
docker-compose.yml
version: '3'
services:
mysql:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
volumes:
- mysql_data:/var/lib/mysql
env_file:
- ../../.env
ports:
- ${MYSQL_PORT}:3306
volumes:
mysql_data:
.env
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=root
MYSQL_DATABASE=final_project_database
MYSQL_USER=db_user
MYSQL_PASSWORD=some_db_user_password
Use --env-file option with docker-compose up command. env_file declared in your MySQL service applies only for container env
Move your .env file to the same directory as the docker-compose file and change the env_file to point to it. That way both docker-compose and the container will use the same environment file.
Right now it's only the container that's using it.
version: '3'
services:
mysql:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
volumes:
- mysql_data:/var/lib/mysql
env_file:
- ./.env
ports:
- ${MYSQL_PORT}:3306
volumes:
mysql_data:
I have a docker-compose file which looks like:
version: '3.8'
services:
scheduler:
image: apache/airflow
command: scheduler
restart: on-failure
env_file:
- .env
volumes:
- ./dags:/opt/airflow/dags
- ./logs:/opt/airflow/logs
webserver:
image: apache/airflow
entrypoint: ./scripts/entrypoint.sh
restart: on-failure
depends_on:
- scheduler
env_file:
- .env
volumes:
- ./dags:/opt/airflow/dags
- ./logs:/opt/airflow/logs
- ./scripts:/opt/airflow/scripts
ports:
- "8080:8080"
It works as expected on running docker-compose up.
But if I remove the env_file option from yml and pass it in CLI - docker-compose --env-file .env up, then it doesn't pick the value of environment variables from the file.
This happens because the --env-file in CLI and env_file in service section are not exactly the same thing.
The former is actually a specification of the file that compose will use to substitute variables inside docker-compose.yml (the default value is .env).
The latter is specifying the file from which the variables will be loaded and passed to the container of the service.
And yes, it's kinda confusing.
Reference:
Read this and this.
I am using docker-compose to run a traefik container. The Domain of this Container should be set by an environment file but everytime i start this service it says:
WARNING: The DOMAIN variable is not set. Defaulting to a blank string
My compose-file setup:
version: '3.5'
networks:
frontend:
name: frontend
backend:
name: backend
services:
Traefik:
image: traefik:latest
command: --api --docker --acme.email="test#test.de"
restart: always
container_name: Traefik
networks:
- backend
- frontend
env_file: ./env.env
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik/traefik.toml:/traefik.toml
- ./traefik/acme.json:/acme.json
ports:
- "80:80"
- "443:443"
labels:
- "traefik.docker.network=frontend"
- "traefik.enable=true"
- "traefik.frontend.rule=Host:traefik.${DOMAIN}"
- "traefik.port=8080"
- "traefik.protocol=http"
My env.env file setup:
DOMAIN=fiture.de
Thanks for your Help!
env_file: ./env.env
The file env.env isn't loaded to parse the compose file, it is loaded to add environment variables within the container being run. At the point docker processes the above instruction, the yaml file has already been loaded and variables have been expanded.
If you are using docker-compose to deploy containers on a single node, you can rename the file .env and docker-compose will load variables from that file before parsing the compose file.
If you are deploying with docker stack deploy, then you need to import the environment variables into your shell yourself. An example of doing that in bash looks like:
set -a && . ./env.env && set +a && docker stack deploy ...
I want to use excel file and some folders with my container .
I am using volumes but , dont know what the problem is with my compose .
seleniumhub:
image: selenium/hub
ports:
- "4444:4444"
firefoxnode:
image: selenium/node-firefox-debug
ports:
- "5901:5900"
links:
- "seleniumhub:hub"
shm_size: '2gb'
environment:
- "NODE_MAX_SESSION=2"
- "NODE_MAX_INSTANCES=2"
chromenode2:
image: selenium/node-chrome-debug
ports:
- "5902:5900"
links:
- "seleniumhub:hub"
shm_size: '2gb'
environment:
- "NODE_MAX_SESSION=2"
- "NODE_MAX_INSTANCES=2"
test:
image: raveena1/dilsel
ports:
- 4579
links:
- "seleniumhub:hub"
container_name: mywebcontainer
**volumes:
- /$$(pwd)/Newfolder/Config/framework-config.properties:/var/lib/docker/**
I want to use the above property file in my container , how can i achieve this ?
I don't think docker-compose can interpret bash command inside a compose file. However, what you can do is use environment variable. In your case, you might want to use $PWD.
[...]
volumes:
- $PWD/Newfolder/Config/framework-config.properties:/var/lib/docker/
[...]
This will interpret the environment variable $PWD (which resolves to your current working directy) and mount this to /var/lib/docker.
Below is an example of using environment variable in docker-compose :
docker-compose.yml:
test:
image: debian:stretch-slim
ports:
- 4579
container_name: mywebcontainer
volumes:
- $PWD/:/current_directory_of_host
entrypoint: "ls -l /current_directory_of_host"
Start this container with docker-compose up. You should see a list of file that is in your current working directory.
You can also use custom environment variable : CUSTOM_ENV=$(pwd) docker-compose up. This will forward CUSTOM_ENV to docker-compose which can be used in your docker-compose.yml.
I'm using docker compose to run my application. And for do that I need to set the hosts inside container (it's depends on the environment i'm running).
My approach was:
Create an environment file and set the variable:
#application.env
SERVER_IP=10.10.9.134
My docker compose file looks like:
version: '2'
services:
api:
container_name: myApplication
env_file:
- application.env
build: ./myApplication/
entrypoint: ./docker/api-startup.sh
ports:
- "8080:8080"
depends_on:
- redis
extra_hosts: &extra_hosts
myip: $SERVER_IP
But my problem is that the variable SERVER_IP is never replaced.
When I run docker-compose config I see:
services:
api:
build:
context: /...../myApplication
container_name: myApplication
depends_on:
- redis
entrypoint: ./docker/api-startup.sh
environment:
SERVER_IP: 10.10.9.134
extra_hosts:
myip: ''
ports:
- 8080:8080
I've tried to replace the variable reference using $SERVER_IP or ${SERVER_IP} but it didn't work.
I created a file .env, added single line HOST=test.example.com, then did this in docker-compose:
extra_hosts:
- myip:${HOST}
docker-compose config then shows
extra_hosts:
myip: test.example.com
To do this I followed the documentation from Docker-compose environment variables the section about .env file
UPDATE
According to the Docker documentation,
Note: If your service specifies a build option, variables defined in
environment files will not be automatically visible during the build.
Use the args sub-option of build to define build-time environment
variables.
It basically means if you place your variables in .env file, you can use them for substitution in docker-compose.yml, but if you use env_file option for the particular container, you can only see the variables inside the Docker container, not during the build. It is also logical, env_file replaces docker run --env-file=FILE ... and nothing else.
So, you can only place your values into .env. Alternatively, as William described, you can use host's environment variables.
EDIT
Try the following:
version: '2'
services:
api:
container_name: myApplication
env_file:
- application.env
build: ./myApplication/
entrypoint: ./docker/api-startup.sh
ports:
- "8080:8080"
depends_on:
- redis
extra_hosts:
- "myip:${SERVER_IP}"
Ensure curly bracers and that the environment variable exists on the host os.