Set extra host in environment variables - docker

I'm using docker compose to run my application. And for do that I need to set the hosts inside container (it's depends on the environment i'm running).
My approach was:
Create an environment file and set the variable:
#application.env
SERVER_IP=10.10.9.134
My docker compose file looks like:
version: '2'
services:
api:
container_name: myApplication
env_file:
- application.env
build: ./myApplication/
entrypoint: ./docker/api-startup.sh
ports:
- "8080:8080"
depends_on:
- redis
extra_hosts: &extra_hosts
myip: $SERVER_IP
But my problem is that the variable SERVER_IP is never replaced.
When I run docker-compose config I see:
services:
api:
build:
context: /...../myApplication
container_name: myApplication
depends_on:
- redis
entrypoint: ./docker/api-startup.sh
environment:
SERVER_IP: 10.10.9.134
extra_hosts:
myip: ''
ports:
- 8080:8080
I've tried to replace the variable reference using $SERVER_IP or ${SERVER_IP} but it didn't work.

I created a file .env, added single line HOST=test.example.com, then did this in docker-compose:
extra_hosts:
- myip:${HOST}
docker-compose config then shows
extra_hosts:
myip: test.example.com
To do this I followed the documentation from Docker-compose environment variables the section about .env file
UPDATE
According to the Docker documentation,
Note: If your service specifies a build option, variables defined in
environment files will not be automatically visible during the build.
Use the args sub-option of build to define build-time environment
variables.
It basically means if you place your variables in .env file, you can use them for substitution in docker-compose.yml, but if you use env_file option for the particular container, you can only see the variables inside the Docker container, not during the build. It is also logical, env_file replaces docker run --env-file=FILE ... and nothing else.
So, you can only place your values into .env. Alternatively, as William described, you can use host's environment variables.

EDIT
Try the following:
version: '2'
services:
api:
container_name: myApplication
env_file:
- application.env
build: ./myApplication/
entrypoint: ./docker/api-startup.sh
ports:
- "8080:8080"
depends_on:
- redis
extra_hosts:
- "myip:${SERVER_IP}"
Ensure curly bracers and that the environment variable exists on the host os.

Related

Cannot interpolate environment variable in docker compose

I have a simple docker compose file to create a Mysql database for my app. But I cannot interpolate the environment variable MYSQL_PORT to set a custom port. Running docker compose up with the configuration below results in a random port being assigned to mysql.
The path to the env file does work, since I have env variables configuring the database.
docker-compose.yml
version: '3'
services:
mysql:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
volumes:
- mysql_data:/var/lib/mysql
env_file:
- ../../.env
ports:
- ${MYSQL_PORT}:3306
volumes:
mysql_data:
.env
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=root
MYSQL_DATABASE=final_project_database
MYSQL_USER=db_user
MYSQL_PASSWORD=some_db_user_password
Use --env-file option with docker-compose up command. env_file declared in your MySQL service applies only for container env
Move your .env file to the same directory as the docker-compose file and change the env_file to point to it. That way both docker-compose and the container will use the same environment file.
Right now it's only the container that's using it.
version: '3'
services:
mysql:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
volumes:
- mysql_data:/var/lib/mysql
env_file:
- ./.env
ports:
- ${MYSQL_PORT}:3306
volumes:
mysql_data:

Issue in docker compose with volume undefined

I get the below error when I run docker-compose up, any pointers why I am getting this error
service "mysqldb-docker" refers to undefined volume mysqldb: invalid compose project
Also, is there a way to pass the $ENV value in CLI to docker-compose up , currently I have a ENV variable that specified dev, uat or prod that I use to specify the db name. Are there better alternatives to do this other than create a .env file explicitly for this
version: '3.8'
services:
mysqldb-docker:
image: '8.0.27'
restart: 'unless-stopped'
ports:
- "3309:3306"
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_PASSWORD=root
- MYSQL_DATABASE=reco-tracker-$ENV
volumes:
- mysqldb:/var/lib/mysql
reco-tracker-docker:
image: 'reco-tracker-docker:v1'
ports:
- "8083:8083"
environment:
- SPRING_DATASOURCE_USERNAME=root
- SPRING_DATASOURCE_PASSWORD=root
- SPRING_DATASOURCE_URL="jdbc:mysql://mysqldb-docker:3309/reco-tracker-$ENV"
depends_on: [mysqldb-docker]
You must define volumes at the top level like this:
version: '3.8'
services:
mysqldb-docker:
# ...
volumes:
- mysqldb:/var/lib/mysql
volumes:
mysqldb:
You can pass environment variables from your shell straight through to a service’s containers with the ‘environment’ key by not giving them a value
https://docs.docker.com/compose/environment-variables/#pass-environment-variables-to-containers
web:
environment:
- ENV
but from my tests you cant write $ENV in the compose file and expect it to read your env
for this you need to call docker-compose that way :
docker-compose run -e ENV web python console.py
see this : https://docs.docker.com/compose/environment-variables/#set-environment-variables-with-docker-compose-run

Docker Compose File cant get .env Variables

I am using docker-compose to run a traefik container. The Domain of this Container should be set by an environment file but everytime i start this service it says:
WARNING: The DOMAIN variable is not set. Defaulting to a blank string
My compose-file setup:
version: '3.5'
networks:
frontend:
name: frontend
backend:
name: backend
services:
Traefik:
image: traefik:latest
command: --api --docker --acme.email="test#test.de"
restart: always
container_name: Traefik
networks:
- backend
- frontend
env_file: ./env.env
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik/traefik.toml:/traefik.toml
- ./traefik/acme.json:/acme.json
ports:
- "80:80"
- "443:443"
labels:
- "traefik.docker.network=frontend"
- "traefik.enable=true"
- "traefik.frontend.rule=Host:traefik.${DOMAIN}"
- "traefik.port=8080"
- "traefik.protocol=http"
My env.env file setup:
DOMAIN=fiture.de
Thanks for your Help!
env_file: ./env.env
The file env.env isn't loaded to parse the compose file, it is loaded to add environment variables within the container being run. At the point docker processes the above instruction, the yaml file has already been loaded and variables have been expanded.
If you are using docker-compose to deploy containers on a single node, you can rename the file .env and docker-compose will load variables from that file before parsing the compose file.
If you are deploying with docker stack deploy, then you need to import the environment variables into your shell yourself. An example of doing that in bash looks like:
set -a && . ./env.env && set +a && docker stack deploy ...

Docker: Why does my project have a .env file?

I'm working on a group project involving Docker that has a .env file, which looks like this:
DATABASE_URL=xxx
DJANGO_SETTINGS_MODULE=xxx
SECRET_KEY=xxx
Couldn't this just be declared inside the Dockerfile? If so, what is the advantage of making a .env file?
Not sure if I'm going in the right direction with this, but this Docker Docs page says (emphasis my own):
Your configuration options can contain environment variables. Compose
uses the variable values from the shell environment in which
docker-compose is run. For example, suppose the shell contains
POSTGRES_VERSION=9.3 and you supply this configuration:
db:
`image: "postgres:${POSTGRES_VERSION}"`
When you run docker-compose up with this configuration, Compose looks for the POSTGRES_VERSION environment variable in the shell and substitutes its value in. For this example, Compose resolves the image to postgres:9.3 before running the configuration.
If an environment variable is not set, Compose substitutes with an empty string. In the example above, if POSTGRES_VERSION is not set, the value for the image option is postgres:.
You can set default values for environment variables using a .env file, which Compose automatically looks for. Values set in the shell environment override those set in the .env file.
If we're using a .env file, then wouldn't I see some ${...} syntax in our docker-compose.yml file? I don't see anything like that, though.
Here's our docker-compose.yml file:
version: '3'
services:
server:
build:
context: ./server
dockerfile: Dockerfile
env_file: .env.dev
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- ./server:/app
ports:
- "8500:8000"
depends_on:
- db
stdin_open: true
tty: true
db:
image: postgres
client:
build:
context: ./client
dockerfile: Dockerfile
command: bash -c "npm install; npm run start"
volumes:
- ./client:/app
- /app/node_modules
ports:
- "3000:3000"
depends_on:
- server
Idea there is probably to have a place to keep secrets separated from docker-compose.yml, which you then can keep in VCS and/or share.

Docker Deploy stack extra hosts ignored

docker stack deploy isnt respecting the extra_hosts parameter in my compose file. when i do a simple docker-compose up the entry is created in the /etc/hosts however when i do docker deploy –compose-file docker-compose.yml myapp it ignores extra_hosts, any insights?
Below is the docker-compose.xml:
version: '3'
services:
web:
image: user-service
deploy:
labels:
- the label
build:
context: ./
environment:
DATABASE_URL: jdbc:postgresql://dbhost:5432/postgres
ports:
- 9002:9002
extra_hosts:
- "dbhost: ${DB_HOST}"
networks:
- wellness_swarm
env_file:
- .env
networks:
wellness_swarm:
external:
name: wellness_swarm
the docker-compose config also displays the compose file properly.
This may not be a direct answer to the question as it doesn't use env variables but what I found was that the extra_hosts block in the compose file was ignored in swarm mode if entered in the format above.
i.e. this works for me and puts entries in /etc/hosts in the container:
extra_hosts:
retisdev: 10.48.161.44
retistesting: 10.48.161.44
whereas when entered in the other format it gets ignored when deploying as a service
extra_hosts:
- "retisdev=10.48.161.44"
- "retistesting=10.48.161.44"
I think it's an ordering issue. The ${} variable you've got in the compose file runs during the YAML processing before the service definition is created. Then stack deploy processes the .env file for running in the container as envvars, but the YAML variable is needed first...
To fix that, you should use the docker-compose config command first, to process the YAML, and then use the output of that to send to the stack deploy.
docker-compose config will show you the output you're likely wanting.
Then do a pipe to get a one-liner.
docker-compose config | docker stack deploy -c - myapp
Note: Ideally you wouldn't use the extra_hosts, but rather put the envvar directly in the connection string. Your way seems like unnecessary complexity and isn't the usual way I see a connection string created.
e.g.
version: '3'
services:
web:
image: user-service
deploy:
labels:
- the label
build:
context: ./
environment:
DATABASE_URL: jdbc:postgresql://${DB_HOST}:5432/postgres
ports:
- 9002:9002
networks:
- wellness_swarm
env_file:
- .env
networks:
wellness_swarm:
external:
name: wellness_swarm
As i see https://github.com/moby/moby/issues/29133 seems like it is by design where in the compose command takes into consideration the environment variables mentioned in .env file however the deploy command ignores that :( why is that so, pretty lame reasons!

Resources