I am trying to use environment variables and extension fields in docker-compose and it doesn't seem to be working.
I am starting the docker-compose with:
docker-compose --env-file ../.env up
The .env has:
DB_HOST=test_db
The docker-compose has:
version: '3.7'
x-common-variables: &common-variables
POSTGRES_HOST: ${DB_HOST}
services:
test_db:
image: postgres:10.5
application:
build:
context: ./app
environment:
<<: *common-variables
When my application starts, it is stuck waiting on the database.
when I go into the application docker instance and run:
printenv | grep POSTGRES_HOST
it returns nothing.
Am I missing something?
Related
I'm working on a group project involving Docker that has a .env file, which looks like this:
DATABASE_URL=xxx
DJANGO_SETTINGS_MODULE=xxx
SECRET_KEY=xxx
Couldn't this just be declared inside the Dockerfile? If so, what is the advantage of making a .env file?
Not sure if I'm going in the right direction with this, but this Docker Docs page says (emphasis my own):
Your configuration options can contain environment variables. Compose
uses the variable values from the shell environment in which
docker-compose is run. For example, suppose the shell contains
POSTGRES_VERSION=9.3 and you supply this configuration:
db:
`image: "postgres:${POSTGRES_VERSION}"`
When you run docker-compose up with this configuration, Compose looks for the POSTGRES_VERSION environment variable in the shell and substitutes its value in. For this example, Compose resolves the image to postgres:9.3 before running the configuration.
If an environment variable is not set, Compose substitutes with an empty string. In the example above, if POSTGRES_VERSION is not set, the value for the image option is postgres:.
You can set default values for environment variables using a .env file, which Compose automatically looks for. Values set in the shell environment override those set in the .env file.
If we're using a .env file, then wouldn't I see some ${...} syntax in our docker-compose.yml file? I don't see anything like that, though.
Here's our docker-compose.yml file:
version: '3'
services:
server:
build:
context: ./server
dockerfile: Dockerfile
env_file: .env.dev
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- ./server:/app
ports:
- "8500:8000"
depends_on:
- db
stdin_open: true
tty: true
db:
image: postgres
client:
build:
context: ./client
dockerfile: Dockerfile
command: bash -c "npm install; npm run start"
volumes:
- ./client:/app
- /app/node_modules
ports:
- "3000:3000"
depends_on:
- server
Idea there is probably to have a place to keep secrets separated from docker-compose.yml, which you then can keep in VCS and/or share.
I have a .env file like below:
# DEV
SALES_DB_HOST=xxx
Then I have a docker-compose.yml file that looks like:
version: "3.1"
services:
web:
image: xxx
build: .
env_file: .env
However, the values for the environment variables when accessed in nodejs like process.env.SALES_DB_HOST it prints undefined.
Output of docker-compose config is:
services:
web:
build:
context: xxxxxxxx
environment:
SALES_DB_HOST: xxx
image: xxxxx
version: '3.1'
So, it looks like docker-compose.yml is formed correctly. But why is process.env not getting this value correctly?
EDIT:
I build the docker image with: docker build -t my_image .
Can you change command on you container configuration in yml file.
You should try to test your environment to understand - where is a problem. In docker or in your code.
Try something like this:
maxantonov : ~/passbolt .$ cat dc.yml
version: '3.4'
services:
db:
image: alpine:latest
container_name: db
hostname: db
env_file:
- env/mysql.env
command: ["printenv"]
maxantonov : ~/passbolt .$ docker-compose -f dc.yml up
Starting db ... done
Attaching to db
db | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
db | HOSTNAME=db
db | MYSQL_ROOT_PASSWORD=test
db | MYSQL_DATABASE=passbolt
db | MYSQL_USER=passbolt
db | MYSQL_PASSWORD=P4ssb0lt
db | HOME=/root
db exited with code 0
It's not a docker problem.
Look to your code:
process.env.process.env.SALES_DB_HOST
It's typo. process.env.process.env
You shold use
process.env.SALES_DB_HOST
I simply want to use a environment variable loaded from file in my docker-compose file. But after running the container, I only got
WARNING: The TESTVAR variable is not set. Defaulting to a blank string.
Only found this topic, but I'm using a later version of docker like there (docker-compose: 1.14.0, docker: 17.05.0-ce). And I changed the encoding to ISO 8859-1, since I found a github issue where strange behavior with encodings was detected. Both doesn't work.
My docker-compose file
version: '2'
services:
mysql:
container_name: test_${TESTVAR}
build: mysql
mem_limit: 1G
env_file:
- credentials.env
credentials.env contains only TESTVAR=test123. To start, I run docker-compose up mysql and I also tried to specify the environment variables directly in the compose file like this:
environment:
- TESTVAR=1234
Not working, too.
If you want to use variables in the docker-compose.yml you can do it with .env file, docker docs
$ cat .env
TAG=v1.5
TESTVAR=123
$ cat docker-compose.yml
version: '3'
services:
web:
image: "webapp:${TAG}"
environment: ["TESTVAR=${TESTVAR}"]
docker-compose.yml
version: '2'
services:
app:
build:
context: .
command: python src/app.py
restart: on-failure
depends_on:
- db
environment:
- TJBOT_DB_HOST=db
- TJBOT_API_KEY
- TJBOT_AUTO_QUESTION_TIME
env_file:
- .env
db:
image: mongo:3.0.14
volumes:
- mongodbdata:/data/db
volumes:
mongodbdata:
If I change the .env file, how could I reload the container to use the new environment variables with minimum downtime?
If you are running the yml with docker-compose, you can just run docker-compose up -d and it will recreate any containers that have changes and leave all unchanged services untouched.
$ cat docker-compose.env2.yml
version: '2'
services:
test:
image: busybox
# command: env
command: tail -f /dev/null
environment:
- MY_VAR=hello
- MY_VAR2=world
test2:
image: busybox
command: tail -f /dev/null
environment:
- MY_VAR=same ole same ole
$ docker-compose -f docker-compose.env2.yml up -d
Creating network "test_default" with the default driver
Creating test_test_1
Creating test_test2_1
$ vi docker-compose.env2.yml # edit the file to change MY_VAR
$ docker-compose -f docker-compose.env2.yml up -d
Recreating test_test_1
test_test2_1 is up-to-date
If you run the containers as a docker stack deploy -c docker-compose.yml with a version 3 file format, you can do a rolling update of the service which will prevent any downtime if you have multiple instances of your service running. This functionality is still very new, you'll want 1.13.1 to fix some of the issues with updates, and as with anything this new, bugs are still being worked out.
When I run docker-compose build && docker-compose up redis, with environment specified in docker-compose.yaml and RUN env in the Dockerfile, the environment variables I set don't get printed.
Why does this not work?
I'm using docker-compose version 1.4.2.
Here are the relevant files:
docker-compose.yaml with environment as a list of KEY=value pairs:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO='bar'
docker-compose.yaml with environment as a dictionary:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO: 'bar'
Dockerfile:
FROM redis:2.6
MAINTAINER me#email.com
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
That's normal
docker-compose only sets the environment variables specified in the environment directive in the docker-compose.yaml file during the run phase of the container, and not during the build phase.
So if you do docker-compose run --entrypoint "/bin/bash" redis -c env you will be able to see your env variables.
If you want to set variables inside your Dockerfile (to be able to see them during the build phase) you can add inside your dockerfile before your RUN env:
ENV FOO bar
Well
I have tested and found following solutions for docker compose with env file or without env file. I will show you two different approach
Lets say you have following docker compose yml file
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
env_file: docker.env
Now you need to setup the postgres variable in a file called docker.env. Remember you need to keep the docker_compose.yml file and docker.env file in same folder.
Next, In the docker.env file you need to have the database variable and value like this:
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=myapp_db
Now hit docker-compose up . It should work.
Lets say now you dont like to specify the env file name in the docker-compose.yml file. So you have to write docker-compose.yml file like this:
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
environments:
- POSTGRES_USER=${PGU}
-POSTGRES_PASSWORD=${PGP}
-POSTGRES_DB=${PGD}
Now your docker.env file should look like this:
PGU=postgres
PGP=postgres
PGD=myapp_db
now hit docker-compose --env-file docker.env up
you are good to go.
This is because you were using environment when (I guess) you wanted to use args inside the build block:
redis:
build:
context: ../storage/redis
args:
- FOO: 'bar'
ports:
- "6379:6379"
Your Dockerfile would define FUN in the (image) environment:
FROM redis:2.6
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
# Read FUN from (build) arguments
# (may define a default: ARG FUN='wow')
ARG FUN
# Define env variable FUN with value from ARG
ENV FUN=$FUN
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
The environment block is used to define variables for the running container (when docker-compose up, NOT when docker-compose build).