When I run docker-compose build && docker-compose up redis, with environment specified in docker-compose.yaml and RUN env in the Dockerfile, the environment variables I set don't get printed.
Why does this not work?
I'm using docker-compose version 1.4.2.
Here are the relevant files:
docker-compose.yaml with environment as a list of KEY=value pairs:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO='bar'
docker-compose.yaml with environment as a dictionary:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO: 'bar'
Dockerfile:
FROM redis:2.6
MAINTAINER me#email.com
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
That's normal
docker-compose only sets the environment variables specified in the environment directive in the docker-compose.yaml file during the run phase of the container, and not during the build phase.
So if you do docker-compose run --entrypoint "/bin/bash" redis -c env you will be able to see your env variables.
If you want to set variables inside your Dockerfile (to be able to see them during the build phase) you can add inside your dockerfile before your RUN env:
ENV FOO bar
Well
I have tested and found following solutions for docker compose with env file or without env file. I will show you two different approach
Lets say you have following docker compose yml file
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
env_file: docker.env
Now you need to setup the postgres variable in a file called docker.env. Remember you need to keep the docker_compose.yml file and docker.env file in same folder.
Next, In the docker.env file you need to have the database variable and value like this:
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=myapp_db
Now hit docker-compose up . It should work.
Lets say now you dont like to specify the env file name in the docker-compose.yml file. So you have to write docker-compose.yml file like this:
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
environments:
- POSTGRES_USER=${PGU}
-POSTGRES_PASSWORD=${PGP}
-POSTGRES_DB=${PGD}
Now your docker.env file should look like this:
PGU=postgres
PGP=postgres
PGD=myapp_db
now hit docker-compose --env-file docker.env up
you are good to go.
This is because you were using environment when (I guess) you wanted to use args inside the build block:
redis:
build:
context: ../storage/redis
args:
- FOO: 'bar'
ports:
- "6379:6379"
Your Dockerfile would define FUN in the (image) environment:
FROM redis:2.6
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
# Read FUN from (build) arguments
# (may define a default: ARG FUN='wow')
ARG FUN
# Define env variable FUN with value from ARG
ENV FUN=$FUN
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
The environment block is used to define variables for the running container (when docker-compose up, NOT when docker-compose build).
Related
I'm looking to be able to run swarm from same docker-compose file which uses env variables. Currently I only achieved that all nodes are replicating Leaders env. Is it possible to let each node start from its own local env var?
My docker-compose
version: '3.1'
networks:
base:
services:
test:
container_name: ${Name}
restart: always
image: ubuntu:latest
environment:
- Name=${Name}
command: sh -c "echo $Name && sleep 30"
networks:
- base
use env_file option
https://docs.docker.com/compose/environment-variables/
# .env, file
Name=<your_name>
# <your_name>.env, file
TEST_ENV=stackoverflow
# docker-compose.yaml, file
version: '3.1'
services:
test:
container_name: ${Name}
restart: always
image: ubuntu:latest
env_file:
- ${Name}.env
command: sh -c "set | grep TEST_ENV && sleep 30"
docker logs <your_name>
# TEST_ENV='stackoverflow'
You can set env_files with different names in different containers.
for example
# docker-compose.yaml, file
version: '3.1'
services:
test1:
container_name: test1
restart: always
image: ubuntu:latest
env_file:
- first.env
command: sh -c "set | grep TEST_FIRST_ENV && sleep 30"
test2:
container_name: test2
restart: always
image: ubuntu:latest
env_file:
- second.env
command: sh -c "set | grep TEST_SECOND_ENV && sleep 30"
Environment variables referenced in the docker-compose.yml file are not resolved on the leader even, they are resolved on whatever jump box you are deploying too the swarm from.
If you want to reference the env vars from the host system, from the command, or entrypoint, iirc you can escape the reference to "$$Name", but this will only make the env variable available to the entrypoint or command script which are evaluated on the host, not to values like the container_name.
Given your specific use case, perhaps service creation templates are what you are looking for: They let you inject per service instance values into hostname, mount and env.
version: '3.8'
services:
test:
env:
MY_HOSTNAME: "{{.Node.Hostname}}"
...
See Create Service Using Templates for the full list of supported values.
docker does really confuse me....
I am trying to use the "environments" variable as well as the .env file to pass variables into the container that will be created. Sadly without success.
What my setup is:
docker-compose.yml
# dockerfile version 3.8 -> 18.07.2020
version: "3.8"
# services basicly means scalable amount of containers
services:
testalpine:
image: testenvalpine
build:
context: .
dockerfile: test-dockerfile
args:
- DB_NAME=nextcloud
- DB_USER=nextcloud
# todo only use https and self sign certificate
ports:
- "80:80"
environment:
- env1=hello
- env2=world
networks:
- nextcloudnetwork
# todo include redis and mariadb
networks:
nextcloudnetwork:
# std driver seems to be overlay?
driver: overlay
Dockerfile:
test-dockerfile
FROM alpine:latest
LABEL maintainer="xddq <donthavemyownemailyet:(((>"
ARG DB_NAME=default
ARG DB_USER=default
ENV env1=dockerfile env2=$DB_NAME
ENTRYPOINT [ "sh", "-c", \
"echo DB_NAME: $DB_NAME DB_USER: $DB_USER env1: $env1 env2: $env2" ]
My .env file
DB_NAME=nextcloud
DB_USER=nextcloud
The output I did EXPECT:
DB_NAME:nextcloud DB_USER: nextcloud env1: hello env2:nextcloud
The output I got:
DB_NAME: DB_USER: env1: dockerfile env2: nextcloud
Does this mean ".env" and ENV variable in docker-compose are completely useless for the env variables inside the container that will be created? I mean I could only get any result using the ARG variable..? :/
greetings
The .env is not automatically passed to the container, you need to declare it in your docker-compose.yml using env_file: Explaination here. The environment inside the dockerfile should be overriden by the ones in your docker-compose file not sure why this is not the case.
I have the following in my docker-compose.yml file
geth-testnet:
build:
context: .
dockerfile: Dockerfile
args:
GETH_REPO: 'https://github.com/ethereum/go-ethereum'
GETH_VERSION: 'v1.8.12'
RPC_LISTEN_PORT: 8546
command: "--rpcport ${RPC_LISTEN_PORT}"
entrypoint:
- "geth"
tty: true
image: geth-node-testnet:v1.8.12
container_name: geth-node-testnet
ports:
- '8546:8546'
volumes:
- /root/.ethereum
When I run, docker-compose up --build, expect it to run the following command:
geth -rpcport 8546
However, I get the following error
flag needs an argument: -rpcport
So, the value for RPC_LISTEN_PORT is not correctly substituted.
I have ARG RPC_LISTEN_PORT in my dockerfile
Double checked your question, seems command: "--rpcport ${RPC_LISTEN_PORT}" cannot utilize the value you put in docker-compose.yml.
So afford two solutions:
export RPC_LISTEN_PORT=8546 in bash before you do compose command.
New a .env file in the same folder, put RPC_LISTEN_PORT=8546 to it.
I'm working on a group project involving Docker that has a .env file, which looks like this:
DATABASE_URL=xxx
DJANGO_SETTINGS_MODULE=xxx
SECRET_KEY=xxx
Couldn't this just be declared inside the Dockerfile? If so, what is the advantage of making a .env file?
Not sure if I'm going in the right direction with this, but this Docker Docs page says (emphasis my own):
Your configuration options can contain environment variables. Compose
uses the variable values from the shell environment in which
docker-compose is run. For example, suppose the shell contains
POSTGRES_VERSION=9.3 and you supply this configuration:
db:
`image: "postgres:${POSTGRES_VERSION}"`
When you run docker-compose up with this configuration, Compose looks for the POSTGRES_VERSION environment variable in the shell and substitutes its value in. For this example, Compose resolves the image to postgres:9.3 before running the configuration.
If an environment variable is not set, Compose substitutes with an empty string. In the example above, if POSTGRES_VERSION is not set, the value for the image option is postgres:.
You can set default values for environment variables using a .env file, which Compose automatically looks for. Values set in the shell environment override those set in the .env file.
If we're using a .env file, then wouldn't I see some ${...} syntax in our docker-compose.yml file? I don't see anything like that, though.
Here's our docker-compose.yml file:
version: '3'
services:
server:
build:
context: ./server
dockerfile: Dockerfile
env_file: .env.dev
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- ./server:/app
ports:
- "8500:8000"
depends_on:
- db
stdin_open: true
tty: true
db:
image: postgres
client:
build:
context: ./client
dockerfile: Dockerfile
command: bash -c "npm install; npm run start"
volumes:
- ./client:/app
- /app/node_modules
ports:
- "3000:3000"
depends_on:
- server
Idea there is probably to have a place to keep secrets separated from docker-compose.yml, which you then can keep in VCS and/or share.
On my image I want to set some environment variables eg: MY_VAR where it will have a static value eg: MY_VAR=12 but I do NOT want to be able to set it via docker's -e param or via docker-compose.yml's environment section.
Furthermore I do not want to be as build argument when i do either docker build or docker-compose build
How can I do that?
You can do that from an entrypoint script.
In your Dockerfile:
ENTRYPOINT ["/entrypoint.sh"]
Example entrypoint.sh:
#!/bin/sh
export VAR=foobar
exec /usr/bin/python "$#"
To be more flexible and allow setting it with the -e option:
export VAR=${VAR:-"foobar"}
...
The best solution for your question is to include an env_file on your docker-compose build
version: '3.2'
services:
db:
restart: always
image: postgres:alpine
volumes:
- backup-data:/var/lib/postgresql/data
env_file:
- ./env/.dev
Then in your env_file:
POSTGRES_USER=my_user
POSTGRES_PASSWORD=my_password
POSTGRES_DB=my_db