On my image I want to set some environment variables eg: MY_VAR where it will have a static value eg: MY_VAR=12 but I do NOT want to be able to set it via docker's -e param or via docker-compose.yml's environment section.
Furthermore I do not want to be as build argument when i do either docker build or docker-compose build
How can I do that?
You can do that from an entrypoint script.
In your Dockerfile:
ENTRYPOINT ["/entrypoint.sh"]
Example entrypoint.sh:
#!/bin/sh
export VAR=foobar
exec /usr/bin/python "$#"
To be more flexible and allow setting it with the -e option:
export VAR=${VAR:-"foobar"}
...
The best solution for your question is to include an env_file on your docker-compose build
version: '3.2'
services:
db:
restart: always
image: postgres:alpine
volumes:
- backup-data:/var/lib/postgresql/data
env_file:
- ./env/.dev
Then in your env_file:
POSTGRES_USER=my_user
POSTGRES_PASSWORD=my_password
POSTGRES_DB=my_db
Related
I am finding my self repeating like this:
services:
noice-service:
environment:
- EUREKA_SERVICE_URL=${EUREKA_SERVICE_URL}
- ZIPKIN_BASE_URL=${ZIPKIN_BASE_URL}
- CONFIG_SERVER_URL=${CONFIG_SERVER_URL}
I have defined these env vars in .env file and some in another scripts and I just want to pass their exact value in container. Is there any way quicker way of achieving this without any custom shell script as entrypoint ?
You can pass the variables directly:
# .env
DOTENV=foo
# docker-compose.yml
version: "3.7"
services:
busybox:
image: busybox
command: ["env"]
environment:
- DOTENV
- ANOTHER
And run ANOTHER=bar docker-compose up.
I'm looking to be able to run swarm from same docker-compose file which uses env variables. Currently I only achieved that all nodes are replicating Leaders env. Is it possible to let each node start from its own local env var?
My docker-compose
version: '3.1'
networks:
base:
services:
test:
container_name: ${Name}
restart: always
image: ubuntu:latest
environment:
- Name=${Name}
command: sh -c "echo $Name && sleep 30"
networks:
- base
use env_file option
https://docs.docker.com/compose/environment-variables/
# .env, file
Name=<your_name>
# <your_name>.env, file
TEST_ENV=stackoverflow
# docker-compose.yaml, file
version: '3.1'
services:
test:
container_name: ${Name}
restart: always
image: ubuntu:latest
env_file:
- ${Name}.env
command: sh -c "set | grep TEST_ENV && sleep 30"
docker logs <your_name>
# TEST_ENV='stackoverflow'
You can set env_files with different names in different containers.
for example
# docker-compose.yaml, file
version: '3.1'
services:
test1:
container_name: test1
restart: always
image: ubuntu:latest
env_file:
- first.env
command: sh -c "set | grep TEST_FIRST_ENV && sleep 30"
test2:
container_name: test2
restart: always
image: ubuntu:latest
env_file:
- second.env
command: sh -c "set | grep TEST_SECOND_ENV && sleep 30"
Environment variables referenced in the docker-compose.yml file are not resolved on the leader even, they are resolved on whatever jump box you are deploying too the swarm from.
If you want to reference the env vars from the host system, from the command, or entrypoint, iirc you can escape the reference to "$$Name", but this will only make the env variable available to the entrypoint or command script which are evaluated on the host, not to values like the container_name.
Given your specific use case, perhaps service creation templates are what you are looking for: They let you inject per service instance values into hostname, mount and env.
version: '3.8'
services:
test:
env:
MY_HOSTNAME: "{{.Node.Hostname}}"
...
See Create Service Using Templates for the full list of supported values.
I'm working on a group project involving Docker that has a .env file, which looks like this:
DATABASE_URL=xxx
DJANGO_SETTINGS_MODULE=xxx
SECRET_KEY=xxx
Couldn't this just be declared inside the Dockerfile? If so, what is the advantage of making a .env file?
Not sure if I'm going in the right direction with this, but this Docker Docs page says (emphasis my own):
Your configuration options can contain environment variables. Compose
uses the variable values from the shell environment in which
docker-compose is run. For example, suppose the shell contains
POSTGRES_VERSION=9.3 and you supply this configuration:
db:
`image: "postgres:${POSTGRES_VERSION}"`
When you run docker-compose up with this configuration, Compose looks for the POSTGRES_VERSION environment variable in the shell and substitutes its value in. For this example, Compose resolves the image to postgres:9.3 before running the configuration.
If an environment variable is not set, Compose substitutes with an empty string. In the example above, if POSTGRES_VERSION is not set, the value for the image option is postgres:.
You can set default values for environment variables using a .env file, which Compose automatically looks for. Values set in the shell environment override those set in the .env file.
If we're using a .env file, then wouldn't I see some ${...} syntax in our docker-compose.yml file? I don't see anything like that, though.
Here's our docker-compose.yml file:
version: '3'
services:
server:
build:
context: ./server
dockerfile: Dockerfile
env_file: .env.dev
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- ./server:/app
ports:
- "8500:8000"
depends_on:
- db
stdin_open: true
tty: true
db:
image: postgres
client:
build:
context: ./client
dockerfile: Dockerfile
command: bash -c "npm install; npm run start"
volumes:
- ./client:/app
- /app/node_modules
ports:
- "3000:3000"
depends_on:
- server
Idea there is probably to have a place to keep secrets separated from docker-compose.yml, which you then can keep in VCS and/or share.
When I run docker-compose build && docker-compose up redis, with environment specified in docker-compose.yaml and RUN env in the Dockerfile, the environment variables I set don't get printed.
Why does this not work?
I'm using docker-compose version 1.4.2.
Here are the relevant files:
docker-compose.yaml with environment as a list of KEY=value pairs:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO='bar'
docker-compose.yaml with environment as a dictionary:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO: 'bar'
Dockerfile:
FROM redis:2.6
MAINTAINER me#email.com
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
That's normal
docker-compose only sets the environment variables specified in the environment directive in the docker-compose.yaml file during the run phase of the container, and not during the build phase.
So if you do docker-compose run --entrypoint "/bin/bash" redis -c env you will be able to see your env variables.
If you want to set variables inside your Dockerfile (to be able to see them during the build phase) you can add inside your dockerfile before your RUN env:
ENV FOO bar
Well
I have tested and found following solutions for docker compose with env file or without env file. I will show you two different approach
Lets say you have following docker compose yml file
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
env_file: docker.env
Now you need to setup the postgres variable in a file called docker.env. Remember you need to keep the docker_compose.yml file and docker.env file in same folder.
Next, In the docker.env file you need to have the database variable and value like this:
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=myapp_db
Now hit docker-compose up . It should work.
Lets say now you dont like to specify the env file name in the docker-compose.yml file. So you have to write docker-compose.yml file like this:
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
environments:
- POSTGRES_USER=${PGU}
-POSTGRES_PASSWORD=${PGP}
-POSTGRES_DB=${PGD}
Now your docker.env file should look like this:
PGU=postgres
PGP=postgres
PGD=myapp_db
now hit docker-compose --env-file docker.env up
you are good to go.
This is because you were using environment when (I guess) you wanted to use args inside the build block:
redis:
build:
context: ../storage/redis
args:
- FOO: 'bar'
ports:
- "6379:6379"
Your Dockerfile would define FUN in the (image) environment:
FROM redis:2.6
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
# Read FUN from (build) arguments
# (may define a default: ARG FUN='wow')
ARG FUN
# Define env variable FUN with value from ARG
ENV FUN=$FUN
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
The environment block is used to define variables for the running container (when docker-compose up, NOT when docker-compose build).
Having the following docker-compose file:
db:
build: .
environment:
- MYSQL_ROOT_PASSWORD=password
- ENV=test
env_file: .env
Is there any way to use the env variables declared in docker-compose.yml (either as environment or declared in the env_file) as part of Dockerfile without declaring them in the Dockerfile? Something like this:
FROM java:7
ADD ${ENV}/data.xml /data/
CMD ["run.sh"]
Although this question was asked long ago, there is an answer to a similar question here: Pass environment variables from docker-compose to container at build stage
Basically, to use variables at the container's build time one has to define the variable in docker-compose.yml:
build:
context: .
args:
MYSQL_ROOT_PASSWORD: password
ENV: test
and then reference it in the Dockerfile using ARG:
ARG MYSQL_ROOT_PASSWORD
ARG ENV
ADD ${ENV}/data.xml /data/
Concerning environment variables defined in an *.env file, I believe that they can't be passed to the container at build time.
It works ok this way:
docker-compose.yml
version: '3.5'
services:
container:
build:
context: .
args:
ENV: ${ENV} # from .env file
env_file:
- .env
Dockerfile
# from compose args
ARG ENV
ADD ${ENV}/data.xml /data/
.env
ENV=myenv
Thus all the values are taken from .env file
This approach goes against the 'build once, run anywhere' theory behind Docker and most DevOps approaches. With this approach you'll need to build a container for every environment you expect to use. By doing so you can't safely say if a container works in the dev environment it will work in staging and production since you aren't using the same container.
You'd be better off adding all config files you need on to the container and writing an entrypoint script that selects/copies the data for that environment to the correct location when the container starts. You can also apply this approach to other config on the container, like templated Apache config using jinja2 templates etc.