Is it possible to specify the env file that docker compose uses for variable substitution? Currently it defaults to ".env" but I have different flavours of compose files which should have different envs.
You can use inheritance for this. If you have one "base" service where you set up the environment, all of your other services can inherit from that.
Example:
version: "2"
services:
base:
env_file:
- my_env.txt
web:
extends:
service: base
image: foo
database:
extends:
service: base
image: foo-db
The above example has everything in the same file, but you can also split this up into multiple files, where the base service would reside in a base.yaml file. You just need to add file: base.yaml to the extends section. Please see the documentation here.
I use this approach for setting the proxy variables for all containers. I have a proxy.yaml file that defines a proxy-app service that picks up the proxy environment variables from the shell. All of my real services extend the proxy-app service and thus inherit the environment settings from that service.
The --env-file command-line argument and the env_file docker-compose.yml variable specify the env file to use for the container, not for the container build. To set a different file (e.g. alt.env) for the build itself, use this:
env $(cat alt.env) docker-compose up --build
According to the documentation, it's now possible to load an environment file (contrary to a per-service file), docker-compose will then export the env variables defined in this env file prior to starting any service, they can then be used in the docker-compose.yml config file itself:
version: "3.7"
services:
node:
environment:
APP_ENV: "${APP_ENV}"
NODE_ENV: "${NODE_ENV}"
ports:
- "${HOST_EXPOSED_NODEJS_DEBUG_PORT}:9229"
volumes:
- type: bind
source: ./project
target: /var/www/project
read_only: false
Since docker-compose 1.25 it's also possible to specify a custom .env file with the --env-file flag (unfortunately it's currently not possible to specify multiple .env files with the --env-file flag)
Related
Is there a way to inject environment variables to all of my docker-compose services, without explicitly declaring them in each service's configuration?
1)using the env_file directive
you can use the env_file directive within your docker-compose.yml file
You can pass multiple environment variables from an external file
through to a service’s containers with the ‘env_file’ option, just
like with docker run --env-file=FILE ...:
you will have to declare the env file for each service that will use it and that is it.
example :
the docker-compose.yml file :
version: "3"
services:
database:
image: "ubuntu"
tty: true
env_file:
- same-variables.env
web:
image: "ubuntu"
tty: true
env_file:
- same-variables.env
the same-variables.env file
IS_DOCKER_COMPOSE=yes
then if you do opening a terminal :
docker exec -it <docker_container> "echo $IS_DOCKER_COMPOSE"
result will be :
yes
2)using the .env file in project root
According to the doc: https://docs.docker.com/compose/environment-variables/#the-env-file
You can set default values for any environment variables referenced in
the Compose file, or used to configure Compose, in an environment file
named .env. The .env file path is as follows
Create an .env file as the root of your project as follow.
example :
your .env file :
TAG=v1.5
your docker-compose.yml file :
version: '3'
services:
web:
image: "webapp:$TAG"
organization of your project should be :
root_folder
|-.env
|-docker-compose.yaml
with the .env file with all your variable
the env file will work for all of them at the same time
I have 2 files:
.env
docker-compose.yml
docker-compose.yml looks like this:
version: '3'
services:
database:
image: mysql:5.7
myapp:
image: me/some-image
depends_on:
- database
env_file: .env
myapp is a web service app that needs a .env file or optionally it can access the environment variables if no .env file is present.
As of now, the myapp is accessing the environment variables because I don't want the .env file to be included in the image build for security reasons. What I did is to pass a env_file: .env to the myapp service in the docker-compose.yml file so it will rely to the environment variables of the service instead of a .env file.
Now, I really want to add a .env file to the myapp service when running docker-compose up. Take note that the myapp web service will throw an error if it didnt find a .env file and the option is to look for a .env file instead of getting from the environment variables of the container.
Is there a way to create a .env file when running docker-compose up and copy the contents of the .env file on the host? Thank you in advance.
You can use bind mount to mount file into the container
change the target location to the one your app requires
version: '3'
services:
database:
image: mysql:5.7
myapp:
image: me/some-image
depends_on:
- database
env_file: .env
volumes:
- type: bind
source: ./.env
target: /envfile/.env
readonly: true
You shouldn't be storing the environment variables in the container.
According to TwelveFactor-
The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.
You are correctly passing the env file. The env file needs to be in the format of <key>=<value>.
For example -
DB_USERNAME=user
DB_PASSWORD=password
DB_SCHEMA=db
DB_PORT=3306
DB_HOST=db
REDIS_HOST=redis
REDIS_PORT=6379
REDIS_DB=0
In your application, you don't need to care about if the .env file is present, you simply read the environment variable.
Example in python -
import os
db_username = os.environ['DB_USERNAME']
You should not be coupling your environment variable to a file. It should read from the runtime environment.
So I am trying to set up a few services in my local docker environment. One of the services, as defined in my docker-compose.yml file, is as follows:
version: "3"
services:
solr:
image: solr:latest
ports:
- "8983:8983"
container_name: solr
...which is pretty straightforward.
However, I also want to set up another service that has its own docker-compose.yml in it. If I simply specify a FROM in the Dockerfile for this service, it doesn't work as expected as the repo's docker-compose.yml includes services from other repositories as well.
What is the right way to configure such a service that has its own docker-compose.yml file?
Use extend in yml file to call another docker-compose file. When defining any service in docker-compose.yml, you can declare that you are extending another service like this:
web:
extends:
file: common-services.yml
service: webapp
Ref : https://docs.docker.com/compose/extends/#understand-the-extends-configuration
Note : extend option is removed from docker-compose version-3 and it is available only in version-2
Ref : https://docs.docker.com/compose/extends/#extending-services
I am working on a docker-compose file, in which I need to specify container_name from an environment variable.
My docker-compose.yml file looks like this:
version: '3.0'
services:
jenkins:
environment:
- INSTANCE_NAME=team_1
image: my_image
container_name: container_$INSTANCE_NAME
ports:
- "80:80"
expose:
- "80"
So, I think, when I run docker-compose up it should create container as name, container_team_1, but instead of that it runs as contaner_
I also tried this thing using .env file, but still, I can not use environment variable in container_name,
although, if I run docker-compose config I can see all variables set like follow,
container_name: container_
environment:
COMPANY_NAME: team_1
but, Actually it is not attaching in container-name.
You cannot use environment variables defined in docker-compose.yml to variable substitution.
Docker Compose uses .env by default so it should work when you define in .env file
INSTANCE_NAME=team_1
And then run docker-compose up
As I can see in variable substitution section of the docker-compose documentation, you will need to set your $INSTANCE_NAME in the shell that is running the docker-compose up, because:
Compose uses the variable values from the shell environment in which docker-compose is run.
First of all, do something like:
export INSTANCE_NAME=my_instance`
and then:
docker-compose up
Best regards.
I have a cache folder and the path is stored in an evironment variable: $LOCAL_CACHE (export LOCAL_CACHE=/home/me/path/to/any/cache/folder)
Now I want to append some additional folders inside my .env file:
My .env file:
LOCAL_COMPOSER_DIR=${LOCAL_CACHE}/composer
LOCAL_NPM_DIR=${LOCAL_CACHE}/npm
LOCAL_BOWER_DIR=${LOCAL_CACHE}/bower
My docker_compose.yml looks like that:
version: '2'
services:
composer:
env_file: .env
image: composer/composer
volumes:
- ./src:${APP_ROOT}
- ${LOCAL_COMPOSER_DIR}:/composer
working_dir: ${APP_ROOT}
When I start the service with docker-compose run composer instal. ... it stops with following error: ERROR: Named volume "$LOCAL_CACHE"/composer":/composer:rw" is used in service "composer" but no declaration was found in the volumes section.
I'm not shure but it seems, the .env file doesn't support the use of variables.
Yes. Into .env file you can not use variables. Suddenly but true. Actual for docker compose version 1.8.
But you can use variables into docker-compose.yml file like
volumes:
- ${LOCAL_COMPOSER_DIR}/${CACHE_DIR}:/composer/${CACHE_DIR}
This answer suggest us another way.