I have a cache folder and the path is stored in an evironment variable: $LOCAL_CACHE (export LOCAL_CACHE=/home/me/path/to/any/cache/folder)
Now I want to append some additional folders inside my .env file:
My .env file:
LOCAL_COMPOSER_DIR=${LOCAL_CACHE}/composer
LOCAL_NPM_DIR=${LOCAL_CACHE}/npm
LOCAL_BOWER_DIR=${LOCAL_CACHE}/bower
My docker_compose.yml looks like that:
version: '2'
services:
composer:
env_file: .env
image: composer/composer
volumes:
- ./src:${APP_ROOT}
- ${LOCAL_COMPOSER_DIR}:/composer
working_dir: ${APP_ROOT}
When I start the service with docker-compose run composer instal. ... it stops with following error: ERROR: Named volume "$LOCAL_CACHE"/composer":/composer:rw" is used in service "composer" but no declaration was found in the volumes section.
I'm not shure but it seems, the .env file doesn't support the use of variables.
Yes. Into .env file you can not use variables. Suddenly but true. Actual for docker compose version 1.8.
But you can use variables into docker-compose.yml file like
volumes:
- ${LOCAL_COMPOSER_DIR}/${CACHE_DIR}:/composer/${CACHE_DIR}
This answer suggest us another way.
Related
Is there a way to inject environment variables to all of my docker-compose services, without explicitly declaring them in each service's configuration?
1)using the env_file directive
you can use the env_file directive within your docker-compose.yml file
You can pass multiple environment variables from an external file
through to a service’s containers with the ‘env_file’ option, just
like with docker run --env-file=FILE ...:
you will have to declare the env file for each service that will use it and that is it.
example :
the docker-compose.yml file :
version: "3"
services:
database:
image: "ubuntu"
tty: true
env_file:
- same-variables.env
web:
image: "ubuntu"
tty: true
env_file:
- same-variables.env
the same-variables.env file
IS_DOCKER_COMPOSE=yes
then if you do opening a terminal :
docker exec -it <docker_container> "echo $IS_DOCKER_COMPOSE"
result will be :
yes
2)using the .env file in project root
According to the doc: https://docs.docker.com/compose/environment-variables/#the-env-file
You can set default values for any environment variables referenced in
the Compose file, or used to configure Compose, in an environment file
named .env. The .env file path is as follows
Create an .env file as the root of your project as follow.
example :
your .env file :
TAG=v1.5
your docker-compose.yml file :
version: '3'
services:
web:
image: "webapp:$TAG"
organization of your project should be :
root_folder
|-.env
|-docker-compose.yaml
with the .env file with all your variable
the env file will work for all of them at the same time
I have a .env file like this:
PROJECT_DIR=./test_dir
SUB_DIR=$(PROJECT_DIR)/sub_dir
And docker-compose.yml file like this:
version: '3.5'
services:
service_nginx:
image: nginx
volumes:
- ${SUB_DIR}:/var/www/html
But I can't use $SUB_DIR in docker-compose.yml file for volume, I got error like this:
ERROR: Named volume "$(PROJECT_DIR)/sub_dir:/var/www/html:rw" is used in service "service_nginx" but no declaration was found in the volumes section.
Update your env file as follows
export PROJECT_DIR=./test_dir
export SUB_DIR=$(PROJECT_DIR)/sub_dir
Then run the following command to read the varialbes;
. ./.env
Then deploy the stack from same terminal session.
I have 2 files:
.env
docker-compose.yml
docker-compose.yml looks like this:
version: '3'
services:
database:
image: mysql:5.7
myapp:
image: me/some-image
depends_on:
- database
env_file: .env
myapp is a web service app that needs a .env file or optionally it can access the environment variables if no .env file is present.
As of now, the myapp is accessing the environment variables because I don't want the .env file to be included in the image build for security reasons. What I did is to pass a env_file: .env to the myapp service in the docker-compose.yml file so it will rely to the environment variables of the service instead of a .env file.
Now, I really want to add a .env file to the myapp service when running docker-compose up. Take note that the myapp web service will throw an error if it didnt find a .env file and the option is to look for a .env file instead of getting from the environment variables of the container.
Is there a way to create a .env file when running docker-compose up and copy the contents of the .env file on the host? Thank you in advance.
You can use bind mount to mount file into the container
change the target location to the one your app requires
version: '3'
services:
database:
image: mysql:5.7
myapp:
image: me/some-image
depends_on:
- database
env_file: .env
volumes:
- type: bind
source: ./.env
target: /envfile/.env
readonly: true
You shouldn't be storing the environment variables in the container.
According to TwelveFactor-
The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.
You are correctly passing the env file. The env file needs to be in the format of <key>=<value>.
For example -
DB_USERNAME=user
DB_PASSWORD=password
DB_SCHEMA=db
DB_PORT=3306
DB_HOST=db
REDIS_HOST=redis
REDIS_PORT=6379
REDIS_DB=0
In your application, you don't need to care about if the .env file is present, you simply read the environment variable.
Example in python -
import os
db_username = os.environ['DB_USERNAME']
You should not be coupling your environment variable to a file. It should read from the runtime environment.
i get crazy with the env_file setting inside my docker-compose.yml.
when i name the file .env and put it in the root of my project everything works fine. the main point of having this env file was to have multiple enviroments (dev, staging, production) where i made my settings.
as soon i name my file different (for example dev.env and put it inside a env folder) and try to load it inside my docker-compose file it won't work.
i make a simple example to make things clear.
for simplicity i use one variable DC_DOCKERFILE where i define what dockerfile to load.
file: .env in the root folder:
COMPOSE_PROJECT_NAME=test
DC_DOCKERFILE=./Dockerfile.bash.dev
file: docker-compose.yml:
version: "2"
services:
bash:
build:
context: .
dockerfile: ${DC_DOCKERFILE}
and docker-compose config shows everything is cool and build works as well:
networks: {}
services:
bash:
build:
context: /devops/myproject
dockerfile: ./Dockerfile.bash.dev
version: '2.0'
volumes: {}
now the point which freaks me out:
create a folder env in the root of the project
move .env to ./env/dev.env
add env_file: ./env/dev.env inside my docker-compose.yml
env content looks same like above so i don't post the content again.
my new docker-compose.yml:
version: "2"
services:
bash:
env_file: ./env/dev.env
build:
context: .
dockerfile: ${DC_DOCKERFILE}
only change, add the env_file settings.
and now i docker-compose config gives me this:
networks: {}
services:
bash:
build:
context: /devops/myproject
dockerfile: ''
environment:
COMPOSE_PROJECT_NAME: test
DC_DOCKERFILE: ./Dockerfile.bash.dev
version: '2.0'
volumes: {}
say what? where the hack is my dockerfile settings?
of course docker-compose build don't like that too and bothers with:
docker-compose build --force-rm --no-cache
WARNING: The DC_DOCKERFILE variable is not set. Defaulting to a blank string.
Building bash
ERROR: Cannot locate specified Dockerfile: Dockerfile
the variables get loaded but somehow not assigned.
so i don't get it.
am i maybe wrong and i can't use the env file for docker-compose stuff inside the docker-compose.yml file. would make sence that it need to be loaded first somehow via docker-compose parameter or something but after i spend so much time i want to know if i can load it inside or not :)
thanks for help and hell yeah for open source and the awesome community.
peace out
When .env file is present in the folder docker-compose command is executed, those environment variables are used as environment variables for docker-compose execution and variable substitution. (https://docs.docker.com/compose/env-file/).
However when you define env_file option to your service, the service will get those variables from the file as environment variables and those are not used for variable substitution.
Is it possible to specify the env file that docker compose uses for variable substitution? Currently it defaults to ".env" but I have different flavours of compose files which should have different envs.
You can use inheritance for this. If you have one "base" service where you set up the environment, all of your other services can inherit from that.
Example:
version: "2"
services:
base:
env_file:
- my_env.txt
web:
extends:
service: base
image: foo
database:
extends:
service: base
image: foo-db
The above example has everything in the same file, but you can also split this up into multiple files, where the base service would reside in a base.yaml file. You just need to add file: base.yaml to the extends section. Please see the documentation here.
I use this approach for setting the proxy variables for all containers. I have a proxy.yaml file that defines a proxy-app service that picks up the proxy environment variables from the shell. All of my real services extend the proxy-app service and thus inherit the environment settings from that service.
The --env-file command-line argument and the env_file docker-compose.yml variable specify the env file to use for the container, not for the container build. To set a different file (e.g. alt.env) for the build itself, use this:
env $(cat alt.env) docker-compose up --build
According to the documentation, it's now possible to load an environment file (contrary to a per-service file), docker-compose will then export the env variables defined in this env file prior to starting any service, they can then be used in the docker-compose.yml config file itself:
version: "3.7"
services:
node:
environment:
APP_ENV: "${APP_ENV}"
NODE_ENV: "${NODE_ENV}"
ports:
- "${HOST_EXPOSED_NODEJS_DEBUG_PORT}:9229"
volumes:
- type: bind
source: ./project
target: /var/www/project
read_only: false
Since docker-compose 1.25 it's also possible to specify a custom .env file with the --env-file flag (unfortunately it's currently not possible to specify multiple .env files with the --env-file flag)