How can I interpolate env vars when using docker-compose . if I have a following vars.env file :
VAR_1=value_1
VAR_2=value_1/value2
VAR_3=value_1/value3
I would like to interpolate using of VAR_1 like this
VAR_1=value_1
VAR_2=${VAR_1}/value2
VAR_3=${VAR_1}/value3
But it is not working. Any idea ?
This is not supported. You can however use an .env file to set the value of $VAR_1: https://docs.docker.com/compose/env-file/
Related
I want to know where can I see the variables I have defined. For example, I have three files:
.env
BLA=1
docker-compose.xml
IS_ONE=${BLA}
Dockerfile
RUN echo "$BLA" >> file.txt
I want to know the relation, for example:
Do I need the variable defined in all files? or there a relation like
docker-compose.xml can only see .evn variables?
Dockerfile can only see docker-compose.xml variables?
And, where do I need to declare a variable to use in a bash script automatically triggered in Dockerfile (if it's possible), or use later in the console of the container.
Lots of examples here: https://docs.docker.com/compose/environment-variables/
If you need to customize docker-compose itself then .env file is used.
For example, I want to include build id in my container name, my .env file would have
BUILD_ID=20
and docker-compose.yml would have
containername: "foo-${BUILD_ID}"
If I want to pass environment vars into container when it runs, I use
environment:
- FOO_VAR=bar
If I want to pass in multiple environment vars into container,
docker-compose.yml would have
env_file: my_container_env
is the easy way if the values of the FOO_VAR are going to be dynamically generated.
Whether to use set env values in docker-compose directly or use file depends on:
Are the values dynamic? If yes, use file and populate correct values
in the file with some script (like jenkins could set my BUILD_ID in
my .env file)
Are they 'secret'? If yes, use file and don't checkin the .env or my_container_env file into the repo - you have to manage those files
separately
I am using a base.env as an env_file for several of my docker services.In this base.env I have several parts of the environment variable that repeat throughout the file. For example, port and ip are the same for three different environment variables.
I would like to specify these in an environment variable and reuse those variables to fill out the other environment variables.
Here is base.env:
### Kafka
# kafka's port is 9092 by default in the docker-compose file
KAFKA_PORT_NUMBER=9092
KAFKA_TOPIC=some-topic
KAFKA_IP=kafka
KAFKA_CONN: //$KAFKA_IP:$KAFKA_PORT_NUMBER/$KAFKA_TOPIC
# kafka topic that is to be created. Note that ':1:3' should remain the same.
KAFKA_CREATE_TOPICS=$KAFKA_TOPIC:1:3
# the url for connecting to kafka
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://$KAFKA_IP:$KAFKA_PORT_NUMBER
I have tried writing
KAFKA_CONN: //$${KAFKA_IP}:$${KAFKA_PORT_NUMBER}/$${KAFKA_TOPIC}
in the environment section of the appropriate service in the docker-compose.yml, but this gets interpreted as a literal string in the container.
Is there a way to do what I want in the base.env file?
Thank you for your help!
You can actually do it like this (at least in vlucas/dotenv package (php), not sure about others, please check it yourself)
MAIL_NAME=${MAIL_FROM}
Read more about it here
There is no way to do this in an env_file since it is not run as a bash command. This means that the variable is not created and then concatenated into the next variable it appears in. The values are just read in as they are in the env_file.
I used $ in Node.js and React.js , and both worked
POSTGRES_PORT=5432
DATABASE_URL="postgresql://root#localhost:${POSTGRES_PORT}/dbname"
and in react
REACT_APP_DOMAIN=domain.com
#API Configurations
REACT_APP_API_DOMAIN=$REACT_APP_DOMAIN
I know that I am a little late to the party, but I had the same question and I found a way to do it. There is a package called env-cmd, which allows you to use a .js file as an .env file. The file simply needs to export an object with the keys being your environment variable names and the values being, well, the values. This now allows you to run javascript before the environment variables are exported and thus use environment variables to set others.
I temporarly managed to deal with this where I create a script to replace env file vars from another env file vars like so:
.env.baseurl:
BASEURL1=http://127.0.0.1
BASEURL2=http://192.168.1.10
.env.uris.default:
URI1=${BASEURL1}/uri1
URI2=${BASEURL2}/uri2
URI3=${BASEURL2}/uri3
convert-env.sh:
#!/bin/bash
# To allow using sed correctly from same file multiple times
cp ./.env.uris.default ./.env.uris
# Go through each variable in .env.baseurl and store them as key value
for VAR in $(cat ./.env.baseurl); do
key=$(echo $VAR | cut -d "=" -f1)
value=$(echo $VAR | cut -d "=" -f2)
# Replace env vars by values in ./.env.uris
sed -i "s/\${$key}/$value/g" ./.env.uris
done
then you can run docker run command to start the container and load it with your env vars (from .env.baseurl to .env.uris) :
docker run -d --env-file "./.env.uris" <image>
This is not the best solution but helped me for now.
Using Nextjs, in the .env.local file I have the following variables:
NEXT_PUBLIC_BASE_URL = http://localhost:5000
NEXT_PUBLIC_API_USERS_URL_REGISTER = ${NEXT_PUBLIC_BASE_URL}/api/users/register
it works well, I used the variable NEXT_PUBLIC_BASE_URL in the variable NEXT_PUBLIC_API_USERS_URL_REGISTER.
There is a simple way to do this you will just need to run:
env >>/root/.bashrc && source /root/.bashrc
this will append all environment variables inside /root/.bashrc then convert those if they have not been converted while passing the env-file
you can use something like this ${yourVar}
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://${KAFKA_IP}:${$KAFKA_PORT_NUMBER}
I test this on PHP / Laravel .env it's working fine
How can I use variable substitution for a list, map, or array value in a docker-compose.yml file.
For example:
graylog:
image: graylog2/server2
extra_hosts: ${EXTRA_HOSTS}
and
export EXTRA_HOSTS="['host1:10.10.10.1','host2:10.10.10.2']"
gives the following error: graylog.extra_hosts must be a mapping
I've tried different variations of the above with no luck.
I do see that there's an open issue about this here: https://github.com/docker/compose/issues/4249
Is it just not possible? Does anyone know of a workaround?
At least as of this time (June 2018), Docker still doesn't support this. I was able to work around the issue utilizing envsubst.
envsubst is part of gettext and it can be used to replace only environment variables you tell it to.
Tweak the docker-compose.yml value to look like an array or map (either brackets or curly braces) but have the value be an environment variable.
For example
graylog:
image: graylog2/server2
extra_hosts: [ ${EXTRA_HOSTS} ]
Then, define your environment variable without brackets or curly braces.
For example:
export EXTRA_HOSTS="'host1:10.10.10.1','host2:10.10.10.2'"
Then utilize envsubst
envsubst '${EXTRA_HOSTS}' < docker-compose.yml > docker-compose.subst.yml && docker stack deploy -c docker-compose.subst.yaml foobar
Notice that you pass '${EXTRA_HOSTS}' to envsubst. This tells it to only replace this environment variable. This ensures it doesn't accidentally replace some other variable that's utilizing the variable substitution syntax of Docker compose files.
I have the following .env file:
# ENV
ENVIRONMENT=local
SITENAME=reo
I have the following docker-compose.yml (part of it) file:
volumes:
${SITENAME}-sync:
external: true
I get the following error:
volumes value '${SITENAME}-sync' does not match any of the regexes: u'^[a-zA-Z0-9._-]+$'
Is it possible to have a variable as a key in a Yaml file?
I noticed that: key: ${SITENAME} does work.
Environment variables are not possible for keys in docker-compose.yml, only for values. See here
Since docker-compose doesn't support interpolation in keys, you'd need to add another level of indirection, e.g. use a templating tool like mustache or handlebars, and generate the docker-compose.yml file yourself from env vars.
I have got a docker-compose.yml file and there I define:
extra_hosts:
- "localhost:${MY_MACHINE_IP}"
It works if I define MY_MACHINE_IP as environment var earlier.
What I want to achieve is to perform action like:
extra_hosts:
- "localhost:<get MY_MACHINE_IP from env if it exists, if not set MY_MACHINE_IP env variable with value <docker-machine-ip>>"
In other words: I want to define it in extra_hosts section, if MY_MACHINE_IP is already specified, get it, if not - set this env. variable with value = my docker machine ip.
Is it possible?
Yes in according to docker documentation
docker-compose run SERVICE env
So i think the variables are not global as you may think. You have to pass them as parameters.
Read this.
You can use the package ruamel.dcw for that (dcw for Docker Compose Wrapper, disclaimer: I am the author of that package). It allows you to create a section with key user-data in your docker-compose.yaml file, which is stripped out before handing the file to the normal docker-compose. That section can look like:
user-data:
author: Your Name <your-name#youremail.com>
description: container for postfix/submission
env-defaults:
PORT: 587 # override during development
NAME: submission
DOCKER_BASE: /data0/DATA
and then you can use {PORT}, {NAME} and {DOCKER_BASE} in the rest of the file, with the option of overriding these default values with environment variables.
The utility also write out a file .dcw_env_vars.inc which you can copy into your container and source to get the appropriate values into scripts you RUN from within the Dockerfile