Is there a way to set a variable from a file in the docker compose?
The following does not work in compose (however it does if you attach to the container):
version: '2'
services:
foo:
image: busybox
command:
- /bin/sh
- -c
- |
echo "hello" > id #this file will be supplied by a volume-from
ls
cat id
TEST=$(cat id) #This line does not work
echo TEST
The error which is produced is:
ERROR: Invalid interpolation format for "command" option in service "foo":
you have to change a bit your config file.
version: '2'
services:
foo:
image: busybox
command:
- /bin/sh
- -c
- |
echo "hello" > id
ls
cat id
TEST=$$(cat id)
echo TEST
I checked against Docker Version Docker version 18.06.0-ce and Docker compose version docker-compose version 1.20.0-rc2,
https://github.com/docker/docker.github.io/blob/master/compose/compose-file/index.md#variable-substitution
You can use a $$ (double-dollar sign) when your configuration needs a
literal dollar sign. This also prevents Compose from interpolating a
value, so a $$ allows you to refer to environment variables that you
don’t want processed by Compose.
Related
Question :
Can I use a "result of a bash command" as env variables in docker-compose.yml file ?
Can I "execute a command" and put the result in env variables during the "docker-compose up" ?
The use Case :
I work with docker & VScode in windows WSL2, and I want send "WSL2 internal IP" to my container to use Xdebug (which need a "remote_host").
The problem are the "WSL2 internal IP" can change on each reboot of the PC.
The commande used :
ip addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}'
Where I am
I have tested to put the command in .env file
MY_WSL_IP=$(ip addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}')
and the docker-compose.yml file
version: "3"
services:
lamp_test:
container_name: lamp_test
image: eurelis/aws-ami-php7:php73
volumes:
[...]
ports:
[...]
env_file: ./drupal.env
links:
[...]
environment:
- WSL_HOST_IP=${MY_WSL_IP}
[...]
In this case, I get the whole command in the container.
Notes
When in the terminal il put play the two command in the terminal, that's work
export MY_WSL_IP=$(ip addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}')
docker-compose -f etc/docker/docker-compose.yml up -d
But there is not the "use case" because variable are created "out side" the docker-compose configuration.
When using the VScode's docker plugin, I can't inject this "result"
I don't have found a proper way to inject the value in the container, nor a explicit "that's not possible".
If someone has a tips
As per docker docs, environment variables in .env file expected to be in in key-val format as VAR=VAL which works fine for sample like foo=bar but no mention of unavoidable special characters e.g. '=', which may confuse for key-val separator OR space both part of valid db connection string as in:
secrets.env file:
connectionString=Data Source=some-server;Initial Catalog=db;User ID=uid;Password=secretpassword
which is referred in docker-compose.debug.yaml file content as:
services:
some-service:
container_name: "service-name"
env_file:
- secrets.env
ports:
- "80:80"
Which is further used to transform into docker-compose.yaml as shown complete flow below:
So the question is - how do you include connection string which has = and Spaces as part of value ?
Need - We have few micro-services within VS solution and looking forward to avoid repetition of same connection strings otherwise needs in service spec of `docker-compose.yaml'
Tried including values in single/double quote but post transformation whatever after = is treated as value including quotes just similar like kubernets yaml file convention
I ran a test without any issues:
$ cat .env
ENV=default
USER_NAME=test2
SPECIAL=field=with=equals;and;semi-colons
$ cat docker-compose.env.yml
version: '2'
services:
test:
image: busybox
command: env
environment:
- SPECIAL
$ docker-compose -f docker-compose.env.yml up
Creating network "test_default" with the default driver
Creating test_test_1_55eac1c3767c ... done
Attaching to test_test_1_d7787ac5bfc0
test_1_d7787ac5bfc0 | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
test_1_d7787ac5bfc0 | HOSTNAME=d249a16a8e09
test_1_d7787ac5bfc0 | SPECIAL=field=with=equals;and;semi-colons
test_1_d7787ac5bfc0 | HOME=/root
test_test_1_d7787ac5bfc0 exited with code 0
I want to print environment variable defined in docker-compose file.
Here is my docker-compose-1.yml:
version: '3.6'
services:
busybox:
image: busybox
command: 'echo $DEBUG'
environment:
- DEBUG='123456'
And, try to print the DEBUG environment variable in docker container:
☁ environment-variables-in-compose [master] ⚡ docker-compose -f docker-compose-1.yml up
WARNING: The DEBUG variable is not set. Defaulting to a blank string.
Creating network "environmentvariablesincompose_default" with the default driver
Creating environmentvariablesincompose_busybox_1 ... done
Attaching to environmentvariablesincompose_busybox_1
busybox_1 |
environmentvariablesincompose_busybox_1 exited with code 0
As you can see, got WARNING: The DEBUG variable is not set. Defaulting to a blank string.
Am I wrong?
update 1:
docker-compose-1.yml:
version: '3.6'
services:
busybox:
image: busybox
command: 'echo $$DEBUG'
environment:
DEBUG: 123456
I change command using double $ sign, but the result is still not what I want.
☁ environment-variables-in-compose [master] ⚡ docker-compose -f docker-compose-1.yml up
Creating network "environmentvariablesincompose_default" with the default driver
Creating environmentvariablesincompose_busybox_1 ... done
Attaching to environmentvariablesincompose_busybox_1
busybox_1 | $DEBUG
environmentvariablesincompose_busybox_1 exited with code 0
As you can see, the environment variable is printed as $DEBUG, not expected value 123456
Compose will expand variables from the environment where you are running compose. To expand the variable inside your container, escape the $ using $$:
command: '/bin/sh -c "echo $$DEBUG"'
For more details, see: https://docs.docker.com/compose/compose-file/#variable-substitution
Edit: I've also added a shell inside the container to expand the variable.
I have an environment variable defined in a file passed in via --env-file like this:
TEST_VAR=The value
Does anybody know whether this is legal? should I place " around the value for this to be interpreted as needed in docker?
Thanks
EDIT: Quotation marks will not be a good solution as it is will be part of the val see reference here.
Lets see the result running the following compose file:
version: "3"
services:
service:
image: alpine
command: env
env_file: env.conf
env.conf:
TEST_VAR1=The value
TEST_VAR2="The value2"
docker-compose up Result:
service_1 | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
service_1 | TEST_VAR2="The value2"
service_1 | TEST_VAR1=The value
service_1 | HOME=/root
Therefore, it is legal to have spaces in the env value.
You can escape the space with a \:
TEST_VAR=The\ value
Edit: This is how I pass them when starting the container (i.e. docker run -e TEST_VAR=The\ value hello-world). If you're using docker-compose or an env file, see the answer by #yamenk.
In Dockerfile use doublequotes, do not use singlequotes because they do not expand variables inside, excerp from passing buildargs/envs to dockerfile and into a python script below:
ARG HOST="welfare-dev testapi"
ENV HOST "${HOST}"
ARG SITENAME="Institusjon"
ENV SITENAME "${SITENAME}"
RUN cd ${TESTDIR}/sensiotools/sensiotools && cd test && \
./testapi-events.py --activate --sitename="${SITENAME}" --host="${HOST}" --dbcheck --debug --wait=0.5 && \
./testapi-events.py --deactivate --sitename="${SITENAME}" --host="${HOST}" --dbcheck --debug
My case with docker-compose, if that can help. I couldn't make use of the suggestions in the other answers.
For a variable in volumes, I could use the .env file:
# .env
LOCAL_DIR=/local/path
while for a variable with spaces (for https://github.com/wolfcw/libfaketime in my case) I had to use command line: FAKETIME_ARG="#2021-02-11 13:23:02" docker-compose up.
The resulting docker-compose file (note the ${} just for LOCAL_DIR) :
# docker-compose.yml
services:
myservice:
build:
context: ./path/to/dir/of/Dockerfile
args:
- FAKETIME_ARG
volumes:
- ${LOCAL_DIR}:/path/in/container
Here is my docker-compose.yml,
elasticsearch:
ports:
- 9200:9200/tcp
image: elasticsearch:2.4
volumes:
- /data/elasticsearch/usr/share/elasticsearch/data:/usr/share/elasticsearch/data
command: /bin/bash -c “echo 'http.cors.enabled: true' > /usr/share/elasticsearch/config/elasticsearch.yml"
it throws the error:
Activating (yaml: [] mapping values are not allowed in this context at line 7, column 49
Looks as if I cannot use the colon sign : in command, is this true?
The colon is how YAML introduces a dictionary. If you have it in a value, you just need to quote the value, for example like this:
image: "elasticsearch:2.4"
Or by using one of the block scalar operators, like this:
command: >
/bin/bash -c “echo 'http.cors.enabled: true' > /usr/share/elasticsearch/config/elasticsearch.yml"
For more information, take a look at the YAML page on Wikipedia. You can always use something like this online YAML parser to test out your YAML syntax.
Properly formatted, your first document should look something like:
elasticsearch:
ports:
- 9200:9200/tcp
image: "elasticsearch:2.4"
volumes:
- /data/elasticsearch/usr/share/elasticsearch/data:/usr/share/elasticsearch/data
command: >
/bin/bash -c “echo 'http.cors.enabled: true' > /usr/share/elasticsearch/config/elasticsearch.yml"
(The indentation of the list markers (-) from the key isn't strictly necessary, but I find that it helps make things easier to read)
A docker container can only run a single command. If you want to run multiple commands, put them in a shell script and copy that into the image.