Running the command docker-compose run -e TYPE=result mongo_db_backup should give me the value of the given TYPE variable:
mongo_db_backup:
image: 'mongo:3.4'
volumes:
- '/backup:/backup'
command: sh -c '$$(echo $TYPE)'
But instead I get the error The TYPE variable is not set. Defaulting to a blank string. What am I doing wrong
It happens that Compose expands $TYPE before it gets to the inside of the container. Compose looks for the $TYPE environment variable in the shell or host environment and substitutes its value in.
This will work with the following terminal command:
docker-compose.yml
command: sh -c 'echo $TYPE'
terminal command
TYPE='hello world' docker-compose run web
When there is no $TYPE environment variable in the host machine, Compose sets the value of $TYPE to an empty string and outputs a warning.
Compose needs to be informed not to expand $TYPE since we want it expanded inside of the shell running in the container.
For this use
docker-compose.yml
command: sh -c "echo $$TYPE"
Prepending a dollar symbol to $TYPE escapes it.
Reference:
Variable Substitution in Compose
Related
I have a docker container writing logfiles to a name volume.
From the host I want to analyce the logfiles and search for given log messages. But when I access the folder which 'docker inspect VOLUMNAME' gives, I get strange behavior, which I do not understand.
e.g. following command does give empty lines as output:
user#docker-host-01:~/docker-server-env/otaya-designdb$ sudo bash -c "for logfile in /var/lib/docker/volumes/design-db-logs/_data/*/*; do echo ${logfile}; done"
user#docker-host-01:~/docker-server-env/otaya-designdb$
What could be the reason?
Your local shell is expanding the variable expansion inside the double quotes before the loop happens. Change the double quotes to single quotes.
That is, when you run
sudo bash -c "for ... ; do echo ${logfile}; done"
first your local shell replaces the variable reference with whatever your local environment has set for $logfile, probably nothing
sudo bash -c 'for ...; do echo ; done'
and then it runs that command. If you change this to single quotes initially
sudo bash -c 'for ... ; do echo ${logfile}; done'
it will avoid this expansion.
You can see this just by putting the word echo at the front of the command: the shell will do its expansion, and then echo will print out the command that would have run.
I need to set some environment variable for all users and processes inside docker container. It should be set at container start, not in Dockerfile, because it depends on running environment.
So the simple Dockerfile
FROM ubuntu
RUN echo 'export TEST=test' >> '/root/.bashrc'
works well for interactive sessions
docker run -ti test bash
then
env
and there is TEST=test
but when docker run -ti test env there is no TEST
I was trying
RUN echo 'export TEST=test' >> '/etc/environment'
RUN echo 'TEST="test"' >> '/etc/environment'
RUN echo 'export TEST=test' >> /etc/profile.d/1.sh
ENTRYPOINT export TEST=test
Nothing helps.
Why I need this. I have http_proxy variable inside container automatically set by docker, I need to set another variables, based on it, i.e. JAVA_OPT, do it system wide, for all users and processes, and in running environment, not at build time.
I would create a script which would be an entrypoint:
#!/bin/bash
# if env variable is not set, set it
if [ -z $VAR ];
then
# env variable is not set
export VAR=$(a command that gives the var value);
fi
# pass the arguments received by the entrypoint.sh
# to /bin/bash with command (-c) option
/bin/bash -c $#
And in Dockerfile I would set the entrypoint:
ENTRYPOINT entrypoint.sh
Now every time I run docker run -it <image> <any command> it uses my script as entrypoint so will always run it before the command then pass the arguments to the right place which is /bin/bash.
Improvements
The above script is enough to work if you are always using the entrypoint with arguments, otherwise your $# variable will be empty and will give you an error /bin/bash: -c: option requires an argument. A easy fix is an if statement:
if [ ! -z $# ];
then
/bin/bash -c $#;
fi
Setting the parameter in ENTRYPOINT would solve this issue.
In docker file pass parameter in ENTRYPOINT
I'm trying to use environment variable substitution inside a Dockerfile paired with environment variable replacement from docker but it looks like the variable replacement takes place after the substitution.
The following Dockefile:
FROM alpine:3.7
ENV name="World"
ENV message="Hello, ${name}"
ENTRYPOINT ["env"]
With the Docker run command:
$ docker run -it --rm -e "name=Marvin" envtest/helloworld
Prints the following environment variables:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=49d702faf257
TERM=xterm
name=Marvin
message=Hello, World
HOME=/root
You can see that even though I replaced the name variable, the message variable is still substituted with the original value from name.
Does anyone know how to do environment variable substitution with dockers environment variable replacement?
EDIT: I found a comment in the Docker forum stating that environment variables are interpreted at build time but can be replaced one by one at runtime. So the documentation is a little misleading.
When you build the Dockerfile, you get an image. And the image has no knowledege about what was written in the Dockerfile.
That means, the docker image has no knowledge weather ENV message="Hello, ${name}" or ENV message="Hello, world" was written in the Dockerfile.
It just has it's environment variable as it is, i.e., name="World", message="Hello, world"
So, when you start your image using $ docker run -it --rm -e "name=Marvin" envtest/helloworld, it overrides the variable name, i.e. now name is Marvin.
But message remains "Hello, world".
Because, inside the image, message is "Hello, world" not "Hello, ${name}"
I am trying to set an arg for docker compose using an output of linux command as my example:
args:
ID_GITLAB: $(id -u $USER)
but when I run my compose I get following error:
ERROR: Invalid interpolation format for "build" option in service "gpc-fontes-ci": "$(id -u $USER)"
Just do
USER_ID=$(id -u $USER) docker-compose
With the Compose file using a regular variable
args:
ID_GITLAB: $USER_ID
You need to escape $ character with another $, e.g.
args:
ID_GITLAB: $$(id -u $USER)
It's the same rule for command.
See: Variable substitution.
You can use a $$ (double-dollar sign) when your configuration needs a literal dollar sign. This also prevents Compose from interpolating a value, so a $$ allows you to refer to environment variables that you don’t want processed by Compose.
For example:
web:
build: .
command: "$$VAR_NOT_INTERPOLATED_BY_COMPOSE"
I am using Docker 17.04.0-ce, build 4845c56 with docker-compose 1.12.0, build b31ff33 on Ubuntu 16.04.2 LTS. I simply want to pass an environment variable and display it from my script running in a container. I am doing this according to the documentation https://docs.docker.com/compose/compose-file/#environment . The problem is that the variable is not passed to the container.
My docker-compose.yml file:
env-file-test:
build: .
dockerfile: Dockerfile
environment:
- DEMO_VAR
My Dockerfile:
FROM alpine
COPY docker-start.sh /
CMD ["/docker-start.sh"]
And the docker-start.sh file:
#!/bin/sh
echo "DEMO_VAR Var Passed in: $DEMO_VAR"
I try to set the variable in my current terminal session and pass it to the container:
$ export DEMO_VAR=aabbdd
$ echo $DEMO_VAR
aabbdd
$ sudo docker-compose up
Starting envfiletest_env-file-test_1
Attaching to envfiletest_env-file-test_1
env-file-test_1 | DEMO_VAR Var Passed in:
envfiletest_env-file-test_1 exited with code 0
So you can see that the variable DEMO_VAR is empty!
I also tried using variables in docker-compose.yml like this: DEMO_VAR=${DEMO_VAR} but then when I run sudo docker-compose up, I get a warning: "WARNING: The DEMO_VAR variable is not set. Defaulting to a blank string.".
What am I doing wrong? What should I do to pass the variable to the container?
I found a solution. Answering my own question...
The problem was with the sudo command. It turned out that it does not pass environment variables by default. There are some possible solutions:
Use sudo -E. Demo:
$ export DEMO_VAR=aabbdd
$ echo $DEMO_VAR
aabbdd
$ sudo -E docker-compose up
env-file-test_1 | DEMO_VAR Var Passed in: aabbdd
Use sudo VAR=value:
sudo DEMO_VAR=$DEMO_VAR docker-compose up
Add environment variables to the sudoers file (https://stackoverflow.com/a/8636711)
Use docker without sudo (https://askubuntu.com/questions/477551/how-can-i-use-docker-without-sudo)
you should use ENV in your Dockerfile, and avoid export.
See the doc
https://docs.docker.com/engine/reference/builder/#env