I have a docker-compose.yml file like this:
version : '2'
services :
s1 :
build : .
environment :
HELLO : world
And a Dockerfile like this:
FROM ubuntu
RUN /bin/bash -c 'echo "$HELLO" > /txt'
How can I end up with an image with a txt file holding the text world in it?
Right now when I test the given example, the file is empty!
[UPDATE]
If I put the environment variable in the Dockerfile it works just fine, Which makes me think it's a docker-compose issue!
FROM ubuntu
ENV HELLO=world
RUN /bin/bash -c 'echo "$HELLO" > /txt'
Environments defined in the docker-compose.yml will not be accessible at build time.
To achieve that, you will have to use the args option of the build field (only supported in version 2 file format of docker-compose). It will let you add build arguments.
https://docs.docker.com/compose/compose-file/#args
Find below how to use it:
docker-compose.yml
services :
s1 :
build :
context: .
args:
HELLO: world
Also, note that you will have to define in your Dockerfile an ARG tag with the same key name.
FROM ubuntu
ARG HELLO
RUN echo "$HELLO" > /txt
Related
I am trying to dynamically generate a json file during my build step of my container via Dockerfile like this:
FROM alpine:3.9
// ... snipped
SHELL ["/bin/bash", "-c"]
RUN echo $'{\n\
"type": "some_type",\n\
"project_id": "$PROJECT_ID",\n\
"private_key_id": "$PRIVATE_KEY_ID"\n\
}' > /etc/my_creds.json
EXPOSE 80
This works fine, so when I shell into my container and cat /etc/my-creds.json file, it appears the environment variables $PROJECT_ID and $PRIVATE_KEY_ID were written literally, they did not get replaced with the environment variable values that were present.
I.e. the file looks like this in the container:
Any ideas what I might be doing wrong here?
The variables will be expanded by the shell if you use double-quotes and escaping, for example:
FROM ubuntu
ENV FOO=bar
RUN echo "{\"foo\": \"$FOO\"}" >foo.json
$ docker build -t foo .
$ docker run foo cat foo.json
{"foo": "bar"}
I have a Dockerfile in which files in a directory are downloaded:
RUN wget https://www.classe.cornell.edu/~cesrulib/downloads/tarballs/ -r -l1 --no-parent -A tgz \
--cut=99 -nH -nv --show-progress --progress=bar:force:noscroll
I know that there is exactly one file here of the form "bmad_dist_YYYY_MMDD.tgz" where "YYYY_MMDD" is a date. For example, the file might be named "bmad_dist_2020_0707.tgz". I want to set a bash variable to the file name without the ".tgz" extension. If this was outside of docker I could use:
FULLNAME=$(ls -1 bmad_dist_*.tgz)
BMADDIST="${FULLNAME%.*}"
So I tried in the dockerfile:
ENV FULLNAME $(ls -1 bmad_dist_*.tgz)
ENV BMADDIST "${FULLNAME%.*}"
But this does not work. Is it possible to do what I want?
Shell expansion does not happen in Dockerfile ENV. Then workaround that you can try is to pass the name during Docker build.
Grab the filename during build name and discard the file or you can try --spider for wget to just get the filename.
ARG FULLNAME
ENV FULLNAME=${FULLNAME}
Then pass the full name dynamically during build time.
For example
docker build --build-args FULLNAME=$(wget -nv https://upload.wikimedia.org/wikipedia/commons/5/54/Golden_Gate_Bridge_0002.jpg 2>&1 |cut -d\" -f2) -t my_image .
The ENV ... ... syntax is mainly for plaintext content, docker build arguments, or other environment variables. It does not support a subshell like your example.
It is also not possible to use RUN export ... and have that variable defined in downstream image layers.
The best route may be to write the name to a file in the filesystem and read from that file instead of an environment variable. Or, if an environment variable is crucial, you could set an environment variable from the contents of that file in an ENTRYPOINT script.
I have a line in my Dockerfile:
&& echo "xdebug.remote_host=192.168.0.216" >> /usr/local/etc/php/conf.d/xdebug.ini`
I want to make the IP dynamic. How would I get the host IP in there?
You need to use build-time variables (–build-arg).
This flag allows you to pass the build-time variables that are accessed like regular environment variables in the RUN instruction of the Dockerfile.
So, Dockerfile is modified to:
ARG IP_ADDRESS
RUN ... && echo "xdebug.remote_host=$IP_ADDRESS" >> /usr/local/etc/php/conf.d/xdebug.ini`
And you just need to define build-time variable IP_ADDRESS during image building:
$ docker build --build-arg IP_ADDRESS=<IP_ADDRESS> .
If you use docker-compose:
1. Create file .env with the following content:
IP_ADDRESS="<IP_ADDRESS>"
You can make it every time like (example is for a linux machine):
IP_ADDRESS=$(ip a | grep <interface> | grep inet | awk '{print $2}' | awk -F'/' '{print $1}')
echo "IP_ADDRESS=$IP_ADDRESS" > .env
2. Use the following docker-compose.yaml to build your image:
version: '3'
services:
myservice:
build:
context: .
args:
IP_ADDRESS: ${IP_ADDRESS}
3. Build the above image:
docker-compose build
There's no simple in built way to get the Docker host IP (unless you are using Docker for Mac)
Entrypoint
It's best not to set a Docker host IP at build time, otherwise the image will be tied to the host it was built on and won't work anywhere else.
An ENTRYPOINT can be used to do the config setup based on an environment variable and then pass through all commands to the container:
#!/bin/sh
if [ -n "$IP_ADDRESS" ]; then
echo "xdebug.remote_host=$IP_ADDRESS" >> /usr/local/etc/php/conf.d/xdebug.ini
else
echo "No environment variable IP_ADDRESS set for xdebug"
fi
exec "$#"
Then run with:
docker run -e IP_ADDRESS=192.168.51.5 me/app-debug
Docker for Mac
On Docker for Mac 17.12+ you can use the host name docker.for.mac.host.internal
Xdebug
Another option is setting xdebug.remote_connect_back = 1 so you don't need a specific remote_host for xdebug.
Build
Nicolay's answer covers the build time setup.
I am using Docker 17.04.0-ce, build 4845c56 with docker-compose 1.12.0, build b31ff33 on Ubuntu 16.04.2 LTS. I simply want to pass an environment variable and display it from my script running in a container. I am doing this according to the documentation https://docs.docker.com/compose/compose-file/#environment . The problem is that the variable is not passed to the container.
My docker-compose.yml file:
env-file-test:
build: .
dockerfile: Dockerfile
environment:
- DEMO_VAR
My Dockerfile:
FROM alpine
COPY docker-start.sh /
CMD ["/docker-start.sh"]
And the docker-start.sh file:
#!/bin/sh
echo "DEMO_VAR Var Passed in: $DEMO_VAR"
I try to set the variable in my current terminal session and pass it to the container:
$ export DEMO_VAR=aabbdd
$ echo $DEMO_VAR
aabbdd
$ sudo docker-compose up
Starting envfiletest_env-file-test_1
Attaching to envfiletest_env-file-test_1
env-file-test_1 | DEMO_VAR Var Passed in:
envfiletest_env-file-test_1 exited with code 0
So you can see that the variable DEMO_VAR is empty!
I also tried using variables in docker-compose.yml like this: DEMO_VAR=${DEMO_VAR} but then when I run sudo docker-compose up, I get a warning: "WARNING: The DEMO_VAR variable is not set. Defaulting to a blank string.".
What am I doing wrong? What should I do to pass the variable to the container?
I found a solution. Answering my own question...
The problem was with the sudo command. It turned out that it does not pass environment variables by default. There are some possible solutions:
Use sudo -E. Demo:
$ export DEMO_VAR=aabbdd
$ echo $DEMO_VAR
aabbdd
$ sudo -E docker-compose up
env-file-test_1 | DEMO_VAR Var Passed in: aabbdd
Use sudo VAR=value:
sudo DEMO_VAR=$DEMO_VAR docker-compose up
Add environment variables to the sudoers file (https://stackoverflow.com/a/8636711)
Use docker without sudo (https://askubuntu.com/questions/477551/how-can-i-use-docker-without-sudo)
you should use ENV in your Dockerfile, and avoid export.
See the doc
https://docs.docker.com/engine/reference/builder/#env
I'm building a Docker image using multiple build args, and was wondering if it was possible to pass them to docker build as a file, in the same way --env-file can be pased to docker run. The env file will be parsed by docker run automatically and the variables made available in the container.
Is it possible to specify a file of build arguments in the same way?
There's no such an option, at least for now. But if you have too many build args and want to save it in a file, you can archive it as follows:
Save the following shell to buildargs.sh, make it executable and put it in your PATH:
#!/bin/bash
awk '{ sub ("\\\\$", " "); printf " --build-arg %s", $0 } END { print "" }' $#
Build your image with argfile like:
docker build $(buildargs.sh argfile) -t your_image .
This code is safe for build-arg's that contain spaces and special characters:
for arg in buildarg1 buildarg2 ; do opts+=(--build-arg "$arg") ; done
...
docker run ... "${opts[#]}"
Just substitute buildarg1 and so on with your build-arg's escaped.
Using linux you can create a file (example: arg_file) with the variables declared:
ARG_VAL_1=Hello
ARG_VAL_2=World
Execute the source command on that file:
source arg_file
Then build a docker image using that variables run this command:
docker build \
--build-arg "ARG_VAL_1=$ARG_VAL_1" \
--build-arg "ARG_VAL_2=$ARG_VAL_2" .