I have the following line in my Dockerfile which is supposed to capture the display number of the host:
RUN DISPLAY_NUMBER="$(echo $DISPLAY | cut -d. -f1 | cut -d: -f2)" && echo $DISPLAY_NUMBER
When I tried to build the Dockerfile, the DISPLAY_NUMBER is empty. But however when I run the same command directly in the terminal I get the see the result. Is there anything that I'm doing wrong here?
Commands specified with RUN are executed when the image is built. There is no display during build hence the output is empty.
You can exchange RUN with ENTRYPOINT then the command is executed when the docker starts.
But how to forward the hosts display to the container is another matter entirely.
Host environment variables cannot be passed during build, only at run-time.
Only build args can be specified by:
first "declaring the arg"
ARG DISPLAY_NUMBER
and then running
docker build . --no-cache -t disp --build-arg DISPLAY_NUMBER=$DISPLAY_NUMBER
You can work around this issue using the envsubst trick
RUN echo $DISPLAY_NUMBER
And on the command line:
envsubst < Dockerfile | docker build . -f -
Which will rewrite the Dockerfile in memory and pass it to Docker with the environment variable changed.
Edit: Note that this solution is pretty useless though, because you probably
want to do this during run-time anyways, because this value should depend on not on where the image is built, but rather where it is run.
I would personally move that logic into your ENTRYPOINT or CMD script.
Related
Snippet from my Dockerfile:
FROM node:12.18.0
RUN echo "hello world"
RUN psql --version
When I run docker build . I don't see any output from these two commands even if they are not cached. The documentation says that docker build is verbose by default. Why am I not seeing the output from commands? I used to see them before.
The output while building:
=> [7/18] RUN echo "hello world" 0.9s
The output I am seeing after building finishes:
=> CACHED [6/18] RUN apt-get install postgresql -y 0.0s
=> [7/18] RUN echo "hello world" 6.4s
=> [8/18] RUN psql --version 17.1s
The Dockerfile is created from node:12.18.0 which is based on Debian 9.
Docker version 19.03.13, build 4484c46d9d.
The output you are showing is from buildkit, which is a replacement for the classic build engine that docker ships with. You can adjust output from this with the --progress option:
--progress string Set type of progress output (auto, plain, tty). Use plain to show container output
(default "auto")
Adding --progress=plain will show the output of the run commands that were not loaded from the cache. This can also be done by setting the BUILDKIT_PROGRESS variable:
export BUILDKIT_PROGRESS=plain
If you are debugging a build, and the steps have already been cached, add --no-cache to your build to rerun the steps and redisplay the output:
docker build --progress=plain --no-cache ...
If you don't want to use buildkit, you can revert to the older build engine by exporting DOCKER_BUILDKIT=0 in your shell, e.g.:
DOCKER_BUILDKIT=0 docker build ...
or
export DOCKER_BUILDKIT=0
docker build ...
Just use this flag --progress=plain after build.
For example:
docker-compose build --progress=plain <container_name>
OR
docker build --progress=plain .
If you don't want to use this flag every time, then permanently tell docker to use this flag by doing:
export BUILDKIT_PROGRESS=plain
Here is the official documentation when you type docker build --help.
--progress string Set type of progress output (auto, plain, tty). Use plain to show container output (default "auto")
In Docker 20.10 i had to use the --no-cache flag, too. Otherwise cached output is not shown.
docker build --progress=plain --no-cache .
As an alternative to specifying the --progress=plain option, you can also permanently disable the "pretty" output by setting this env variable in your shell config:
export BUILDKIT_PROGRESS=plain
Do 2 things
Instead of docker build . use this
docker build . --progress=plain
Add random junk to your RUN command every build (this tricks docker into thinking it hasn't seen the command before, so it doesn't use the cached version)
Example. If your command is RUN ls use this instead RUN ls && echo sdfjskdflsjdf (change the sdfjskdflsjdf to something else each time you build).
Why this works
I tried other answers and they all presented problems and imperfections. It's highly frustrating that Docker doesn't have some simple functionality like --verbose=true.
Here's what I ended up using (it's ludicrous but it works).
Suppose you want to see the output of ls command, this won't work docker build .
RUN ls
but this will print the output docker build --progress=plain:
RUN ls
now try again, it won't print! - that's because docker caches the unchanged layer, so the trick is to alter the command each time by adding some nonsense to it && echo sdfljsdfljksdfljk, and changing the nonsense each time docker build --progress=plain:
# This prints
RUN ls && echo sdfljsdfljksdfljk
# Next time you run it use a different token
RUN ls && echo sdlfkjsldfkjlskj
So each and every time, I mash the keyboard and come up with a new token. Stupifying. (note that I tried something like && openssl rand -base64 12 to generate a random string, but docker realises the code hasn't changed that doesn't work).
This solution is highly inferior to genuine docker support for printing output to console.
If your error looks something like this:
#7 0.584 /bin/sh: 1: /install.sh: not found
it's telling you the error is in line number 1. you are running into windows line endings
I was using VS code and I solved it pretty easily by converting the file from CRLF to LF using VS code.
just click on the CRLF button in the bottom right corner of the editor and save the file.
everything should work fine when you build the image now.
Is there a way we can pass a variable lets say in this example I want to pass a list of animals into an entrypoint.sh file using ENV animals="turtle, monkey, goose"
But I want to be able to pass different animals when running the container for example docker run -t image animals="mouse,rat,kangaroo"
How do you go about passing arguments when running the docker run command?
The goal is to take that variable when using the docker run command and insert them into that entrypoint.sh file
Right now i hard code that in my Dockerfile. But i want to be able to do this when running the docker run command so I dont always have to change the Dockerfile.
FROM anapsix/alpine-java:8u121b13_jdk
ENV FILE_NAME="file_to_run.zip"
ENV animals="turtle, monkey, goose"
ADD ${FILE_NAME} .
RUN echo "${FILENAME} ${animals}" > ./entrypoint.sh
CMD [ "/bin/ash", "./entrypoint.sh" ]
It looks like you might be confusing the image build with the container run. If the difference between the two isn't immediately clear, I'd recommend reviewing some other questions and docs like:
In Docker, what's the difference between a container and an image?
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
RUN echo "${FILENAME} ${animals}" > ./entrypoint.sh
With the above, the variables will be expanded during the image build. The entrypoint.sh will not contain ${FILENAME} ${animals}. Instead, it will contain
file_to_run.zip turtle, monkey, goose
After the build, the docker run command will create a container from that image and run the above script with the environment variables defined but never used since the script already has the variables expanded. To prevent the variable expansion, you need to escape the $ or use single quotes to prevent the expansion, e.g.
RUN echo "\${FILENAME} \${animals}" > ./entrypoint.sh
or
RUN echo '${FILENAME} ${animals}' > ./entrypoint.sh
I would also recommend being explicit with a #!/bin/ash at the top of this script. Then when you run the script, do not override the command with parameters after the image name. Instead set the environment variables with the appropriate flag to run:
docker run -it -e animals="mouse,rat,kangaroo" image
Simplest way, forward individual variables:
docker run ... --env animals="turtle, monkey, goose" --env FILE_NAME="file_to_run.zip"
Forward several variables using file:
Or if you need to grab all your environment variables from outside, you can do something like this first:
printenv | grep -E 'animals|FILE_NAME' > my-env
The grep is because Docker doesn't like some variables, e.g. with spaces in them, which you might possibly have in your real environment.
Then use that file in your Docker command:
docker run ... --env-file ./my-env
The latter is also useful if you want to avoid sending environment variables to logs (like for sensitive variables). I use this approach in a CI/CD pipeline that runs some scripts.
Using variables inside Docker:
With either approach, the environment variables actually become available to scripts running inside the container to use.
#BMitch's answer has more complete details about how to achieve this in your case, where you have related logic in both build and execution.
Reference
See docs here.
How can I set an ENV var in my Dockerfile based on the hostname? I tried this:
RUN if [ hostname = "foo" ]; then ENV BAR "BAZ"; else ENV BAR "BIFF"; fi
But that failed with
ENV: not found
RUN if [ hostname = "foo" ]; then ENV BAR "BAZ"; else ENV BAR "BIFF"; fi
You can't nest docker build instructions, everything after the RUN instruction gets executed in the image context, docker build commands don't
exist there. So that explains the error you are seeing.
Even you if you translated that to proper shell code BAR would only be active for that single RUN instruction during the build.
Either orchestrate on the host and pass BAR via run -e to your container or add a startup script to the image that sets BAR as needed on container start:
FROM foo
COPY my-start.sh /
CMD ["/my-start.sh"]
First of all, you can't embed Docker build command into shell of RUN, the shell will run inside the intermediate container during build process, and Docker build commands will be ran by Docker build engine, they're different things. And besides, Docker does not support conditional commands like IF or something like that. Docker is about immutable infrastructure, Dockerfile is the definition of your image and it's supposed to be able to generate the same image no matter what build context it is in. And from the delivery perspective of view, the image is your deliverable build artifacts, if you want to deliver different stuff, then use different Dockerfile to build different images, otherwise if the differences is about the runtime, I think you could really consider postpone the env definition to the runtime with -e option of docker run.
The reason why your build is failing has been explained by #shizhz & #Erik Dannenberk.
However, if you do really need that behavior I suggest you make a little script to do that:
export BAR=`[[ hostname = "foo" ]] && echo "BAZ" || echo "BIFF"`
docker build -t hello/hi - <<EOF
FROM alpine
ENV BAR $BAR
CMD echo $BAR
EOF
I need to fill a variable in dockerfile with the result of a command
Like in bash var=$(date)
EDIT 1
date is a example.
in my case i use FROM phusion/baseimage:0.9.17 so i want at each building use the last version so i use this
curl -v --silent api.github.com/repos/phusion/baseimage-docker/tags 2>&1 | grep -oh 'rel-.*",' | head -1 | sed 's/",//' | sed 's/rel-//' ==> 0.9.17.
but i don't know how i parse it in var with dockerfile for this result
ENV verbaseimage=curl...
FROM phusion/baseimage:$verbaseimage
RESULT
In my use case
FROM phusion/baseimage:latest
But the question remains unresolved for other case
I had same issue and found way to set environment variable as result of function by using RUN command in dockerfile.
For example i need to set SECRET_KEY_BASE for Rails app just once without changing as would when i run:
docker run -e SECRET_KEY_BASE="$(openssl rand -hex 64)"
Instead it i write to Dockerfile string like:
RUN bash -l -c 'echo export SECRET_KEY_BASE="$(openssl rand -hex 64)" >> /etc/bash.bashrc'
and my env variable available from root, even after bash login.
or may be
RUN /bin/bash -l -c 'echo export SECRET_KEY_BASE="$(openssl rand -hex 64)" > /etc/profile.d/docker_init.sh'
then it variable available in CMD and ENTRYPOINT commands
Docker cache it as layer and change only if you change some strings before it.
You also can try different ways to set environment variable.
The old workaround is mentioned here (issue 2637: Feature request: expand Dockerfile ENV $VARIABLES in WORKDIR):
One work around that I've used, is to have a file in my context called "build-env". What I do is source it and run my desired command in the same RUN step. So for example:
build-env:
VERSION=stable
Dockerfile:
FROM radial/axle-base:latest
ADD build-env /build-env
RUN source build-env && mkdir /$VERSION
RUN ls /
But for date, that might not be as precise as you want.
Other workarounds are in issue 2022 "Dockerfile with variable interpolation".
In docker 1.9 (end of October 2015), you will have "support for build-time environment variables to the 'build' API (PR 9176)" and "Support for passing build-time variables in build context (PR 15182)".
docker build --build-arg=[]: Set build-time variables
You can use ENV instructions in a Dockerfile to define variable values. These values persist in the built image. However, often persistence is not what you want. Users want to specify variables differently depending on which host they build an image on.
A good example is http_proxy or source versions for pulling intermediate files. The ARG instruction lets Dockerfile authors define values that users can set at build-time using the ---build-arg flag:
$ docker build --build-arg HTTP_PROXY=http://10.20.30.2:1234 .
This flag allows you to pass the build-time variables that are accessed like regular environment variables in the RUN instruction of the Dockerfile.
Also, these values don't persist in the intermediate or final images like ENV values do.
so I want at each building use the last version so I use this
curl -v --silent api.github.com/repos/phusion/baseimage-docker/tags 2>&1 | grep -oh 'rel-.*",' | head -1 | sed 's/",//' | sed 's/rel-//' ==> 0.9.17.
If you want to use the last version of that image, all you need to do is use the tag 'latest' with the FROM directive:
FROM phusion/baseimage:latest
See also "The misunderstood Docker tag: latest": it doesn't always reference the actual latest build, but in this instance, it should work.
If you really want to use the curl|parse option, use it to generate a Dockerfile with the right value (as in a template processed to generate the right file).
Don't try to use it directly in the Dockerfile.
I wanted to set an ENV or LABEL variable from a computation in the Dockerfile, e.g. to make some computed installation options visible in docker inspect.
There does not seem to be any way to do that, and this issue suggests that it's a security design choice.
A Dockerfile can set an ENV variable to $X, ${X:-default}, or ${X:+substitute} where that $X must be another ENV or ARG variable.
A single RUN command can set and use shell variables, but that goes away at the end of the RUN command when that container layer shuts down.
A RUN command can write computed data into files, but the Dockerfile still can't get that data into an ENV or LABEL even if the file is ~/.bashrc. (File contents can, of course, be used by code running in the Container.)
The build can at least RUN echo $X to record choices to the build log -- unless that step comes from the build cache, in which case the RUN step doesn't run.
Please do correct me if there's a way out.
Partially connected to question. If one wants to use the result of some command later on it is possible within single RUN statement as follows:
RUN CUR_DIR=`pwd` && \
echo $CUR_DIR
I'm trying to write a little docker file that sets a User and just echos the current user as a little example to prove to myself it is working. I've tried a number of variants and couldn't find much help in the documentation.
FROM ubuntu
USER daemon
# ENTRYPOINT ["echo", "$USER"]
# just gives "$USER"
# ENTRYPOINT ["echo", "-e", "${USER}"]
# just gives "$USER"
# ENTRYPOINT echo $USER
# gives empty string
# ENTRYPOINT ["/bin/echo", "$USER"]
# just gives "$USER"
I'm running docker build . on the dockerfile and then running docker run <image-id> and getting the results
Expected result is daemon, or without the USER daemon line, I expect root. Probably a really simple answer.
This is the expected behavior, as weird as it seems!
When ENTRYPOINT is a list (as in ENTRYPOINT ["echo", "$USER"]), it is used as-is, without further parsing or interpretation. So $USER remains $USER, because there is no shell involved in the process to replace it with the value of the USER environment variable.
Now, when ENTRYPOINT is a string (as in ENTRYPOINT echo $USER), what is actually executed is sh -c "echo $USER", and $USER is replaced with the value of the environment variable (as you would expect).
However, the environment variable USER is not set by default. It is set by the login process; and when you just run sh -c ... the login process is not involved.
Compare the environment when running docker run -t -i ubuntu bash and docker run -t -i ubuntu login -f root. In the former case, you will get a very basic environment; in the latter case, you will get the complete environment that you are used to (including USERvariable).
Couldn't you set, in the Dockerfile, the ENV command to a default value, and then, when run-ning a container, use the -e, --env dictionary to override what would be interpreted by the:
ENTRYPOINT echo $SOMEENVVAR
form of ENTRYPOINT?
I think there´s a series of issues here.
when I
docker run -i -t ubuntu /bin/bash
echo $USER
set
I don´t see $USER set at all - whoami does report daemon though.
additionally, I have the suspicion (but have not looked at the code yet) that ENV vars in the Dockerfile are escaped, to avoid their use (many people assume that they can export host variables to the built container, but this is something that the docker guys would like to avoid)