I have a Dockerfile with the following line:
USER lodgepole
CMD ["tail", "-f", "~/block.txt"]
But container exits with error that the file ~/block.txt is not found
The following does work as expected:
CMD ["tail", "-f", "/home/lodgepole/block.txt"]
Is this a bug in Docker or is there a limitation related to tilde expansion that I'm not aware of?
Using Docker 20.10.9 on Linux.
Tilde expansion is done by the shell. When you use the exec form of CMD, there is no shell to do the expansion.
You can use the shell form of CMD instead, which runs the command through a shell, like this
CMD tail -f ~/block.txt
or you can use the exec form and run the shell explicitly, like this
CMD ["/bin/sh", "-c", "tail -f ~/block.txt"]
The two commands do the exact same thing. If you use the shell form, it'll run the shell in the same way that the exec form does.
More info here: https://docs.docker.com/engine/reference/builder/#cmd
Related
I am trying to write ssh keys to docker image using CMD.
I have docker file like below.
FROM public.ecr.aws/ubuntu/ubuntu:18.04_stable
CMD ["sh", "-c", "echo $PUBLIC_KEY >> ./.ssh/id_rsa.pub"]
CMD ["sh", "-c", "echo $PRIVATE_KEY >> ./.ssh/id_rsa"]
I run the container with env var like so:
docker run -it -d -e PUBLIC_KEY="key1" -e PRIVATE_KEY="key2" my-image
As result, writing both of them doesn't work. However, when I manually docker exec these 2 cmd against the running container, it will write both public key and private key to the correct location.
Can anyone explain this? How should I make the CMD work?
CMD is a way to define a default command when starting a container. There can only be one default command. In the example you have given, the second CMD will be the default command, and the first CMD will not run. The default command will run only when you do not specify a command to run on the command line, i.e. as part of the command line
docker run [OPTIONS] IMAGE[:TAG|#DIGEST] [COMMAND] [ARG...]
if you provide a COMMAND, the CMD in the dockerfile will not be run.
When you issue docker exec, you explicitly run the command line, so it will always run.
I'm using docker 3.5. In this docker version I've got an issue with nodes dependencies at start time. I tried to resolve it as it was recommended using external sh script coping into docker file. It lead to more issues. For example `script is present but execution was not detected, executed, but program was not started. My docker-compose is up, but swarm mode is failed and so on...
I think I'm not clear with Docker life cycle. Lets imagine we have Dockerfile, docker-compose.yml and docker-swarm.yml. Each of them has an CMD and ENTRYPOINT instruction.
Starting docker-compose I can detect that my service waits for required one (because of waiting script). In case I'm using swarm mode i'm getting fails and my service can't start correctly.
Can you please help with considering lifecycle?
there are instructions:
CMD (docker file)
ENTRYPOINT (docker file)
entrypoint (docker-compose)
command (docker-compose)
entrypoint (docker-swarm)
command (docker-swarm)
Is it possible to have information about execution order of specified instructions for different scenarios?
There is no "execution order" between an entrypoint and a command, regardless of whether it is defined in your image (Dockerfile) or overridden at runtime (with a compose file or cli argument). There is only one command that docker will run to start your container, and when that command exits, the container exits.
If you only define an entrypoint or a command, docker will run that. If you define both an entrypoint and a command, docker will append the command as an argument to the entrypoint. So if you have:
ENTRYPOINT ["/bin/app", "arg1"]
CMD ["script.sh", "arg2"]
Docker will run your container with the command:
/bin/app arg1 script.sh arg2
meaning that script.sh is passed as a cli argument to /bin/app.
If you use the shell/string syntax instead of the exec/json syntax, this can get a bit strange since the shell syntax wraps your command with a /bin/sh -c "$string", and more importantly, the -c arg to /bin/sh only takes a single argument. That means:
ENTRYPOINT /bin/app arg1
CMD script.sh arg2
Will run:
/bin/sh -c "/bin/app arg1" /bin/sh -c "script.sh arg2"
which will ultimately run:
/bin/app arg1
The standard workflow to call a command after running your entrypoint script, is to include the following line a the end of the entrypoint.sh script:
exec "$#"
which will run any cli args to the entrypoint script, typically the value of CMD, as the new pid 1.
I'm trying to use ENTRYPOINT and CMD such that ENTRYPOINT is the script I am calling and CMD provides the default arguments to the ENTRYPOINT command but will be overridden by any arguments given to docker run.
The part I'm struggling with is how to have environment variable expanded in my default arguments using CMD.
For example. Given this dockerfile built as tag test:
FROM busybox
ENV AVAR=hello
ENTRYPOINT ["/bin/sh", "-c", "exec echo \"$#\""]
CMD ["${AVAR}"]
I am expecting the following results:
docker run -it --rm test
> hello
docker run -it --rm test world
> world
Note: I'm just using echo here as an example. In my actual Dockerfile I'll be calling ./bin/somescript.sh which is a script to launch an application I have no control over and is what I am trying to pass arguments to.
This question is similar but is asking about expanding variables in the ENTRYPOINT, I'm trying to expand variables in CMD.
I've tried many combinations of shell/exec form for both ENTRYPOINT and CMD but I just can't seem to find the magic combination:
FROM busybox
ENV AVAR=hello
ENTRYPOINT ["/bin/sh", "-c", "exec echo \"$#\""]
CMD ${AVAR}
docker run -it --rm test
> -c ${AVAR}
Is what I'm trying to do possible?
Many more failed attempts
This is the closest I can get:
FROM busybox
ENV AVAR=hello
ENV AVAR2=world
ENTRYPOINT ["/bin/sh", "-c", "echo $#", "$#"]
CMD ["${AVAR}", "${AVAR2}"]
This works fine when I pass in an argument to the run command:
docker run -it --rm test world
> world
But it doesn't expand the default arguments when not given a command:
docker run -it --rm test
> ${AVAR} ${AVAR2}
Found the magic. I don't completely follow what's going on here but I'll try to explain it
FROM busybox
ENV AVAR=hello
ENV AVAR2=world
ENTRYPOINT ["/bin/sh", "-c", "echo $(eval echo $#)", "$#"]
CMD ["${AVAR}", "${AVAR2}"]
docker run -it --rm test
> hello world
docker run -it --rm test world
> world
My attempt at explanation (I'm really not sure if this is right):
CMD in exec form is passed as an argument to ENTRYPOINT without shell substitution. I'm taking those values and passing them as positional arguments to /bin/sh -c ... which is why I need the "extra" "$#" at the end of the ENTRYPOINT array.
Within ENTRYPOINT I need to expand $# and do parameter substitution on the result of the expansion. So in a subshell ($(...)) I call eval to do parameter substitution and then echo the result which ends up just being the contents of CMD but with variables substituted.
If I pass in an argument to docker run it simply takes place of CMD and is evaluated correctly.
https://docs.docker.com/engine/reference/builder/#cmd says that in a Dockerfile
CMD ["executable","param1","param2"]
is the 'exec form' and
CMD command param1 param2
is the 'shell form', and they are run slightly differently.
Note: Unlike the shell form, the exec form does not invoke a command shell
But when you are passing in the command with docker run, I can find any way to make it run without a command shell.
docker run container-name echo test # shell form I guess
What is the exec form?
Why guessing? Just check.
docker run ubuntu sh -c 'echo "Hello world!"'
creates a container one may discover with docker container ls -a
and inspect by id.
The inspect command reveals that no additional sh -c was added except the explicit one:
"Cmd": [
"sh",
"-c",
"echo \"Hello world!\""
],
Thus current hypothesis is that it uses exec form.
Just to mention: my tests for docker-compose run have shown the same results.
This is very poorly documented both for docker and docker-compose, that's true. The --entrypoint option is slightly luckier and is documented at dockerfile docs:
You can override the ENTRYPOINT setting using --entrypoint, but this can only set the binary to exec (no sh -c will be used).
The documentation for the run command follows the following syntax:
docker run [OPTIONS] IMAGE[:TAG|#DIGEST] [COMMAND] [ARG...]
however I've found at times that I want to pass a flag to [COMMAND].
For example, I've been working with this image, where the [COMMAND] as specified in the Dockerfile is:
CMD ["/bin/bash", "-c", "/opt/solr/bin/solr -f"]
Is there any way to tack on flags to "/opt/solr/bin/solr -f" so that it's in the form "/opt/solr/bin/solr -f [-MY FLAGS]"?
Do I need to edit the DockerFile or is there some built in functionality for this?
There is a special directive ENTRYPOINT which fits your needs. Unlike CMD it will add additional flags at the end of your command.
For example, you can write
ENTRYPOINT ["python"]
and run it with
docker run <image_name> -c "print(1)"
Note, that this only will work if you write command in exec form (via ["...", "..."]), otherwise ENTRYPOINT will invoke shell and pass your args there, not to your script.
More generally, you can combine ENTRYPOINT and CMD
ENTRYPOINT ["ping"]
CMD ["www.google.com"]
Where CMD means default args for your ENTRYPOINT. Now you can run both of
docker run <image_name>
docker run <image_name> yandex.ru
and only CMD will be replaced.
Full reference about how ENTRYPOINT and CMD interact can be found here
The CMD directive of a Dockerfile is the command that would be run when the container starts if no command was specified in the docker run command.
The main purpose of a CMD is to provide defaults for an executing container.
In your case, just use the docker run command as follow to override the default command specified in the Dockerfile:
docker run makuk66/docker-solr /bin/bash -c "/opt/solr/bin/solr -f [your flags]"