I am creating a Dockerfile that needs to source a script before a shell is run.
ENTRYPOINT ["/bin/bash", "-rcfile","<(echo '. ./mydir/scripttosource.sh')"]
However, the script isn't sourced as expected.
Combining these parameters on a command line (normal Linux instance, outside of any Docker container), it works properly, for example:
$ /bin/bash -rcfile <(echo '. ./mydir/scripttosource.sh')
So I took a look at what was actually used by the container when it was run.
$ docker ps --format "table {{.ID}} \t {{.Names}} \t {{.Command}}" --no-trunc
CONTAINER ID NAMES COMMAND
70a5f846787075bd9bd55432dc17366268c33c1ab06fb36b23a50f5c3aef19bb happy_cray "/bin/bash -rcfile '<(echo '. ./mydir/scripttosource.sh')'"
Besides the fact that it properly identified the emotional state of Cray computers, Docker seems to be sneaking in undesired single quotes into the third parameter to ENTRYPOINT.
'<(echo '. ./mydir/scripttosource.sh')'
Thus the command actually being executed is:
$ /bin/bash -rcfile '<(echo '. ./mydir/scripttosource.sh')'
Which doesn't work...
Now I realize there are more ways to skin this cat, and I could make this work a different way, I am curious about the insertion of single quotes to the third argument to ENTRYPOINT. Is there a way to avoid this?
Thank you,
At a super low level, the Unix execve(2) function launches a process by taking a sequence of words, where the first word is the actual command to run and the remaining words are its arguments. When you run a command interactively, the shell breaks it into words, usually at spaces, and then calls an exec-type function to run it. The shell also does other processing like replacing $VARIABLE references or the bash-specific <(subprocess) construct; all of these are at layers above simply "run a process".
The Dockerfile ENTRYPOINT (and also CMD, and less frequently RUN) has two forms. You're using the JSON-array exec form. If you do this, you're telling Docker that you want to run the main container command with exactly these three literal strings as arguments. In particular the <(...) string is passed as a literal argument to bash --rcfile, and nothing actually executes it.
The obvious answer here is to use the string-syntax shell form instead
ENTRYPOINT /bin/bash -rcfile <(echo '. ./mydir/scripttosource.sh')
Docker wraps this in an invocation of sh -c (or the Dockerfile SHELL). That causes a shell to preprocess the command string, break it into words, and execute it. Assuming the SHELL is bash and not a pure POSIX shell, this will handle the substitution.
However, there are some downsides to this, most notably that the sh -c invocation "eats" all of the arguments that might be passed in the CMD. If you want your main container process to be anything other than an interactive shell, this won't work.
This brings you to the point of trying to find simpler alternatives to doing this. One specific observation is that the substitution here isn't doing anything; <(echo something) will always produce the fixed string something and you can do it without the substitution. If you can avoid the substitution then you don't need the shell either:
ENTRYPOINT ["/bin/bash", "--rcfile", "./mydir/scripttosource.sh"]
Another sensible approach here is to use an entrypoint wrapper script. This uses the ENTRYPOINT to run a shell script that does whatever initialization is needed, then exec "$#" to run the main container command. In particular, if you use the shell . command to set environment variables (equivalent to the bash-specific source) those will "stick" for the main container process.
#!/bin/sh
# entrypoint.sh
# read the file that sets variables
. ./mydir/scripttosource.sh
# run the main container command
exec "$#"
# Dockerfile
COPY entrypoint.sh ./ # may be part of some other COPY
ENTRYPOINT ["./entrypoint.sh"] # must be JSON-array syntax
CMD ???
This should have the same net effect. If you get a debugging shell with docker run --rm -it your-image bash, it will run under the entrypoint wrapper and see the environment variables. You can do other setup in the wrapper script if required. This particular setup also doesn't use any bash-specific options, and might run better under minimal Alpine-based images.
insertion of single quotes can be avoided by using escape characters in the third argument to ENTRYPOINT.
ENTRYPOINT ["/bin/bash", "-rcfile","$(echo '. ./mydir/scripttosource.sh')"]
Related
Can I use environment variables in my CMD stanza in a Dockerfile?
I want to do something like this:
CMD ["myserver", "--arg=$ARG", "--memcache=$MEMCACHE_11211_TCP_ADDR:$MEMCACHE_11211_TCP_PORT"]
Where $MEMCACHE_11211_TCP_* would be set automatically by the inclusion of the --link parameter of my docker run command. And $ARG would be configurable by the user at runtime, maybe by the "-e" parameter?
This doesn't seem to be working for me, it seems to be literally passing through the string "$ARG" for example.
This answer may be a little late. But environment for CMD is interpreted slightly differently depending on how you write the arguments. If you pass the CMD as a string (not inside an array), it gets launched as a shell instead of exec. See https://docs.docker.com/engine/reference/builder/#cmd.
You may try the CMD without the array syntax to run as a shell:
CMD myserver --arg=$ARG --memcache=$MEMCACHE_11211_TCP_ADDR:$MEMCACHE_11211_TCP_PORT
CMD ["sh", "-c", "echo ${MY_HOME}"]
Answer from sffits here.
Both Andys had it right. The json syntax bypasses the entrypoint. When you use CMD as in their example, it is considered as an argument to the default entrypoint: /bin/sh -c which will interpret the environement variables.
Docker does not evaluate the variables in CMD in either case. In the former, the command is directly called so nothing gets interpreted, in the later, the variables are interpreted by sh.
I can't speak to how it is supposed to work, but I think if you called this as a shell script, e.g. CMD runmyserver.sh, then the interpretation of the shell variables would be deferred until the CMD actually ran.
So, try
myserver --arg=$ARG --memcache=$MEMCACHE_11211_TCP_ADDR:$MEMCACHE_11211_TCP_PORT``
as a shell script?
My Dockerfile:
FROM my-image-base
COPY src src
RUN chmod +x src/script.sh
ENTRYPOINT ['/src/script.sh']
After successful build:
docker run created_image
/bin/sh: [/src/script.sh]: No such file or directory
docker run --entrypoint /src/script.sh created_image
script runs successfully
I feel like I'm overwriting the default ENTRYPOINT with the same thing but it behaves differently. Am I misunderstanding something?
No, it's not the same thing, because your ENTRYPOINT is invalid.
ENTRYPOINT has two forms:
The shell command form:
ENTRYPOINT command param1 param2
and the exec form:
ENTRYPOINT ["executable", "param1", "param2"]
Which of the two forms are you using here? If you answer "the second", you have been subtly led astray, because you have single-quotes ' there instead of double-quotes "!
As specified in the docs:
Note: The exec form is parsed as a JSON array, which means that you must use double-quotes (") around words not single-quotes (').
So, you actually have the first form, the shell command form.
Docker will pass your "command" to the shell, like this:
/bin/sh -c "['/src/script.sh']"
Due to the way quotes work when parsing shell command lines, this ends up being equivalent to typing [/src/script.sh] into your shell. The shell then rightfully complains that that's not a valid command.
If I want to run, for example wget, in a Docker file, I can type this:
RUN wget http://example.com
If I want do an echo command I could do this
RUN echo 'Hello' >> /home/file.text
But I've also seen this:
RUN bash -c 'echo $USERNAME:ros | chpasswd'
If I want to run a shell script, I could do this
RUN 'bash ./install_foo.sh'
I also was recommended this:
RUN . /home/ros/.bashrc
I think there are some invalid examples above and others that have subtle differing semantics. I would like to
Understand it so I can learn
What the right one is to use when I want to run a shell script
Here's a brain dump of related one-line answers:
Every RUN command launches a new shell (in a new container even) with a new clean environment and doesn't read any dotfiles. RUN export ... and RUN . ... are both no-ops that will have no effect on later steps.
Many standard Docker paths (like docker run ... some command) don't involve a shell at all, so if you create a .bashrc or .profile file it will be ignored in many common cases.
Unquoted RUN some command, CMD some command, and ENTRYPOINT some command are all automatically wrapped in sh -c '...' and you basically never need to say this explicitly. (In the case of ENTRYPOINT using the unquoted form is probably a bug.) Forms like CMD ["some", "command"] do not implicitly involve a shell (and don't expand environment variables).
GNU bash has several vendor extensions that unfortunately are in widespread use; Alpine base images don't include bash. In particular never say source when . is in the standard and does the same thing.
If you're installing software in an image, your best choice is to install it in a "system" location (pip install without an active virtual environment, npm install -g, ./configure --prefix=/usr/local); if you must install it somewhere else, use the Dockerfile ENV directive to set any environment variables that are needed; and if you can't do that, an ENTRYPOINT wrapper script can programmatically set the environment for the main process (but not any docker exec shells).
Just in general, ./foo.sh will run a shell script (provided it is executable and starts with a #!/bin/sh line); bash foo.sh will as well (but doesn't require it to be executable and explicitly specifies which shell to use); and . ./foo.sh runs it in the context of the current shell (only this form can change environment variables for example).
Is there a way we can pass a variable lets say in this example I want to pass a list of animals into an entrypoint.sh file using ENV animals="turtle, monkey, goose"
But I want to be able to pass different animals when running the container for example docker run -t image animals="mouse,rat,kangaroo"
How do you go about passing arguments when running the docker run command?
The goal is to take that variable when using the docker run command and insert them into that entrypoint.sh file
Right now i hard code that in my Dockerfile. But i want to be able to do this when running the docker run command so I dont always have to change the Dockerfile.
FROM anapsix/alpine-java:8u121b13_jdk
ENV FILE_NAME="file_to_run.zip"
ENV animals="turtle, monkey, goose"
ADD ${FILE_NAME} .
RUN echo "${FILENAME} ${animals}" > ./entrypoint.sh
CMD [ "/bin/ash", "./entrypoint.sh" ]
It looks like you might be confusing the image build with the container run. If the difference between the two isn't immediately clear, I'd recommend reviewing some other questions and docs like:
In Docker, what's the difference between a container and an image?
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
RUN echo "${FILENAME} ${animals}" > ./entrypoint.sh
With the above, the variables will be expanded during the image build. The entrypoint.sh will not contain ${FILENAME} ${animals}. Instead, it will contain
file_to_run.zip turtle, monkey, goose
After the build, the docker run command will create a container from that image and run the above script with the environment variables defined but never used since the script already has the variables expanded. To prevent the variable expansion, you need to escape the $ or use single quotes to prevent the expansion, e.g.
RUN echo "\${FILENAME} \${animals}" > ./entrypoint.sh
or
RUN echo '${FILENAME} ${animals}' > ./entrypoint.sh
I would also recommend being explicit with a #!/bin/ash at the top of this script. Then when you run the script, do not override the command with parameters after the image name. Instead set the environment variables with the appropriate flag to run:
docker run -it -e animals="mouse,rat,kangaroo" image
Simplest way, forward individual variables:
docker run ... --env animals="turtle, monkey, goose" --env FILE_NAME="file_to_run.zip"
Forward several variables using file:
Or if you need to grab all your environment variables from outside, you can do something like this first:
printenv | grep -E 'animals|FILE_NAME' > my-env
The grep is because Docker doesn't like some variables, e.g. with spaces in them, which you might possibly have in your real environment.
Then use that file in your Docker command:
docker run ... --env-file ./my-env
The latter is also useful if you want to avoid sending environment variables to logs (like for sensitive variables). I use this approach in a CI/CD pipeline that runs some scripts.
Using variables inside Docker:
With either approach, the environment variables actually become available to scripts running inside the container to use.
#BMitch's answer has more complete details about how to achieve this in your case, where you have related logic in both build and execution.
Reference
See docs here.
Can I use environment variables in my CMD stanza in a Dockerfile?
I want to do something like this:
CMD ["myserver", "--arg=$ARG", "--memcache=$MEMCACHE_11211_TCP_ADDR:$MEMCACHE_11211_TCP_PORT"]
Where $MEMCACHE_11211_TCP_* would be set automatically by the inclusion of the --link parameter of my docker run command. And $ARG would be configurable by the user at runtime, maybe by the "-e" parameter?
This doesn't seem to be working for me, it seems to be literally passing through the string "$ARG" for example.
This answer may be a little late. But environment for CMD is interpreted slightly differently depending on how you write the arguments. If you pass the CMD as a string (not inside an array), it gets launched as a shell instead of exec. See https://docs.docker.com/engine/reference/builder/#cmd.
You may try the CMD without the array syntax to run as a shell:
CMD myserver --arg=$ARG --memcache=$MEMCACHE_11211_TCP_ADDR:$MEMCACHE_11211_TCP_PORT
CMD ["sh", "-c", "echo ${MY_HOME}"]
Answer from sffits here.
Both Andys had it right. The json syntax bypasses the entrypoint. When you use CMD as in their example, it is considered as an argument to the default entrypoint: /bin/sh -c which will interpret the environement variables.
Docker does not evaluate the variables in CMD in either case. In the former, the command is directly called so nothing gets interpreted, in the later, the variables are interpreted by sh.
I can't speak to how it is supposed to work, but I think if you called this as a shell script, e.g. CMD runmyserver.sh, then the interpretation of the shell variables would be deferred until the CMD actually ran.
So, try
myserver --arg=$ARG --memcache=$MEMCACHE_11211_TCP_ADDR:$MEMCACHE_11211_TCP_PORT``
as a shell script?