Some Dockerfiles have
RUN bash -c "apt-get update -qq && ... \"
while others write without quotes like
RUN apt-get update -qq && ... \
What is the difference between these variants? Is one of them more preferable over another one?
You should just write RUN apt-get update ... without manually inserting a sh -c wrapper.
The RUN, CMD, and ENTRYPOINT directives all share the same syntax. It's best documented for ENTRYPOINT but all three commands work the same way. There are two ways to write commands for them: you can either provide a specific set of command words as a JSON array (exec form), or you can write a string and Docker will automatically wrap it sh -c (shell form). For example:
# Create a directory with a space in its name
RUN mkdir "a directory"
# JSON-array form: each array element is a shell word
RUN ["ls", "-ld", "a directory"]
# String form: Docker provides a shell, so these two are equivalent
RUN ls -ld 'a directory'
RUN ["/bin/sh", "-c", "ls -ld 'a directory'"]
This makes your first form redundant: if you RUN bash -c '...', it's a string, and Docker automatically wraps it in sh -c. So you get in effect
# RUN bash -c '...'
RUN ["/bin/sh", "-c", "bash -c '...'"]
GNU bash has a number of extensions that are not POSIX-standard syntax and it's possible to run into trouble with these, particularly on an Alpine-based image where /bin/sh is a minimal shell from the BusyBox toolset. I could see this as an attempt to force a shell command to run using bash rather than the default shell. For most things that appear in Dockerfiles, they won't usually be so complex that they can't be easily rewritten in standard syntax.
# Needs bash for the non-standard `source` syntax
RUN bash -c 'source ./venv/bin/activate && pip list'
# But you can use the standard `.` instead
RUN . ./venv/bin/activate && pip list
If you must have bash interpreting RUN lines then I'd suggest using the SHELL directive to change the command that's used to interpret bare strings.
Style-wise, I also occasionally see JSON-array syntax that begins with an explicit CMD ["/bin/sh", "-c", "..."]. There's no reason to write this out; it's shorter and no less clear to use the string form.
Related
I am new to the docker world. I have to invoke a shell script that takes command line arguments through a docker container.
Ex: My shell script looks like:
#!bin/bash
echo $1
Dockerfile looks like this:
FROM ubuntu:14.04
COPY ./file.sh /
CMD /bin/bash file.sh
I am not sure how to pass the arguments while running the container
with this script in file.sh
#!/bin/bash
echo Your container args are: "$#"
and this Dockerfile
FROM ubuntu:14.04
COPY ./file.sh /
ENTRYPOINT ["/file.sh"]
you should be able to:
% docker build -t test .
% docker run test hello world
Your container args are: hello world
Use the same file.sh
#!/bin/bash
echo $1
Build the image using the existing Dockerfile:
docker build -t test .
Run the image with arguments abc or xyz or something else.
docker run -ti --rm test /file.sh abc
docker run -ti --rm test /file.sh xyz
There are a few things interacting here:
docker run your_image arg1 arg2 will replace the value of CMD with arg1 arg2. That's a full replacement of the CMD, not appending more values to it. This is why you often see docker run some_image /bin/bash to run a bash shell in the container.
When you have both an ENTRYPOINT and a CMD value defined, docker starts the container by concatenating the two and running that concatenated command. So if you define your entrypoint to be file.sh, you can now run the container with additional args that will be passed as args to file.sh.
Entrypoints and Commands in docker have two syntaxes, a string syntax that will launch a shell, and a json syntax that will perform an exec. The shell is useful to handle things like IO redirection, chaining multiple commands together (with things like &&), variable substitution, etc. However, that shell gets in the way with signal handling (if you've ever seen a 10 second delay to stop a container, this is often the cause) and with concatenating an entrypoint and command together. If you define your entrypoint as a string, it would run /bin/sh -c "file.sh", which alone is fine. But if you have a command defined as a string too, you'll see something like /bin/sh -c "file.sh" /bin/sh -c "arg1 arg2" as the command being launched inside your container, not so good. See the table here for more on how these two options interact
The shell -c option only takes a single argument. Everything after that would get passed as $1, $2, etc, to that single argument, but not into an embedded shell script unless you explicitly passed the args. I.e. /bin/sh -c "file.sh $1 $2" "arg1" "arg2" would work, but /bin/sh -c "file.sh" "arg1" "arg2" would not since file.sh would be called with no args.
Putting that all together, the common design is:
FROM ubuntu:14.04
COPY ./file.sh /
RUN chmod 755 /file.sh
# Note the json syntax on this next line is strict, double quotes, and any syntax
# error will result in a shell being used to run the line.
ENTRYPOINT ["file.sh"]
And you then run that with:
docker run your_image arg1 arg2
There's a fair bit more detail on this at:
https://docs.docker.com/engine/reference/run/#cmd-default-command-or-options
https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example
With Docker, the proper way to pass this sort of information is through environment variables.
So with the same Dockerfile, change the script to
#!/bin/bash
echo $FOO
After building, use the following docker command:
docker run -e FOO="hello world!" test
What I have is a script file that actually runs things. This scrip file might be relatively complicated. Let's call it "run_container". This script takes arguments from the command line:
run_container p1 p2 p3
A simple run_container might be:
#!/bin/bash
echo "argc = ${#*}"
echo "argv = ${*}"
What I want to do is, after "dockering" this I would like to be able to startup this container with the parameters on the docker command line like this:
docker run image_name p1 p2 p3
and have the run_container script be run with p1 p2 p3 as the parameters.
This is my solution:
Dockerfile:
FROM docker.io/ubuntu
ADD run_container /
ENTRYPOINT ["/bin/bash", "-c", "/run_container \"$#\"", "--"]
If you want to run it #build time :
CMD /bin/bash /file.sh arg1
if you want to run it #run time :
ENTRYPOINT ["/bin/bash"]
CMD ["/file.sh", "arg1"]
Then in the host shell
docker build -t test .
docker run -i -t test
I wanted to use the string version of ENTRYPOINT so I could use the interactive shell.
FROM docker.io/ubuntu
...
ENTRYPOINT python -m server "$#"
And then the command to run (note the --):
docker run -it server -- --my_server_flag
The way this works is that the string version of ENTRYPOINT runs a shell with the command specified as the value of the -c flag. Arguments passed to the shell after -- are provided as arguments to the command where "$#" is located. See the table here: https://tldp.org/LDP/abs/html/options.html
(Credit to #jkh and #BMitch answers for helping me understand what's happening.)
Another option...
To make this works
docker run -d --rm $IMG_NAME "bash:command1&&command2&&command3"
in dockerfile
ENTRYPOINT ["/entrypoint.sh"]
in entrypoint.sh
#!/bin/sh
entrypoint_params=$1
printf "==>[entrypoint.sh] %s\n" "entry_point_param is $entrypoint_params"
PARAM1=$(echo $entrypoint_params | cut -d':' -f1) # output is 1 must be 'bash' it will be tested
PARAM2=$(echo $entrypoint_params | cut -d':' -f2) # the real command separated by &&
printf "==>[entrypoint.sh] %s\n" "PARAM1=$PARAM1"
printf "==>[entrypoint.sh] %s\n" "PARAM2=$PARAM2"
if [ "$PARAM1" = "bash" ];
then
printf "==>[entrypoint.sh] %s\n" "about to running $PARAM2 command"
echo $PARAM2 | tr '&&' '\n' | while read cmd; do
$cmd
done
fi
In my docker file I have a variable defined for a version number with dots which I want to replace with underscores for a further usage.
ARG ABC_VERSION=1.2.3
ARG SOME_OTHER_VARIABLE=/dir_name/abc_${ABC_VERSION//./_}
Unfortunately, I get the following error.
failed to process "/dir_name/abc_${ABC_VERSION//./_}": missing ':' in substitution
I need the version number several times within the dockerfile multiple times with the '.' and one time with the '_' and I do not like to define two variables.
Does someone know how to solve this problem?
Edit:
One portion of the actual code where I would like to make use of the feature to replace characters is looking like this.
ARG EXPAT_VERSION=2.1.0
# ...
RUN wget https://github.com/libexpat/libexpat/releases/download/R_${EXPAT_VERSION//./_}/expat-${EXPAT_VERSION}.tar.gz \
&& tar xzf expat-${EXPAT_VERSION}.tar.gz \
&& cp -R expat-${EXPAT_VERSION}/lib ./xmp_sdk/third-party/expat \
&& rm -r expat-${EXPAT_VERSION} && rm expat-${EXPAT_VERSION}.tar.gz
I saw that something similiar is used in the tensorflow-gpu dockerfiles:
ARG CUDA=10.1
#...
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
cuda-command-line-tools-${CUDA/./-} #...
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/dockerfiles/dockerfiles/devel-gpu.Dockerfile
The ${parameter/pattern/string} syntax is actually a Bash feature (cf. shell parameter expansion), not a POSIX feature.
According to the official documentation, the Dockerfile directives only supports:
$var
${var}
${var:-word} → if var is not set then word is the result;
${var:+word} → if var is set then word is the result, otherwise the empty string
Workaround 1
So the problem does not have a "direct" solution, but if the variable you would like to substitute will be used, in the end, in some shell command (in a RUN, ENTRYPOINT or CMD directive), you could just as well keep the initial value as is (with no substitution), then substitute it later on?
I mean for example, the following Dockerfile:
FROM debian
ARG ABC_VERSION=1.2.3
ENV SOME_OTHER_VARIABLE=/app/${ABC_VERSION}
WORKDIR /app
RUN /bin/bash -c 'touch "${SOME_OTHER_VARIABLE//./_}"'
# RUN touch "${SOME_OTHER_VARIABLE//./_}"
# would raise /bin/sh: 1: Bad substitution
CMD ["/bin/bash", "-c", "ls -hal \"${SOME_OTHER_VARIABLE//./_}\""]
As an aside:
I replaced ARG SOME_OTHER_VARIABLE with ENV SOME_OTHER_VARIABLE just to be able to use it from CMD.
It can be recalled that ENTRYPOINT and CMD directives should rather be written in exec form − CMD ["…", "…"] − rather in shell form (see e.g. that question: CMD doesn't run after ENTRYPOINT in Dockerfile).
Workaround 2
Or as an alternative workaround, you may want to split your version number in major, minor, patch, to write something like this?
ARG MAJOR=1
ARG MINOR=2
ARG PATCH=3
ARG ABC_VERSION=$MAJOR.$MINOR.$PATCH
ARG SOME_OTHER_VARIABLE=/dir_name/abc_${MAJOR}_${MINOR}_${PATCH}
…
A more concise syntax for workaround 1
Following the OP's edit, I guess one concern is the relative verbosity of this line that I mentioned in the "workaround 1":
…
RUN /bin/bash -c 'touch "${SOME_OTHER_VARIABLE//./_}"'
To alleviate this, Docker allows one to replace the implied shell (by default sh) with Bash, which does support the shell parameter expansion you are interested in. The key point is the following directive that has to be written before the RUN command (and which was precisely part of the Dockerfile the OP mentioned):
SHELL ["/bin/bash", "-c"]
Thus, the Dockerfile becomes:
…
ARG ABC_VERSION=1.2.3
SHELL ["/bin/bash", "-c"]
RUN touch "/dir_name/abc_${ABC_VERSION//./_}" \
&& ls -hal "/dir_name/abc_${ABC_VERSION//./_}"
or taking advantage of some temporary environment variable:
…
ARG ABC_VERSION=1.2.3
SHELL ["/bin/bash", "-c"]
RUN export SOME_OTHER_VARIABLE="/dir_name/abc_${ABC_VERSION//./_}" \
&& touch "$SOME_OTHER_VARIABLE" \
&& ls -hal "$SOME_OTHER_VARIABLE"
I want to access the value of one of environment variable in my dockerfile , and pass it as first argument to the main script in docker ENTRYPOINT.
I came across this so link which shows two ways to do it. one with exec form and one with shell form.
The exec form worked fine to echo the environment variable with ["sh", "-c", "echo $VARIABLE"] but when I tried with my custom entrypoint script ENTRYPOINT ["/bin/customentrypoint.sh", "$VARIABLE"] it is not able to get the value for variable, instead its just taking it as constant $VARIABLE.
So I went with shell form approach and just called ENTRYPOINT /bin/customentrypoing "$VARIABLE", and it worked fine to get the value of $VARIABLE but It seems that its restricting the no of command line arguments in this case. as I am getting only one value of $# even after passing other command line arguments from docker run.Can someone please help me if I am doing something wrong , or I should tackle this in different way.Thanks in Advance.
docker looks is similar to
#!/usr/bin/env bash
...
ENV VARIABLE NO
...
RUN echo "#!/bin/bash" > /bin/customentrypoint.sh
RUN echo "if [ "\"\$1\"" = 'YES' ] ; then ; python ${LOCATION}/main.py" \"\$#\" "; else ; echo Please select -e VARIABLE=YES ; fi" >> /bin/customentrypoint.sh
RUN chmod +x /bin/customentrypoint.sh
RUN ln -s -T /bin/customentrypoint.sh /bin/customentrypoint
WORKDIR ${LOCATION}
ENTRYPOINT /bin/customentrypoint "$VARIABLE" # - works fine but limits no of command line arguments
# ENTRYPOINT ["bin/customentrypoint", "$VARIABLE"] # not able to get value of $VARIABLE instead taking as constant.
command I am using
docker run --rm -v $PWD:/mnt -e VARIABLE=VALUE docker_image:tag entrypoint -d /mnt/tmp -i /mnt/input_file
The environment for CMD is interpreted slightly differently depending on how you write the arguments. If you pass the CMD as a string (not inside an array), it gets launched as a shell instead of exec. See https://docs.docker.com/engine/reference/builder/#cmd.
What you can try if you want to use array is
ENTRYPOINT ["/bin/sh", "-c", "echo ${VARIABLE}"]
I am creating a Nginx docker image that I'll be using as a reverse proxy component in ECS/Fargate in AWS. I'm using the official Nginx image as the base image (1.17.5).
When the container starts I'm trying to run a bash script from an ENTRYPOINT to go out to the AWS Parameter Store and retrieve certificate info. This work fine, however when I try to add a parameter to pass to the bash script (e.g. ENTRYPOINT ["installcerts.sh", "AppName"] it executes the script but the container terminates without error.
I want the container to continue on to start up Nginx after the parameterized batch script.
Here is my Docker File:
FROM nginx:1.17.5
# Install AWS CLI/BOTO3, JQ
RUN apt-get update && apt-get install -y && apt-get install awscli -y && apt-get install jq -y
# Copy Nginx config to etc/nginx
COPY proxy_ssl.conf /etc/nginx/conf.d/
VOLUME ["/etc/nginx/conf/d"]
# Copy entrypoint bash script to install certs from the AWS Parameter Store
COPY installcerts.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/installcerts.sh
#Pull certs from Parameter Store
ENTRYPOINT ["/usr/local/bin/installcerts.sh", "AppName"]
CMD ["nginx", "-g", "daemon off;"]
And here is my "installcerts.sh" script showing utilizing the parameter passed in from ENTRYPOINT.
#!/usr/bin/env bash
-e
echo Installing certs...
aws ssm get-parameters --name /Certificate/$1/CRT | jq '.Parameters[0].Value' -r > /etc/nginx/conf.d/app.crt
aws ssm get-parameters --name /Certificate/$1/KEY | jq '.Parameters[0].Value' -r > /etc/nginx/conf.d/app.key
echo Exiting script.
exec "$#"
The "exec "$#" in the bash script is needed, but honestly, I don't completely understand how or why this works even after hours of trying to track it down.
The short story is:
If I use this, the container does what i want it to do but I can't send a parameter to the bash script.
ENTRYPOINT ["/usr/local/bin/installcerts.sh"]
But If I use this, the script will run WITH the parameter successfully, but the container exits and Nginx doesn't start up.
ENTRYPOINT ["/usr/local/bin/installcerts.sh", "AppName"]
What am I doing wrong?
When you set an ENTRYPOINT on your image, then docker passes that script the value of CMD (or whatever you pass on the command line after the image name). For example, if you have:
ENTRYPOINT ["/entrypoint.sh"]
CMD ["/usr/bin/myprogram"]
Then docker runs, effectively:
/entrypoint.sh /usr/bin/myprogram
That is, Docker itself never runs /usr/bin/myprogram: it is entirely up to the ENTRYPOINT script to do that. That is what the exec "$#" is for. This is a shell variable the evaluates to:
Expands to the positional parameters, starting from one.
(bash(1) man page, in the "Special Parameters" section)
In our example, this would evaluate to:
exec /usr/bin/myprogram
...which replaces the current script with /usr/bin/myprogram. But if we were to set ENTRYPOINT as you have in your question:
ENTRYPOINT ["/entrypoint.sh", "appName"]
Then exec "$#" will in fact evaluate to:
exec appName /usr/bin/myprogram
And since appName isn't a valid command, the container will simply fail.
There are a few ways of dealing with this:
Do you really need to pass parameters to your ENTRYPOINT script? What about using environment variables instead?
If you always pass parameters to your script, you can use the shift shell command to drop those from the positional parameters before using $#. For example, for a script that expects two parameters:
param1=$1
param2=$2
shift 2
...do stuff here...
exec "$#"
...but this only works if you always pass two parameters.
You can implement command line option processing in your script using e.g. the getopts command:
while getopts a:b: ch; do
case $ch in
(a) param1=$OPTARG
;;
(b) param2=$OPTARG
;;
esac
done
shift $(( $OPTIND - 1 ))
...do stuff here...
exec "$#"
Having said that: I would opt for option 1 (use environment variables) as being the simplest solution:
docker run -e PARAM1="some value" ...
And then in your ENTRYPOINT script you can just use the $PARAM1 variable where you need it.
If I set an environment variable, say ENV ADDRESSEE=world, and I want to use it in the entry point script concatenated into a fixed string like:
ENTRYPOINT ["./greeting", "--message", "Hello, world!"]
with world being the value of the environment varible, how do I do it? I tried using "Hello, $ADDRESSEE" but that doesn't seem to work, as it takes the $ADDRESSEE literally.
You're using the exec form of ENTRYPOINT. Unlike the shell form, the exec form does not invoke a command shell. This means that normal shell processing does not happen. For example, ENTRYPOINT [ "echo", "$HOME" ] will not do variable substitution on $HOME. If you want shell processing then either use the shell form or execute a shell directly, for example: ENTRYPOINT [ "sh", "-c", "echo $HOME" ].
When using the exec form and executing a shell directly, as in the case for the shell form, it is the shell that is doing the environment variable expansion, not docker.(from Dockerfile reference)
In your case, I would use shell form
ENTRYPOINT ./greeting --message "Hello, $ADDRESSEE\!"
After much pain, and great assistance from #vitr et al above, i decided to try
standard bash substitution
shell form of ENTRYPOINT (great tip from above)
and that worked.
ENV LISTEN_PORT=""
ENTRYPOINT java -cp "app:app/lib/*" hello.Application --server.port=${LISTEN_PORT:-80}
e.g.
docker run --rm -p 8080:8080 -d --env LISTEN_PORT=8080 my-image
and
docker run --rm -p 8080:80 -d my-image
both set the port correctly in my container
Refs
see https://www.cyberciti.biz/tips/bash-shell-parameter-substitution-2.html
I tried to resolve with the suggested answer and still ran into some issues...
This was a solution to my problem:
ARG APP_EXE="AppName.exe"
ENV _EXE=${APP_EXE}
# Build a shell script because the ENTRYPOINT command doesn't like using ENV
RUN echo "#!/bin/bash \n mono ${_EXE}" > ./entrypoint.sh
RUN chmod +x ./entrypoint.sh
# Run the generated shell script.
ENTRYPOINT ["./entrypoint.sh"]
Specifically targeting your problem:
RUN echo "#!/bin/bash \n ./greeting --message ${ADDRESSEE}" > ./entrypoint.sh
RUN chmod +x ./entrypoint.sh
ENTRYPOINT ["./entrypoint.sh"]
I SOLVED THIS VERY SIMPLY!
IMPORTANT: The variable which you wish to use in the ENTRYPOINT MUST be ENV type (and not ARG type).
EXAMPLE #1:
ARG APP_NAME=app.jar # $APP_NAME can be ARG or ENV type.
ENV APP_PATH=app-directory/$APP_NAME # $APP_PATH must be ENV type.
ENTRYPOINT java -jar $APP_PATH
This will result with executing:
java -jar app-directory/app.jar
EXAMPLE #2 (YOUR QUESTION):
ARG ADDRESSEE="world" # $ADDRESSEE can be ARG or ENV type.
ENV MESSAGE="Hello, $ADDRESSEE!" # $MESSAGE must be ENV type.
ENTRYPOINT ./greeting --message $MESSAGE
This will result with executing:
./greeting --message Hello, world!
Please verify to be sure, whether you need quotation-marks "" when assigning string variables.
MY TIP: Use ENV instead of ARG whenever possible to avoid confusion on your part or the SHELL side.
For me, I wanted to store the name of the script in a variable and still use the exec form.
Note: Make sure, the variable you are trying to use is declared an environment variable either from the commandline or via the ENV directive.
Initially I did something like:
ENTRYPOINT [ "${BASE_FOLDER}/scripts/entrypoint.sh" ]
But obviously this didn't work because we are using the shell form and the first program listed needs to be an executable on the PATH. So to fix this, this is what I ended up doing:
ENTRYPOINT [ "/bin/bash", "-c", "exec ${BASE_FOLDER}/scripts/entrypoint.sh \"${#}\"", "--" ]
Note the double quotes are required
What this does is to allow us to take whatever extra args were passed to /bin/bash, and supply those same arguments to our script after the name has been resolved by bash.
man 7 bash
-- A -- signals the end of options and disables further
option processing. Any arguments after the -- are treated
as filenames and arguments. An argument of - is
equivalent to --.
In my case worked this way: (for Spring boot app in docker)
ENTRYPOINT java -DidMachine=${IDMACHINE} -jar my-app-name
and passing the params on docker run
docker run --env IDMACHINE=Idmachine -p 8383:8383 my-app-name
I solved the problem using a variation on "create a custom script" approach above. Like this:
FROM hairyhenderson/figlet
ENV GREETING="Hello"
RUN printf '#!/bin/sh\nfiglet -W \${GREETING} \$#\n' > /runme && chmod +x /runme
ENTRYPOINT ["/runme"]
CMD ["World"]
Run like
docker container run -it --rm -e GREETING="G'Day" dockerfornovices/figlet-greeter Alec
If someone wants to pass an ARG or ENV variable to exec form of ENTRYPOINT then a temp file created during image building process might be used.
In my case I had to start the app differently depending on whether the .NET app has been published as self-contained or not.
What I did is I created the temp file and I used its name in the if statement of my bash script.
Part of my dockerfile:
ARG SELF_CONTAINED=true #ENV SELF_CONTAINED=true also works
# File has to be used as a variable as it's impossible to pass variable do ENTRYPOINT using Exec form. File name allows to check whether app is self-contained
RUN touch ${SELF_CONTAINED}.txt
COPY run-dotnet-app.sh .
ENTRYPOINT ["./run-dotnet-app.sh", "MyApp" ]
run-dotnet-app.sh:
#!/bin/sh
FILENAME=$1
if [ -f "true.txt" ]; then
./"${FILENAME}"
else
dotnet "${FILENAME}".dll
fi
Here is what worked for me:
ENTRYPOINT [ "/bin/bash", "-c", "source ~/.bashrc && ./entrypoint.sh ${#}", "--" ]
Now you can supply whatever arguments to the docker run command and still read all environment variables.