I have a Dockerfile with Entrypoint where I specify the config file variable and the executable file but it looks like Docker or Entrypoint doesn't recognize it. My main.py has to be executed with the config file.
ENTRYPOINT ["CONFIG_FILE=path/to/config.file ./main.py"]
always reproduce no such file or directory: OCI not found
Note: I have copied all the files in the current work directory already. main.py is an executable file. So I guess the problem is the config variable appended before the executable file. Does anyone know what is going on there? Also changing from ENTRYPOINT to CMD does not help as well.
Dockerfile
FROM registry.fedoraproject.org/fedora:34
WORKDIR /home
COPY . /home
ENTRYPOINT ["CONFIG_FILE=path/to/config.file ./main.py"]
If you just need to set an environment variable to a static string, use the Dockerfile ENV directive.
ENV CONFIG_FILE=/path/to/config.file
CMD ["./main.py"]
The Dockerfile ENTRYPOINT and CMD directives (and also RUN) have two forms. You've used the JSON-array form; in that form, there is not a shell involved and you have to manually split out words. (You are trying to run a command CONFIG_FILE=... ./main.py, where the executable file itself needs to include the = and space.) If you don't use the JSON-array form you can use the shell form instead, and this form should work:
CMD CONFIG_FILE=/path/to/config.file ./main.py
In general you should prefer CMD to ENTRYPOINT. There's a fairly standard pattern of using ENTRYPOINT to do first-time setup and then execute the CMD. For example, if you expected the configuration file to be bind-mounted in, but want to set the variable only if it exists, you could write a shell script:
#!/bin/sh
# entrypoint.sh
#
# If the config file exists, set it as an environment variable.
CONFIG_FILE=/path/to/config.file
if [ -f "$CONFIG_FILE" ]; then
export CONFIG_FILE
else
unset CONFIG_FILE
fi
# Run the main container CMD.
exec "$#"
Then you can specify both the ENTRYPOINT (which sets up the environment variables) and the CMD (which says what to actually do)
# ENTRYPOINT must be JSON-array form for this to work
ENTRYPOINT ["./entrypoint.sh"]
# Any valid CMD syntax is fine
CMD ["./main.py"]
You can double-check the environment variable setting by providing an alternate docker run command
# (Make sure to quote things so the host shell doesn't expand them first)
docker run --rm my-image sh -c 'echo $CONFIG_FILE'
docker run --rm -v "$PWD:/path/to" my-image sh -c 'echo $CONFIG_FILE'
If having the same environment in one-off debugging shells launched by docker exec is important to you, of these approaches, only Dockerfile ENV will make the variable visible there. In the other cases the environment variable is only visible in the main container process and its children, but the docker exec process isn't a child of the main process.
Related
I have a Dockerfile where I start a executable with default arguments like this:
ENTRYPOINT ["executable", "cmd"]
CMD ["--param1=1", "--param2=2"]
This works fine and I can run the container with default arguments:
docker run image_name
or with custom arguments:
docker run image_name --param1=a --param2=2
Now i would like to have a default parameter depend on a environment variable or default to the deafult value (1) like this:
--param1='${PARAM1:-1}'
I Understand that
ENTRYPOINT ["executable", "cmd"]
CMD ["--param1='${PARAM1:-1}'", "--param2=2"]
does not work since CMD is in exec form and does not invoke a command shell and cannot substitute environment variables.
But if I use CMD in shell form:
ENTRYPOINT ["executable", "cmd"]
CMD "--param1='${PARAM1:-1}' --param2=2"
I get no such option: -c
So my question is:
How get I archive environment variable substitution within the default arguments in CMD for my ENTRYPOINT?
One way would be to lose the CMD and wrap all the defaults up in a custom entrypoint. I try to avoid doing this, but sometimes it seems like the cleanest way, and you can be a lot more flexible:
Dockerfile:
COPY 'my-entrypoint.sh' '/somewhere/in/path/my-entrypoint'
ENTRYPOINT ['my-entrypoint']
my-entrypoint.sh
#!/bin/sh
ARGS="${#}"
if [ -z "${ARGS}" ]; then
ARGS="--param1=${PARAM1:-1} --param2=2"
fi
executable cmd $ARGS
You can't do this the way you describe, for the reasons you've laid out in the question. The ENTRYPOINT and CMD simply get concatenated together to form a single command line, and if either or both of those parts is a string rather than a JSON array it gets automatically converted to sh -c 'the string'.
ENTRYPOINT ["executable", "cmd"]
CMD "--param1='${PARAM1:-1}' --param2=2"
# Equivalently:
ENTRYPOINT ["executable", "cmd", "/bin/sh", "-c", "\"--param1=...\""]
CMD []
There are two techniques I'd suggest to work around this problem, though both require potentially substantial changes in the setup.
In Docker and Kubernetes, it turns out to generally be more convenient to pass options via environment variables than on the command line. This means your application needs to know to look for those variables, and supply some of the defaults you describe here. Some argument-parsing libraries support this out-of-the-box, but not all. Python's standard argparse library, for example, doesn't directly have environment-variable support, but you can still easily support them:
import argparse
import os
parser = argparse.ArgumentParser()
parser.add_argument('param1', default=os.environ.get('PARAM1', '1'))
args = parser.parse_args()
print(args.param1)
# Uses --param1 option, or else $PARAM1 variable, or else default "1"
The other approach I generally recommend is to make CMD a well-formed shell command; don't try to split the command between CMD and ENTRYPOINT. This avoids the problem of Docker inserting the sh -c wrapper in the middle of the line.
# no ENTRYPOINT
CMD executable cmd --param1="${PARAM1:-1}" --param2=2
The ENTRYPOINT pattern that I do find useful is to use a wrapper script to provide defaults and do other first-time setup. If that script is a Bourne shell script and ends with exec "$#", then it will run the CMD as the main container process.
#!/bin/sh
# docker-entrypoint.sh
# In Docker specifically, default $PARAM1 to "docker", not "1".
: ${PARAM1:=docker}
# Run the main container command.
exec "$#"
ENTRYPOINT ["/docker-entrypoint.sh"] # must be a JSON array
CMD executable cmd --param2=2
(There is no requirement to have an ENTRYPOINT. Making ENTRYPOINT be an interpreter and putting the script name in CMD doesn't bring any benefit, and makes it harder to run debugging commands like docker run --rm my-image ls -l /app.)
Set up
I set up GitLab Runner with a KubernetesExecutor and want to create a custom helper image which adds some extra files to the build container.
The current set up is quite basic:
A Dockerfile, which adds some files (start.sh and Dockerfile) into the container.
A start.sh file which is present in the helper image. This should be executed when the helper is run.
Code
start.sh
#!/bin/bash
printenv > /test.out # Check whether the script is run.
cp /var/Dockerfile /builds/Dockerfile # Copy file to shared volume.
exec "$#"
Dockerfile
FROM gitlab/gitlab-runner-helper:x86_64-latest
ADD templates/Dockerfile /var/Dockerfile
ADD start.sh /var/run/start.sh
ENTRYPOINT ["sh", "/var/run/start.sh"]
The shared volume between the containers is: /builds. As such, I'd like to copy /var/Dockerfile to /builds/Dockerfile.
Problem
I can't seem to find a way to (even) run start.sh when the helper image is executed. Using kubectl exec -it pod-id -c build bash and kubectl exec -it pod-id -c helper bash, I verify whether the files are created. When I run start.sh (manually) from the latter command, the files are copied. However, neither /test.out nor /builds/Dockerfile are available when logging in to the helper image initially.
Attempts
I've tried setting up a different CMD (/var/run/start.sh), but it seems like it simply doesn't run the sh file.
I am trying to deploy an app in payara micro based on payara dockerimage and I need to pass one arguement snapshotversion in ENTRYPOINT(basically i want to access the build args in ENTRYFORM) exec form, as exec form of ENTRYPOINT is preferred: my docker file is as follows:
FROM payara/micro:5.193.1
ARG snapshotversion
ENV snapshotvs=$snapshotversion
RUN jar xf payara-micro.jar
COPY /service/war/target/app-emailverification-service-war-${snapshotversion}.war ${DEPLOY_DIR}/
COPY ojdbc6.jar ${PAYARA_HOME}/
COPY --chown=payara domain.xml /opt/payara/MICRO-INF/domain/domain.xml
RUN cd /opt/payara/MICRO-INF/domain && ls -lrt
#ENTRYPOINT ["java", "-jar", "/opt/payara/payara-micro.jar", "--deploy", "/opt/payara/deployments/app-service-war-$snapshotvs.war", "--domainConfig", "/opt/payara/MICRO-INF/domain/domain.xml","--addLibs", "/opt/payara/ojdbc6.jar"]
ENTRYPOINT java -jar /opt/payara/payara-micro.jar --deploy /opt/payara/deployments/app-service-war-$snapshotvs.war --domainConfig /opt/payara/MICRO-INF/domain/domain.xml --addLibs /opt/payara/ojdbc6.jar
The commented ENTRYPOINT does not work. Container logs says invalid deployment. What am i missing here? Also how can I use CMD with this. Can someone post an example.
The commented line doesn't work, because it is an exec form of ENTRYPOINT, which doesn't invoke shell (/bin/sh -c), so variable substitution doesn't happening.
If you want to use an exec form and environment variables you need to specify it directly:
ENTRYPOINT ["sh", "-c", "your command with env variable"]
To your question about how can you use CMD with this, for example like this:
ENTRYPOINT ["sh", "-c"]
CMD ["your command with env variable"]
You mentioned, that you want to use build args in ENTRYPOINT instruction. It's not really possible, because nor ARG nor ENV are expanded in ENTRYPOINT or CMD: https://docs.docker.com/engine/reference/builder/#environment-replacement, https://docs.docker.com/engine/reference/builder/#scope
Also you could take a look at great page with best practices for writing Dockerfile and ENTRYPOINT instructions specifically.
Two suggestions that complement each other:
If you're COPYing a file into the image, you can give it a fixed name inside the image. That avoids this problem.
WORKDIR /opt/payara
COPY service/war/target/app-emailverification-service-war-${snapshotversion}.war deployments/app-service.war
If you have a particularly long or involved command that you're trying to make be the main container process, wrap it in a shell script. You want to make sure to exec the main container process to avoid some trouble around signal handling (resulting in docker stop pausing for 10 seconds and then hard-killing your actual process).
#!/bin/sh
exec java \
-jar /opt/payara/payara-micro.jar \
--deploy /opt/payara/deployments/app-service.war \
--domainConfig /opt/payara/MICRO-INF/domain/domain.xml \
--addLibs /opt/payara/ojdbc6.jar
COPY launch.sh ./
RUN chmod +x launch.sh
CMD ["/opt/payara/launch.sh"]
In this second case, it's a shell script, so you can have ordinary shell variable substitutions.
I have a Dockerfile as follows.
ENV SPRING_ENV="local"
ENV APP_OPTS "-Xmx8144m"
RUN echo "/usr/lib/jvm/java-1.8-openjdk/bin/java ${APP_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /apps/demo/demo-fe.jar --spring.config.location=file:///apps/demo/conf/ump.properties -Dspring.profiles.active=${SPRING_ENV} &" > /apps/demo/entrypoint.sh
RUN chmod +x /apps/demo/entrypoint.sh
When I run the dockerfile, I see a file 'entrypoint.sh' with the java command that I specified in the Dockerfile.
But I want to change the java max memory depending on the environment. So I am running like this.
docker run -it <image_id> sh -e "APP_OPTS=-Xmx9144m" -e "SPRING_ENV=dev"
But when I run it, i check the entrypoint.sh, i don't see the environment variables replaced. Am I missing something?
Does it replace only on the fly when I actually run the container?
You need to escape the $ in ${APP_OPTS} (i.e., change it to \${APP_OPTS}) -- during docker build, the variable is getting replaced with the "current" environment variable, which would be whatever is in your env output (otherwise null). Calling docker run ... -e "APP_OPTS=-Xmx9144m" won't do anything at this point because ${APP_OPTS} has been replaced after the image was created.
Otherwise, you could try saving the entrypoint.sh file and put it in the same folder as your Dockerfile instead of having your Dockerfile create it (and use COPY instead to put it where you want it). That way, the ${APP_OPTS} environment variable won't get replaced during docker build
The Dockerfile (and the RUN command) are only executed when you build the image. SPRING_ENV and APP_UMPFE_OPTS are being evaluated only once and during the build.
When you run the image, the --env=KEY=VALUE are passed to the shell (!) running the process defined in the ENTRYPOINT or CMD (which you need but do not have).
You're missing a FROM ... statement near the top of the Dockerfile too.
You will need to define (recommend the shell-form of) ENTRYPOINT that invokes the java runtime, passes the environment variables and runs your code, perhaps (have not tried this):
FROM ???
ENV SPRING_ENV="local"
ENV APP_OPTS "-Xmx8144m"
ENTRYPOINT /usr/lib/jvm/java-1.8-openjdk/bin/java ${APP_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /apps/demo/demo-fe.jar --spring.config.location=file:///apps/demo/conf/ump.properties -Dspring.profiles.active=${SPRING_ENV}
Example:
FROM busybox
ENV DOG=Freddie
ENTRYPOINT echo ${DOG}
Then:
docker build --tag=58208029 --file=./Dockerfile .
docker run -it 58208029:latest
Freddie
docker run -it --env=DOG=Henry 58208029:latest
Henry
HTH!
The entrypoint.sh is being written when you build the image, so that RUN statement won't be executed again when you run the container. So the entrypoint.sh file itself will not be updated.
Another issue is that when you do the docker run, the -e options need to be before the image name and command:
docker run -it -e "APP_OPTS=-Xmx9144m" -e "SPRING_ENV=dev" <image_id> sh
Otherwise those are just being passed as arguments to the entrypoint/command
Also, in your Dockerfile, you probably want single quotes around your entrypoint script so that it doesn't interpolate the values at build time.
RUN echo '/usr/lib/jvm/java-1.8-openjdk/bin/java ${APP_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /apps/demo/demo-fe.jar --spring.config.location=file:///apps/demo/conf/ump.properties -Dspring.profiles.active=${SPRING_ENV} &' > /apps/demo/entrypoint.sh
Then when you run the container, the entrypoint script should read the variable values at run time from the environment.
As per Docker documentation:
There can only be one CMD instruction in a Dockerfile. If you list more than one CMD then only the last CMD will take effect.
I wish to execute a simple bash script(which processes docker environment variable) before the CMD command(which is init in my case).
Is there any way to do this?
Use a custom entrypoint
Make a custom entrypoint which does what you want, and then exec's your CMD at the end.
NOTE: if your image already defines a custom entrypoint, you may need to extend it rather than replace it, or you may change behavior you need.
entrypoint.sh:
#!/bin/sh
## Do whatever you need with env vars here ...
# Hand off to the CMD
exec "$#"
Dockerfile:
COPY entrypoint.sh /entrypoint.sh
RUN chmod 755 /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
Docker will run your entrypoint, using CMD as arguments. If your CMD is init, then:
/entrypoint.sh init
The exec at the end of the entrypoint script takes care of handing off to CMD when the entrypoint is done with what it needed to do.
Why this works
The use of ENTRYPOINT and CMD frequently confuses people new to Docker. In comments, you expressed confusion about it. Here is how it works and why.
The ENTRYPOINT is the initial thing run inside the container. It takes the CMD as an argument list. Therefore, in this example, what is run in the container is this argument list:
# ENTRYPOINT = /entrypoint.sh
# CMD = init
["/entrypoint.sh", "init"]
# or shown in a simpler form:
/entrypoint.sh init
It is not required that an image have an ENTRYPOINT. If you don't define one, Docker has a default: /bin/sh -c.
So with your original situation, no ENTRYPOINT, and using a CMD of init, Docker would have run this:
/bin/sh -c 'init'
^--------^ ^--^
| \------- CMD
\--------------- ENTRYPOINT
In the beginning, Docker offered only CMD, and /bin/sh -c was hard-coded as the ENTRYPOINT (you could not change it). At some point along the way, people had use cases where they had to do more custom things, and Docker exposed ENTRYPOINT so you could change it to anything you want.
In the example I show above, the ENTRYPOINT is replaced with a custom script. (Though it is still ultimately being run by sh, because it starts with #!/bin/sh.)
That ENTRYPOINT takes the CMD as is argument. At the end of the entrypoint.sh script is exec "$#". Since $# expands to the list of arguments given to the script, this is turned into
exec "init"
And therefore, when the script is finished, it goes away and is replaced by init as PID 1. (That's what exec does - it replaces the current process with a different command.)
How to include CMD
In the comments, you asked about adding CMD in the Dockerfile. Yes, you can do that.
Dockerfile:
CMD ["init"]
Or if there is more to your command, e.g. arguments like init -a -b, would look like this:
CMD ["init", "-a", "-b"]
Dan's answer was correct, but I found it rather confusing to implement. For those in the same situation, here are code examples of how I implemented his explanation of the use of ENTRYPOINT instead of CMD.
Here are the last few lines in my Dockerfile:
#change directory where the mergeandlaunch script is located.
WORKDIR /home/connextcms
ENTRYPOINT ["./mergeandlaunch", "node", "keystone.js"]
Here are the contents of the mergeandlaunch bash shell script:
#!/bin/bash
#This script should be edited to execute any merge scripts needed to
#merge plugins and theme files before starting ConnextCMS/KeystoneJS.
echo Running mergeandlaunch script
#Execute merge scripts. Put in path to each merge script you want to run here.
cd ~/theme/rtb4/
./merge-plugin
#Launch KeystoneJS and ConnextCMS
cd ~/myCMS
exec "$#"
Here is how the code gets executed:
The ENTRYPOINT command kicks off the mergeandlaunch shell script
The two arguments 'node' and 'keystone.js' are passed along to the shell script.
At the end of the script, the arguments are passed on to the exec command.
The exec command then launched my node program the same way the Docker command CMD would do.
Thanks to Dan for his answer.
Although I found I had to do something like this within the Dockerfile:
WORKDIR /
COPY startup.sh /
RUN chmod 755 /startup.sh
ENTRYPOINT sh /startup.sh /usr/sbin/init
NOTE: I named the script startup.sh as opposed to entrypoint.sh
The key here was that I needed to provide 'sh' otherwise I kept getting "no such file..." errors coming out of 'docker logs -f container_name'.
See:
https://github.com/docker/compose/issues/3876