I am trying to find a way to delay the docker container to be up until the task in ENTRYPOINT is completed. To explain it further, I have a docker file which has the entry point
ENTRYPOINT ["bash", "-c", "tox", "-e", "docker-server"]
When I run the container using
Docker run -d -t -p 127.0.0.1:8882:8882 datawarehouse
it immediately makes the container up where as tox command is still building the environment. The problem with this is that, if I trigger a cron job or run a python code immediately it will fail because the tox environment is still in the build phase. I want to avoid running anything until the ENTRYPOINT task is complete, can this be achieved in the docker file or in the run command?
yes , in the docker-compose file you can set it to sleep or you can define dependencies.
https://docs.docker.com/compose/startup-order/
https://8thlight.com/blog/dariusz-pasciak/2016/10/17/docker-compose-wait-for-dependencies.html
I dont have an elegant solution, but here is what I did.
RUN <your dependencies>
# Then add a second RUN command with a sleep at the beginning:
RUN sleep 400 && gcloud dataproc jobs submit xxxxxx
Each RUN command will run at a separate container layer on a clean slate, hence the sleep && the actual entry-point command goes together as one logical command.
But as you can see this was Hard coded, change the sleep duration accordingly.
I think that this in an incorrect approach. When a container "start" we need to avoid install dependencies, libraries, etc. The build image process is the moment to do that: We ensure that an image "AAAA" always will "works" if we install any dependencies, build any code in the "build" images process. When a container run, is only for do that, just "run".
Related
I would like to have a script for my projekt that starts everytime my container start or restart. The script should run in the console some statments like Console/cake schema etc.
I am using Cakephp 2.x for my projekt and docker.
I tried in the docker file with CMD oder ENTRYPOINT, but the process ends if the script is finished. My container stopped.
CMD ["/bin/bash", "/starter.sh"]
or
ENTRYPOINT ["/starter.sh"]
or
COPY starter.sh /
RUN chmod +x /starter.sh
With the first and second i get the script but the main prozess isnt running.
At the third try the script doesnt run, but my projekt runs.
From the documentation:
An ENTRYPOINT allows you to configure a container that will run as an executable.
And for CMD as well:
The main purpose of a CMD is to provide defaults for an executing container. These defaults can include an executable, or they can omit the executable, in which case you must specify an ENTRYPOINT instruction as well.
So these are used for the main executable. So your third attempt was in the right direction, you only forgot one important step: Executing your script after making it executable with chmod +x
If you include another RUN statement like:
RUN /starter.sh
your script will run
Note what the documentation says about RUN:
The RUN instruction will execute any commands in a new layer on top of the current image and commit the results. The resulting committed image will be used for the next step in the Dockerfile.
This means that it will cache the results of Run if docker does not see the results as different, which means that the script only runs whenever the image is created the first time or whenever your script changes. Most likely this is the behaviour that you want anyway, but you can run it without this caching behaviour:
The cache for RUN instructions isn’t invalidated automatically during the next build. The cache for an instruction like RUN apt-get dist-upgrade -y will be reused during the next build. The cache for RUN instructions can be invalidated by using the --no-cache flag, for example docker build --no-cache.
I also suggest checking out the best practices for the RUN statement: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#run
I think you can check out RUN and CMD in the manual. But I also notice that you said that the process stopped when your script finished. Are you sure that your script won't stop? If your process will stop then no matter what you do, the container will stop at the end.
I'm confused about when should I use CMD vs RUN. For example, to execute bash/shell commands (i.e. ls -la) I would always use CMD or is there a situation where I would use RUN? Trying to understand the best practices about these two similar Dockerfile directives.
RUN is an image build step, the state of the container after a RUN command will be committed to the container image. A Dockerfile can have many RUN steps that layer on top of one another to build the image.
CMD is the command the container executes by default when you launch the built image. A Dockerfile will only use the final CMD defined. The CMD can be overridden when starting a container with docker run $image $other_command.
ENTRYPOINT is also closely related to CMD and can modify the way a container is started from an image.
RUN - command triggers while we build the docker image.
CMD - command triggers while we launch the created docker image.
I found this article very helpful to understand the difference between them:
RUN -
RUN instruction allows you to install your application and packages
required for it. It executes any commands on top of the current image
and creates a new layer by committing the results. Often you will find
multiple RUN instructions in a Dockerfile.
CMD -
CMD instruction allows you to set a default command, which will be
executed only when you run container without specifying a command.
If Docker container runs with a command, the default command will be
ignored. If Dockerfile has more than one CMD instruction, all but last
CMD instructions are ignored.
The existing answers cover most of what anyone looking at this question would need. So I'll just cover some niche areas for CMD and RUN.
CMD: Duplicates are Allowed but Wasteful
GingerBeer makes an important point: you won't get any errors if you put in more than one CMD - but it's wasteful to do so. I'd like to elaborate with an example:
FROM busybox
CMD echo "Executing CMD"
CMD echo "Executing CMD 2"
If you build this into an image and run a container in this image, then as GingerBeer states, only the last CMD will be heeded. So the output of that container will be:
Executing CMD 2
The way I think of it is that "CMD" is setting a single global variable for the entire image that is being built, so successive "CMD" statements simply overwrite any previous writes to that global variable, and in the final image that's built the last one to write wins. Since a Dockerfile executes in order from top to bottom, we know that the bottom-most CMD is the one gets this final "write" (metaphorically speaking).
RUN: Commands May not Execute if Images are Cached
A subtle point to notice about RUN is that it's treated as a pure function even if there are side-effects, and is thus cached. What this means is that if RUN had some side effects that don't change the resultant image, and that image has already been cached, the RUN won't be executed again and so the side effects won't happen on subsequent builds. For example, take this Dockerfile:
FROM busybox
RUN echo "Just echo while you work"
First time you run it, you'll get output such as this, with different alphanumeric IDs:
docker build -t example/run-echo .
Sending build context to Docker daemon 9.216kB
Step 1/2 : FROM busybox
---> be5888e67be6
Step 2/2 : RUN echo "Just echo while you work"
---> Running in ed37d558c505
Just echo while you work
Removing intermediate container ed37d558c505
---> 6f46f7a393d8
Successfully built 6f46f7a393d8
Successfully tagged example/run-echo:latest
Notice that the echo statement was executed in the above. Second time you run it, it uses the cache, and you won't see any echo in the output of the build:
docker build -t example/run-echo .
Sending build context to Docker daemon 9.216kB
Step 1/2 : FROM busybox
---> be5888e67be6
Step 2/2 : RUN echo "Just echo while you work"
---> Using cache
---> 6f46f7a393d8
Successfully built 6f46f7a393d8
Successfully tagged example/run-echo:latest
RUN - Install Python , your container now has python burnt into its image
CMD - python hello.py , run your favourite script
Note: Don’t confuse RUN with CMD. RUN actually runs a command and
commits the result; CMD does not execute anything at build time, but
specifies the intended command for the image.
from docker file reference
https://docs.docker.com/engine/reference/builder/#cmd
RUN Command:
RUN command will basically, execute the default command, when we are building the image. It also will commit the image changes for next step.
There can be more than 1 RUN command, to aid in process of building a new image.
CMD Command:
CMD commands will just set the default command for the new container. This will not be executed at build time.
If a docker file has more than 1 CMD commands then all of them are ignored except the last one. As this command will not execute anything but just set the default command.
RUN: Can be many, and it is used in build process, e.g. install multiple libraries
CMD: Can only have 1, which is your execute start point (e.g. ["npm", "start"], ["node", "app.js"])
There has been enough answers on RUN and CMD. I just want to add a few words on ENTRYPOINT. CMD arguments can be overwritten by command line arguments, while ENTRYPOINT arguments are always used.
This article is a good source of information.
I'm trying to setup a Dockerfile for keycloak and I want to run some commands once my container has started
The reason for this is once the server is started, I want to add some custom configuration each time the Dockerfile is run. I've tried using the "RUN" command however since my container hasn't started when I use the run command, it causes the whole Dockerfile to bomb out
I thought to run a command after the container has started, I could use "CMD" however when I even try running CMD ["echo", "hi"] or CMD ["sh", "echo", "hi"], I get an error "invalid option echo"
Is there a way to get commands to run once a container is running and if so how?
The way to define what your container does when you start it is to specify either CMD or ENTRYPOINT. These commands are executed when you use docker run. You can use RUN to perform various tasks during the build phase. Depending on what you want to do it may or may not be appropriate.
Try CMD sh -c 'echo hi' or CMD ["sh", "-c", "echo hi"]
The exec (list style) format is preferred but the shell format is also acceptable.
Also, keep in mind that the Dockerfile is used only for the build process. Containers are generally designed to be stateless. You shouldn't have to rebuild every time you change something in your application config.
I have built my docker image using openjdk.
# config Dockerfile
FROM openjdk:8
COPY . /usr/src/myapp
WORKDIR /usr/src/myapp
# build image
docker build -t shantanuo/dbt .
It is working as expected using this command...
docker run -p 8081:8080 -it shantanuo/dbt
Once I log-in, I have to run this command...
sh bin/startup.sh
My Question: Is it possible to add the startup command to dockerfile? I tried adding this line in my dockerfile.
CMD ["sh", "bin/startup.sh"]
But after building the image, I can not use -d parameter to start the container.
You can use the entrypoint to run the startup script. In the entrypoint you can specify your custom script, and then run catlina.sh.
Example:
ENTRYPOINT "bin/startup.sh && catalina.sh run"
This will run your startup script and then start your tomcat server, and it won't exit the container.
This is addressed in the documentation here: https://docs.docker.com/config/containers/multi-service_container/
If one of your processes depends on the main process, then start your helper process FIRST with a script like wait-for-it, then start the main process SECOND and remove the fg %1 line.
#!/bin/bash
# turn on bash's job control
set -m
# Start the primary process and put it in the background
./my_main_process &
# Start the helper process
./my_helper_process
# the my_helper_process might need to know how to wait on the
# primary process to start before it does its work and returns
# now we bring the primary process back into the foreground
# and leave it there
fg %1
A docker container must have a dedicated task. It is important that this task/startup script does not terminate. When it does, the task is done and everything for docker is done right.
It makes no sense to start a container only with the JDK. You have to put your application in it.
I think it would help when you will post what you exactly want to do.
The Docker reference is always a good place to look at: https://docs.docker.com/engine/reference/builder/#entrypoint
I'm confused about when should I use CMD vs RUN. For example, to execute bash/shell commands (i.e. ls -la) I would always use CMD or is there a situation where I would use RUN? Trying to understand the best practices about these two similar Dockerfile directives.
RUN is an image build step, the state of the container after a RUN command will be committed to the container image. A Dockerfile can have many RUN steps that layer on top of one another to build the image.
CMD is the command the container executes by default when you launch the built image. A Dockerfile will only use the final CMD defined. The CMD can be overridden when starting a container with docker run $image $other_command.
ENTRYPOINT is also closely related to CMD and can modify the way a container is started from an image.
RUN - command triggers while we build the docker image.
CMD - command triggers while we launch the created docker image.
I found this article very helpful to understand the difference between them:
RUN -
RUN instruction allows you to install your application and packages
required for it. It executes any commands on top of the current image
and creates a new layer by committing the results. Often you will find
multiple RUN instructions in a Dockerfile.
CMD -
CMD instruction allows you to set a default command, which will be
executed only when you run container without specifying a command.
If Docker container runs with a command, the default command will be
ignored. If Dockerfile has more than one CMD instruction, all but last
CMD instructions are ignored.
The existing answers cover most of what anyone looking at this question would need. So I'll just cover some niche areas for CMD and RUN.
CMD: Duplicates are Allowed but Wasteful
GingerBeer makes an important point: you won't get any errors if you put in more than one CMD - but it's wasteful to do so. I'd like to elaborate with an example:
FROM busybox
CMD echo "Executing CMD"
CMD echo "Executing CMD 2"
If you build this into an image and run a container in this image, then as GingerBeer states, only the last CMD will be heeded. So the output of that container will be:
Executing CMD 2
The way I think of it is that "CMD" is setting a single global variable for the entire image that is being built, so successive "CMD" statements simply overwrite any previous writes to that global variable, and in the final image that's built the last one to write wins. Since a Dockerfile executes in order from top to bottom, we know that the bottom-most CMD is the one gets this final "write" (metaphorically speaking).
RUN: Commands May not Execute if Images are Cached
A subtle point to notice about RUN is that it's treated as a pure function even if there are side-effects, and is thus cached. What this means is that if RUN had some side effects that don't change the resultant image, and that image has already been cached, the RUN won't be executed again and so the side effects won't happen on subsequent builds. For example, take this Dockerfile:
FROM busybox
RUN echo "Just echo while you work"
First time you run it, you'll get output such as this, with different alphanumeric IDs:
docker build -t example/run-echo .
Sending build context to Docker daemon 9.216kB
Step 1/2 : FROM busybox
---> be5888e67be6
Step 2/2 : RUN echo "Just echo while you work"
---> Running in ed37d558c505
Just echo while you work
Removing intermediate container ed37d558c505
---> 6f46f7a393d8
Successfully built 6f46f7a393d8
Successfully tagged example/run-echo:latest
Notice that the echo statement was executed in the above. Second time you run it, it uses the cache, and you won't see any echo in the output of the build:
docker build -t example/run-echo .
Sending build context to Docker daemon 9.216kB
Step 1/2 : FROM busybox
---> be5888e67be6
Step 2/2 : RUN echo "Just echo while you work"
---> Using cache
---> 6f46f7a393d8
Successfully built 6f46f7a393d8
Successfully tagged example/run-echo:latest
RUN - Install Python , your container now has python burnt into its image
CMD - python hello.py , run your favourite script
Note: Don’t confuse RUN with CMD. RUN actually runs a command and
commits the result; CMD does not execute anything at build time, but
specifies the intended command for the image.
from docker file reference
https://docs.docker.com/engine/reference/builder/#cmd
RUN Command:
RUN command will basically, execute the default command, when we are building the image. It also will commit the image changes for next step.
There can be more than 1 RUN command, to aid in process of building a new image.
CMD Command:
CMD commands will just set the default command for the new container. This will not be executed at build time.
If a docker file has more than 1 CMD commands then all of them are ignored except the last one. As this command will not execute anything but just set the default command.
RUN: Can be many, and it is used in build process, e.g. install multiple libraries
CMD: Can only have 1, which is your execute start point (e.g. ["npm", "start"], ["node", "app.js"])
There has been enough answers on RUN and CMD. I just want to add a few words on ENTRYPOINT. CMD arguments can be overwritten by command line arguments, while ENTRYPOINT arguments are always used.
This article is a good source of information.