I have an existential question about docker. Given this dockerfile:
FROM someImage
MAINTAINER abc
ENV something=somehow
RUN sh calculatePi.sh
ENV somethingElse=somehow2
"Calculate Pi" is a "continuous" program that never ends and needs to be ran on the background. It calculates all the digits of PI (3.1415.....) and save it to a txt file.
My question:
Is this dockerfile even plausible?
If Yes, When I run a container based on this image, what is the saved state? In other words, if I open the txt file, what would I see?
When Docker builds an image, each instruction in the Dockerfile gets executed in an interim container, run from the preceding image layer. So if your calculatePi.sh ran endlessly then your image would never build - it would stick at the RUN instruction waiting for it to complete.
In practice, it's more likely that you'll max out disk or CPU resource and take down the machine if you tried to build it. Either way, you wouldn't get a completed image that you could run.
No, that Dockerfile won't work. RUN instructions need to complete before Docker can create an image from them. Perhaps you want to make that a CMD instruction instead?
May be you can write your docker file like this:
FROM someImage
MAINTAINER abc
ENV something=somehow
ENV somethingElse=somehow2
ENTRYPOINT ["/bin/bash"]
CMD ["calculatePi.sh"]
Then when you run this image
docker run -d thisImage
The script calculatePi.sh will run in your container as an App.
Related
I'm trying to learn how to write a Dockerfile. Currently my strategy is as follows:
Guess what commands are correct to write based documentation.
Run sudo docker-compose up --build -d to build a docker container
Wait ~5 minutes for my anaconda packages to install
Find that I made a mistake on step 15, and go back to step 1.
Is there a way to interactively enter the commands for a Dockerfile, or to cache the first 14 successful steps so I don't need to rebuild the entire file? I saw something about docker exec but it seems that's only for running containers. I also want to try and use the same syntax as I use in the dockerfile (i.e. ENTRYPOINT and ENV) since I'm not sure what the bash equivalent is/if it exists.
you can run docker-compose without the --build flag that way you don't have to rebuild the image every time, although as you are testing the Dockerfile i don't know if you have much options here; the docker should cache automatically the builds but only if there's no changes from the last time that you made a build, and there's no way to build a image interactively, docker doesn't work like that, lastly, the docker exec is just to run commands inside the container that was created from the build.
some references for you: docker cache, docker file best practices
I've been attempting to solve this problem for what seems like an eternity, but it seems as though Docker for Windows has only recently been able to successfully bind volumes from the Windows host to its Docker containers. However, I'm unable to get the same functionality working via docker-compose. Does anyone have any definitive information surrounding this, or thoughts on how I might go about getting this to work?
I have read a painful amount of GitHub issues about Windows volume mounting, but it seems as though most of the answers are related to getting Docker to mount the volume. I'm having no issues there, just docker-compose.
To illustrate my issue, the following command gives me the following output, which is the correct contents of my local directory, mounted in Docker:
$ docker run --rm -v c:/Users/synta/go/src/github.com/syntaqx/example:/go/src/github.com/syntaqx/example alpine ls /go/src/github.com/syntaqx/example
LICENSE
README.md
cmd
docker-compose.yml
docs
go.mod
go.sum
example.go
However, given the same implementation within a docker-compose.yml:
version: '3.6'
services:
example:
build:
context: .
dockerfile: Dockerfile
volumes:
- /c/Users/syntaqx/go/src/github.com/syntaqx/example:/go/src/github.com/syntaqx/example
And the following Dockerfile:
ARG GO_VERSION=1.11
ARG ALPINE_VERSION=3.8
FROM golang:${GO_VERSION}-alpine${ALPINE_VERSION} AS builder
WORKDIR /go/src/github.com/syntaqx/example
RUN ls -alh
ENTRYPOINT ["go"]
I'm given literally no output (ls -alh gives . and .., letting me know the directory is definitely there via WORKDIR, but not populated from the binding):
$ docker-compose up -d --build
Building example
Step 1/6 : ARG GO_VERSION=1.11
Step 2/6 : ARG ALPINE_VERSION=3.8
Step 3/6 : FROM golang:${GO_VERSION}-alpine${ALPINE_VERSION} AS builder
---> be1230a1b343
Step 4/6 : WORKDIR /go/src/github.com/syntaqx/example
---> Using cache
---> a0ccb401a86a
Step 5/6 : RUN ls /go/src/github.com/syntaqx/example
---> Running in 0cd52d44a637
Removing intermediate container 0cd52d44a637
---> 4f362f738e49
Step 6/6 : ENTRYPOINT ["go"]
---> Running in 5d7a1e841bfe
Removing intermediate container 5d7a1e841bfe
---> 9da9cfcf372a
...
Perhaps I'm missing something obvious here, but I've tried a dozen ways of expressing the volume PATH (., C:\, /c/, ///c), with relative paths apparently being very broken in Docker Windows, and the other paths not changing the result.
Also very sorry if this is a duplicate, but from what I've seen, most other questions are very specifically trying to mount host volumes in Docker, period. Or, the issue with relative paths, which I'm happy to side step for the time being. To repeat, all of that is working fine for me as shown in my example, just docker-compose seems broken.
Hope one of you guys can help me! Feeling very out of my depth here.
Version
What a long road that was.
But, I learned something: I actually had a fundamental misunderstanding of how volume mounts work within Docker. To my surprise, everything, including relative volume paths are working as expected.
The fundamental misunderstanding I had for volume mounts is that they are not available during build steps, but the mount is attached to the final container. Docker imposes this requirement, as without it, builds would not be repeatable.
So, in the usecase I provided above, the only difference (albeit not obvious until I understood this) between the Dockerfile and the docker run command, is that my Dockerfile is actively building a container, and attempting to execute commands, where the docker run is only running the command on the prebuilt alpine.
So, is it possible to do what I was doing? Sort of
I ran into a couple of cheese solutions that utilize a decorator-ish entrypoint.sh chaining pattern, allowing you to chain multiple Dockerfile build outputs into their final container, and when it's run, executes chained commands before the given CMD. Simply put, you have a bunch of dockerfiles that all end with ENTRYPOINT ["entrypoint.sh"], and then outline the steps you would previously leverage RUN for:
ls.dockerfile
mod.dockerfile
final.dockerfile
#!/bin/sh
set -eu
until /go/src/github.com/syntaqx/example && go mod download
do
echo "Waiting for mount..."
done
sh -c "go $*"
Then, you can have each dockerfile shove commands on top of the last one.
Note: I did not end up using this once, as I describe below, so this shell may not work, but I thought it was a cool implementation thought, so I wanted to nod to it.
However, for my use case, all of this was very unnecessary. Knowing why the commands are failing, I'm I'm able to simply defer the RUN commands until the final ENTRYPOINT, create a singular entrypoint.sh, and then run them in the order I wanted during the build. I don't benefit from dependency caching this way, but I also don't need it for this particular use case.
Thanks for reading, and I hope you learned something too!
If I want to run a python script in my container what is the point in having the RUN command, if I can pass in an argument at build along with running the script?
Each time I run the container I want x.py to be run on an ENV variable passed in during the build stage.
If I were to use Swarm, and the only goal was to run the x.py script, swarm would only be building nodes, rather than building and eventually running, since the CMD and ENTRYPOINT instructions only happen at run time.
Am I missing something?
The docker build command creates an immutable image. The docker run command creates a container that uses the image as a base filesystem, and other metadata from the image is used as defaults to run that image.
Each RUN line in a Dockerfile is used to add a layer to the image filesystem in docker. Docker actually performs that task in a temporary container, hence the selection of the confusing "run" term. The only thing preserved from that RUN command are the filesystem changes, running processes, changes to environment variables, shell settings like the current working directory, are all lost when the temporary container is cleaned up at the completion of the RUN command.
The ENTRYPOINT and CMD value are used to specify the default command to run when the container is started. When both are defined, the result is the value of the entrypoint is run with the value of the cmd appended as a command line argument. The value of CMD is easily overridden at the end of the docker run command line, so by using both you can get easy to reconfigure containers that run the same command with different user input parameters.
If the command you are trying to run needs to be performed every time the container starts, rather than being stored in the immutable image, then you need to perform that command in your ENTRYPOINT or CMD. This will add to the container startup time, so if the result of that command can be stored as a filesystem change and cached for all future containers being run, you want to make that setting in a RUN line.
If an image is currently being built out of a Dockerfile and I perform some changes on that Dockerfile itself, will it affect subsequent steps of the build?
Not until you re-run it. The first version of the Dockerfile would be in memory and have no awareness of your changes.
You should see the Dockerfile as the source code of your container (like C, C++ or Go) and see the docker image as a compiled binary, actually the docker image is just a binary file, so the Dockerfile matters only when you do a docker build -t containerName . command, after all the container is just a file that you can run or move to anywhere where you can do a docker run containerName.
So short answer it doesn't matter at all :)
Is it possible to add instructions like RUN in Dockerfile that, instead of run on docker build command, execute when a new container is created with docker run? I think this can be useful to initialize a volume attached to host file system.
Take a look at the ENTRYPOINT command. This specifies a command to run when the container starts, regardless of what someone provides as a command on the docker run command line. In fact, it is the job of the ENTRYPOINT script to interpret any command passed to docker run.
I think you are looking for the CMD
https://docs.docker.com/reference/builder/#cmd
The main purpose of a CMD is to provide defaults for an executing
container. These defaults can include an executable, or they can omit
the executable, in which case you must specify an ENTRYPOINT
instruction as well.
Note: don't confuse RUN with CMD. RUN actually runs a command and
commits the result; CMD does not execute anything at build time, but
specifies the intended command for the image.
You should also look into using Data Containers see this excellent Blog post.
Persistent volumes with Docker - Data-only container pattern
http://container42.com/2013/12/16/persistent-volumes-with-docker-container-as-volume-pattern/