I would like to pass the coverage report to codecov during docker build phase. Like this:
ci_env=`bash <(curl -s https://codecov.io/env)`
docker build . --build-arg ci_env=$ci_env
This is currently not possible since ci_env is a list of environment variables.
In the documentation they describe how to achieve it with docker run but not with docker build.
Does anyone know how to do this?
Related
My docker repo has various folders like mongo,maven each has separate dockerfiles.
I need to execute the jenkins pipeline for all the folders ie dockerfiles...
My current setup is using parameter i am selecting individual folder like maven to build the maven image.
How to build all the dockerfiles?
I think a simple shell script should do what you need. Get all folders and execute docker build there. For example something like:
ls -d */ | xargs -I {} bash -c "cd '{}' && docker build ."
I Have a dockerfile and in one of last steps, I download a WAR file from artifactory, so I can use it in the containers in the webapps/ directory.
Thing is I don´t want to show user and pass of curl -u command. How can I hide both users and password in following command? Is there a way in docker to hide/encrypt passwords?
RUN curl -u user:pass -O "https://artifactory.xxxx.com:443/artifactory/api/api-0.0.1-SNAPSHOT.war"
You can use multi stage build to achieve a lightweight image, but you have to use one single docker build, instead of two. Like this single Dockerfile:
FROM maven as build
(... Your app build....)
FROM tomcat
COPY --from=build artifact.war /dest/dir
Everything before the second FROM is discarded from the resulting image, so it will contain Tomcat, not Maven, and your copied artifact.
You can't hide the contents of RUN commands from docker history.
You should download the artifact outside the Dockerfile. A script that did the curl command and then ran docker build might simplify things for you. In addition to not exposing the credentials via docker history, this also avoids committing them to source control, makes it possible to build the image even if your Artifactory is unreachable, and simplifies a developer-oriented sequence where they need to build a temporary image out of something they've built themselves.
According to the documentation, it could be done with this new BuildKit feature RUN --mount=type=secret:
# syntax=docker/dockerfile:1.2
FROM ubuntu:22.04
RUN --mount=type=secret,id=creds,target=/tmp/artifactory.credentials \
curl -k /tmp/artifactory.credentials -O <URL>
# REST of the DockerFile
Then you can build this image like this:
docker build -secret id=creds,src=$HOME/creds/artifactory.curl .
Content of the $HOME/creds/artifactory.curl:
-u USER:PASS
For additional info on -k, please follow the official documentation
The official cUrl documentation says -K not -k
I have a bunch of Dockerfiles that are build from a common automated place using the same build command:
docker build -t $name:$tag --build-arg BRANCH=$branch .
Some of the Dockerfiles contain this:
ARG BRANCH=master
And that argument is used for some steps of the image build.
But for some Dockerfiles which doesn't need that argument I get this error at the end:
One or more build-args [BRANCH] were not consumed, failing build.
How can I overcome this problem without including the argument to all the Dockerfiles?
Have you considered grepping your Dockerfile for BRANCH and using it result to decide if you should supply your ARG or not?
You could replace your automation build trigger with something like:
if grep BRANCH Dockerfile; then docker build -t $name:$tag --build-arg BRANCH=$branch .; else docker build -t $name:$tag . ; fi
I don't see any documented way to avoid this error without changing your input or your Dockerfile. robertobado already covers changing your input. As a second option, you can include an effectively unused build arg at the end of your Dockerfile which would have a very minor impact on your build.
ARG BRANCH=undefined
RUN echo "Built from branch ${BRANCH}"
Since this doesn't modify the filesystem, I believe the image checksum will be identical.
I know docker has a --no-cache=true option to force a clean build of a docker image. For me however, all I'd really like to do is force the last step to run in my dockerfile, which is a CMD command that runs a shell script.
For whatever reason, when I modify that script and save it, a typical docker build will reuse the cached version of that step. Is there a way to force docker not to do so, just on that one portion?
Note that this would invalidate the cache for all Dockerfile directives after that line. This is requested in Issue 1996 (not yet implemented, and now (2021) closed), and issue 42799 (mentioned by ub-marco in the comments).
The current workaround is:
FROM foo
ARG CACHE_DATE=2016-01-01
<your command without cache>
docker build --build-arg CACHE_DATE=$(date) ....
That would invalidate cache after the ARG CACHE_DATE line for every build.
acdcjunior reports in the comments having to use:
docker build --build-arg CACHE_DATE=$(date +%Y-%m-%d_%H:%M:%S)
Another workaround from azul:
Here's what I am using to rebuild in CI if changes in git happened:
export LAST_SERVER_COMMIT=`git ls-remote $REPO "refs/heads/$BRANCH" | grep -o "^\S\+"`
docker build --build-arg LAST_SERVER_COMMIT="$LAST_SERVER_COMMIT"
And then in the Dockerfile:
ARG LAST_SERVER_COMMIT
RUN git clone ...
This will only rebuild the following layers if the git repo actually changed.
My specific use case is that I want to organize some data about the EC2 instance a container is running on and make i available as an environment variable. I'd like to do this when the container is built.
I was hoping to be able to do something like ENV VAR_NAME $(./script/that/gets/var) in my Dockerfile, but unsurprisingly that does not work (you just get the string $(./script...).
I should mention that I know the docker run --env... will do this, but I specifically want it to be built into the container.
Am I missing something obvious? Is this even possible?
Docker v1.9 or newer
If you are using Docker v1.9 or newer, this is possible via support for build time arguments. Arguments are declared in the Dockerfile by using the ARG statement.
ARG REQUIRED_ARGUMENT
ARG OPTIONAL_ARGUMENT=default_value
When you later actually build your image using docker build you can pass arguments via the flag --build-arg as described in the docker docs.
$ docker build --build-arg REQUIRED_ARGUMENT=this-is-required .
Please note that it is not recommended to use build-time variables for passwords or secrets such as keys or credentials.
Furthermore, build-time variables may have great impact on caching. Therefore the Dockerfile should be constructed with great care to be able to utilize caching as much as possible and therein speed up the building process.
Edit: the "docker newer than v1.9"-part was added after input from leedm777:s answer.
Docker before v1.9
If you are using a Docker-version before 1.9, the ARG/--build-arg approach was not possible. You couldn't resolve this kind of info during the build so you had to pass them as parameters to the docker run command.
Docker images are to be consistent over time whereas containers can be tweaked and considered as "throw away processes".
More info about ENV
A docker discussion about dynamic builds
The old solution to this problem was to use templating. This is not a neat solution but was one of very few viable options at the time. (Inspiration from this discussion).
save all your dynamic data in a json or yaml file
create a docker file "template" where the dynamic can later be expanded
write a script that creates a Dockerfile from the config data using some templating library that you are familiar with
Docker 1.9 has added support for build time arguments.
In your Dockerfile, you add an ARG statement, which has a similar syntax to ENV.
ARG FOO_REQUIRED
ARG BAR_OPTIONAL=something
At build time, you can pass pass a --build-arg argument to set the argument for that build. Any ARG that was not given a default value in the Dockerfile must be specified.
$ docker build --build-arg FOO_REQUIRED=best-foo-ever .
To build ENV VAR_NAME $(./script/that/gets/var) into the container, create a dynamic Dockerfile at build time:
$ docker build -t awesome -f Dockerfile .
$ # get VAR_NAME value:
$ VAR_VALUE=`docker run --rm awesome \
bash -c 'echo $(./script/that/gets/var)'`
$ # use dynamic Dockerfile:
$ {
echo "FROM awesome"
echo "ENV VAR_NAME $VAR_VALUE"
} | docker build -t awesome -
https://github.com/42ua/docker-autobuild/blob/master/emscripten-sdk/README.md#build-docker-image