Im running a VSTS build which contains a docker build task. I pass --no-cache argument in the build args field. Unfortunately, durring the build I receive a message that this arg was ignored. Did anyone had the same problem?
The reason of it is that, the docker build task add --no-cache argument after --build-arg, that can’t be consumed.
The workaround is that you can add additional arguments, such as test=test --no-cache (the warning will be [test] were not consumed.
On the other hand, you also can call docker build command through Command Line task.
adding --no-cache did not work for me
I added a marker in the Dockerfile before the COPY statement I did not want to be cached
FROM microsoft/azure-functions-dotnet-core2.0:2.0-nanoserver-1803
ARG CACHEBUSTER=0
COPY ./FunctionApp/bin/Release/netstandard2.0/Publish /approot
and then placed a RegEx Replace task before the docker build task and replace ARG CACHEBUSTER=0 with something unique e.g. ARG CACHEBUSTER=$(Build.BuildNumber)
Using "azure-pipelines.yml" in the Azure Build Pipelines fixes this problem:
script: docker build -t $(dockerId)/$(imageName) . # add options to this command to meet your needs
Build, test, and push Docker container apps in Azure Pipelines - Build an image
Example:
pool:
name: MarkusMeyer
demands:
- node.js
- Agent.OSVersion -equals 10.0.17134
variables:
imageName: 'your-container-image-name:$(build.buildId)'
steps:
- script: docker build --no-cache -f Dockerfile -t $(imageName) .
displayName: 'docker build'
Related
I want to just print HOMEPATH variable from my machine:
This is my Dockerfile:
FROM python:3.8-slim
ARG some_path=$HOMEPATH
RUN echo "this is some_path: $some_path"
but receiving empty string, during image creation - docker build -t test_image . :
What I am doing wrong ?
Docker doesn't expose the builder to this sort of security risk. The build environment could have sensitive data in environment variables that they do not want exposed when the build arbitrary docker images. To inject a variables during build, you must explicitly pass it on the build command line with:
docker build --build-arg "some_path=$HOMEPATH" -t test_image .
Note that the path on the build server is typically not something you would want to inject in a build. The same build should work regardless of where you perform the build.
The problem is i cannot get the docker build arg value in the shell script while running the docker build.
My docker build command:
DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain \
-t test \
--build-arg WHL_PATH=/fake/path \
.
Dockerfile
ARG WHL_PATH
FROM python:3.8.8
COPY test.sh .
RUN ./test.sh $WHL_PATH
and in the test.sh the "$1" is empty...., if in the Dockerfile i put some constant value then i will be able to see that value in the $1, but with docker build arg or set the build arg as ENV VAR are always empty...
Where am i doing wrong, how should i achieve this?
Docker version 20.10.5, build 55c4c88
Build args are scoped. Before the first FROM step they only apply to the FROM steps to adjust the image you use. Within each stage, an ARG step applies to the remaining steps within that stage. So the fix is to reorder your steps:
FROM python:3.8.8
COPY test.sh .
ARG WHL_PATH
RUN ./test.sh $WHL_PATH
Oops, i never realised the position of the ARG instruction matters, basically:
any ARG before the first FROM can be used in any FROM line
any ARG within a build stage (after a FROM) can be used in that build stage
After i moved the ARG WHL_PATH after the line FROM xxx it works perfectly, hope this can save some of your time in the future.
And i was inspired by this answer actually: https://stackoverflow.com/a/50292255/7658313
I received a project that is built with a number of Makefiles which run a number of docker commands. I want to build it on Cloud Build.
I know that Cloud Build handles Docker natively, but must work with the existing Makefile-structure.
I tried the make Custom Build step, but Cloud Build failed on lack of docker and kubectl. So I could add these to the Dockerfile of the make Custom Build step, but it seems like that is wrong, because then make runs Docker-in-Docker and this causes problems -- for example, gcloud and kubectl permissions and context are missing.
As a small part of one such a Makefile, here is a target that calls docker build and docker push.
build/myapp/deployer: ...
docker build \
--build-arg REGISTRY="$(REGISTRY)/myapp" \
--tag "$(APP_DEPLOYER_IMAGE)" \
-f deployer/Dockerfile \
.
docker push "$(APP_DEPLOYER_IMAGE)"
#touch "$#"
How do I run these Makefiles in Cloud Build?
To answer to your comment: "How to run docker inside a docker container?", you have to see your custom cloud builder like an runner. You can use it like this in Cloud Build
- steps:
- name: gcr.io/your-project/your-custom-builder
entrypoint: "bash"
args:
- "-c"
- |
Makefile
<All the script lines that you want which use binaries installed in your custom builder>
Here the custom Cloud Builder run the command but the result is stored in your Cloud Build workspace, not in the custom Cloud Builder container.
I have a bunch of Dockerfiles that are build from a common automated place using the same build command:
docker build -t $name:$tag --build-arg BRANCH=$branch .
Some of the Dockerfiles contain this:
ARG BRANCH=master
And that argument is used for some steps of the image build.
But for some Dockerfiles which doesn't need that argument I get this error at the end:
One or more build-args [BRANCH] were not consumed, failing build.
How can I overcome this problem without including the argument to all the Dockerfiles?
Have you considered grepping your Dockerfile for BRANCH and using it result to decide if you should supply your ARG or not?
You could replace your automation build trigger with something like:
if grep BRANCH Dockerfile; then docker build -t $name:$tag --build-arg BRANCH=$branch .; else docker build -t $name:$tag . ; fi
I don't see any documented way to avoid this error without changing your input or your Dockerfile. robertobado already covers changing your input. As a second option, you can include an effectively unused build arg at the end of your Dockerfile which would have a very minor impact on your build.
ARG BRANCH=undefined
RUN echo "Built from branch ${BRANCH}"
Since this doesn't modify the filesystem, I believe the image checksum will be identical.
I know docker has a --no-cache=true option to force a clean build of a docker image. For me however, all I'd really like to do is force the last step to run in my dockerfile, which is a CMD command that runs a shell script.
For whatever reason, when I modify that script and save it, a typical docker build will reuse the cached version of that step. Is there a way to force docker not to do so, just on that one portion?
Note that this would invalidate the cache for all Dockerfile directives after that line. This is requested in Issue 1996 (not yet implemented, and now (2021) closed), and issue 42799 (mentioned by ub-marco in the comments).
The current workaround is:
FROM foo
ARG CACHE_DATE=2016-01-01
<your command without cache>
docker build --build-arg CACHE_DATE=$(date) ....
That would invalidate cache after the ARG CACHE_DATE line for every build.
acdcjunior reports in the comments having to use:
docker build --build-arg CACHE_DATE=$(date +%Y-%m-%d_%H:%M:%S)
Another workaround from azul:
Here's what I am using to rebuild in CI if changes in git happened:
export LAST_SERVER_COMMIT=`git ls-remote $REPO "refs/heads/$BRANCH" | grep -o "^\S\+"`
docker build --build-arg LAST_SERVER_COMMIT="$LAST_SERVER_COMMIT"
And then in the Dockerfile:
ARG LAST_SERVER_COMMIT
RUN git clone ...
This will only rebuild the following layers if the git repo actually changed.