My project structure:
projectRoot/
- service/
- data.json
- Dockerfile
In that Dockerfile :
FROM node:16.14-alpine3.14
ENV THE_DATA=/tmp/data.json
COPY data.json /tmp/data.json
Under project root, if I build the image:
docker build -t service:data_tmp -f service/Dockerfile .
I get error:
=> ERROR [2/2] COPY data.json /tmp/data.json
...
failed to compute cache key: "/data.json" not found: not found
I guess the error is due to the last . indicates build context is project root, that's why the data.json can't be located.
(My 2nd try) Then, I changed the Dockerfile to:
FROM node:16.14-alpine3.14
ENV THE_DATA=/tmp/data.json
COPY ./service/data.json /tmp/data.json
But get error:
=> ERROR [2/2] COPY ./service/data.json /tmp/data.json
...
failed to compute cache key: "/service/data.json" not found: not found
(My 3rd try successful) I managed to make it eventually build successfully by changing the build context to /service/:
docker build -t service:data_tmp -f service/Dockerfile /service/
But I don't get why my 2nd try above is not working? I mean in my 2nd try, even though build context is still . meaning current directory, meaning project root, then, the path COPY from ./service/data.json should be correct. Why I still get error there?
When files that exist on the filesystem are not found in a COPY step, check two things:
Your context, which was done here. That's the . at the end of the build command saying the context is the current directory. If you pass a different directory, that is the source for COPY steps (at least those that don't change the source with a --from).
A .dockerignore file. This is in the root of the context, and has a syntax similar to .gitignore. When changing the context, you change the location docker checks for the .dockerignore file.
A common pattern for a minimal docker build is to specify the .dockerignore file with:
*
!src
# ...
Which tells docker to exclude everything on the first line, and then reinclude src on the second line. You would add additional lines to reinclude the folder here with !service.
Related
I have a folder tmpdata with slightly different permissions at the top level of my project, which is used by a separate container to be able to restart with data.
I list this tmpdata/ folder in my .dockerignore file because I don't want it to be part of the build for the main app.
In my Dockerfile, I do have a COPY . . directive though. I thought it would copy everything except whatever's listed in .dockerignore, but when I try to build, I get:
failed to solve with frontend dockerfile.v0: failed to read dockerfile: error from sender: open tmpdata: permission denied
My question is, why isn't the .dockerignore file preventing the build process from even trying to touch this folder? Is there a way to have the COPY command respect (ignore) entries in the .dockerignore file?
UPDATE even after refactoring the dockerfile to use selective COPY commands that only copy needed files, docker build still fails, complaining about the same tmpdata folder.
UPDATE 2 my docker ignore file:
.dockerignore
docker-compose.yml
.env
.env.*
.git
test/
src/test/
**/*.spec.*
**/*.test.*
**/node_modules/
npm-debug.log
.gitignore
.cache
.DS_Store
.thumbsdb
dist/
_unused/
.vscode/
**/*.md
**/.git
tmpdata/
I am setting up a docker image, in the dockerfile I have an ADD command where source of the ADD command is a variable.
Dockerfile takes a build argument, I want to use that arg as source of the ADD command.
But ADD command is not expanding the variable and I get an error
Please share any workaround that comes in your mind
FROM ubuntu
ARG source_dir
RUN echo ${source_dir}
ADD ${source_dir} ./ContainerDir
Build command
docker build . -t image --build-arg source_dir=/home/john/Desktop/
data
Error
Step 3/3 : ADD ${source_dir} ./ContainerDir ADD failed: stat /var/lib/docker/tmp/docker-builder311119108/home/john/Desktop/
data: no such file or directory
However, the directory (/home/john/Desktop/
data) exists
From the error message, the variable expanded and complained that you don't have the path in your build context:
stat /var/lib/docker/tmp/docker-builder311119108/a/b/c: no such file or directory
In your example, the build context is . (the current directory) so you need a/b/c in the current directory for this to not error. That also need to not be in any ./.dockerignore file if you have one.
From your second edit:
docker build . -t image --build-arg source_dir=/home/john/Desktop/data
It looks like you are trying to include a directory inside your build from outside of the build context. That is explicitly not allowed in docker builds. All files needed for the ADD and COPY commands need to be included in your context, and the entire content of the context is sent to the build server in the first step, so you want to keep this small (rather than sending the entire home directory). The source is always relative to this context, so /home is looking for ./home since your context is . in the build command.
The fix is to move the data directory to be a sub directory of . where you are building your docker images. You can also switch to COPY since there is no functionality of ADD that you need.
Disclaimer: there are two pieces of over simplification here:
The COPY command can include files from different contexts using the --from option to COPY.
The entire context is sent before the build starts with the classic build command. The newer BuildKit implementation is much more selective about how much and what parts of the context to send.
I'm having some trouble building a docker image, because the way the code has been structured. The code is written in C#, and in a solution there is a lot of projects that "support" the application i want to build.
My problem is if i put the dockerfile into the root i can build it, without any problem, and it's okay but i don't think it's the optimal way, because we have some other dockerfiles we also need to build and if i put them all into the root folder i think it will end up messy.
So if i put the dockerfile into the folder with the application, how do i navigate into the root folder to grab the folders i need?
I tried with "../" but from my point of view it didn't seem to work. Is there any way to do it, or what is best practice in this scenario?
TL;DR
run it from the root directory:
docker build . -f ./path/to/dockerfile
the long answer:
in dockerfile you cant really go up.
why
when the docker daemon is building you image, it uses 2 parameters:
your Dockerfile
the context
the context is what you refer to as . in the dockerfile. (for example as COPY . /app)
both of them affect the final image - the dockerfile determines what is going to happen. the context tells docker on which files it should perform the operations you've specified in that dockerfile.
thats how the docs put it:
A build’s context is the set of files located in the
specified PATH or URL. The build process can refer to any of the files
in the context. For example, your build can use a COPY instruction to
reference a file in the context.
so, usually the context is the directory where the Dockerfile is placed. my suggestion is to leave it where it belongs. name your dockerfiles after their role (Dockerfile.dev,Dockerfile.prod, etc) thats ok to have a few of them in the same dir.
the context can still be changed:
after all, you are the one that specify the context. since the docker build command accepts the context and the dockerfile path. when i run:
docker build .
i am actually giving it the context of my current directory, (ive omitted the dockerfile path so it defaults to PATH/Dockerfile)
so if you have a dockerfile in dockerfiles/Dockerfile.dev, you shoul place youself in the directory you want as context, and you run:
docker build . -f dockerfiles/Dockerfile.dev
same applies to docker-compose build section (you specify there a context and the dockerfile path)
hope that made sense.
You can use RUN command and after & do whatever you want.
RUN cd ../ &
Dockerfile is failing on the following line:
ADD ./test-web-app/build/libs/test-web*.war /app/test-web.war
Error Step 8/29 : COPY ./test-web-app/build/libs/test-web*.war
/app/micro-service.war No source files were specified
This is the first time I am working on Docker builds. How do I debug this issue? Is there a way to echo if the host file is existing by a command ?
be sure that the path of the file is accessible where the Dockerfile is. When you run the build, the . folder is where the Dockerfile is. So you directory structure has to be something similar to this:
.
..
Dockerfile
test-web-app (folder)
To be sure that the war file is accessible try to list the file (on your host machine) for example.
$ ls ./test-web-app/build/libs/test-web*.war
I am trying to use docker hub to automatically build something that builds fine locally. It fails saying:
Build process failed: stat /var/lib/docker/aufs/mnt/1be9db483fa6f3de2596b5261e7c450de8df503185e579278396f14ba179c257/bin/run.sh: not a directory
You can view the build itself here:
https://hub.docker.com/r/zbyte64/rethinkdb-tlsproxy/builds/bjclhq33kgwxxvn6nbfsgyh/
run.sh is in the same directory as Dockerfile, it seems the build path on dockerhub is different then where it stores the Dockerfile.
I have tried the following variations:
COPY run.sh /bin
ADD ./run.sh /bin
The COPY command (on Dockerhub's Docker version) expects the target file on the right hand side, not just the target directory. The following command should work for you even on Dockerhub.
COPY run.sh /bin/run.sh
Or if you want to use ADD, include the trailing slash.
ADD ./run.sh /bin/
What is actually happening? From https://docs.docker.com/engine/reference/builder/#add :
ADD src dest
"If dest does not end with a trailing slash, it will be considered a regular file and the contents of src will be written at dest."
Without the trailing slash on /bin, it expects run.sh to be a directory being copied to directory /bin.
I don't know why, but dockerhub wants the first argument of COPY or ADD to be a directory - not a file. I am running Docker 1.9.1 locally and that is not the case. I switched the Dockerfile to copy a resource directory instead of individual files and things started to work.