COPY/ADD command in Dockerfile is not copying .gitignored folder - docker

I have a folder called public
which is gitignored.
I have this Dockerfile
FROM nginx
ADD public /usr/share/nginx/public/
# COPY public /usr/share/nginx/public/
COPY nginx.conf /etc/nginx/conf.d/default.conf
Now when I create image, I get this error
Step 2/3 : COPY public /usr/share/nginx/public/
COPY failed: file not found in build context or excluded by .dockerignore: stat public: file does not exist
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1
I figured out if I exclude public folder from gitignore then all is good.
I checked there is no .dockerignore file.
Just make sure if it's fallback issue, I created a new .dockerignore file with empty content

Finally figure it out, I was using gcloud build command to create docker file, and seems google by default ignores gitignore files.
https://cloud.google.com/sdk/gcloud/reference/topic/gcloudignore
so I created a .gcloudignore file with below content
*
!public

Related

Docker build context puzzle

My project structure:
projectRoot/
- service/
- data.json
- Dockerfile
In that Dockerfile :
FROM node:16.14-alpine3.14
ENV THE_DATA=/tmp/data.json
COPY data.json /tmp/data.json
Under project root, if I build the image:
docker build -t service:data_tmp -f service/Dockerfile .
I get error:
=> ERROR [2/2] COPY data.json /tmp/data.json
...
failed to compute cache key: "/data.json" not found: not found
I guess the error is due to the last . indicates build context is project root, that's why the data.json can't be located.
(My 2nd try) Then, I changed the Dockerfile to:
FROM node:16.14-alpine3.14
ENV THE_DATA=/tmp/data.json
COPY ./service/data.json /tmp/data.json
But get error:
=> ERROR [2/2] COPY ./service/data.json /tmp/data.json
...
failed to compute cache key: "/service/data.json" not found: not found
(My 3rd try successful) I managed to make it eventually build successfully by changing the build context to /service/:
docker build -t service:data_tmp -f service/Dockerfile /service/
But I don't get why my 2nd try above is not working? I mean in my 2nd try, even though build context is still . meaning current directory, meaning project root, then, the path COPY from ./service/data.json should be correct. Why I still get error there?
When files that exist on the filesystem are not found in a COPY step, check two things:
Your context, which was done here. That's the . at the end of the build command saying the context is the current directory. If you pass a different directory, that is the source for COPY steps (at least those that don't change the source with a --from).
A .dockerignore file. This is in the root of the context, and has a syntax similar to .gitignore. When changing the context, you change the location docker checks for the .dockerignore file.
A common pattern for a minimal docker build is to specify the .dockerignore file with:
*
!src
# ...
Which tells docker to exclude everything on the first line, and then reinclude src on the second line. You would add additional lines to reinclude the folder here with !service.

Docker COPY cmd that respects .dockerignore?

I have a folder tmpdata with slightly different permissions at the top level of my project, which is used by a separate container to be able to restart with data.
I list this tmpdata/ folder in my .dockerignore file because I don't want it to be part of the build for the main app.
In my Dockerfile, I do have a COPY . . directive though. I thought it would copy everything except whatever's listed in .dockerignore, but when I try to build, I get:
failed to solve with frontend dockerfile.v0: failed to read dockerfile: error from sender: open tmpdata: permission denied
My question is, why isn't the .dockerignore file preventing the build process from even trying to touch this folder? Is there a way to have the COPY command respect (ignore) entries in the .dockerignore file?
UPDATE even after refactoring the dockerfile to use selective COPY commands that only copy needed files, docker build still fails, complaining about the same tmpdata folder.
UPDATE 2 my docker ignore file:
.dockerignore
docker-compose.yml
.env
.env.*
.git
test/
src/test/
**/*.spec.*
**/*.test.*
**/node_modules/
npm-debug.log
.gitignore
.cache
.DS_Store
.thumbsdb
dist/
_unused/
.vscode/
**/*.md
**/.git
tmpdata/

Docker ADD giving error "No source files were specified"

Dockerfile is failing on the following line:
ADD ./test-web-app/build/libs/test-web*.war /app/test-web.war
Error Step 8/29 : COPY ./test-web-app/build/libs/test-web*.war
/app/micro-service.war No source files were specified
This is the first time I am working on Docker builds. How do I debug this issue? Is there a way to echo if the host file is existing by a command ?
be sure that the path of the file is accessible where the Dockerfile is. When you run the build, the . folder is where the Dockerfile is. So you directory structure has to be something similar to this:
.
..
Dockerfile
test-web-app (folder)
To be sure that the war file is accessible try to list the file (on your host machine) for example.
$ ls ./test-web-app/build/libs/test-web*.war

Unable to replace files in docker image with ADD or COPY

I am trying to replace some files in a folder in a docker image. I am using the following command inside Dockerfile:
COPY /home/adietz/Work/20_BSP/Jenkins/venvs/linux_selenium/lib/python2.7/site-packages/browsermobproxy /usr/local/lib/python2.7/dist-packages/browsermobproxy
which results in an error
Step 4/12 : COPY /home/adietz/Work/20_BSP/Jenkins/venvs/linux_selenium/lib/python2.7/site-packages/browsermobproxy /usr/local/lib/python2.7/dist-packages/browsermobproxy
lstat home/adietz/Work/20_BSP/Jenkins/venvs/linux_selenium/lib/python2.7/site-packages/browsermobproxy: no such file or directory
Replacing COPY with ADD results in the same error. Also the following command
COPY /home/adietz/Work/20_BSP/Jenkins/venvs/linux_selenium/lib/python2.7/site-packages/browsermobproxy /usr/local/lib/python2.7/dist-packages/
and gives identical(!) error.
Both paths are folders. The folder in the docker image already exists; I just want to replace the files.
What am I doing wrong here...?
It seems you cannot use absolute paths in the COPY command AND you can only copy files which are inside the folder you are running the docker command.
So to copy these files you have to do e.g. the following steps
cp -r /home/adietz/Work/20_BSP/Jenkins/venvs/linux_selenium/lib/python2.7/site-packages/browsermobproxy .
and then add to the Dockerfile:
COPY browsermobproxy/ /usr/local/lib/python2.7/dist-packages/
A symbolic link also does not work...
I had this same issue and realized that COPY or ADD won't work with node_modules that are referenced instead of directly installed into the project. When I switched this it worked for me.

Dockerhub automated build fails, "not a directory" when adding file

I am trying to use docker hub to automatically build something that builds fine locally. It fails saying:
Build process failed: stat /var/lib/docker/aufs/mnt/1be9db483fa6f3de2596b5261e7c450de8df503185e579278396f14ba179c257/bin/run.sh: not a directory
You can view the build itself here:
https://hub.docker.com/r/zbyte64/rethinkdb-tlsproxy/builds/bjclhq33kgwxxvn6nbfsgyh/
run.sh is in the same directory as Dockerfile, it seems the build path on dockerhub is different then where it stores the Dockerfile.
I have tried the following variations:
COPY run.sh /bin
ADD ./run.sh /bin
The COPY command (on Dockerhub's Docker version) expects the target file on the right hand side, not just the target directory. The following command should work for you even on Dockerhub.
COPY run.sh /bin/run.sh
Or if you want to use ADD, include the trailing slash.
ADD ./run.sh /bin/
What is actually happening? From https://docs.docker.com/engine/reference/builder/#add :
ADD src dest
"If dest does not end with a trailing slash, it will be considered a regular file and the contents of src will be written at dest."
Without the trailing slash on /bin, it expects run.sh to be a directory being copied to directory /bin.
I don't know why, but dockerhub wants the first argument of COPY or ADD to be a directory - not a file. I am running Docker 1.9.1 locally and that is not the case. I switched the Dockerfile to copy a resource directory instead of individual files and things started to work.

Resources