I have a folder tmpdata with slightly different permissions at the top level of my project, which is used by a separate container to be able to restart with data.
I list this tmpdata/ folder in my .dockerignore file because I don't want it to be part of the build for the main app.
In my Dockerfile, I do have a COPY . . directive though. I thought it would copy everything except whatever's listed in .dockerignore, but when I try to build, I get:
failed to solve with frontend dockerfile.v0: failed to read dockerfile: error from sender: open tmpdata: permission denied
My question is, why isn't the .dockerignore file preventing the build process from even trying to touch this folder? Is there a way to have the COPY command respect (ignore) entries in the .dockerignore file?
UPDATE even after refactoring the dockerfile to use selective COPY commands that only copy needed files, docker build still fails, complaining about the same tmpdata folder.
UPDATE 2 my docker ignore file:
.dockerignore
docker-compose.yml
.env
.env.*
.git
test/
src/test/
**/*.spec.*
**/*.test.*
**/node_modules/
npm-debug.log
.gitignore
.cache
.DS_Store
.thumbsdb
dist/
_unused/
.vscode/
**/*.md
**/.git
tmpdata/
Related
My project structure:
projectRoot/
- service/
- data.json
- Dockerfile
In that Dockerfile :
FROM node:16.14-alpine3.14
ENV THE_DATA=/tmp/data.json
COPY data.json /tmp/data.json
Under project root, if I build the image:
docker build -t service:data_tmp -f service/Dockerfile .
I get error:
=> ERROR [2/2] COPY data.json /tmp/data.json
...
failed to compute cache key: "/data.json" not found: not found
I guess the error is due to the last . indicates build context is project root, that's why the data.json can't be located.
(My 2nd try) Then, I changed the Dockerfile to:
FROM node:16.14-alpine3.14
ENV THE_DATA=/tmp/data.json
COPY ./service/data.json /tmp/data.json
But get error:
=> ERROR [2/2] COPY ./service/data.json /tmp/data.json
...
failed to compute cache key: "/service/data.json" not found: not found
(My 3rd try successful) I managed to make it eventually build successfully by changing the build context to /service/:
docker build -t service:data_tmp -f service/Dockerfile /service/
But I don't get why my 2nd try above is not working? I mean in my 2nd try, even though build context is still . meaning current directory, meaning project root, then, the path COPY from ./service/data.json should be correct. Why I still get error there?
When files that exist on the filesystem are not found in a COPY step, check two things:
Your context, which was done here. That's the . at the end of the build command saying the context is the current directory. If you pass a different directory, that is the source for COPY steps (at least those that don't change the source with a --from).
A .dockerignore file. This is in the root of the context, and has a syntax similar to .gitignore. When changing the context, you change the location docker checks for the .dockerignore file.
A common pattern for a minimal docker build is to specify the .dockerignore file with:
*
!src
# ...
Which tells docker to exclude everything on the first line, and then reinclude src on the second line. You would add additional lines to reinclude the folder here with !service.
I am trying to replace some files in a folder in a docker image. I am using the following command inside Dockerfile:
COPY /home/adietz/Work/20_BSP/Jenkins/venvs/linux_selenium/lib/python2.7/site-packages/browsermobproxy /usr/local/lib/python2.7/dist-packages/browsermobproxy
which results in an error
Step 4/12 : COPY /home/adietz/Work/20_BSP/Jenkins/venvs/linux_selenium/lib/python2.7/site-packages/browsermobproxy /usr/local/lib/python2.7/dist-packages/browsermobproxy
lstat home/adietz/Work/20_BSP/Jenkins/venvs/linux_selenium/lib/python2.7/site-packages/browsermobproxy: no such file or directory
Replacing COPY with ADD results in the same error. Also the following command
COPY /home/adietz/Work/20_BSP/Jenkins/venvs/linux_selenium/lib/python2.7/site-packages/browsermobproxy /usr/local/lib/python2.7/dist-packages/
and gives identical(!) error.
Both paths are folders. The folder in the docker image already exists; I just want to replace the files.
What am I doing wrong here...?
It seems you cannot use absolute paths in the COPY command AND you can only copy files which are inside the folder you are running the docker command.
So to copy these files you have to do e.g. the following steps
cp -r /home/adietz/Work/20_BSP/Jenkins/venvs/linux_selenium/lib/python2.7/site-packages/browsermobproxy .
and then add to the Dockerfile:
COPY browsermobproxy/ /usr/local/lib/python2.7/dist-packages/
A symbolic link also does not work...
I had this same issue and realized that COPY or ADD won't work with node_modules that are referenced instead of directly installed into the project. When I switched this it worked for me.
Is it possible to copy multiple files to different locations in a Dockerfile?
I'm looking to go from:
COPY outputs/output/build/.tools /root/.tools
COPY outputs/output/build/configuration /root/configuration
COPY outputs/output/build/database/postgres /root/database/postgres
I have tried the following, but no joy:
COPY ["outputs/output/build/.tools /root/.tools","outputs/output/build/configuration /root/configuration","outputs/output/build/database/postgres /root/database/postgres"]
Not sure if this is even possible.
Create a file .dockerignore in your docker build context directory. Create a soft link (ln) of the root directory you want to copy and exclude the directories you don't want in your .dockerignore.
In your case, you don't need the soft link as the directory is already in the docker build context. Now, add the directories in the .dockerignore file that you don't want eg. if you don't want bin directory you can do that as,
# this is .dockerignore file
outputs/output/build/bin*
Finally in your Dockerfile,
COPY outputs/output/build/ /root/
Details are here.
it looks like you are trying to do bash style [] brace expansion
RUN command uses sh NOT bash
see this SO article discussing it
Bash brace expansion not working on Dockerfile RUN command
or the docker reference
https://docs.docker.com/engine/reference/builder/#/run
I have to create a docker file that copies MyApp directory into the image.
MyApp
-libs
-classes
-resources
Libs directory has around 50 MB and it is not frequently changing where as Classes directory is around 1 MB and it is subjected to frequent changes. To optimize the docker build, I planned to add Libs directory at the beginning of the Dockerfile and add other directories at the end of the Dockerfile. My current approach is like this
ADD MyApp/libs /opt/MyApp/libs
## do other operations
ADD MyApp/classes /opt/MyApp/resources
ADD MyApp/classes /opt/MyApp/classes
This is not a maintainable format as in future I may have some other directories in the MyApp directory to be copied into the docker image. My target is to write a docker file like this
ADD MyApp/libs /opt/MyApp/libs
## do other operations
ADD MyApp -exclude MyApp/libs /opt/MyApp
Is there a similar command to exclude some files in a directory which is copied into the docker image?
I considered the method explained by #nwinkler and added few steps to make the build consistent.
Now my context directory structure is as follows
-Dockerfile
-MyApp
-libs
-classes
-resources
-.dockerignore
-libs
I copied the libs directory to the outer of the MyApp directory. Added a .dockerignore file which contains following line
MyApp/libs/*
Updated the Dockerfile as this
ADD libs /opt/MyApp/libs
## do other operations
ADD MyApp /opt/MyApp
Because dockerignore file ignores MyApp/lib directory, there is no risk in over-writing libs directory I have copied earlier.
This is not possible out of the box with the current version of Docker. Using the .dockerignore file will also not work, since it would always exclude the libs folder.
What you can do is wrapping your docker build in a shell script and copy the MyApp folder (minus the libs folder), and the libs folder into temporary directories before calling docker build.
Something like this:
#!/usr/bin/env bash
rm -rf temp
mkdir -p temp
# Copy the whole folder
cp -r MyApp temp
# Move the libs folder up one level
mv temp/MyApp/libs temp
# Now build the Docker image
docker build ...
Then you could change your Dockerfile to copy from the temp directory:
ADD temp/libs /opt/MyApp/libs
## do other operations
ADD temp/MyApp /opt/MyApp
There's a risk that the second ADD command will remove the /opt/MyApp/libs folder from the image. If that happens, you might have to reverse the ADD commands and add the libs folder after everything else.
I am trying to use docker hub to automatically build something that builds fine locally. It fails saying:
Build process failed: stat /var/lib/docker/aufs/mnt/1be9db483fa6f3de2596b5261e7c450de8df503185e579278396f14ba179c257/bin/run.sh: not a directory
You can view the build itself here:
https://hub.docker.com/r/zbyte64/rethinkdb-tlsproxy/builds/bjclhq33kgwxxvn6nbfsgyh/
run.sh is in the same directory as Dockerfile, it seems the build path on dockerhub is different then where it stores the Dockerfile.
I have tried the following variations:
COPY run.sh /bin
ADD ./run.sh /bin
The COPY command (on Dockerhub's Docker version) expects the target file on the right hand side, not just the target directory. The following command should work for you even on Dockerhub.
COPY run.sh /bin/run.sh
Or if you want to use ADD, include the trailing slash.
ADD ./run.sh /bin/
What is actually happening? From https://docs.docker.com/engine/reference/builder/#add :
ADD src dest
"If dest does not end with a trailing slash, it will be considered a regular file and the contents of src will be written at dest."
Without the trailing slash on /bin, it expects run.sh to be a directory being copied to directory /bin.
I don't know why, but dockerhub wants the first argument of COPY or ADD to be a directory - not a file. I am running Docker 1.9.1 locally and that is not the case. I switched the Dockerfile to copy a resource directory instead of individual files and things started to work.