I have a project that need an sdk to be compiled.
This sdk is shared on a server with other users as everyone need it. (It is located under /opt/my-sdk.tar.gz)
My dockerfile lines are :
RUN mkdir /sdk
COPY /opt/my-sdk.tar.gz /sdk
RUN cd /sdk && tar xvf my-sdk.tar.gz
Then I type in my dockerfile directory :
docker build -t myImage .
But this error appears :
COPY failed: file not found in build context or excluded by .dockerignore: stat opt/my-sdk.tar.gz: file does not exist
I think that my docker context doesn't know about /opt but how to solve this as my competences are limited.
Your docker file should be like this:
RUN mkdir /sdk
COPY my-sdk.tar.gz /sdk
WORKDIR /sdk
RUN tar xvf my-sdk.tar.gz
But, before you build this, my-sdk.tar.gz has to be in the same path with Dockerfile. You can not copy things from any other locations
Related
I have a dockerfile
FROM myregistry.de/public/12/s11
LABEL maintainer="Me"
COPY ./somefile.txt /opt
CMD /bin/sh
the file I want to copy to Docker Container is in the same folder as the Dockerfile
But when I do docker build -t 'name:tag4' .
I get the error message :
COPY failed: file not found in build context or excluded by .dockerignore: stat somefile.txt: **file does not exist**
So why is this simple copy not working ?
Please check what you get in build context directory by creating another Dockerfile with the following content:
FROM myregistry.de/public/12/s11
RUN mkdir /tmp/build
COPY . /tmp/build
CMD /bin/sh
Then build and run the container and check /tmp/build folder contents.
I want to copy the file to a folder location that is outside the working directory. I used the following lines in my docker file, but the files are not there when I look in the container.
WORKDIR /app
RUN cd ../opt/venv/lib/python3.7/site-packages/xxx/
COPY ./resources/abc.py .
When look a that /opt/venv/lib/python3.7/site-packages/xxx/ location the abc.py is not there
What is the issue with my approach? Appreciate your inputs.
You can't COPY a file from outside the build context. So if you are trying to COPY /opt/venv/lib/python3.7/site-packages/xxx/resources/abc.py into your docker image, and that is not in your build context, it will fail. Full stop.
Here's some annotated code.
# change to the /app directory in the container
WORKDIR /app
# run the command cd in the container. cd is a shell builtin, and after
# this command finishes you will still be inside the /app directory in
# your container.
RUN cd ../opt/venv/lib/python3.7/site-packages/xxx/
# Attempt to copy ./resources/abc.py from your host's build context
# (NOT /opt/venv/lib/python3.7/site-packages/xxx/) into the container.
COPY ./resources/abc.py .
The basic fix for this is to first copy abc.py into your build directory. Then you will be able to copy it into your docker container during your build like so:
WORKDIR /app
COPY abc.py .
# /app/abc.py now exists in your container
Note on cd
cd is a shell builtin that changes the working directory of the shell. When you execute it inside a script (or in this case a docker RUN) it only changes the working directory for that process, which ends when the script ends. After which your working directory will be the one you started in. So you cannot use it in the way you were intending. Much better explanation here.
Take this Dockerfile for example:
FROM alpine:latest
RUN cd /opt # cd to /opt
RUN pwd # check current directory, you're STILL in '/'
RUN cd /opt && \
pwd # works as expected because you're still in the same process that cd ran in.
# But once you exit this RUN command you will be back in '/'
# Use WORKDIR to set the working directory in a dockerfile
Here's the output of building that Dockerfile (removed noisy docker output):
$ docker build --no-cache .
Sending build context to Docker daemon 3.584kB
Step 1/4 : FROM alpine:latest
Step 2/4 : RUN cd /opt
Step 3/4 : RUN pwd
/
Step 4/4 : RUN cd /opt && pwd
/opt
From what I understand, you're trying to copy a file into a specific location (/opt/venv/lib/python3.7/site-packages/xxx/) in your Docker image that is outside the WORKDIR you defined in the Dockerfile for your image.
You can easily do this by specifying the absolute destination path in the COPY command:
WORKDIR /app
COPY ./resources/abc.py /opt/venv/lib/python3.7/site-packages/xxx/abc.py
RUN adduser -D appUser
RUN mkdir /usr/share/app
RUN mkdir /logs
ADD Function/target/foo.jar /usr/share/app
WORKDIR /usr/share/app
RUN chown -R appUser /usr/share/app
RUN chown -R appUser /logs
USER appUser
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "foo.jar"]`
I've got this weird issue I can't seem to work my head around.
My root folder contains two directories, (both with subdirectories) 'Demo/Dockerfile' and 'Function/target/foo.jar'
I have a copy command in my Dockerfile that reads
COPY Function/target/foo.bar /usr/share/app
but when I run docker build -f Demo/Dockerfile from the root folder, I get an error
stat /var/lib/docker/tmp/docker-builder238823934/Function/target/foo.jar: no such file or directory
I find this a bit strange because when I edit the copy command to read COPY /target/foo.bar /usr/share/app and then I cd into the Function directory and run
docker build -f ../Demo/Dockerfile
it builds successfully, or if I edit the Dockerfile to read COPY foo.bar /usr/share/app and then cd into the target directory and run docker build -f ../../Demo/Dockerfile, this also works.
Is there an explanation for this sort of behavior?
This is what my dockerignore file looks like
!**/Dockerfile
!DockerServiceDescription/**
!Function/target/*.war
!server.xml
!tomcat-users.xml
Docker uses context directory and children only and does not allow using any files outside for security reasons.
You should show context directory to docker using '.' or so:
cd myproject
docker build -f Demo/Dockerfile .
below is the content of the dockerfile
FROM golang:1.8 as goimage
ENV SRC=/go/src/
RUN mkdir -p /go/src/
RUN mkdir /go/src/go_docker
WORKDIR /go/src/go_docker
RUN cd /go/src/go_docker
COPY StoreImage.go .
RUN go build -o /bin/go_docker
CMD ["/bin/go_docker"]
Docker build is successful for the above content.. But dont see the binary file generated in /bin/go_docker
Can someone please help me with this.
Make sure StoreImage.go can copy into docker
Binary file is created in /bin location and go_docker is the created binary file.
.exe will not be the extension in linux whereas .exe will be seen only in windows
I've got a repo set up like this:
/config
config.json
/worker-a
Dockerfile
<symlink to config.json>
/code
/worker-b
Dockerfile
<symlink to config.json>
/code
However, building the images fails, because Docker can't handle the symlinks. I should mention my project is far more complicated than this, so restructuring directories isn't a great option. How do I deal with this situation?
Docker doesn't support symlinking files outside the build context.
Here are some different methods for using a shared file in a container:
Build Time
Copy from a config image (Docker buildkit)
Recent versions of Docker allow RUN steps to bind mount from a named image or previous build stage with the --mount=type=bind,target=/dir,source=/dir,from=image-or-stage-name
Create a Dockerfile for the base me/worker-config image that includes the shared config/files.
FROM scratch
COPY config.json /config.json
Build and tag the config image me/worker-config
docker build -t me/worker-config:latest .
Mount the me/worker-config image during the real build
RUN --mount=type=bind,target=/worker-config,source=/,from=me/worker-config:latest \
cp /worker-config/config.json /app/config.json;
Share a base image
Create a Dockerfile for the base me/worker-config image that includes the shared config/files.
COPY config.json /config.json
Build and tag the image me/worker-config
docker build -t me/worker-config:latest .
Source the base me/worker-config image for all your worker Dockerfiles
FROM me/worker-config:latest
Build script
Use a script to push the common config to each of your worker containers.
./build worker-n
#!/bin/sh
set -uex
rundir=$(readlink -f "${0%/*}")
container=$(shift)
cd "$rundir/$container"
cp ../config/config.json ./config-docker.json
docker build "$#" .
Build from URL
Pull the config from a common URL for all worker-n builds.
ADD http://somehost/config.json /
Increase the scope of the image build context
Include the symlink target files in the build context by building from a parent directory that includes both the shared files and specific container files.
cd ..
docker build -f worker-a/Dockerfile .
All the source paths you reference in a Dockerfile must also change to match the new build context:
COPY workerathing /app
becomes
COPY worker-a/workerathing /app
Using this method can make all build contexts large if you have one large build context, as they all become shared. It can slow down builds, especially to remote Docker build servers. Note that only the .dockerignore file from the base of the build context is referenced.
Alternate build that can mount volumes
Other projects that strive for Dockerfile compatibility may support volumes at build time. For example a podman build / buildah support a --volume option to bind mount files from the host into a build container.
podman build --volume /project/config:/worker-config:ro,Z -t me/worker-a .
Then the build can reference the mounted volume
COPY /worker-config/config.json /app
Run time
Mount a config directory from a named volume
Volumes like this only work as directories, so you can't specify a file like you could when mounting a file from the host to container.
docker volume create --name=worker-cfg-vol
docker run -v worker-cfg-vol:/config worker-config cp config.json /config
docker run -v worker-cfg-vol:/config:/config worker-a
Mount config directory from data container
Again, directories only as it's basically the same as above. This will automatically copy files from the destination directory into the newly created shared volume though.
docker create --name wcc -v /config worker-config /bin/true
docker run --volumes-from wcc worker-a
Mount config file from host at runtime
docker run -v /app/config/config.json:/config.json worker-a
Node.js-specific solution
I also ran into this problem, and would like to share another method that hasn't been mentioned above. Instead of using npm link in my Dockerfile, I used yalc.
Install yalc in your container, e.g. RUN npm i -g yalc.
Build your library in Docker, and run yalc publish (add the --private flag if your shared lib is private). This will 'publish' your library locally.
Run yalc add my-lib in each repo that would normally use npm link before running npm install. It will create a local .yalc folder in your Docker container, create a symlink in node_modules that works inside Docker to this folder, and rewrite your package.json to refer to this folder too, so you can safely run install.
Optionally, if you do a two stage build, make sure that you also copy the .yalc folder to your final image.
Below an example Dockerfile, assuming you have a mono repository with three packages: models, gui and server, and the models repository must be shared and named my-models.
# You can access the container using:
# docker run -it my-name sh
# To start it stand-alone:
# docker run -it -p 8888:3000 my-name
FROM node:alpine AS builder
# Install yalc globally (the apk add... line is only needed if your installation requires it)
RUN apk add --no-cache --virtual .gyp python make g++ && \
npm i -g yalc
RUN mkdir /packages && \
mkdir /packages/models && \
mkdir /packages/gui && \
mkdir /packages/server
COPY ./packages/models /packages/models
WORKDIR /packages/models
RUN npm install && \
npm run build && \
yalc publish --private
COPY ./packages/gui /packages/gui
WORKDIR /packages/gui
RUN yalc add my-models && \
npm install && \
npm run build
COPY ./packages/server /packages/server
WORKDIR /packages/server
RUN yalc add my-models && \
npm install && \
npm run build
FROM node:alpine
RUN mkdir -p /app
COPY --from=builder /packages/server/package.json /app/package.json
COPY --from=builder /packages/server/dist /app/dist
# Make sure you copy the yalc registry too.
COPY --from=builder /packages/server/.yalc /app/.yalc
COPY --from=builder /packages/server/node_modules /app/node_modules
COPY --from=builder /packages/gui/dist /app/dist/public
WORKDIR /app
EXPOSE 3000
CMD ["node", "./dist/index.js"]
Hope that helps...
The docker build CLI command sends the specified directory (typically .) as the "build context" to the Docker Engine (daemon). Instead of specifying the build context as /worker-a, specify the build context as the root directory, and use the -f argument to specify the path to the Dockerfile in one of the child directories.
docker build -f worker-a/Dockerfile .
docker build -f worker-b/Dockerfile .
You'll have to rework your Dockerfiles slightly, to point them to ../config/config.json, but that is pretty trivial to fix.
Also check out this question/answer, which I think addresses the exact same problem that you're experiencing.
How to include files outside of Docker's build context?
Hope this helps! Cheers
An alternative solution is to upgrade all your soft links into hard links.