COPY command fails - docker

Been stuck on this for the last 3 days. I'm building an image in a docker and
copy command fails due to not finding the right directory.
FROM python:3.6.7-alpine
WORKDIR /usr/src/app
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip3 install -r requirements.txt
COPY . /usr/src/app
CMD python3 manage.py run -h 0.0.0.0
which is run by this docker-dev file:
version: '3.7'
services:
users:
build:
context: ./services/users
dockerfile: Dockerfile-dev
volumes:
- './services/users:/usr/src/app'
ports:
- 5001:5000
environment:
- FLASK_APP=project/__init__.py
- FLASK_ENV=development
and getting this error:
Building users
Step 1/6 : FROM python:3.6.7-alpine
---> cb04a359db13
Step 2/6 : WORKDIR /usr/src/app
---> Using cache
---> 06bb39a49444
Step 3/6 : COPY ./requirements.txt /usr/src/app/requirements.txt
ERROR: Service 'users' failed to build: COPY failed: stat /var/snap/docker/common/var-lib-docker/tmp/docker-builder353668631/requirements.txt: no such file or directory
I don't even know where to start with debugging this. When I tried to access the directory it gave me permission error. So I tried to run the command with sudo which didn't help. Any thoughts ?

Little late to reply, but second COPY command COPY . /usr/src/app replaces the /usr/src/app content generated by RUN pip3 install -r requirements.txt.
Try
FROM python:3.6.7-alpine
WORKDIR /usr/src/app
# install in temp directory
RUN mkdir /dependencies
COPY ./requirements.txt /dependencies/requirements.txt
RUN cd /dependencies && pip3 install -r requirements.txt
COPY . /usr/src/app
# copy generated dependencies
RUN cp -r /dependencies/* /usr/src/app/
CMD python3 manage.py run -h 0.0.0.0

As larsks suggests in his comment, you need the file in the services/users directory. To understand why, an understanding of the "context" is useful.
Docker does not build on the client, it does not see your current directory, or other files on your filesystem. Instead, the last argument to the build command is passed as the build context. With docker-compose, this context defaults to the current directory, which you will often see as . in a docker build command, but you can override that as you've done here with ./services/users as your context. When you run a build, the very first step is to send that build context from the docker client to the server. Even when the client and server are on the same host (a common default, especially for desktop environments), this same process happens. Files listed in .dockerignore, and files in parent directories to the build context are not sent to the docker server.
When you run a COPY or ADD command, the first argument (or all but the last argument when you have multiple) refer to files from the build context, and the last argument is the destination file or directory inside the image.
Therefore, when you put together this compose file entry:
build:
context: ./services/users
dockerfile: Dockerfile-dev
with this COPY command:
COPY ./requirements.txt /usr/src/app/requirements.txt
the COPY will try to copy the requirements.txt file from the build context generated from ./services/users, meaning ./services/users/requirements.txt needs to exist, and not be excluded by a .dockerignore file in ./services/users.

I had a similar problem building an image with beryllium, and I solved this deleting it into the .dockerignore
$ sudo docker build -t apache .
Sending build context to Docker daemon
10.55MB Step 1/4 : FROM centos ---> 9f38484d220f Step 2/4 :
RUN yum install httpd -y
---> Using cache ---> ccdafc4ae476 Step 3/4 :
**COPY ./**beryllium** /var/www/html COPY failed: stat /var/snap/docker/common/var-lib-docker/tmp/docker-builder04301**
$nano .dockerignore
startbootstrap-freelancer-master
run.sh
pro
fruit
beryllium
Bell.zip
remove beryllium from that file
$ sudo docker build -t apache .
Sending build context to Docker daemon 12.92MB
Step 1/4 : FROM centos
---> 9f38484d220f
Step 2/4 : RUN yum install httpd -y
---> Using cache
---> ccdafc4ae476
Step 3/4 : COPY ./beryllium /var/www/HTML
---> 40ebc02992a9
Step 4/4 : CMD apachectl -DFOREGROUND
---> Running in dab0a406c89e
Removing intermediate container dab0a406c89e
---> 1bea741cfb65
Successfully built 1bea741cfb65
Successfully tagged apache:latest

Related

Error copying folder using COPY --from from another layer

The Dockerfile uses the COPY --from command from the other build Node layer, but the generated directory is not found.
Note 1: This Dockerfile works locally on my machine doing builds normally.
Note 2: In the execution log it mentions the removal of an intermediate container, is that it? Would it be possible to preserve this container so that the copy works?
FROM node:16.16 as build
# USER node
WORKDIR /app
COPY package.json /app
RUN npm install --location=global npm#latest && npm install --silent
COPY . .
ARG SCRIPT
ENV SCRIPT=$SCRIPT
ARG API_URL
ENV API_URL=$API_URL
ARG API_SERVER
ENV API_SERVER=$API_SERVER
CMD ["/bin/sh", "-c", "envsubst < src/proxy.conf.template.js > src/proxy.conf.js"]
RUN npm run ${SCRIPT}
FROM nginx:1.23
VOLUME /var/cache/nginx
EXPOSE 80
COPY --from=build /app/dist/siai-spa /usr/share/nginx/html
COPY ./config/nginx-template.conf /etc/nginx/nginx-template.conf
b9ed43dcc388: Pull complete
Digest: sha256:db345982a2f2a4257c6f699a499feb1d79451a1305e8022f16456ddc3ad6b94c
Status: Downloaded newer image for nginx:1.23
---> 41b0e86104ba
Step 15/24 : VOLUME /var/cache/nginx
---> Running in dc0e24ae6e51
Removing intermediate container dc0e24ae6e51
---> 3b2799dad197
Step 16/24 : EXPOSE 80
---> Running in f30edd617285
Removing intermediate container f30edd617285
---> 21985745ce49
Step 17/24 : COPY --from=build /app/dist/siai-spa /usr/share/nginx/html
COPY failed: stat app/dist/siai-spa: file does not exist
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1
I guess, you should use CMD instead of RUN while npm run ${SCRIPT} as this needs to be executed during container running time rather than image build time.
Solved problem!
The difference was that locally I used docker-compose which captures the build arguments from the .env file. The npm run command did not override the ${SCRIPT} variable as the docker command does not use the env file, required to pass through the --build-arg parameters.

docker-compose.yml with entrypoint. file not found

Tried a lot already, but doesn't want to run docker-compose with entrypoint...
I'm new to docker and trying to figure it out, but I can't seem to run the program through entrypoint.sh
Dockerfile
FROM python:3.7-alpine
WORKDIR /app
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev
ENV FLASK_APP=app.py
ENV FLASK_RUN_HOST=0.0.0.0
RUN /usr/local/bin/python3 -m pip install --upgrade pip
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY /web .
COPY entrypoint.sh entrypoint.sh
RUN chmod u+x ./entrypoint.sh
docker-compose.yml
version: "3.9"
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- ./web:/app
environment:
PG_HOST: pg
PG_USER: ${POSTGRES_USER}
PG_PASSWORD: ${POSTGRES_PASSWORD}
PG_DB: ${POSTGRES_DB}
depends_on:
- pg
entrypoint: entrypoint.sh
pg:
image: "postgres:alpine"
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
# PGDATA: /var/lib/postgresql/data/pgdata
# volumes:
# - ./data/pgdata:/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
entrypoint.sh
#python3 manage.py db upgrade
flask run --host=0.0.0.0
ERROR:
Step 1/11 : FROM python:3.7-alpine
---> c051b8594107
Step 2/11 : WORKDIR /app
---> Using cache
---> d19c508ff35c
Step 3/11 : RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev
---> Using cache
---> 19b5e197621a
Step 4/11 : ENV FLASK_APP=app.py
---> Using cache
---> af02528555c4
Step 5/11 : ENV FLASK_RUN_HOST=0.0.0.0
---> Using cache
---> 4029b777d985
Step 6/11 : RUN /usr/local/bin/python3 -m pip install --upgrade pip
---> Using cache
---> e94f1f70106b
Step 7/11 : COPY requirements.txt requirements.txt
---> Using cache
---> fb4ad7239fa2
Step 8/11 : RUN pip install -r requirements.txt
---> Using cache
---> 1f912edd8219
Step 9/11 : COPY /web .
---> Using cache
---> 32966afa52ee
Step 10/11 : COPY entrypoint.sh entrypoint.sh
---> Using cache
---> 11525954905f
Step 11/11 : RUN chmod u+x ./entrypoint.sh
---> Using cache
---> 575c41093e1f
...
Successfully built 89b11d4d5f71
Successfully tagged rodauth_web:latest
rodauth_pg_1 is up-to-date
Recreating 1e4f3f2f6898_rodauth_web_1 ...
Recreating 1e4f3f2f6898_rodauth_web_1 ... error
ERROR: for 1e4f3f2f6898_rodauth_web_1 Cannot start service web: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "entrypoint.sh": executable file not found in $PATH: unknown
ERROR: for web Cannot start service web: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "entrypoint.sh": executable file not found in $PATH: unknown
Encountered errors while bringing up the project.
tried adding command RUN chmod u+x ./entrypoint.sh but
the problem is not gone.
Added the build log for COPY entrypoint.sh entrypoint.sh part.
Your entrypoint.sh file get removed from the container when you mount ./web to /app folder as a volume in docker-compose file. so copy entrypoint to root
COPY entrypoint.sh /entrypoint.sh
RUN chmod u+x /entrypoint.sh
And set entrypoint in docker-compose as
entrypoint: /entrypoint.sh
instead of COPY /web ., just do COPY . /app where you are copying the same structure into /app. as you mentioned, workdir /app you are already in that app folder.
or you can just change the last line, COPY entrypoint.sh .
simple docker example : https://github.com/cerofrais/baiscs/tree/master/basic_docker
First, if you want the script to be the normal entrypoint for the image, I'd move that into the Dockerfile with the following:
ENTRYPOINT ["./entrypoint.sh"]
As for why you'd see a file not found error on a script, several potential reasons:
The first line, e.g. #!/bin/bash, points to a command that does not exist in the image you are using. You need to update your script to use the right shell, or change base images to one that includes your shell.
The script is formatted with Windows linefeeds that include a CR before the LF. That turns /bin/sh into /bin/sh^M (sometimes shown as /bin/sh\r) which doesn't exist in the filesystem. This can be fixed by changing the line feed format in your editor, fixing Git to change how linefeeds are automatically changed, or using a tool like dos2unix.
If you run an image for a different platform, and have qemu binfmt_misc installed but without the --fix-binary option, it will look for the interpreter for your platform inside the image rather that from the host. Often this can be fixed with a newer version of binfmt_misc.
The exec format error mentioned in the comments indicates you are running an image designed for another platform on your host. Docker will attempt to pull the proper platform if you have a multi-platform image. However if the image is only built for one platform, and you try to run it on a different platform without something like qemu's binfmt_misc properly configured, you'll get errors. And the fix for that is to change base images to one that supports that CPU architecture and binary format where you are running the image.
This is how I customized Ravisha Hesh's answer and it worked for me.
COPY entrypoint.sh /entrypoint.sh
RUN chmod u+x /entrypoint.sh
CMD ["/entrypoint.sh"]

COPY failed: no such file or directory

I am trying to build an image by simply copying requirement.txt into it. My directory structure is as below:
- app
- Dockerfile
- requirement.txt
And my dockerfile looks like this :
FROM python:3.7
WORKDIR /home/app
COPY requirements.txt /home/app/
RUN pip install --no-cache-dir -r requirements.txt
I am ensuring that the requirement.txt file is inside docker build context and there is no .dockerignore file which could be possibly ignoring this file. But yet I am getting this below error.
Sending build context to Docker daemon 3.072kB
Step 1/4 : FROM python:3.7
---> 11c6e5fd966a
Step 2/4 : WORKDIR /home/app
---> Running in 5ea181f45c01
Removing intermediate container 5ea181f45c01
---> c150cd1dea37
Step 3/4 : COPY requirements.txt /home/app/
COPY failed: stat /var/lib/docker/tmp/docker-builder580673490/requirements.txt: no such file or directory
Command used for building the image : docker build -t test_app:0.1 .
Am I missing anything here?
WORKDIR switches working directory in docker image, but not in host.
COPY app/requirements.txt /home/app/
It works correctly.

no such file or directory in COPY dockerfile

I have made the following dockerfile to contain my node js application, the problem is that an error appears when building the dockerfile:
Sending build context to Docker daemon 2.048kB
Step 1/7 : FROM node:10
---> 0d5ae56139bd
Step 2/7 : WORKDIR /usr/src/app
---> Using cache
---> 5bfc0405d8fa
Step 3/7 : COPY package.json ./
COPY failed: stat /var/lib/docker/tmp/docker-
builder803334317/package.json: no such file or directory
this is my dockerfile:
FROM node:10
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
CMD [ "npm", "start" ]
i executed the command:
sudo docker build - < Dockerfile
into my root project folder.
My project folder is simple, like this:
-Project
-app.js
-Dockerfile
-package.json
-package-lock.json
-README.md
I am doing something wrong?
When you use the Dockerfile-on-stdin syntax
sudo docker build - < Dockerfile
the build sequence runs in a context that only has the Dockerfile, and no other files on disk.
The directory layout you show is pretty typical, and pointing docker build at that directory should work better
sudo docker build .
(This is the same rule as the "Dockerfiles can't access files in parent directories" rule, but instead of giving the current directory as the base directory to Docker, you're giving no directory at all, so it can't even access files in the current directory.)

Dockerfile not using cache in RUN composer install command

I thought i understand Docker already, but today i found some problem about utilizing docker cache.
Here is my dockerfile
FROM quay.io/my_company/phpjenkins
WORKDIR /usr/src/my_project
ADD composer.json composer.json
ADD composer.lock composer.lock
RUN composer install -o
ADD . .
RUN mkdir -p temp/unittest/cache log
RUN cp app/config/config.unittest.template.neon app/config/config.unittest.neon
CMD ["tail", "-f", "/dev/null"]
I expect docker to use the cache until ADD . .
However, every build, look like docker try to do composer install every time.
Here is some output
+ docker-compose -f docker-compose.yml run app vendor/bin/phpunit -d memory_limit=2048M
Creating network "xxx_default" with the default driver
Creating xxx_rabbitmq_1
Creating xxx_mysql_1
Building app
Step 1/9 : FROM quay.io/my_company/phpjenkins
---> f10ea65fb7df
Step 2/9 : WORKDIR /usr/src/my_project
---> Using cache
---> 07ad76770cd2
Step 3/9 : ADD composer.json composer.json
---> Using cache
---> 0d22314b81af
Step 4/9 : ADD composer.lock composer.lock
---> Using cache
---> 3d41825efcb3
Step 5/9 : RUN composer install -o
---> Running in 38de5f08eb46
Warning: This development build of composer is over 60 days old. It is recommended to update it by running "/usr/local/bin/composer self-update" to get the latest version.
Do not run Composer as root/super user! See https://getcomposer.org/root for details ....
...
---> aa05dc9ddc5f
Removing intermediate container 581aa7e4b00f
Step 6/9 : ADD . .
---> 8796a9235b9a
Removing intermediate container b7354231fbd7
I run out of lead, what could be possible thing that dockerfile didn't use cache for RUN composer install command
I'm using Docker version 17.05.0-ce, build 89658be on Debian, if this help for investigation.
Please advise.
As a work-around you could create two Dockerfiles. One that creates an image at the point where you would like to cache. The second Dockerfile can then use the first image as its base and make modifications as required.
FROM quay.io/my_company/phpjenkins
WORKDIR /usr/src/my_project
ADD composer.json composer.json
ADD composer.lock composer.lock
RUN composer install -o
CMD ["tail", "-f", "/dev/null"]
Build this file to mycomposerimage using
docker build -t mycomposerimage .
Then second dockerfile picks up from there
FROM mycomposerimage
WORKDIR /usr/src/my_project
ADD . .
RUN mkdir -p temp/unittest/cache log
RUN cp app/config/config.unittest.template.neon app/config/config.unittest.neon
CMD ["tail", "-f", "/dev/null"]

Resources