This question is asked before yet After reviewing the answers I am still not able to copy the solution.
I am still new to docker and after watching tutorials and following articles I was able to create a Dockerfile for an existing GitHub repository.
I started by using the nearest available image as a base then adding what I need.
from what I read the problem is in WORKDIR and CMD commands
This is error message:
python: can't open file 'save_model.py': [Errno 2] No such file or directory*
This is my Dockerfile:
# syntax=docker/dockerfile:1
FROM tensorflow/serving:2.3.0-rc0-devel-gpu
WORKDIR app
COPY requirements-gpu.txt .
# install dependencies
RUN pip install -r requirements-gpu.txt
# copy the content of the local src directory to the working directory
COPY /home/pc/Desktop/yolo4_deep .
# command to run on container start
CMD ["python","./app/save_model.py","./app/object_tracker.py" ]
src
save_model.py
object_tracker.py
...
requirements.txt
Dockerfile
I tried WORKDIR command to set the absolute path: WORKDIR /home/pc/Desktop/yolo4_Deep_sort_nojupitor the result was Same Error.
I see multiple issues in your Dockerfile.
COPY /home/pc/Desktop/yolo4_deep .
The COPY command copies files from your local machine to the container. The path on your local machine must be path relative to your build context. The build context is the path you pass in when you run docker build . — in this case the . (the current directory) is the build context. Also the local machine path can only reference files located under the build context — i.e. paths containing .. (parent directory) or / (root directory) are not allowed.
WORKDIR app
WORKDIR sets the path inside the container not on your local machine. So WORKDIR /app means that all commands — RUN, CMD, ENTRYPOINT — will be executed from the /app directory.
CMD ["python","./app/save_model.py","./app/object_tracker.py" ]
As mentioned above WORKDIR /app causes all operations to be executed from the /app directory. So ./app/save_model.py is actually translated as /app/app/save_model.py.
Thanks for help Everyone.
As I mentioned earlier I'm beginner in the docker world. I solved the issue by editing the copy command.
# syntax=docker/dockerfile:1
FROM tensorflow/serving:2.3.0-rc0-devel-gpu
WORKDIR /home/pc/Desktop/yolo4_deep
COPY requirements-gpu.txt .
# install dependencies
RUN pip install -r requirements-gpu.txt
# copy the content of the local src directory to the working directory
COPY src/ .
# command to run on container start
ENTRYPOINT ["./start.sh"]
Related
So I am learning out docker for the first time and was wondering if I am doing this in the correct format for my flask app, as a lot of documentation online for the WOKRDIR command is changing dir into "/app" however my main file to run the app is run.py which would be the same directory as the actual docker file. However, WORKDIR doesn't let me do "WORKDIR ." to use the current DIR.
Can someone clarify if I have my docker file set up correctly?
(I also plan to host this on Heroku if that matters)
File structure
Docker file
# start by pulling the python image
FROM python:3.8-alpine
# copy the requirements file into the image
COPY ./requirements.txt /requirements.txt
# Don't need to switch working directory
# WORKDIR
# install the dependencies and packages in the requirements file
RUN pip install -r requirements.txt
# copy every content from the local file to the image
COPY . /app
# configure the container to run in an executed manner
ENTRYPOINT [ "python" ]
CMD ["run.py" ]
I want to copy the file to a folder location that is outside the working directory. I used the following lines in my docker file, but the files are not there when I look in the container.
WORKDIR /app
RUN cd ../opt/venv/lib/python3.7/site-packages/xxx/
COPY ./resources/abc.py .
When look a that /opt/venv/lib/python3.7/site-packages/xxx/ location the abc.py is not there
What is the issue with my approach? Appreciate your inputs.
You can't COPY a file from outside the build context. So if you are trying to COPY /opt/venv/lib/python3.7/site-packages/xxx/resources/abc.py into your docker image, and that is not in your build context, it will fail. Full stop.
Here's some annotated code.
# change to the /app directory in the container
WORKDIR /app
# run the command cd in the container. cd is a shell builtin, and after
# this command finishes you will still be inside the /app directory in
# your container.
RUN cd ../opt/venv/lib/python3.7/site-packages/xxx/
# Attempt to copy ./resources/abc.py from your host's build context
# (NOT /opt/venv/lib/python3.7/site-packages/xxx/) into the container.
COPY ./resources/abc.py .
The basic fix for this is to first copy abc.py into your build directory. Then you will be able to copy it into your docker container during your build like so:
WORKDIR /app
COPY abc.py .
# /app/abc.py now exists in your container
Note on cd
cd is a shell builtin that changes the working directory of the shell. When you execute it inside a script (or in this case a docker RUN) it only changes the working directory for that process, which ends when the script ends. After which your working directory will be the one you started in. So you cannot use it in the way you were intending. Much better explanation here.
Take this Dockerfile for example:
FROM alpine:latest
RUN cd /opt # cd to /opt
RUN pwd # check current directory, you're STILL in '/'
RUN cd /opt && \
pwd # works as expected because you're still in the same process that cd ran in.
# But once you exit this RUN command you will be back in '/'
# Use WORKDIR to set the working directory in a dockerfile
Here's the output of building that Dockerfile (removed noisy docker output):
$ docker build --no-cache .
Sending build context to Docker daemon 3.584kB
Step 1/4 : FROM alpine:latest
Step 2/4 : RUN cd /opt
Step 3/4 : RUN pwd
/
Step 4/4 : RUN cd /opt && pwd
/opt
From what I understand, you're trying to copy a file into a specific location (/opt/venv/lib/python3.7/site-packages/xxx/) in your Docker image that is outside the WORKDIR you defined in the Dockerfile for your image.
You can easily do this by specifying the absolute destination path in the COPY command:
WORKDIR /app
COPY ./resources/abc.py /opt/venv/lib/python3.7/site-packages/xxx/abc.py
I have a Dockerfile that explicitly defines which directores and files from the context directory are copied to the app directory. But regardless of this Docker tries to copy all files in the context directory.
The Dockerfile is in the context directory.
My test code and data files are in directories directly below the context directory. It attempts to copy everything in the context directory, not just the directories and files specified by my COPY commands. So I get a few hundred of these following ERROR messages, except specifying each and every file in every directory and sub directory:
ERRO[0043] Can't add file /home/david/gitlab/etl/testdata/test_s3_fetched.csv to tar: archive/tar: missed writing 12029507 bytes
...
ERRO[0043] Can't close tar writer: archive/tar: missed writing 12029507 bytes
Sending build context to Docker daemon 1.164GB
Error response from daemon: Error processing tar file(exit status 1): unexpected EOF
My reading of the reference is that it only copies all files and directories if there are no ADD or COPY directives.
I have tried with the following COPY patterns
COPY ./name/ /app/name
COPY name/ /app/name
COPY name /app/name
WORKDIR /app
COPY ./name/ /name
WORKDIR /app
COPY name/ /name
WORKDIR /app
COPY name /name
My Dockerfile:
FROM python3.7.3-alpine3.9
RUN apk update && apk upgrade && apk add bash
# Copy app
WORKDIR /app
COPY app /app
COPY configfiles /configfiles
COPY logs /logs/
COPY errorfiles /errorfiles
COPY shell /shell
COPY ./*.py .
WORKDIR ../
COPY requirements.txt /tmp/
RUN pip install -U pip && pip install -U sphinx && pip install -r /tmp/requirements.txt
EXPOSE 22 80 8887
I expect it to only copy my files without the errors associated with trying to copy files I have not specified in COPY commands. Because the Docker output scrolls off my terminal window due to aqll thew error messages I cannot see if it succeeded with my COPY commands.
All files at and below the build directory are coppied into the initial layer of the docker build context.
Consider using a .dockerignore file to exclude files and directories from the build.
Try to copy the files in the following manner-
# set working directory
WORKDIR /usr/src/app
# add and install requirements
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
# add app
COPY ./errorfiles /usr/src/app
Also, you will have to make sure that your docker-compose.yml file is correctly built-
version: "3.6"
services:
users:
build:
context: ./app
dockerfile: Dockerfile
volumes:
- "./app:/usr/src/app"
Here, I'm assuming that your docker-compose.yml file is inside the parent directory of your app.
See if this works. :)
In the docker docs getting started tutorial part 2, it has one make a Dockerfile. It instructs to add the following lines:
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
COPY . /app
What is /app, and why is this a necessary step?
There are two important directories when building a docker image:
the build context directory.
the WORKDIR directory.
Build context directory
It's the directory on the host machine where docker will get the files to build the image. It is passed to the docker build command as the last argument. (Instead of a PATH on the host machine it can be a URL). Simple example:
docker build -t myimage .
Here the current dir (.) is the build context dir. In this case, docker build will use Dockerfile located in that dir. All files from that dir will be visible to docker build.
The build context dir is not necessarily where the Dockerfile is located. Dockerfile location defaults to current dir and is otherwise indicated by the -f otpion. Example:
docker build -t myimage -f ./rest-adapter/docker/Dockerfile ./rest-adapter
Here build context dir is ./rest-adapter, a subdirectory of where you call docker build; the Dokerfile location is indicated by -f.
WORKDIR
It's a directory inside your container image that can be set with the WORKDIR instruction in the Dockerfile. It is optional (default is /, but base image might have set it), but considered a good practice. Subsequent instructions in the Dockerfile, such as RUN, CMD and ENTRYPOINT will operate in this dir. As for COPY and ADD, they use both...
COPY and ADD use both dirs
These two commands have <src> and <dest>.
<src> is relative to the build context directory.
<dest> is relative to the WORKDIR directory.
For example, if your Dockerfile contains...
WORKDIR /myapp
COPY . .
then the contents of your build context directory will be copied to the /myapp dir inside your docker image.
WORKDIR is a good practice because you can set a directory as the main directory, then you can work on it using COPY, ENTRYPOINT, CMD commands, because them will execute pointing to this PATH.
Docker documentation: https://docs.docker.com/engine/reference/builder/
The WORKDIR instruction sets the working directory for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile. If the WORKDIR doesn’t exist, it will be created even if it’s not used in any subsequent Dockerfile instruction.
The WORKDIR instruction can be used multiple times in a Dockerfile. If a relative path is provided, it will be relative to the path of the previous WORKDIR instruction.
Dockerfile Example:
FROM node:alpine
WORKDIR '/app'
COPY ./package.json ./
RUN npm install
COPY . .
CMD ["npm", "run", "start"]
A alpine node.js was created and the workdir is /app, then al files are copied them into /app
Finally npm run start command is running into /app folder inside the container.
You should exec the following command in the case you have sh or bash tty:
docker exec -it <container-id> sh
or
docker exec -it <container-id> bash
After that you can do ls command and you will can see the WORKDIR folder.
I hope it may help you
You need to declare a working directory and move your code into it, because your code has to live somewhere. Otherwise your code wouldn't be present and your app wouldn't run. Then when commands like RUN, CMD, ENTRYPOINT, COPY, and ADD are used, they are executed in the context of WORKDIR.
/app is an arbitrary choice of working directory. You could use anything you like (foo, bar, or baz), but app is nice since it's self-descriptive and commonly used.
docker build Dockerfile .//running it correctly.
1.) I have mentioned in the comments each command will execute as written, Is that correct working of this Dockerfile?
2.)These commands will be used to make the image when I ran docker build, so
[ec2-user#ip-xx-xx-xx-xx ~]$cd /project/p1
[ec2-user#ip-xx-xx-xx-xx p1]$ls
Dockerfile a b c d
My Dockerfile consists of following commands.
Dockerfile
node 8.1.0 //puls the image from hub
RUN mkdir -p /etc/x/y //make directory in the host at path /etc/x/y
RUN mkdir /app //make directory in the host at path /app
COPY . /app //copy all the files that is
WORKDIR /app //cd /app; now the working directory will be /app for next commands i.e npm install.
RUN npm install
EXPOSE 3000 //what this will do?
Question 1: how to run docker build?
docker build Dockerfile . # am I running it correctly.
No, you run it with docker build . and docker will automatically look for the Dockerfile in the current directory. Or you use docker build -f Path_to_the_docker_file/DockerFile where you clearly specify the path to the DockerFile.
Question 2: Fixing errors and clarifying commands
There are few mistakes in the Dockerfile, check the edited comments:
# pulls the image from dockerhub : YES
# Needs to be preceeded with FROM
FROM node 8.1.0
# all directories are made inside the docker image
# make directory in the image at path /etc/x/y : YES
RUN mkdir -p /etc/x/y
# make directory in the image at path /app : YES
RUN mkdir /app
COPY . /app # copy all the files that is : YES
WORKDIR /app # cd /app; now the working directory will be /app for next commands i.e npm install. : YES
RUN npm install
EXPOSE 3000 # what this will do? => tells all docker instances of this image to listen on port 3000.