I tried to copy all files from some package into my WORKDIR in my Dockerfile
My Dockerfile look like this:
FROM python:3.8
WORKDIR /yahoo_finance_app
COPY requirements.txt /yahoo_finance_app/requirements.txt
COPY /manage_db/* /yahoo_finance_app/manage_db/
RUN pip3 install -r requirements.txt
COPY .. .
And I execute it with docker-compose which looks like this:
version: "3.8"
services:
py-api-yahoo-finance:
build: ./api-yahoo-finance/yahoo
ports:
- "5000:5000"
container_name: api_yahoo_finance
command: python manage.py runserver 0.0.0.0:5000
The file tree looks like this:
├── api_yahoo_finance
├── yahoo
├── Dockerfile
├── manage_db
I try to copy all the files from manage_db into my WORKDIR in my Dockerfile, but I got the following error:
=> ERROR [5/7] COPY /manage_db/* /yahoo_finance_app/manage_db/ 0.0s
[5/7] COPY /manage_db/* /yahoo_finance_app/manage_db/:
lstat /var/lib/docker/tmp/buildkit-mount563896323/manage_db: no such file or directory
ERROR: Service 'py-api-yahoo-finance' failed to build : Build failed
And I take the relative path, so the folder is existing.
Thanks to all the helpers.
Dockerfiles are built in a variety of environments.
As a security and practical requirement Dockerfile commands can only access files that are in the context directory, either directly or in one of its children.
To keep Dockerfiles many levels deep, you can do this:
py-api-yahoo-finance:
build:
context: .
file: ./api-yahoo-finance/yahoo/Dockerfile
and then all paths in the Dockerfile will be relative to the project root where the compose.yml is.
Related
I have a simple Dockerfile that is in a directory called /App when I build my docker container using a docker-compose yaml file the dockerfile copies files from one level up, outside the /App folder into the container.
Here is my dockerfile
FROM python:3.8
ENV PYTHONUNBUFFERED 1
WORKDIR /code
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
and here is my docker-compose file
version: '3'
services:
dash:
build:
context: ./App
dockerfile: Dockerfile.dash
container_name: dash_dash
command: ls
volumes:
- .:/code
ports:
- "80:8080"
When I build and run the container the ls command shows that it copied the directory one level above the /App directory, such that the /App directory is included but is not the main directory.
The volumes section of your docker-compose.yml is overriding the working directory:
volumes:
- .:/code
is copying the whole folder where the docker-compose.yml is (so it is copying the /App folder entirely). As a result, your files in your working directory (/code) are overridden.
You should remove the volumes section of your docker-compose.yml. The ls command will then show the contents of the App directory, copied by the COPY . . section of your Dockerfile.
I created a docker file.
FROM node:13.6.0-alpine3.10
WORKDIR /src
RUN apk add --no-cache bash
COPY ./package.json .
COPY ./package-lock.json .
RUN npm install
COPY . .
EXPOSE 8081
CMD npm run start:dev
Structure of my project.
.
└── my-app
└── docker-compose.yml
└── ...
└── server
└── docker
└── Dockerfile
└── src
└── ....
├── package.json
├── package-lock.json
└── ...
When I build the container, I get an error.
ERROR: Service 'server' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder791989542/package.json: no such file or directory
I understand that my files are in a different directory. I tried to solve my problems like this.
COPY ./../package.json .
COPY ./../package-lock.json .
But I also got an error
ERROR: Service 'server' failed to build: COPY failed: Forbidden path outside the build context: ../package.json ()
How to tell the docker so that he is looking for files not in the docker folder where the dockerfile file is located, but in the src folder, where my package-lock.json and package.json files are located
I created a docker-compose.yml file.
version: "3.3"
services:
server:
container_name: server
command:
- npm
- run
- start:dev
build:
context: ./server/docker
dockerfile: dockerfile
environment:
PORT: 8081
ports:
- 8081:8081
restart: on-failure
volumes:
- ./server:/src/
As explained earlier in comments and previous answers, you cannot copy files which are outside of the build context. So you either need to change the context up the directory path or to move the needed files inside the current context.
Looking at your layout, I would go for the first solution and by default use your base directory as context (once you understand the solution, you can adapt context and dockerfile to fit your exact needs).
In your Dockerfile, change the following lines:
COPY ./server/package.json .
COPY ./server/package-lock.json .
Note: the line COPY . . should be removed (or you need to explain a littke better what you actually try to achieve with this)
The build section in your docker-compose.yml file should become:
build:
context: .
dockerfile: server/docker/Dockerfile
The equivalent manual build would be
cd /path/to/my-app
docker build -f server/docker/Dockerfile .
You cannot do that, because first step of build, is to build the context dirs. You need to change the context.
Straight from docker docs,
COPY obeys the following rules:
The path must be inside the context of the build;
you cannot COPY ../something /something, because the first step of a docker build is to send the context directory (and subdirectories) to the docker daemon.
I am getting this error when trying to run my docker with docker compose
=> ERROR [6/9] COPY .npmrc .npmrc
0.0s => ERROR [7/9] ADD package.json /Users/gs/Documents/Dev/DockerDi 0.0s
version: "3"
services:
gs-service:
build:
context: .
dockerfile: ./gs/Dockerfile
ports:
- 8081:8081
ui-service:
build:
context: .
dockerfile: ./ui/Dockerfile
ports:
- 8083:8083
Here is the detail of the Dockerfile which is failing. This is a subdirectory. For example /user/gs/dockerexperiment/ui
docker-compose file is in parent directory. For example
/user/gs/dockerexperiment
Dockerfile is in subdirectory. For example
/user/gs/dockerexperiment/ui
I know something is going wrong with the class path but I am having a difficult time figuring out what I am missing
A Dockerfile operates on a build context; this is a copy of the directory tree you name as the build: { context: } in the docker-compose.yml file (or, if you're using docker build directly, its directory argument).
build:
context: .
In the Dockerfile, any COPY (or ADD) instructions are interpreted as relative to this directory tree. (If it looks like an absolute path, it's still relative to this directory; any .. path steps that would step outside this directory get ignored.)
COPY package*.json ./
# copies ./package.json, relative to the context directory
But, you clarify in a comment, this isn't actually where your files are. Relative to the docker-compose.yml file, you need ./ui/package.json; the Dockerfile is also in the same subdirectory.
The Dockerfile you show doesn't seem to need any content from outside its immediate tree, so the easiest way to address this is to set the ui directory as the build context:
version: '3.8'
services:
ui:
build: ./ui
# Same as
# build:
# context: ./ui
# dockerfile: Dockerfile # relative to the context directory
ports:
- 8085:8085
In the Dockerfile, you can remove the duplicate ADD package.json line, and you should remove the dangerous RUN npm set config line. You can also set a much shorter WORKDIR, something like /app would be typical.
If you don't want to change the docker-compose.yml file, the other possible change is to adjust the Dockerfile so that its COPY paths are relative to the (parent) build-context directory.
COPY ui/package*.json .
The advantage of doing this is that you can access files from the other application, if you need to, or you can store shared files in that parent directory.
FROM node:14
# Add the public TLS key for our internal repository
# (from the parent directory)
COPY repository.example.com.crt /usr/local/share/ca-certificates
RUN update-ca-certificates
WORKDIR /app
# Also get our local npm configuration (from the parent directory)
COPY .npmrc .
# Install the ./ui application
COPY ui/package*.json .
RUN npm ci
COPY ui/src ./src
EXPOSE 8085
CMD ["npm", "start"]
I have a monorepo that has holds various Go services and libraries. The directory structure is like the following:
monorepo
services
service-a
- Dockerfile
go.mod
go.sum
This go.mod file resides in the root of the monorepo directory and the services use the dependencies stated in that file.
I build the Docker image with this command:
docker build -t some:tag ./services/service-a/
When I try to build my Docker image from the root of monorepo directory with the above docker command I get the following error:
COPY failed: Forbidden path outside the build context: ../../go.mod ()
Below is my Dockerfile
FROM golang:1.14.1-alpine3.11
RUN apk add --no-cache ca-certificates git
# Enable Go Modules
ENV GO111MODULE=on
# Set the Current Working Directory inside the container
WORKDIR /app
# Copy go mod and sum files
COPY ../../go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o service-a
ENTRYPOINT ["/app/service-a"]
Is there something I have to do to be able to add files into my Docker image that aren't in the current directory without having to have a separate go.mod and go.sum in each service within the monorepo?
Docker only allows adding files to the image from the context, which is by default the directory containing the Dockerfile. You can specify a different context when you build, but again, it won't let you include files outside that context:
docker build -f ./services/service-a/Dockerfile .
This should use the current directory as the context.
Alternatively, you can create a temp directory, copy all the artifacts there and use that as the build context. This can be automated by a makefile or build script.
You can build and manage your docker containers using docker-compose, then this problem can be solved with the help context directive, for example:
project_folder
├─── src
│ └── folder1
│ └── folder2
│ └── Dockerfile
├── docker-compose.yaml
└── copied_file.ext
docker-compose.yaml
version: '3'
services:
your_service_name:
build:
context: ./ #project_folder for this case
dockerfile: ./src/folder1/folder2/Dockefile
Dockerfile
FROM xxx
COPY copied_file.ext /target_folder/
build or rebuild services:
docker-compose build
run a one-off command on a service:
docker-compose run your_service_name <command> [arguments]
I created a docker file.
FROM node:13.6.0-alpine3.10
WORKDIR /src
RUN apk add --no-cache bash
COPY ./package.json .
COPY ./package-lock.json .
RUN npm install
COPY . .
EXPOSE 8081
CMD npm run start:dev
Structure of my project.
.
└── my-app
└── docker-compose.yml
└── ...
└── server
└── docker
└── Dockerfile
└── src
└── ....
├── package.json
├── package-lock.json
└── ...
When I build the container, I get an error.
ERROR: Service 'server' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder791989542/package.json: no such file or directory
I understand that my files are in a different directory. I tried to solve my problems like this.
COPY ./../package.json .
COPY ./../package-lock.json .
But I also got an error
ERROR: Service 'server' failed to build: COPY failed: Forbidden path outside the build context: ../package.json ()
How to tell the docker so that he is looking for files not in the docker folder where the dockerfile file is located, but in the src folder, where my package-lock.json and package.json files are located
I created a docker-compose.yml file.
version: "3.3"
services:
server:
container_name: server
command:
- npm
- run
- start:dev
build:
context: ./server/docker
dockerfile: dockerfile
environment:
PORT: 8081
ports:
- 8081:8081
restart: on-failure
volumes:
- ./server:/src/
As explained earlier in comments and previous answers, you cannot copy files which are outside of the build context. So you either need to change the context up the directory path or to move the needed files inside the current context.
Looking at your layout, I would go for the first solution and by default use your base directory as context (once you understand the solution, you can adapt context and dockerfile to fit your exact needs).
In your Dockerfile, change the following lines:
COPY ./server/package.json .
COPY ./server/package-lock.json .
Note: the line COPY . . should be removed (or you need to explain a littke better what you actually try to achieve with this)
The build section in your docker-compose.yml file should become:
build:
context: .
dockerfile: server/docker/Dockerfile
The equivalent manual build would be
cd /path/to/my-app
docker build -f server/docker/Dockerfile .
You cannot do that, because first step of build, is to build the context dirs. You need to change the context.
Straight from docker docs,
COPY obeys the following rules:
The path must be inside the context of the build;
you cannot COPY ../something /something, because the first step of a docker build is to send the context directory (and subdirectories) to the docker daemon.