This question already has answers here:
How to include files outside of Docker's build context?
(19 answers)
Closed 6 months ago.
I have a project structure as below
├───Project
│ ├───ServiceA
│ │ DockerFile
│ │ main.py
│ └───CommonFilesFolder
│ utils.py
and my DockerFile looks as below
FROM <base-image>
COPY . /app/
COPY ../CommonFilesFolder/ /app/CommonFilesFolder
ENV PYTHONPATH /app/
RUN pip install -U -r requirements.txt
WORKDIR /app/
CMD [ "python", "main.py"]
I understand that the line COPY . /app/ will copy all the contents of ServiceA Folder (The folder in which the DockerFile exists) to the container's app folder.
Similarly, I also want to copy the folder CommonFilesFolder into the /app, so I tried the line
COPY ../CommonFilesFolder/ /app/CommonFilesFolder
which is throwing me the error during the build
error: failed to solve: failed to compute cache key: "/CommonFilesFolder" not
found: not found.
Failed to build container
How exactly can I change the line COPY ../CommonFilesFolder/ /app/CommonFilesFolder to copy to the container.
When building the container, you have to specify build context. I would say you use docker build .. That means your build context is current directory and you can't use anything outside of it.
Official documentation: https://docs.docker.com/engine/reference/commandline/build/#description
Check this answer how to resolve this: https://stackoverflow.com/a/51013639/12118546
Related
I have read countless posts and can't figure out why COPY isn't working or permissions/ownership aren't changing.
My dockerfile is as follow (simplified)
FROM somealpineimage as prod
USER root
RUN mkdir -p /home/node/app/test
WORKDIR /home/node/app
COPY package*json ./
COPY ./src/folder/tests /home/node/app/test # <- this guy here nothing happens tried multiple variations
RUN npm install
COPY . . # I assumed this command would copy everything from project directory to image but doesn't
FROM prod
RUN npm prune --production
COPY . .
USER node
CMD ["npm", "run", "start"]
This particular folder I'm trying to copy is to resolve a permissions issue.
My initial thought was to copy the contents of tests folder to test with added --chown=node:node to set the correct ownership. But I can't seem to get the ownership to change.
I've tried chmod -R 777 as root user didn't work either.
Like so:
USER root
RUN chmod -R 777 /home/node/app/tests
# or with a higher folder
RUN chmod -R 777 /home/node # with -R it should recursively change permissions but it did nothing
The dockerfile is in the root directory of the project
project
├── Dockerfile
├── src
│ ├── begin-with-the-crazy-ideas.textile
│ └── isn't-docker-supposed-to-make-it-easier.markdown
├── tests
│ ├── test1.test
└── test2.test
The reason I need these files with different ownership is my Node app can't edit/open/unzip/zip them since they're owned by root and nodejs app runs under the user node. Don't have the option to run as root and it would be bad practice.
Any help I'd appreciate. For now I'll go research.
Note : the project is written/tested on a M1 MacOS Docker but the official container runs on Kubernetes/Linux I have permissions issues only when deployed.
I am new to Next.js and Docker. What I am trying to achieve is to essentially deploy a Next.js project with Docker. I am in the process of creating the Dockerfile and docker-compose.yml files. However, the project has some custom packages that it uses outside of the source folder (on the root level). My build step is failing because it cannot resolve these packages.
ModuleNotFoundError: Module not found: Error: Can't resolve '#custompackage/themes/src/Simply/containers' in '/opt/app/src/pages'
This is what the imports look like
import Theme, { theme } from '#custompackage/themes/src/Simply';
import {
Navbar,
Copyright,
Welcome,
Services,
About,
Pricing,
Clients,
Contact,
} from '#custompackage/themes/src/Simply/containers';
import preview from '#custompackage/themes/src/Simply/assets/preview.jpg';
This is my Dockerfile
# Install dependencies only when needed
FROM node:16-alpine AS deps
WORKDIR /opt/app
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile
# Rebuild the source code only when needed
# This is where because may be the case that you would try
# to build the app based on some `X_TAG` in my case (Git commit hash)
# but the code hasn't changed.
FROM node:16-alpine AS builder
ENV NODE_ENV=production
WORKDIR /opt/app
COPY . .
COPY --from=deps /opt/app/node_modules ./node_modules
RUN yarn build
# Production image, copy all the files and run next
FROM node:16-alpine AS runner
ARG X_TAG
WORKDIR /opt/app
ENV NODE_ENV=production
COPY --from=builder /opt/app/next.config.js ./
COPY --from=builder /opt/app/public ./public
COPY --from=builder /opt/app/.next ./.next
COPY --from=builder /opt/app/node_modules ./node_modules
CMD ["node_modules/.bin/next", "start"]
Folder structure
I have tried to use the COPY command in the Dockerfile before build step to copy the packages content into the /opt/app folder so that they can be resolved. However, I wasn't exactly sure if I was doing it right and kept getting nowhere.
You could sure find a way to make your app work without changing the directory structure, but I don't think you should.
Module imported import keyword should fall into one of this two category:
Application code, located in your source folder
Dependencies in usually in a node_modules folder
I you want to have multiple packages into one repository, you should look at the monorepo pattern.
Since you are using yarn, you can take a look at Yarn Workspace which is the solution provided by Yarn to build a monorepo.
You might want to slightly change you directory structure to something like that:
├── app
│ ├── src
│ └── package.json (next.js)
├── packages
│ ├── custom-package-1
│ │ ├── src
│ │ └── package.json
│ └── custom-package-2
│ ├── src
│ └── package.json
└── package.json (main)
In the package.json you will add custom-package-1 to your dependencies and your monorepo tool will do some magic to include custom-package-1 in your dependencies (mainly by creating some symlinks).
I'm running into a problem with docker that I can't fix by looking at other references, documentation, etc. and since i'm a beginner with Docker I try my luck here. I'm working in a Next.js project that is using Docker to build the app. I'm using the example documentation of Next.js, and that works if I have my Dockerfile in the root of my project. However, I want to put it in a folder called etc and use it from there. This is giving me problems, because docker can't find the files that i'm trying to copy to the working directory, see error below.
Structure
.
├── etc
│ └── Dockerfile
├── package.json
└── yarn.lock
Command
docker build etc/
Error
failed to compute cache key: "/yarn.lock" not found: not found
Dockerfile
FROM node:16-alpine AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile
I've tried a bunch of things, such as changing the files and paths. Also, in the documentation they mention the -f flag, but that doesn't work for me either because I get the error "docker build" requires exactly 1 argument. when running docker build -f etc/Dockerfile. Is that outdated? Anyway, my question is how to build my app with docker when my dockerfile is not in the root of the project but in a child folder like etc/.
You have forgotten the dot at the end of the command docker build -f etc/Dockerfile .
Currently, In my dockerfile i am using multiple COPY commands to copy directories from my repository.
COPY requirements.txt requirements.txt
COPY validation /opt/validation
COPY templates /opt/templates
COPY goss /opt/goss
COPY newman /opt/newman
COPY conftest.py /opt/validation/conftest.py
How can i achieve the same results as above using a single COPY command. Is there a way?
There is a little hack with the scratch image:
FROM scratch as tmp
COPY foo /opt/some/path/foo
COPY bar /usr/share/tmp/bar
FROM debian:buster
COPY --from=tmp / /
CMD bash -c "ls /opt/some/path /usr/share/tmp"
❯ docker build -t test . && docker run --rm test
/opt/some/path:
foo
/usr/share/tmp:
bar
scratch is a pseudo-image, it is much like an empty directory. The hack is to copy everything there as it should be in the final image, then merge root directories. The merge produces a single layer.
❯ docker inspect --format '{{.RootFS}}' test
{layers [
sha256:c2ddc1bc2645ab5d982c60434d8bbc6aecee1bd4e8eee0df7fd08c96df2d58bb
sha256:fd35279adf8471b9a168ec75e3ef830046d0d7944fe11570eef4d09e0edde936
] }
If you just want to copy things to the same folder /opt, maybe simply using next:
Folder structure:
.
├── conftest.py
├── Dockerfile
├── .dockerignore
├── goss
├── newman
├── requirements.txt
├── templates
└── validation
Dockerfile:
FROM alpine
COPY . /opt
#RUN mv /opt/conftest.py /opt/validation
RUN ls /opt
.dockerignore:
Dockerfile
Execution:
$ docker build -t abc:1 . --no-cache
Sending build context to Docker daemon 6.144kB
Step 1/3 : FROM alpine
---> 28f6e2705743
Step 2/3 : COPY . /opt
---> 8beb53be958c
Step 3/3 : RUN ls /opt
---> Running in cfc9228124fb
conftest.py
goss
newman
requirements.txt
templates
validation
Removing intermediate container cfc9228124fb
---> 4cdb9275d6f4
Successfully built 4cdb9275d6f4
Successfully tagged abc:1
Here, we use COPY . /opt to copy all things in current folder to /opt/ of container. We use .dockerignore to ignore the files/folders which won't want to copy to containers.
Additional, not sure the rules for COPY conftest.py /opt/validation/conftest.py correct or not, if it's correct, you may have to use RUN mv to move it to specified folder.
I want to deploy a simple JS Boilerplate to Docker Cloud. I use a Dockerfile that I already used for a different Boilerplate and image. The Dockerfile is pretty simple. It is just based on the official nginx, adds two config files and then the output folder of my gulp boilerplate to the nginx root. So I copied it from the one directory to the new boilerplate since I want to try this one.
The error I'm getting is this (last line)
Sending build context to Docker daemon 277.5 kB
Step 1 : FROM nginx
---> af4b3d7d5401
Step 2 : MAINTAINER Ole Bjarnstroem
---> Using cache
---> f57bc23d9444
Step 3 : ENV LANG en_US.UTF-8
---> Using cache
---> f6f4a76092dd
Step 4 : COPY ./nginx/nginx.conf /etc/nginx/nginx.conf
---> Using cache
---> c4f83a39ba73
Step 5 : COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
---> Using cache
---> 6fe5a6b61d9f
Step 6 : ADD ./dist /usr/share/nginx/html
lstat dist: no such file or directory
But the dist folder is there.
.
├── Dockerfile
├── JSCS.intellij.formatter.xml
├── README.md
├── app
├── dist
├── gulpfile.babel.js
├── jspm.conf.js
├── jspm_packages
├── karma.conf.js
├── nginx
├── node_modules
├── package.json
├── tsconfig.json
├── tslint.json
├── typings
└── typings.json
It might be noteworthy that the folder to be copied was called ./public So I could imagine that this is some kind of weird Docker Cache issue.
My Dockerfile:
FROM nginx
ENV LANG en_US.UTF-8
# Copy configuration files
COPY ./nginx/nginx.conf /etc/nginx/nginx.conf
COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
# Add Gulp output folder to server root
ADD ./dist /usr/share/nginx/html
# Port configuration
EXPOSE 8080
What I tried so far:
Deleting dangling and unused images
Deleting the image that was produced by the same docker file before
Using a different tag
My build command:
docker build -t my_repo/my_app .
Thanks for your help!
Edit: Every other folder works. It is also not a problem of file permissions. It seems, that Docker just doesn't like the dist folder. Which sucks.
Well, stupid me. There was a .dockerignore file with dist in the project folder... Case closed
I had the same issue, but it wasn't the .dockerignore, I forgot to specify the directory to run docker in. In my case that directory was . My full command before was
docker build - < Dockerfile
and after was
docker build . < Dockerfile
I put the directory after the build command used -f to specify the dockerfile
eg:
sudo docker build . -t test:i386 -f mydockerfile
The dot after build is the directory to build from, in this case present dir.
I also had the same issue, the problem wasn't my .dockerignore but my .gitignore, as I couldn't remove dist from my gitgnore I've added cp command in my Dockerfile:
....
WORKDIR /
RUN cp -r public/dist/* www/
EXPOSE 80
(credits: https://serverfault.com/a/666154/152918)
The files you want to copy must be inside the Docker image directory. You cannot reference files anywhere on your file system.
I had this issue, and the problem turned out to be that I had inlined a comment, e.g.
COPY file1.txt dest/ # comment
Turns out you can't do that.
A related bug in the Google App Engine SDK version 138 resulted in the same error message. This bug has been fixed in version 139 of the SDK. You can upgrade to the newest version with the following command:
gcloud components upgrade
For following docker build error,
COPY failed: stat /<**path**> :no such file or directory
I got it around by restarting docker service.
sudo service docker restart
A bit about how I got this error:
I was running in gitlab-ci, and I git this in the logs:
And so when I got to the stage where I run docker build, there was no Dockerfile in there, so I got this error.
There are two ways to solve it:
You can setup somewhere in the project settings to use git strategy of fetch/clone. Im not sure which setting it is - so play around until you get it. This will configure the git strategy for the entire project.
You can also setup git strategy just for the build - you can add a variable like this in the top of your gitlab-ci.yml file:
variables:
GIT_STRATEGY: clone
And it should solve it.
On Mac OS Big Sur I had to just restart my Docker Service then worked again for me(Always happens after changing my .env)