I'm running into a problem with docker that I can't fix by looking at other references, documentation, etc. and since i'm a beginner with Docker I try my luck here. I'm working in a Next.js project that is using Docker to build the app. I'm using the example documentation of Next.js, and that works if I have my Dockerfile in the root of my project. However, I want to put it in a folder called etc and use it from there. This is giving me problems, because docker can't find the files that i'm trying to copy to the working directory, see error below.
Structure
.
├── etc
│ └── Dockerfile
├── package.json
└── yarn.lock
Command
docker build etc/
Error
failed to compute cache key: "/yarn.lock" not found: not found
Dockerfile
FROM node:16-alpine AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile
I've tried a bunch of things, such as changing the files and paths. Also, in the documentation they mention the -f flag, but that doesn't work for me either because I get the error "docker build" requires exactly 1 argument. when running docker build -f etc/Dockerfile. Is that outdated? Anyway, my question is how to build my app with docker when my dockerfile is not in the root of the project but in a child folder like etc/.
You have forgotten the dot at the end of the command docker build -f etc/Dockerfile .
Related
I have two go modules github.com/myuser/mymainrepo and github.com/myuser/commonrepo
Here is how i have the files in my local computer
- allmyrepos
- mymainrepo
- Dockerfile
- go.mod
- commonrepo
- go.mod
mymainrepo/go.mod
...
require (
github.com/myuser/commonrepo
)
replace (
github.com/myuser/commonrepo => ../commonrepo
)
It works well i can do local development with it. Problem happens when i'm building docker image of mymainrepo
mymainrepo/Dockerfile
...
WORKDIR /go/src/mymainrepo
COPY go.mod go.sum ./
RUN go mod download
COPY ./ ./
RUN go build -o appbinary
...
Here replace replaces github.com/myuser/commonrepo with ../commonrepo but in Docker /go/src/commonrepo does not exists.
I'm building the Docker image on CI/CD which needs to fetch directly from remote github url but i also need to do local development on commonrepo. How can i do both ?
I tried to put all my files in GOPATH so it's ~/go/src/github.com/myuser/commonrepo and go/src/github.com/myuser/mymainrepo. And i removed the replace directive. But it looks for commonrepo inside ~/go/pkg/mod/... that's downloaded from github.
Create two go.mod files: one for local development, and one for your build. You can name it go.build.mod for example.
Keep the replace directive in your go.mod file but remove it from go.build.mod.
Finally, in your Dockerfile:
COPY go.build.mod ./go.mod
COPY go.sum ./
I still can't find other better solution even the voted answer doesn't work for me. Here a trick I've done that workaround for me. This is an example structure for doing this:
|---sample
| |---...
| |---go.mod
| |---Dockerfile
|---core
| |---...
| |---go.mod
We know that docker build error when it can't find our local module. Let's make one in the builder process:
# Use the offical golang image to create a binary.
# This is based on Debian and sets the GOPATH to /go.
# https://hub.docker.com/_/golang
FROM golang:1.16.3-buster AS builder
# Copy core library
RUN mkdir /core
COPY core/ /core
# Create and change to the app directory.
WORKDIR /app
# Retrieve application dependencies.
# This allows the container build to reuse cached dependencies.
# Expecting to copy go.mod and if present go.sum.
COPY go.* ./
RUN go mod download
# Copy local code to the container image.
COPY . ./
# Build the binary
RUN go build -o /app/sample cmd/main.go
...
...
Ok, our working dir is /app and our core lib placed next to it /core.
Let's make a trick when build a docker image! Yeah, you know it.
cp -R ../core . && docker build --tag sample-service . && rm -R core/
Update
A way better, create a Makefile next to Dockerfile, with content below:
build:
cp -R ../core .
docker build -t sample-service .
rm -R core/
Then command, make build in the sample directory.
You can create make submit or make deploy commands as you like to.
=> Production ready!
Be aware that if there's an error occurs during docker build process, it won't delete back the core folder we have copied to sample.
Pls let me know if you find any better solution. ;)
I want to dockerize my Nest API. I'm completely new to Docker so I created a fresh Nest project with the CLI. I created a .dockerignore and added every file that shouldn't live in the Docker image.
.git
.gitignore
coverage
LICENSE
README.md
CONTRIBUTING.md
docker-compose.yml
Dockerfile
node_modules/
.github
.vscode
npm-debug.log
npm-debug.log.*
Next I started with the Dockerfile.
FROM node:12.13-alpine As api
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
ADD . /usr/src/app
CMD npm start
I'm wondering why the image has a size of 321 MB. Does someone know how to improve this? I don't need fancy stuff for development and testing purposes etc. I just would like to get into Docker by starting with a small "clean" image to setup the docker-compose file for the TypeORM database support.
If you don't need development and testing stuff, enhance project dependency installation in Dockerfile next way:
RUN npm install --production
We are working with two project at the moment:
1 C++ based project
2 Nodejs based project
These two projectes are separated which means they have different codebase(git repoitory) and working directory.
C++ project will produce a node binding file .node which will be used by Nodejs project.
And we try to build an docker image for the Nodejs project with multi-stage like this:
from ubuntu:18.04 as u
WORKDIR /app
RUN apt-get........
copy (?) . #1 copy the c++ source codes
RUN make
from node:10
WORKDIR /app
copy (?) . #1 copy the nodejs cource codes
RUN npm install
copy --from=u /app/dist/xx.node ./lib/
node index.js
And I will build the image by docker build -t xx (?) #2.
However as commented in the dockerfile and the command, how to setup the context directory(see comment #2)? Since it will affect the path in the dockerfile (see comment #1).
Also which project should I put inside for the above dockerfile?
You will have two options on this, as the limiting factor is that Docker only allows copying from the same directory as the Dockerfile:
Create a new Repository
You can either create a new repo and use your repos as submodules or just for the Dockerfile (Than you would have to copy both repos into the root folder at build time). In the End what you want to achieve is the following structure:
/ (root)
|-- C-plus-plus-Repo
|-- |-- <Files>
|-- Node-Repo
|-- |-- <Files>
|-- Dockerfile
Than you can build your project with:
from ubuntu:18.04 as u
WORKDIR /app
RUN apt-get........
#1 copy the c++ source files
copy ./C-plus-plus-Repo .
RUN make
from node:10
WORKDIR /app
#1 copy the nodejs cource codes
copy ./Node-Repo .
RUN npm install
copy --from=u /app/dist/xx.node ./lib/
node index.js
In the root Directory execute:
docker build -t xx .
Build your staging containers extra
Docker allows to copy from an external container as stage.
So you can build the C++ container in your C++ Repo root
from ubuntu:18.04 as u
WORKDIR /app
RUN apt-get........
#1 copy the c++ source files
copy . .
RUN make
and Tag it:
# Build your C++ Container in root of the c++ repo
docker build . -t c-stage
then copy the files from it, using the tag (in your node Repo root):
from node:10
WORKDIR /app
#1 copy the nodejs source files
copy . .
RUN npm install
# Use the Tag-Name of the already build container "c-stage"
copy --from=c-stage /app/dist/xx.node ./lib/
node index.js
Both build steps can be executed from their respective repo roots.
Creating a deploy project with git submodules
How about creating a deploy project using git submodules?
This project would only exist for building the docker image and contains the Dockerfile and both of your projects as git submodules.
Since you not just copy the two projects, but manage them with git, you can always keep them up to date with git submodules update --remote, but note that this leaves your submodule in a detached head state. However this is not a problem as long as you do not try to update your C++ project or the node project from the deploy project.
You can create the project with the following commands:
mkdir deploy_project && cd deploy_project
git init
git submodule add git#your-gitserver.com:YourName/YourCppProject.git cpp_project
git submodule add git#your-gitserver.com:YourName/YourNodeProject.git nodejs_project
Then you can simply add the paths to the subprojects to your dockerfile and build the image in the root directory of the deploy project.
The dockerfile would look like this
FROM ubuntu:18.04 as u
WORKDIR /app
RUN apt-get........
COPY cpp_project/ . #1 copy the c++ source codes
RUN make
FROM node:10
WORKDIR /app
COPY nodejs_project/ . #1 copy the nodejs cource codes
RUN npm install
COPY --from=u /app/dist/xx.node ./lib/
You can use ADD command (her context watching to host directory where Dockerfile placed. It will copy everything placed in same directory as Dockerfile in host machine (in this case content of cpp_app directory) into the docker container.
...
ADD cpp_app /place/to/build
WORKDIR /place/to/build
RUN make
RUN mv result_file /place/where/result_file/have/to/be
WORKDIR /place/where/result_file/have/to/be
... execute your nodejs stuff
I have a Go project with a large vendor/ directory which almost never changes.
I am trying to use the new go 1.10 build cache feature to speed up my builds in Docker engine locally.
Avoiding recompilation of my vendor/ directory would be enough optimization. So I'm trying to do Go equivalent of this common Dockerfile pattern for Python:
FROM python
COPY requirements.txt . # <-- copy your dependency list
RUN pip install -r requirements.txt # <-- install dependencies
COPY ./src ... # <-- your actual code (everything above is cached)
Similarly I tried:
FROM golang:1.10-alpine
COPY ./vendor ./src/myproject/vendor
RUN go build -v myproject/vendor/... # <-- pre-build & cache "vendor/"
COPY . ./src/myproject
However this is giving "cannot find package" error (likely because you cannot build stuff in vendor/ directly normally either).
Has anyone been able to figure this out?
Here's something that works for me:
FROM golang:1.10-alpine
WORKDIR /usr/local/go/src/github.com/myorg/myproject/
COPY vendor vendor
RUN find vendor -maxdepth 2 -mindepth 2 -type d -exec sh -c 'go install -i github.com/myorg/myproject/{}/... || true' \;
COPY main.go .
RUN go build main.go
It makes sure the vendored libraries are installed first. As long as you don't change a library, you're good.
Just use go install -i ./vendor/....
Consider the following Dockerfile:
FROM golang:1.10-alpine
ARG APP
ENV PTH $GOPATH/src/$APP
WORKDIR $PTH
# Pre-compile vendors.
COPY vendor/ $PTH/vendor/
RUN go install -i ./vendor/...
ADD . $PTH/
# Display time taken and the list of the packages being compiled.
RUN time go build -v
You can test it doing something like:
docker build -t test --build-arg APP=$(go list .) .
On the project I am working on, without pre-compile, it takes ~12sec with 90+ package each time, after, it take ~1.2s with only 3 (only the local ones).
If you still have "cannot find package", it means there are missing vendors. Re-run dep ensure should fix it.
An other tip, unrelated to Go is to have your .dockerignore start with *. i.e. ignore everything and then whitelist what you need.
As of Go 1.11, you would use go modules to accomplish this;
FROM golang
WORKDIR /src/myproject
COPY go.mod go.sum ./ # <-- copy your dependency list
RUN go mod download # <-- install dependencies
COPY . . # <-- your actual code (everything above is cached)
As long as go.sum doesn't change, the image layer created by go mod download shall be reused from cache.
Using go mod download alone didn't do the trick for me.
What ended up working for me was to leverage buildkit to mount Go's build cache.
This was the article that lead me to this feature.
My Dockerfile looks something like this
FROM golang AS build
WORKDIR /go/src/app
COPY go.mod ./
COPY go.sum ./
RUN go mod download
COPY . ./
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
go build -o /my-happy-app
For local development this did quite a different for me, changing buildtimes from 1.5 minutes to 3 seconds.
I'm using colima (to run Docker on my Mac. Just mentioning this since I'm not using Docker for Mac, but it should work just the same) with
colima version 0.4.2
git commit: f112f336d05926d62eb6134ee3d00f206560493b
runtime: docker
arch: x86_64
client: v20.10.9
server: v20.10.11
And golang 1.17, so this is not a 1.10 specific answer.
I took this setup from docker compose's cli dockerfile here.
Your mileage may vary.
I want to deploy a simple JS Boilerplate to Docker Cloud. I use a Dockerfile that I already used for a different Boilerplate and image. The Dockerfile is pretty simple. It is just based on the official nginx, adds two config files and then the output folder of my gulp boilerplate to the nginx root. So I copied it from the one directory to the new boilerplate since I want to try this one.
The error I'm getting is this (last line)
Sending build context to Docker daemon 277.5 kB
Step 1 : FROM nginx
---> af4b3d7d5401
Step 2 : MAINTAINER Ole Bjarnstroem
---> Using cache
---> f57bc23d9444
Step 3 : ENV LANG en_US.UTF-8
---> Using cache
---> f6f4a76092dd
Step 4 : COPY ./nginx/nginx.conf /etc/nginx/nginx.conf
---> Using cache
---> c4f83a39ba73
Step 5 : COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
---> Using cache
---> 6fe5a6b61d9f
Step 6 : ADD ./dist /usr/share/nginx/html
lstat dist: no such file or directory
But the dist folder is there.
.
├── Dockerfile
├── JSCS.intellij.formatter.xml
├── README.md
├── app
├── dist
├── gulpfile.babel.js
├── jspm.conf.js
├── jspm_packages
├── karma.conf.js
├── nginx
├── node_modules
├── package.json
├── tsconfig.json
├── tslint.json
├── typings
└── typings.json
It might be noteworthy that the folder to be copied was called ./public So I could imagine that this is some kind of weird Docker Cache issue.
My Dockerfile:
FROM nginx
ENV LANG en_US.UTF-8
# Copy configuration files
COPY ./nginx/nginx.conf /etc/nginx/nginx.conf
COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
# Add Gulp output folder to server root
ADD ./dist /usr/share/nginx/html
# Port configuration
EXPOSE 8080
What I tried so far:
Deleting dangling and unused images
Deleting the image that was produced by the same docker file before
Using a different tag
My build command:
docker build -t my_repo/my_app .
Thanks for your help!
Edit: Every other folder works. It is also not a problem of file permissions. It seems, that Docker just doesn't like the dist folder. Which sucks.
Well, stupid me. There was a .dockerignore file with dist in the project folder... Case closed
I had the same issue, but it wasn't the .dockerignore, I forgot to specify the directory to run docker in. In my case that directory was . My full command before was
docker build - < Dockerfile
and after was
docker build . < Dockerfile
I put the directory after the build command used -f to specify the dockerfile
eg:
sudo docker build . -t test:i386 -f mydockerfile
The dot after build is the directory to build from, in this case present dir.
I also had the same issue, the problem wasn't my .dockerignore but my .gitignore, as I couldn't remove dist from my gitgnore I've added cp command in my Dockerfile:
....
WORKDIR /
RUN cp -r public/dist/* www/
EXPOSE 80
(credits: https://serverfault.com/a/666154/152918)
The files you want to copy must be inside the Docker image directory. You cannot reference files anywhere on your file system.
I had this issue, and the problem turned out to be that I had inlined a comment, e.g.
COPY file1.txt dest/ # comment
Turns out you can't do that.
A related bug in the Google App Engine SDK version 138 resulted in the same error message. This bug has been fixed in version 139 of the SDK. You can upgrade to the newest version with the following command:
gcloud components upgrade
For following docker build error,
COPY failed: stat /<**path**> :no such file or directory
I got it around by restarting docker service.
sudo service docker restart
A bit about how I got this error:
I was running in gitlab-ci, and I git this in the logs:
And so when I got to the stage where I run docker build, there was no Dockerfile in there, so I got this error.
There are two ways to solve it:
You can setup somewhere in the project settings to use git strategy of fetch/clone. Im not sure which setting it is - so play around until you get it. This will configure the git strategy for the entire project.
You can also setup git strategy just for the build - you can add a variable like this in the top of your gitlab-ci.yml file:
variables:
GIT_STRATEGY: clone
And it should solve it.
On Mac OS Big Sur I had to just restart my Docker Service then worked again for me(Always happens after changing my .env)