I have a folder structure as below:
Folder
|- folder1.framework
|- bin/...
|- <other files/folders>
|- folder2.tests
|- bin/...
|- <other files/folders>
|- folder3.tests
|- bin/...
|- <other files/folders>
|- Dockerfile
I want to copy just the files/folders in bin directory that is part of all .tests subfolders when building the docker image. In my dockerfile I have this line:
COPY **tests/bin/ /app
But I get an error saying "no source files were specified". For some reason wildcards are not working in copy command. I even tried:
COPY *.tests/bin/ /app
COPY **.tests/bin/ /app
What am I doing wrong?
You will need to use ADD instead of COPY:
ADD *.tests/bin/ /app
docs: https://docs.docker.com/engine/reference/builder/#add
Sorry, COPY also just works fine, I tried it:
COPY *.tests/bin/ /app
So please share your Dockerfile and logs that your docker build is producing, also docker version, os version just in case.
Related
I have a directory:
and a Docerfile:
from rocker/rstudio
COPY . .
when I look in the root directory, I cannot find any copied files in the root, nor in the rstudio directory
what is going on?
I looked at other links Docker copy command does not copy file, Dockerfile COPY instruction failing?, dockerfile COPY not copying file, but cannot find an answer.
Have I missed something obvious?
Being new to docker files. I am trying to understand how docker build is working here.
I currently have a folder that has two files in it. The folder is called foo
The structure of the folder is as follows. It has two files in it the first file is main.go and the other file next to it is Dockerfile as shown in the diagram below
Foo
|_main.go
|_go.mod
|_go.sum
|_Dockerfile
My Dockerfile currently looks like this and the command docker build -t .. is called from inside the folder Foo.
FROM golang:1.18-alpine AS builder
# Create and change to the app directory.
WORKDIR /app
COPY go.* ./
RUN go mod download
# Copy local code to the container image.
COPY . ./
RUN go build -v -o my app
....
Now this is a fairly simple Dockerfile. Here is what I understand.
1- golang:1.18-alpine is the base image
2- In the container a folder called app will be created
3- go.mod and go.sum will be copied to ./ path of the container (probably home) of the container.
4- Run go mod download will be called from inside the /app folder correct ?
My question is basically for no. 4. go mod download called from inside /app folder ? If so how does that work because from my understanding is that the /app folder is so far empty ? When did the app folder get populated ? In the statement COPY go.* ./ what path is ./ is that the home ?
In line WORKDIR /app, your current path will be set to /app. If the directory don't exist then it will be created beforehand.
Next COPY go.* ./, this matches all of files start with go. will be copied to /app directory in the docker container. So your docker /app should look like this :
/app
| go.mod
| go.sum
Again with COPY . ./, you are copying all files from current directory to /app directory of your docker container. It will replace already existing files in the contrainer. /app will look like this:
/app
| main.go
| go.mod
| go.sum
| Dockerfile
Last with RUN go build -v -o myapp, you are building the app using go and saving binary file myapp.
I have two go modules github.com/myuser/mymainrepo and github.com/myuser/commonrepo
Here is how i have the files in my local computer
- allmyrepos
- mymainrepo
- Dockerfile
- go.mod
- commonrepo
- go.mod
mymainrepo/go.mod
...
require (
github.com/myuser/commonrepo
)
replace (
github.com/myuser/commonrepo => ../commonrepo
)
It works well i can do local development with it. Problem happens when i'm building docker image of mymainrepo
mymainrepo/Dockerfile
...
WORKDIR /go/src/mymainrepo
COPY go.mod go.sum ./
RUN go mod download
COPY ./ ./
RUN go build -o appbinary
...
Here replace replaces github.com/myuser/commonrepo with ../commonrepo but in Docker /go/src/commonrepo does not exists.
I'm building the Docker image on CI/CD which needs to fetch directly from remote github url but i also need to do local development on commonrepo. How can i do both ?
I tried to put all my files in GOPATH so it's ~/go/src/github.com/myuser/commonrepo and go/src/github.com/myuser/mymainrepo. And i removed the replace directive. But it looks for commonrepo inside ~/go/pkg/mod/... that's downloaded from github.
Create two go.mod files: one for local development, and one for your build. You can name it go.build.mod for example.
Keep the replace directive in your go.mod file but remove it from go.build.mod.
Finally, in your Dockerfile:
COPY go.build.mod ./go.mod
COPY go.sum ./
I still can't find other better solution even the voted answer doesn't work for me. Here a trick I've done that workaround for me. This is an example structure for doing this:
|---sample
| |---...
| |---go.mod
| |---Dockerfile
|---core
| |---...
| |---go.mod
We know that docker build error when it can't find our local module. Let's make one in the builder process:
# Use the offical golang image to create a binary.
# This is based on Debian and sets the GOPATH to /go.
# https://hub.docker.com/_/golang
FROM golang:1.16.3-buster AS builder
# Copy core library
RUN mkdir /core
COPY core/ /core
# Create and change to the app directory.
WORKDIR /app
# Retrieve application dependencies.
# This allows the container build to reuse cached dependencies.
# Expecting to copy go.mod and if present go.sum.
COPY go.* ./
RUN go mod download
# Copy local code to the container image.
COPY . ./
# Build the binary
RUN go build -o /app/sample cmd/main.go
...
...
Ok, our working dir is /app and our core lib placed next to it /core.
Let's make a trick when build a docker image! Yeah, you know it.
cp -R ../core . && docker build --tag sample-service . && rm -R core/
Update
A way better, create a Makefile next to Dockerfile, with content below:
build:
cp -R ../core .
docker build -t sample-service .
rm -R core/
Then command, make build in the sample directory.
You can create make submit or make deploy commands as you like to.
=> Production ready!
Be aware that if there's an error occurs during docker build process, it won't delete back the core folder we have copied to sample.
Pls let me know if you find any better solution. ;)
Suppose I have a very nested folder structure with lots of project files:
src
projectA
projectA.csproj
someFile.txt
projectB
projectB.csproj
someFile.txt
projectC
projectC.csproj
someFile.txt
In this case I want my DockerFile to copy over the full folder structure, but only include .csproj files:
src
projectA
projectA.csproj
projectB
projectB.csproj
projectC
projectC.csproj
I can do this for each file line by line, but is there a cleaner way?
COPY src/projectA/projectA.csproj src/projectA/projectA.csproj
COPY src/projectB/projectB.csproj src/projectB/projectB.csproj
COPY src/projectC/projectC.csproj src/projectC/projectC.csproj
I've faced a similar situation and the only solution I've found was to prepare a .tgz file containing what I needed and copy it in the docker image using the ADD directive.
e.g.
this is a run.sh script similar to what I used:
#!/bin/bash
tar cvfz csproj.tgz $( find src -name "*.csproj" )
docker build -t test .
docker run -it --rm test
this is a test Dockerfile:
FROM alpine
RUN mkdir /src
ADD csproj.tgz /src
CMD ls -alR /src
This solution is not very pleasant but it did do what I needed at the time.
The ADD directive (src: https://docs.docker.com/engine/reference/builder/#add) is able to copy files (like the COPY directive) and
If is a local tar archive in a recognized compression format (identity, gzip, bzip2 or xz) then it is unpacked as a directory. Resources from remote URLs are not decompressed.
I want to deploy a simple JS Boilerplate to Docker Cloud. I use a Dockerfile that I already used for a different Boilerplate and image. The Dockerfile is pretty simple. It is just based on the official nginx, adds two config files and then the output folder of my gulp boilerplate to the nginx root. So I copied it from the one directory to the new boilerplate since I want to try this one.
The error I'm getting is this (last line)
Sending build context to Docker daemon 277.5 kB
Step 1 : FROM nginx
---> af4b3d7d5401
Step 2 : MAINTAINER Ole Bjarnstroem
---> Using cache
---> f57bc23d9444
Step 3 : ENV LANG en_US.UTF-8
---> Using cache
---> f6f4a76092dd
Step 4 : COPY ./nginx/nginx.conf /etc/nginx/nginx.conf
---> Using cache
---> c4f83a39ba73
Step 5 : COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
---> Using cache
---> 6fe5a6b61d9f
Step 6 : ADD ./dist /usr/share/nginx/html
lstat dist: no such file or directory
But the dist folder is there.
.
├── Dockerfile
├── JSCS.intellij.formatter.xml
├── README.md
├── app
├── dist
├── gulpfile.babel.js
├── jspm.conf.js
├── jspm_packages
├── karma.conf.js
├── nginx
├── node_modules
├── package.json
├── tsconfig.json
├── tslint.json
├── typings
└── typings.json
It might be noteworthy that the folder to be copied was called ./public So I could imagine that this is some kind of weird Docker Cache issue.
My Dockerfile:
FROM nginx
ENV LANG en_US.UTF-8
# Copy configuration files
COPY ./nginx/nginx.conf /etc/nginx/nginx.conf
COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
# Add Gulp output folder to server root
ADD ./dist /usr/share/nginx/html
# Port configuration
EXPOSE 8080
What I tried so far:
Deleting dangling and unused images
Deleting the image that was produced by the same docker file before
Using a different tag
My build command:
docker build -t my_repo/my_app .
Thanks for your help!
Edit: Every other folder works. It is also not a problem of file permissions. It seems, that Docker just doesn't like the dist folder. Which sucks.
Well, stupid me. There was a .dockerignore file with dist in the project folder... Case closed
I had the same issue, but it wasn't the .dockerignore, I forgot to specify the directory to run docker in. In my case that directory was . My full command before was
docker build - < Dockerfile
and after was
docker build . < Dockerfile
I put the directory after the build command used -f to specify the dockerfile
eg:
sudo docker build . -t test:i386 -f mydockerfile
The dot after build is the directory to build from, in this case present dir.
I also had the same issue, the problem wasn't my .dockerignore but my .gitignore, as I couldn't remove dist from my gitgnore I've added cp command in my Dockerfile:
....
WORKDIR /
RUN cp -r public/dist/* www/
EXPOSE 80
(credits: https://serverfault.com/a/666154/152918)
The files you want to copy must be inside the Docker image directory. You cannot reference files anywhere on your file system.
I had this issue, and the problem turned out to be that I had inlined a comment, e.g.
COPY file1.txt dest/ # comment
Turns out you can't do that.
A related bug in the Google App Engine SDK version 138 resulted in the same error message. This bug has been fixed in version 139 of the SDK. You can upgrade to the newest version with the following command:
gcloud components upgrade
For following docker build error,
COPY failed: stat /<**path**> :no such file or directory
I got it around by restarting docker service.
sudo service docker restart
A bit about how I got this error:
I was running in gitlab-ci, and I git this in the logs:
And so when I got to the stage where I run docker build, there was no Dockerfile in there, so I got this error.
There are two ways to solve it:
You can setup somewhere in the project settings to use git strategy of fetch/clone. Im not sure which setting it is - so play around until you get it. This will configure the git strategy for the entire project.
You can also setup git strategy just for the build - you can add a variable like this in the top of your gitlab-ci.yml file:
variables:
GIT_STRATEGY: clone
And it should solve it.
On Mac OS Big Sur I had to just restart my Docker Service then worked again for me(Always happens after changing my .env)