Being new to docker files. I am trying to understand how docker build is working here.
I currently have a folder that has two files in it. The folder is called foo
The structure of the folder is as follows. It has two files in it the first file is main.go and the other file next to it is Dockerfile as shown in the diagram below
Foo
|_main.go
|_go.mod
|_go.sum
|_Dockerfile
My Dockerfile currently looks like this and the command docker build -t .. is called from inside the folder Foo.
FROM golang:1.18-alpine AS builder
# Create and change to the app directory.
WORKDIR /app
COPY go.* ./
RUN go mod download
# Copy local code to the container image.
COPY . ./
RUN go build -v -o my app
....
Now this is a fairly simple Dockerfile. Here is what I understand.
1- golang:1.18-alpine is the base image
2- In the container a folder called app will be created
3- go.mod and go.sum will be copied to ./ path of the container (probably home) of the container.
4- Run go mod download will be called from inside the /app folder correct ?
My question is basically for no. 4. go mod download called from inside /app folder ? If so how does that work because from my understanding is that the /app folder is so far empty ? When did the app folder get populated ? In the statement COPY go.* ./ what path is ./ is that the home ?
In line WORKDIR /app, your current path will be set to /app. If the directory don't exist then it will be created beforehand.
Next COPY go.* ./, this matches all of files start with go. will be copied to /app directory in the docker container. So your docker /app should look like this :
/app
| go.mod
| go.sum
Again with COPY . ./, you are copying all files from current directory to /app directory of your docker container. It will replace already existing files in the contrainer. /app will look like this:
/app
| main.go
| go.mod
| go.sum
| Dockerfile
Last with RUN go build -v -o myapp, you are building the app using go and saving binary file myapp.
Related
My Dockerfile is below:
# syntax=docker/dockerfile:1
FROM golang:1.18-alpine
WORKDIR /app
COPY go.mod ./
COPY go.sum ./
RUN go mod download
COPY *.go ./
RUN go build -o /datapuller
EXPOSE 8080
CMD [ "/datapuller" ]
I tried to build with $ docker build --tag datapuller .
But got error:
main.go:13:2: no required module provides package gitlab.com/mycorp/mycompany/data/datapuller/dbutil; to add it:
go get gitlab.com/mycorp/mycompany/data/datapuller/dbutil
main.go:14:2: no required module provides package gitlab.com/mycorp/mycompany/data/datapuller/models; to add it:
go get gitlab.com/mycorp/mycompany/data/datapuller/models
How to solve this, I can run directly with go run main.go just fine.
My main.go's import is below. I think the imports caused this problem:
package main
import (
"encoding/json"
client "github.com/bozd4g/go-http-client"
"github.com/robfig/cron/v3"
"github.com/xuri/excelize/v2"
"gitlab.com/mycorp/mycompany/data/datapuller/dbutil"
"gitlab.com/mycorp/mycompany/data/datapuller/models"
"gorm.io/gorm"
)
func main() {
...
Because the associated package needs to be pulled when building.
Docker may be missing the necessary environment variables to pull these packages.
It is recommended that you use the go mod vendor command,then build image
FROM golang:1.18-alpine
ADD . /go/src/<project name>
WORKDIR /go/src/<project name>
RUN go build -mod=vendor -v -o /go/src/bin/main main.go
RUN rm -rf /go/src/<project name>
WORKDIR /go/src/bin
CMD ["/go/src/bin/main"]
When you copy your source code into the image, you only copy files in the current directory
COPY *.go ./ # just the current directory's *.go, not any subdirectories
It's usually more common to copy in the entire host source tree, maybe using a .dockerignore file to cause some of the source tree to be ignored
COPY ./ ./
Otherwise you need to copy the specific subdirectories you need into the image (each directory needs a separate COPY command)
COPY *.go ./
COPY dbutil/ dbutil/
COPY models/ models/
My dockerfile:
FROM golang:1.14
RUN mkdir /app
ADD . /app
WORKDIR /app
RUN go build -o main .
CMD ["/app/main"]
error:
main.go:11:2: cannot find package "github.com/gorilla/mux" in any of:
/usr/local/go/src/github.com/gorilla/mux (from $GOROOT)
/go/src/github.com/gorilla/mux (from $GOPATH)
My PATH in GOPATH is
GOPATH=/Users/pstrom/go
I'm coming from a javascript background and there you run NPM INSTALL which adds all external packages to directory node_modules in same directory as the project.
Is there any similar command in Go? Can't find any. I don't want add any PATH in docker, because I wanna run it from anywhere.
How do I handle external packages in Docker in Go?
See the comments too.
It's possible you need to create a go.mod file which functions like package.json. If you don't have a go.mod file but just want to get going, you can go mod init x in the directory alongside main.go and Dockerfile. Then, to force packages to be added to go.mod, you can just go run . (or go run main.go).
Then:
FROM golang:1.15
WORKDIR /app
COPY go.mod .
RUN go mod download
COPY . .
RUN go build -o main .
ENTRYPOINT ["/app/main"]
I recommend bumping to Go 1.15
WORKDIR creates the directory if not present so you skip the mkdir
/app is outside of ${GOPATH} which is correct when using modules
COPY >> ADD (my preference)
go mod download gets dependencies defined in go.mod
COPY . . everything else, may just need to be COPY main.go .
ENTRYPOINT >> CMD and the container will default to running your binary
I have two go modules github.com/myuser/mymainrepo and github.com/myuser/commonrepo
Here is how i have the files in my local computer
- allmyrepos
- mymainrepo
- Dockerfile
- go.mod
- commonrepo
- go.mod
mymainrepo/go.mod
...
require (
github.com/myuser/commonrepo
)
replace (
github.com/myuser/commonrepo => ../commonrepo
)
It works well i can do local development with it. Problem happens when i'm building docker image of mymainrepo
mymainrepo/Dockerfile
...
WORKDIR /go/src/mymainrepo
COPY go.mod go.sum ./
RUN go mod download
COPY ./ ./
RUN go build -o appbinary
...
Here replace replaces github.com/myuser/commonrepo with ../commonrepo but in Docker /go/src/commonrepo does not exists.
I'm building the Docker image on CI/CD which needs to fetch directly from remote github url but i also need to do local development on commonrepo. How can i do both ?
I tried to put all my files in GOPATH so it's ~/go/src/github.com/myuser/commonrepo and go/src/github.com/myuser/mymainrepo. And i removed the replace directive. But it looks for commonrepo inside ~/go/pkg/mod/... that's downloaded from github.
Create two go.mod files: one for local development, and one for your build. You can name it go.build.mod for example.
Keep the replace directive in your go.mod file but remove it from go.build.mod.
Finally, in your Dockerfile:
COPY go.build.mod ./go.mod
COPY go.sum ./
I still can't find other better solution even the voted answer doesn't work for me. Here a trick I've done that workaround for me. This is an example structure for doing this:
|---sample
| |---...
| |---go.mod
| |---Dockerfile
|---core
| |---...
| |---go.mod
We know that docker build error when it can't find our local module. Let's make one in the builder process:
# Use the offical golang image to create a binary.
# This is based on Debian and sets the GOPATH to /go.
# https://hub.docker.com/_/golang
FROM golang:1.16.3-buster AS builder
# Copy core library
RUN mkdir /core
COPY core/ /core
# Create and change to the app directory.
WORKDIR /app
# Retrieve application dependencies.
# This allows the container build to reuse cached dependencies.
# Expecting to copy go.mod and if present go.sum.
COPY go.* ./
RUN go mod download
# Copy local code to the container image.
COPY . ./
# Build the binary
RUN go build -o /app/sample cmd/main.go
...
...
Ok, our working dir is /app and our core lib placed next to it /core.
Let's make a trick when build a docker image! Yeah, you know it.
cp -R ../core . && docker build --tag sample-service . && rm -R core/
Update
A way better, create a Makefile next to Dockerfile, with content below:
build:
cp -R ../core .
docker build -t sample-service .
rm -R core/
Then command, make build in the sample directory.
You can create make submit or make deploy commands as you like to.
=> Production ready!
Be aware that if there's an error occurs during docker build process, it won't delete back the core folder we have copied to sample.
Pls let me know if you find any better solution. ;)
Hello i am trying to recreate inside a docker image my host folder that contains:
Publish (folder containing a .NET app )
dockerfile
conf.json
dockerfile
FROM microsoft/aspnetcore
WORKDIR /app
ADD . /app
ENTRYPOINT ["dotnet","/publish/Bench.dll"]
EXPOSE 8300
When i am trying to see what it created using docker exec -it <id> bash it just takes all the content of publish and throws it inside app without copying conf.json.
I have also tried with
COPY . /app,
COPY /publish /app+COPY conf.json /app to no avail.
Whatever i am trying it won't copy the folder as-is and it wont put the json file beside it.
What am i doing wrong?
So I tested this out. I had the publish folder dockerfile and conf.json in the same directory where I build the image from. That is the key. Here I am using nginx as my base image. The following is the command I used to build the nginx image
docker build -t test/nginx .
and the dockerfile is as below. So I create the app directory by using the RUN command. You will have to use the similar command in .net to create that directory if it doesn't exist. Also pay attention to the docker build logs. It will tell you things that are important. Also you could add an ls command in the dockerfile to list the files in the folder /app if you want to.
FROM nginx
RUN mkdir /app
WORKDIR /app
ADD . /app
After I create a container from the image I built, i can navigate to /app folder and view my
Dockerfile, config.json and publish folder in there. Hope this helps
Let me know if you have any questions
I have the docker file as follows:
FROM node:8 as builder
WORKDIR /usr/src/app
COPY ./src/register_form/package*.json .
RUN npm install
COPY ./src/register_form .
RUN yarn build
FROM tensorflow/tensorflow:1.10.0-gpu-py3
COPY --from=builder /usr/src/app/register_form/build/index.html /app/src/
WORKDIR /app
ENTRYPOINT ["python3"]
CMD ["/app/src/main.pyc"]
However, it cannot copy the index.html from the builder stage. Although when I list the folder in the first stage, the files are there.
The error is:
Step 8/22 : COPY --from=builder ./register_form/build/ /app/src/
COPY failed: stat /var/lib/docker/overlay2/5470e05501898502b3aa437639f975ca3e4bfb5a1e897281e62e07ab89866304/merged/register_form/build: no such file or directory
How can I fix this problem - the COPY --from=builder docker command?
I think you are misusing COPY command. As it is told in docs:
If src is a directory, the entire contents of the directory are
copied, including filesystem metadata.
Note: The directory itself is not copied, just its contents.
So your command COPY ./src/register_form . does NOT create register_form folder in container, but instead copies all contents. You can try adding:
RUN ls .
to your Dockerfile to make sure.
As noticed by #BMitch in comments, you can explicitly set destination folder name to achieve expected results:
COPY ./src/register_form/ register_form/