I have my Dockerfile in the root of directory with src/myapp folder, myapp contains myapp.go with main package.
Dockerfile looks like following:
FROM golang:1.9.2
ADD . /
RUN go build myapp;
ENTRYPOINT ["/go/bin/myapp"]
I get following error:
can't load package: package myapp: cannot find package "myapp" in any of:
/usr/local/go/src/myapp (from $GOROOT)
/go/src/myapp (from $GOPATH)
What am I doing wrong? Can I log ls command after docker has done ADD?
You are copying all the files to Image root directory, Didn't installed any dependencies, Trying to Build it and then run the binary from /go/bin/app. The binary doesn't exists in that directory and it's generating errors.
I would recommend using a Dockerfile like this,
FROM golang:1.9.2
ADD . /go/src/myapp
WORKDIR /go/src/myapp
RUN go get myapp
RUN go install
ENTRYPOINT ["/go/bin/myapp"]
This'll do the following.
Copy project files to /go/src/myapp.
Set Working directory to /go/src/myapp.
Install dependencies, I used go get but replace it with which ever dependency management tool you are using.
Install/build the binary.
Set entry point.
You can run ls or any other command using docker exec.
Example:
docker exec <image name/hash> ls
You can also enter the shell in the generated image to understand it well using
docker run --rm -it <image hash/name> /bin/sh
After experiments I've come to this way of building Golang apps.
This way has several advantages:
dependencies are installed on build stage
if you need you may uncomment test options
build first fully-functional image about 800 MB
copies your program to an fresh empty image and produces very small image about 10 MB
Dockerfile:
# Two-stage build:
# first FROM prepares a binary file in full environment ~780MB
# second FROM takes only binary file ~10MB
FROM golang:1.9 AS builder
RUN go version
COPY . "/go/src/github.com/your-login/your-project"
WORKDIR "/go/src/github.com/your-login/your-project"
#RUN go get -v -t .
RUN set -x && \
#go get github.com/2tvenom/go-test-teamcity && \
go get github.com/golang/dep/cmd/dep && \
dep ensure -v
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /your-app
CMD ["/your-app"]
EXPOSE 8000
#########
# second stage to obtain a very small image
FROM scratch
COPY --from=builder /your-app .
EXPOSE 8000
CMD ["/your-app"]
For go 1.11 , you can use go module, the following is example
FROM alpine AS base
RUN apk add --no-cache curl wget
FROM golang:1.11 AS go-builder
WORKDIR /go/app
COPY . /go/app
RUN GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /go/app/main /go/app/cmd/myapp/main.go
FROM base
COPY --from=go-builder /go/app/main /main
CMD ["/main"]
The official docs suggests the following Dockerfile:
FROM golang:1.8
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
Please, visit https://hub.docker.com/_/golang for more info
myapp needs to be in /go/src/myapp as suggested, or in /usr/local/go/src/myapp. You can add it in ADD section.
If the objective is to create a container that simply runs your binary, I would take different approach.
First build the binary for linux:
GOOS=linux CGO_ENABLED=0 go build -a -installsuffix cgo
Then build a lightweight docker image from scratch:
FROM scratch
COPY myApp
CMD ["/myApp"]
Related
There is a library writing in Go. Its folder structure is:
lib/
golang/
go.mod
lib.go
example/
golang/
proto/
protofile
go.mod
main.go
Dockerfile
example is folder to show how to use this lib, so it has a main.go which can build and run. In order to use the lib, the example/golang/go.mod is:
module golang
go 1.15
require (
lib/golang v0.0.0
other stuff...
)
replace lib/golang => ../../golang
Now I want to pack the runnable example into a docker image, the Dockerfile is:
FROM golang:1.15
WORKDIR /go/src/app
COPY . .
RUN go env -w GO111MODULE=auto
RUN go env -w GOPROXY=https://goproxy.io,direct
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
Then I cd into example/golang run docker build -t example ., the error log is
open /go/golang/go.mod: no such file or directory
It seems can not access lib/go.mod file, then I cd into lib folder and run docker build -t server -f examples/golang/Dockerfile ., however this will affect the import of main.go:
import "golang/proto"
error log is:
golang/proto: unrecognized import path "golang/proto"
How should I fix this to make the docker image?
==========================================
After I spend some time to read docker doc, here is the summary about this problem:
docker build command follow a PATH argument, that is the dot . at the end. That control the content of building, it decides what files docker build can access. So the reason of first error is that the pwd is lib/example/golang, and docker build command path is ., then docker build can not access parent files of lib/example/golang and they are required in main.go as a lib.
the command should be: docker build -t example ../../ However, docker build seek Dockerfile only in root of content path, so use -f to tell it where the Dockerfile located: docker build -t example -f ./Dockerfile ../../
if pwd is lib/, command is docker build -t server -f examples/golang/Dockerfile .
Be short:
If the dockerfile is not located at project root path, use -f and PATH of docker build command to give it enough access of files. If using go module, make sure PATH contain a go.mod file
If main.go is located in sub folder, make sure the WORKDIR in dockerfile same as the sub folder after COPY all need, or else go build or go install fail to compile
Your Dockerfile is too nested. Since your go build relies on relative paths - paths that are in parent directories - a docker build . will not see any parent-directory files.
Move the Dockerfile to the top e.g.
Dockerile
lib/
and update to build the nested build directory:
FROM golang:1.15
WORKDIR /go/src/app
# just need to copy lib tree
COPY lib .
# working from here now
WORKDIR /go/src/app/lib/golang/example/golang
RUN go env -w GO111MODULE=auto
RUN go env -w GOPROXY=https://goproxy.io,direct
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
You can use go mod vendor before to build with Docker, it will centralise all your mod in vendor folder.
I did a a new file called build.sh with this inside:
#! /bin/sh
go mod vendor
docker build . -t myapp/myservice
rm -rf ./vendor
and run it whenever i need to build and by deleting vendor i can still use go run *.go with fresh version of my libraries
Everytime I build the container I have to wait for apk add docker to finish which takes a long time.
Since everytime it downloads the same thing, can I somehow force Docker to cache apk's downloads for development purposes?
Here's my Dockerfile:
FROM golang:1.13.5-alpine
WORKDIR /go/src/app
COPY src .
RUN go get -d -v ./...
RUN go install -v ./...
RUN apk add --update docker
CMD ["app"]
BTW, I am using this part volumes: - /var/run/docker.sock:/var/run/docker.sock in my docker-compose.yml to use sibling containers, if that matters.
EDIT: I've found google to copy docker.tgz in Chromium:
# add docker client -- do not install docker via apk -- it will try to install
# docker engine which takes a lot of space as well (we don't need it, we need
# only the small client to communicate with the host's docker server)
ADD build/docker/docker.tgz /
What is that docker.tgz? How can I get it?
Reorder your Dockerfile and it should work.
FROM golang:1.13.5-alpine
RUN apk add --update docker
WORKDIR /go/src/app
COPY src .
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
As you are copying before installation, so whenever you change something in src the cache will invalidate for docker installtion.
Whenever you have a COPY command, if any of the files involve change, it causes every command after that to get re-run. If you move your RUN apk add ... command to the start of the file before it COPYs anything, it will get cached across runs.
A fairly generic recipe for most Dockerfiles to accommodate this pattern looks like:
FROM some-base-image
# Install OS-level dependencies
RUN apk add or apt-get install ...
WORKDIR /app
# Install language-level dependencies
COPY requirements.txt requirements.lock ./
RUN something install -r requirements.txt
# Install the rest of the application
COPY main.app ./
COPY src src/
# Set up standard run-time metadata
EXPOSE 12345
CMD ["/app/main.app"]
(Go and Java applications need the additional step of compiling the application, which often lends itself to a multi-stage build, but this same pattern can be repeated in both stages.)
You can download Docker x86_64 binaries for mac, linux, windows and unzip/untar and make it executable.
Whenever you are installing any packages in Docker container those should go at the beginning of Dockerfile, so it won’t ask you again to install same packages and COPY command part must be at the end of Dockerfile.
I'm beginner with Docker, and I'm trying to build an image in two stages, as explained here: https://docs.docker.com/develop/develop-images/multistage-build/
You can selectively copy artifacts from one stage to another
Looking at the examples given there, I had thought that one could build some files during a first stage, and then make them available for the next one:
FROM golang:1.7.3 AS builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
(Example taken from the above-linked page)
Isn't that what the COPY app.go . and the COPY --from=builder /go/src/github.com/alexellis/href-counter/app . are supposed to do?
I probably have a complete misunderstanding of what is going on, because when I try to do something similar (see below), it seems that the COPY command from the first stage is not able to see the files that have just been built (I can confirm that they have been actually built using a RUN ls step, but then I get a lstat <the file>: no such file or directory error).
And indeed, most other information I can gather regarding COPY (except the examples in the above link) rather suggest that COPY is actually meant to copy files from the directory where the docker build command was launched, not from within the build environment.
Here is my Dockerfile:
FROM haskell:8.6.5 as haskell
RUN git clone https://gitlab+deploy-token-75:sakyTxfe-PxPHDwqsoGm#gitlab.pasteur.fr/bli/bioinfo_utils.git
WORKDIR bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
RUN stack --resolver ghc-8.6.5 build && \
stack --resolver ghc-8.6.5 install --local-bin-path .
RUN pwd; echo "---"; ls
COPY remove-duplicates-from-sorted-fastq .
FROM python:3.7-buster
RUN python3.7 -m pip install snakemake
RUN mkdir -p /opt/bin
COPY --from=haskell /bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell/remove-duplicates-from-sorted-fastq /opt/bin/remove-duplicates-from-sorted-fastq
CMD ["/bin/bash"]
And here is how the build ends when I run docker build . from the directory containing the Dockerfile:
Step 5/11 : RUN pwd; echo "---"; ls
---> Running in 28ff49fe9150
/bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
---
LICENSE
Setup.hs
install.sh
remove-duplicates-from-sorted-fastq
remove-duplicates-from-sorted-fastq.cabal
src
stack.yaml
---> f551efc6bba2
Removing intermediate container 28ff49fe9150
Step 6/11 : COPY remove-duplicates-from-sorted-fastq .
lstat remove-duplicates-from-sorted-fastq: no such file or directory
How am I supposed to proceed to have the built file available for the next stage?
Well, apparently, I was mislead by the COPY step used in the first stage in the doc example. In my case, this is actually useless, and I can just COPY --from=haskell in my second stage, without any COPY in the first stage.
The following Dockerfile builds without issues:
FROM haskell:8.6.5 as haskell
RUN git clone https://gitlab+deploy-token-75:sakyTxfe-PxPHDwqsoGm#gitlab.pasteur.fr/bli/bioinfo_utils.git
WORKDIR bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
RUN stack --resolver ghc-8.6.5 build && \
stack --resolver ghc-8.6.5 install --local-bin-path .
FROM python:3.7-buster
RUN python3.7 -m pip install snakemake
RUN mkdir -p /opt/bin
COPY --from=haskell /bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell/remove-duplicates-from-sorted-fastq /opt/bin/remove-duplicates-from-sorted-fastq
CMD ["/bin/bash"]
I have a Dockerfile with the following:
FROM i386/alpine
WORKDIR /tmp/src/mybuild
ADD . /tmp/src/mybuild
FROM travisci/travis-build
My goal is to end up with a 32 bit image which contains the stuff from both images. After running sudo docker build --rm --tag travis-32 ., the "image" is built, but when I run sudo docker run -it travis-32 /bin/bash, I end up in a bash terminal and typing uname -m gives me x86_64, which is clearly not 32 bit as I was expecting.
How can I make this work?
what's wrong with the Dockerfile:
The Dokcerfile mentioned in the question is a multi-stage build file. Every stage starts with FROM, where one is supposed to copy the artifacts from the first build stage into the second, in the goal to achieve a smaller docker image to use in production at the end.
in the mentioned Dokcerfile:
the first stage copies some files to an i386/alpine image:
FROM i386/alpine
WORKDIR /tmp/src/mybuild
ADD . /tmp/src/mybuild
then everything done is ignored, and another image is taken instead:
FROM travisci/travis-build
so the end result is an exact copy of travisci/travis-build.
Regarding the 32b,64b question:
Usually what is compiled under 32b works only under 32b, and in order for it to run under 64b, you need to compile it under 64b (except for some languages like go, where you can define the target platform), so one need to be careful.
Example:
Take a look, and notice where the artifacts are moved from stage to another using the COPY --from= statement:
FROM golang:1.7.3 as builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
More info can be found in the official docs: docker multistage-builds
hope this helps.
im trying to build docker image with my golang project
I use the following
#build stage
FROM golang:alpine as builder
WORKDIR /go/src/app
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main .
RUN apk add --no-cache git
#final stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY --from=builder /go/bin/app /app
ENTRYPOINT ./app
LABEL Name=fzr-dbc Version=0.0.1
EXPOSE 3000
This build is failing in my main.go file which is looks like following
package main
import (
"fzr-dbc/cmd/tsr”
)
func main() {
tsr.Execute()
}
when I run the command
docker build -t fzr .
The error is:
main.go:4:2: cannot find package "fzr-dbc/cmd/tsr” in any of:
/go/src/app/vendor/fzr-dbc/cmd/tsr (vendor tree)
/usr/local/go/src/fzr-dbc/cmd/tsr (from $GOROOT)
/go/src/fzr-dbc/cmd/tsrs (from $GOPATH)
The error since it’s not finding my project path , what could be missing here ?
The docker file is in my root project fzr and I run the docker build from there
You are running go build in your Dockerfile:
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main .
This requires you to install all your dependencies for building it.
To fix this, either:
Build the binary on your host and COPY the binary directly into the docker; or,
Install dependencies inside your docker as part of RUN processes:
For Go before 1.11, you need to move your project source code into proper directory in GOPATH. Then, on that folder, run go get -u ./...
For Go 1.11+, you need to use Go 1.11 with go.mod and go.sum properly defined. Then you can use the command go mod download to download all the dependencies to the Go package cache inside your docker image.