I am trying to use CompileDaemon to hot reload a go project using Docker.
My folder structure looks like the below
my-api
- server
- main.go
- Dockerfile
- docker-compose.yml
- Makefile
This is the error I am getting:
go build github.com/firstApi/test-platform/lib/my-api/server: build output "server" already exists and is a directory
This is what my dockerfile looks like
FROM golang:1.12-stretch
ENV GO111MODULE=on
WORKDIR /go/src
COPY go.mod .
COPY go.sum .
RUN go mod download
COPY . .
RUN ["go", "get", "github.com/githubnemo/CompileDaemon"]
RUN go test ./... \
&& CGO_ENABLED=0 go build -v -a -installsuffix cgo -o /main server/main.go
ENTRYPOINT CompileDaemon -log-prefix=false -build="go build ./server" -command="./main"
any ideas what I am doing wrong and what I need to change in order to fix this issue?
UPDATE*****
I tried the solution as suggested by the only answer but I now get the following error:
Could not start command:%!(EXTRA *errors.errorString=can't start command: fork/exec ./server: permission denied)
Your default go build is attempting to output the same name as the directory. You could change your build and ENTRYPOINT line to refer to "go build -o apiserver".
Related
I have a problem when I build with docker compose an application with local dependencies to create a docker image.
My Dockerfile:
FROM golang:alpine AS build
ENV GOPATH=$GOPATH
#GOPROXY
ENV GOPROXY=http://proxy.golang.org
ENV GO111MODULE=on
WORKDIR $GOPATH/src/github.com/julianskyline/motorcars-core-business
COPY . .
# Set OS as linux
RUN GOOS=linux go build -o $GOPATH/bin/github.com/julianskyline/motorcars-core-business main.go
FROM alpine
COPY --from=build $GOPATH/bin/github.com/julianskyline/motorcars-core-business $GOPATH/bin/github.com/julianskyline/motorcars-core-business
ENTRYPOINT [ "/go/bin/motorcars-core-business" ]
My go.mod
module github.com/julianskyline/motorcars-core-business
go 1.15
replace (
github.com/julianskyline/errors => /home/julianmarin/proyectos/go/src/github.com/julianskyline/errors
github.com/julianskyline/motorcars-db => /home/julianmarin/proyectos/go/src/github.com/julianskyline/motorcars-db
github.com/julianskyline/motorcars-models => /home/julianmarin/proyectos/go/src/github.com/julianskyline/motorcars-models)
Projects are in the same folder:
$GOPATH/src/github.com/julianskyline/errors
$GOPATH/src/github.com/julianskyline/motorcars-core-business
enter image description here
The go build/run work fine.
Error sudo docker-compose build:
Step 6/9 : RUN GOOS=linux go build -o $GOPATH/bin/github.com/julianskyline/motorcars-core-business main.go
---> Running in 45227441dfdd
go: github.com/julianskyline/errors#v0.0.0-00010101000000-000000000000 (replaced by /home/julianmarin/proyectos/go/src/github.com/julianskyline/errors): reading /home/julianmarin/proyectos/go/src/github.com/julianskyline/errors/go.mod: open /home/julianmarin/proyectos/go/src/github.com/julianskyline/errors/go.mod: no such file or directory
The command '/bin/sh -c GOOS=linux go build -o $GOPATH/bin/github.com/julianskyline/motorcars-core-business main.go' returned a non-zero code: 1
ERROR: Service 'api' failed to build
NOTE: The file /home/julianmarin/proyectos/go/src/github.com/julianskyline/errors/go.mod exists!
The Dockerfile is in $GOPATH/src/github.com/julianskyline/motorcars-core-business which means that the COPY . . within it will only copy $GOPATH/src/github.com/julianskyline/motorcars-core-business into the docker image.
The go.mod contains replace directives that reference folders not under $GOPATH/src/github.com/julianskyline/motorcars-core-business (e.g. $GOPATH/src/github.com/julianskyline/errors); this leads to compilation errors because those folders are not present within the docker image.
To resolve this you can:
Copy the entire julianskyline folder into the image (by moving Docker file into the parent folder, specifying the context on the command line or using docker-compose).
Remove the replace directives and letting go pull the files from github.
Posting answer as this was requested in the comments; I believe the comments provided sufficient info for the OP.
There is a library writing in Go. Its folder structure is:
lib/
golang/
go.mod
lib.go
example/
golang/
proto/
protofile
go.mod
main.go
Dockerfile
example is folder to show how to use this lib, so it has a main.go which can build and run. In order to use the lib, the example/golang/go.mod is:
module golang
go 1.15
require (
lib/golang v0.0.0
other stuff...
)
replace lib/golang => ../../golang
Now I want to pack the runnable example into a docker image, the Dockerfile is:
FROM golang:1.15
WORKDIR /go/src/app
COPY . .
RUN go env -w GO111MODULE=auto
RUN go env -w GOPROXY=https://goproxy.io,direct
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
Then I cd into example/golang run docker build -t example ., the error log is
open /go/golang/go.mod: no such file or directory
It seems can not access lib/go.mod file, then I cd into lib folder and run docker build -t server -f examples/golang/Dockerfile ., however this will affect the import of main.go:
import "golang/proto"
error log is:
golang/proto: unrecognized import path "golang/proto"
How should I fix this to make the docker image?
==========================================
After I spend some time to read docker doc, here is the summary about this problem:
docker build command follow a PATH argument, that is the dot . at the end. That control the content of building, it decides what files docker build can access. So the reason of first error is that the pwd is lib/example/golang, and docker build command path is ., then docker build can not access parent files of lib/example/golang and they are required in main.go as a lib.
the command should be: docker build -t example ../../ However, docker build seek Dockerfile only in root of content path, so use -f to tell it where the Dockerfile located: docker build -t example -f ./Dockerfile ../../
if pwd is lib/, command is docker build -t server -f examples/golang/Dockerfile .
Be short:
If the dockerfile is not located at project root path, use -f and PATH of docker build command to give it enough access of files. If using go module, make sure PATH contain a go.mod file
If main.go is located in sub folder, make sure the WORKDIR in dockerfile same as the sub folder after COPY all need, or else go build or go install fail to compile
Your Dockerfile is too nested. Since your go build relies on relative paths - paths that are in parent directories - a docker build . will not see any parent-directory files.
Move the Dockerfile to the top e.g.
Dockerile
lib/
and update to build the nested build directory:
FROM golang:1.15
WORKDIR /go/src/app
# just need to copy lib tree
COPY lib .
# working from here now
WORKDIR /go/src/app/lib/golang/example/golang
RUN go env -w GO111MODULE=auto
RUN go env -w GOPROXY=https://goproxy.io,direct
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
You can use go mod vendor before to build with Docker, it will centralise all your mod in vendor folder.
I did a a new file called build.sh with this inside:
#! /bin/sh
go mod vendor
docker build . -t myapp/myservice
rm -rf ./vendor
and run it whenever i need to build and by deleting vendor i can still use go run *.go with fresh version of my libraries
When deploying a simple API with a Docker multi-stage build the following error is being thrown:
$ docker build -t api:latest .
[...]
$ docker run -p 4880:4880 --rm --name=api api:latest
2019/01/29 23:02:15 exec: "go": executable file not found in $PATH
As my application is not calling go from anywhere (or any exec.Command) I ran a go mod vendor so I could check the dependency source code and found the following:
vendor/github.com/lib/pq/oid/gen.go
12: "os/exec"
64: cmd := exec.Command("gofmt")
Could this be the issue and if so how can I fix it? I am sure I have managed to deploy github.com/lib/pq in a multi-stage build before but I am currently stuck.
Many thanks and Happy Hacking.
Dockerfile
FROM golang:1-alpine AS builder
ENV GO111MODULE=on
RUN apk --no-cache add ca-certificates git
WORKDIR /app
# Copy go module info first...
COPY go.mod .
COPY go.sum .
# ...then download to cache dependencies between builds
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -installsuffix cgo -ldflags '-extldflags "-static"' -o api cmd/api/
FROM scratch
WORKDIR /app
COPY --from=builder /app/api /app/
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
EXPOSE 8080
ENTRYPOINT ["./api"]
Your actual build line:
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -installsuffix cgo -ldflags '-extldflags "-static"' -o api cmd/api/
is missing a relative path in front of cmd, making it instead be ./cmd/api will result in a successful build/run. cmd/api is a real thing apparently, https://golang.org/cmd/api/, so it's not erring out, but also not doing what you are expecting it to do. If you type go run cmd/api in your terminal you should see a ton of output unrelated to your application.
im trying to build docker image with my golang project
I use the following
#build stage
FROM golang:alpine as builder
WORKDIR /go/src/app
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main .
RUN apk add --no-cache git
#final stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY --from=builder /go/bin/app /app
ENTRYPOINT ./app
LABEL Name=fzr-dbc Version=0.0.1
EXPOSE 3000
This build is failing in my main.go file which is looks like following
package main
import (
"fzr-dbc/cmd/tsr”
)
func main() {
tsr.Execute()
}
when I run the command
docker build -t fzr .
The error is:
main.go:4:2: cannot find package "fzr-dbc/cmd/tsr” in any of:
/go/src/app/vendor/fzr-dbc/cmd/tsr (vendor tree)
/usr/local/go/src/fzr-dbc/cmd/tsr (from $GOROOT)
/go/src/fzr-dbc/cmd/tsrs (from $GOPATH)
The error since it’s not finding my project path , what could be missing here ?
The docker file is in my root project fzr and I run the docker build from there
You are running go build in your Dockerfile:
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main .
This requires you to install all your dependencies for building it.
To fix this, either:
Build the binary on your host and COPY the binary directly into the docker; or,
Install dependencies inside your docker as part of RUN processes:
For Go before 1.11, you need to move your project source code into proper directory in GOPATH. Then, on that folder, run go get -u ./...
For Go 1.11+, you need to use Go 1.11 with go.mod and go.sum properly defined. Then you can use the command go mod download to download all the dependencies to the Go package cache inside your docker image.
I have my Dockerfile in the root of directory with src/myapp folder, myapp contains myapp.go with main package.
Dockerfile looks like following:
FROM golang:1.9.2
ADD . /
RUN go build myapp;
ENTRYPOINT ["/go/bin/myapp"]
I get following error:
can't load package: package myapp: cannot find package "myapp" in any of:
/usr/local/go/src/myapp (from $GOROOT)
/go/src/myapp (from $GOPATH)
What am I doing wrong? Can I log ls command after docker has done ADD?
You are copying all the files to Image root directory, Didn't installed any dependencies, Trying to Build it and then run the binary from /go/bin/app. The binary doesn't exists in that directory and it's generating errors.
I would recommend using a Dockerfile like this,
FROM golang:1.9.2
ADD . /go/src/myapp
WORKDIR /go/src/myapp
RUN go get myapp
RUN go install
ENTRYPOINT ["/go/bin/myapp"]
This'll do the following.
Copy project files to /go/src/myapp.
Set Working directory to /go/src/myapp.
Install dependencies, I used go get but replace it with which ever dependency management tool you are using.
Install/build the binary.
Set entry point.
You can run ls or any other command using docker exec.
Example:
docker exec <image name/hash> ls
You can also enter the shell in the generated image to understand it well using
docker run --rm -it <image hash/name> /bin/sh
After experiments I've come to this way of building Golang apps.
This way has several advantages:
dependencies are installed on build stage
if you need you may uncomment test options
build first fully-functional image about 800 MB
copies your program to an fresh empty image and produces very small image about 10 MB
Dockerfile:
# Two-stage build:
# first FROM prepares a binary file in full environment ~780MB
# second FROM takes only binary file ~10MB
FROM golang:1.9 AS builder
RUN go version
COPY . "/go/src/github.com/your-login/your-project"
WORKDIR "/go/src/github.com/your-login/your-project"
#RUN go get -v -t .
RUN set -x && \
#go get github.com/2tvenom/go-test-teamcity && \
go get github.com/golang/dep/cmd/dep && \
dep ensure -v
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /your-app
CMD ["/your-app"]
EXPOSE 8000
#########
# second stage to obtain a very small image
FROM scratch
COPY --from=builder /your-app .
EXPOSE 8000
CMD ["/your-app"]
For go 1.11 , you can use go module, the following is example
FROM alpine AS base
RUN apk add --no-cache curl wget
FROM golang:1.11 AS go-builder
WORKDIR /go/app
COPY . /go/app
RUN GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /go/app/main /go/app/cmd/myapp/main.go
FROM base
COPY --from=go-builder /go/app/main /main
CMD ["/main"]
The official docs suggests the following Dockerfile:
FROM golang:1.8
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
Please, visit https://hub.docker.com/_/golang for more info
myapp needs to be in /go/src/myapp as suggested, or in /usr/local/go/src/myapp. You can add it in ADD section.
If the objective is to create a container that simply runs your binary, I would take different approach.
First build the binary for linux:
GOOS=linux CGO_ENABLED=0 go build -a -installsuffix cgo
Then build a lightweight docker image from scratch:
FROM scratch
COPY myApp
CMD ["/myApp"]