I have go web application and im trying to deploy on Docker but im keep getting this messages. Im Running this on Windows 10 enterprise 11th Gen Intel(R) Core(TM) i7-1185G7 # 3.00GHz 64 bit
Docker file
FROM golang:latest
RUN mkdir /app
COPY bin/main /app/main
WORKDIR /app
CMD ["/app/main"]
MAKEFILE
GOOS=linux
GOARCH=amd64
build:
go build -o bin/main main.go
run:
go run main.go
compile:
echo "Compiling for every OS and Platform"
GOOS=linux GOARCH=arm go build -o bin/main-linux-arm main.go
GOOS=linux GOARCH=arm64 go build -o bin/main-linux-arm64 main.go
GOOS=freebsd GOARCH=386 go build -o bin/main-freebsd-386 main.go
im running this commands
go build -o bin/main main.go
docker build -t tiny .
docker run -p 127.0.0.1:8080:8080 tiny
error im getting: exec /app/main: exec format error
go file: log.Fatal(http.ListenAndServe (":8080",nil))
When your Dockerfile says
COPY bin/main /app/main
you're copying a main binary built on your (Windows) host system into your (Linux) Docker image. Since the operating systems don't match, you're getting that "exec format error" message.
It's very common when you're building a Docker image to do the actual compilation step in the Dockerfile itself. In a compiled language like Go, a further extremely common approach is to use a multi-stage build so that the Go toolchain itself isn't part of the final image; this gives you a much smaller image and doesn't republish your source code to end users.
FROM golang:1.19-bullseye AS build
WORKDIR /app
COPY ./ ./
RUN go build -o main ./
FROM debian:bullseye
COPY --from=build /app/main /usr/local/bin/main
CMD ["main"]
This approach ignores your Makefile, which isn't really using any Make features (for example, there is nowhere one rule depends on another, and there is nowhere you might skip a rule execution if its output file already exists). This particular Dockerfile builds a dynamically-linked binary, and makes sure the build and execution phase are using the same major version of the same Linux distribution, and therefore the same system C library.
Related
I can't remember where I saw it, I thought it was on Datadog or on NewRelic or maybe CloudFlare? but I remember someone mentioning that with Golang, they run release binaries in production (of course), and within their Docker containers, they also include a separate file containing debug symbols in case a crash occurs so as to be able to see what happened.
Background
I'm building and running in Docker with a Dockerfile like this:
# do all of our docker building in one image
FROM golang:latest as build
WORKDIR /usr/src/api
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# build the application with relevant flags to make it completely self-contained for a scratch container
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags "-s" -a -installsuffix cgo -o app
# and then copy the built binary to an empty image
FROM ubuntu:latest
COPY --from=build /usr/src/api/app /
COPY --from=build /usr/src/api/config.defaults.json /config.json
COPY --from=build /usr/src/api/logo.png /
# default to running in a dev environment
ENV ENV=dev
EXPOSE 8080
ENTRYPOINT ["/bin/bash"]
If I don't use the flags above, the binary will fail to execute in alpine and scratch base images:
standard_init_linux.go:219: exec user process caused: no such file or directory
Running this in ubuntu:latest just works, so the above compile flags seem to fix the issue for alpine and scratch.
Question
With this environment in mind, is it possible to have go build emit debug symbols into a separate file to live alongside my static binary in the Docker image?
Use go tool compile using -E flag to Debug symbol export. Is that what you need?
$ go tool compile -E *.go
Type:
go tool compile
for more help regarding how to use it and what are the options available.
Reference:
https://golang.org/cmd/compile/
You don't need to use " -a -installsuffix cgo" flags when building with CGO_ENABLED=0 -- just setting the environment variable will do the trick.
You are building with "-ldflags -s", which is going to strip out all debug symbols and ELF symbol table information. Instead of doing that, do a regular build, archive that executable (in case you need the symbols later) and then remove symbols using strip. E.g.
$ CGO_ENABLED=0 GOOS=linux go build -o app.withsymbols
$ cp app.withsymbols /my/archive/for/debugging/production/issues
$ strip app.withsymbols -o app.stripped
$ cp app.stripped /production/bin
That should give you the behavior you're asking for (e.g. a small production binary, but also a backup binary with symbols for debugging problems in production).
I'm running into a seemingly impossible situation: I am building a .Net application with the following Dockerfile:
FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019 AS netbuild
WORKDIR /app
COPY . ./
RUN dotnet publish -c Debug -o out
FROM mcr.microsoft.com/dotnet/framework/runtime:4.8-windowsservercore-ltsc2019
WORKDIR /app
COPY --from=netbuild /app/MyProject/out .
#Some other things and an entrypoint
And my build command is
docker build -t myapp:test -f Dockerfile-backend-only .
On my local machine this works perfectly fine. However, when I run it on a different machine, which serves as a build agent, dotnet publish decides to put the result of the build into C:\app\out instead of C:\app\MyProject\out.
Now I already read that the behaviour of the -o option for relative paths changed in some releases of the build tools, but this is a Dockerfile, so this shouldn't have any effect. Also both machines are running Win 10 Pro and have up-to-date Docker installations.
When deploying a simple API with a Docker multi-stage build the following error is being thrown:
$ docker build -t api:latest .
[...]
$ docker run -p 4880:4880 --rm --name=api api:latest
2019/01/29 23:02:15 exec: "go": executable file not found in $PATH
As my application is not calling go from anywhere (or any exec.Command) I ran a go mod vendor so I could check the dependency source code and found the following:
vendor/github.com/lib/pq/oid/gen.go
12: "os/exec"
64: cmd := exec.Command("gofmt")
Could this be the issue and if so how can I fix it? I am sure I have managed to deploy github.com/lib/pq in a multi-stage build before but I am currently stuck.
Many thanks and Happy Hacking.
Dockerfile
FROM golang:1-alpine AS builder
ENV GO111MODULE=on
RUN apk --no-cache add ca-certificates git
WORKDIR /app
# Copy go module info first...
COPY go.mod .
COPY go.sum .
# ...then download to cache dependencies between builds
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -installsuffix cgo -ldflags '-extldflags "-static"' -o api cmd/api/
FROM scratch
WORKDIR /app
COPY --from=builder /app/api /app/
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
EXPOSE 8080
ENTRYPOINT ["./api"]
Your actual build line:
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -installsuffix cgo -ldflags '-extldflags "-static"' -o api cmd/api/
is missing a relative path in front of cmd, making it instead be ./cmd/api will result in a successful build/run. cmd/api is a real thing apparently, https://golang.org/cmd/api/, so it's not erring out, but also not doing what you are expecting it to do. If you type go run cmd/api in your terminal you should see a ton of output unrelated to your application.
im trying to build docker image with my golang project
I use the following
#build stage
FROM golang:alpine as builder
WORKDIR /go/src/app
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main .
RUN apk add --no-cache git
#final stage
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY --from=builder /go/bin/app /app
ENTRYPOINT ./app
LABEL Name=fzr-dbc Version=0.0.1
EXPOSE 3000
This build is failing in my main.go file which is looks like following
package main
import (
"fzr-dbc/cmd/tsr”
)
func main() {
tsr.Execute()
}
when I run the command
docker build -t fzr .
The error is:
main.go:4:2: cannot find package "fzr-dbc/cmd/tsr” in any of:
/go/src/app/vendor/fzr-dbc/cmd/tsr (vendor tree)
/usr/local/go/src/fzr-dbc/cmd/tsr (from $GOROOT)
/go/src/fzr-dbc/cmd/tsrs (from $GOPATH)
The error since it’s not finding my project path , what could be missing here ?
The docker file is in my root project fzr and I run the docker build from there
You are running go build in your Dockerfile:
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o main .
This requires you to install all your dependencies for building it.
To fix this, either:
Build the binary on your host and COPY the binary directly into the docker; or,
Install dependencies inside your docker as part of RUN processes:
For Go before 1.11, you need to move your project source code into proper directory in GOPATH. Then, on that folder, run go get -u ./...
For Go 1.11+, you need to use Go 1.11 with go.mod and go.sum properly defined. Then you can use the command go mod download to download all the dependencies to the Go package cache inside your docker image.
I have my Dockerfile in the root of directory with src/myapp folder, myapp contains myapp.go with main package.
Dockerfile looks like following:
FROM golang:1.9.2
ADD . /
RUN go build myapp;
ENTRYPOINT ["/go/bin/myapp"]
I get following error:
can't load package: package myapp: cannot find package "myapp" in any of:
/usr/local/go/src/myapp (from $GOROOT)
/go/src/myapp (from $GOPATH)
What am I doing wrong? Can I log ls command after docker has done ADD?
You are copying all the files to Image root directory, Didn't installed any dependencies, Trying to Build it and then run the binary from /go/bin/app. The binary doesn't exists in that directory and it's generating errors.
I would recommend using a Dockerfile like this,
FROM golang:1.9.2
ADD . /go/src/myapp
WORKDIR /go/src/myapp
RUN go get myapp
RUN go install
ENTRYPOINT ["/go/bin/myapp"]
This'll do the following.
Copy project files to /go/src/myapp.
Set Working directory to /go/src/myapp.
Install dependencies, I used go get but replace it with which ever dependency management tool you are using.
Install/build the binary.
Set entry point.
You can run ls or any other command using docker exec.
Example:
docker exec <image name/hash> ls
You can also enter the shell in the generated image to understand it well using
docker run --rm -it <image hash/name> /bin/sh
After experiments I've come to this way of building Golang apps.
This way has several advantages:
dependencies are installed on build stage
if you need you may uncomment test options
build first fully-functional image about 800 MB
copies your program to an fresh empty image and produces very small image about 10 MB
Dockerfile:
# Two-stage build:
# first FROM prepares a binary file in full environment ~780MB
# second FROM takes only binary file ~10MB
FROM golang:1.9 AS builder
RUN go version
COPY . "/go/src/github.com/your-login/your-project"
WORKDIR "/go/src/github.com/your-login/your-project"
#RUN go get -v -t .
RUN set -x && \
#go get github.com/2tvenom/go-test-teamcity && \
go get github.com/golang/dep/cmd/dep && \
dep ensure -v
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /your-app
CMD ["/your-app"]
EXPOSE 8000
#########
# second stage to obtain a very small image
FROM scratch
COPY --from=builder /your-app .
EXPOSE 8000
CMD ["/your-app"]
For go 1.11 , you can use go module, the following is example
FROM alpine AS base
RUN apk add --no-cache curl wget
FROM golang:1.11 AS go-builder
WORKDIR /go/app
COPY . /go/app
RUN GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /go/app/main /go/app/cmd/myapp/main.go
FROM base
COPY --from=go-builder /go/app/main /main
CMD ["/main"]
The official docs suggests the following Dockerfile:
FROM golang:1.8
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
Please, visit https://hub.docker.com/_/golang for more info
myapp needs to be in /go/src/myapp as suggested, or in /usr/local/go/src/myapp. You can add it in ADD section.
If the objective is to create a container that simply runs your binary, I would take different approach.
First build the binary for linux:
GOOS=linux CGO_ENABLED=0 go build -a -installsuffix cgo
Then build a lightweight docker image from scratch:
FROM scratch
COPY myApp
CMD ["/myApp"]