Need help setting up golang dockerfile - docker

I'm having difficulties setting up a golang dockerfile for a personal project.
project structure is:
Project
|
+-- log.go (contains main)
|
+-- go.mod
|
+-- hash
|
+-- hash.go
The app prints a random hash every 5 seconds and appends a timestamp to it.
File contents:
log.go
package main
import (
"fmt"
"github.com/postelniql/logger-output/hash"
"time"
)
func log() string {
dt := time.Now()
hash := hash.NewSHA1Hash()
return dt.Format("01-02-2006T15:04:05.000Z") + ": " + hash
}
func main() {
fmt.Println(log())
tick := time.Tick(5000 * time.Millisecond)
for range tick {
fmt.Println(log())
}
}
go.mod:
module github.com/postelniql/logger-output
go 1.19
hash.go:
package hash
import (
"crypto/sha1"
"fmt"
"math/rand"
"time"
)
func init() {
rand.Seed(time.Now().UnixNano())
}
func NewSHA1Hash(n ...int) string {
noRandomCharacters := 32
if len(n) > 0 {
noRandomCharacters = n[0]
}
randString := randomString(noRandomCharacters)
hash := sha1.New()
hash.Write([]byte(randString))
bs := hash.Sum(nil)
return fmt.Sprintf("%x", bs)
}
var characterRunes =
[]rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789")
// RandomString generates a random string of n length
func randomString(n int) string {
b := make([]rune, n)
for i := range b {
b[i] = characterRunes[rand.Intn(len(characterRunes))]
}
return string(b)
}
I scraped together the following Dockerfile:
FROM golang:1.19-alpine
WORKDIR /app
COPY go.mod ./
RUN apk add git
RUN go get github.com/postelniql/logger-output/hash
COPY *.go ./
RUN go build -o /logger-output-app
EXPOSE 8080
CMD [ "/logger-output-app" ]
However I keep getting this error (and similar sort of errors):
------
> [6/8] RUN go get github.com/postelniql/logger-output/hash:
#10 2.105 go: github.com/postelniql/logger-output/hash: no matching versions for
query "upgrade"
------
executor failed running [/bin/sh -c go get github.com/postelniql/logger-
output/hash]: exit code: 1
I've searched the web for hours trying to fix this, I genuinely don't understand what's wrong with it. I suspect I'm doing something wrong when it comes to dependency management in the Dockerfile.
I mention I'm a noob in go and am coding this as part of my learning process.
Please help me write a dockerfile that builds and runs.
Thanks!

This should work, explanation below.
#build stage
FROM golang:alpine AS builder
RUN apk add --no-cache git
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o /go/bin/app -v .
#final stage
FROM alpine:latest
RUN addgroup -S app && adduser -S app -G app
COPY --from=builder --chown=app /go/bin/app /app
USER app
ENTRYPOINT ["/app"]
Place the Dockerfile in your project so that you don't have to clone the project inside (makes no sense).
Use 2-stage build to have a clean final image.
Use a different user than root to run the final binary.
Don't expose any port since your app is not listening on any port.
Use ENTRYPOINT instead of CMD. This way you can later pass arguments on the docker run command line.

You don't need to go get github.com/postelniql/logger-output/hash: this is part of your local source tree and you have it locally. You do need to make sure you COPY it into your image.
The changes in your Dockerfile aren't complicated, but your Dockerfile isn't that large to start with:
FROM golang:1.19-alpine
WORKDIR /app
COPY go.mod go.sum ./ # add `go.sum`
RUN go mod download # add: downloads external dependencies
# RUN apk add git # delete
# RUN go get github.com/postelniql/logger-output/hash # delete
COPY *.go ./
COPY hash/ hash/ # add
RUN go build -o /logger-output-app
EXPOSE 8080
CMD [ "/logger-output-app" ]
The Dockerfile in #Mihai's answer should work too, since it also deletes the go get your-own-application line. The multi-stage build setup deletes the Go toolchain from the final build so you get a much smaller image out, but it's also more complex and a little harder to debug.

Related

Error when trying to use go mod download on docker

I'm making an app using docker and Postgres and gorm (go ORM library)
I have this Error (some kind of EOF?) while I want to build my docker image
My code is so simple and runs correctly without docker.
package main
import (
"fmt"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
func main() {
dsn := "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable TimeZone=Asia/Shanghai"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
fmt.Println("Error connecting to postgres", err)
}
fmt.Printf("%+v\n", db)
}
Also, it's my Dockerfile:
FROM golang:1.16.4-alpine3.13 AS build
RUN mkdir /app
COPY src/go.mod /app
COPY src/go.sum /app
WORKDIR /app
RUN go mod download
COPY src /app
RUN go build -o main .
FROM alpine AS final
COPY --from=build /app /app
ENTRYPOINT ["/app/main"]
This looks like a networking issue.
When you are running outside of Docker, it is likely that you don't see these errors because the affected modules are already in your local module cache from a previous build, whereas your Dockerfile is starting from a clean cache and redownloading the module source every time.

Create docker container to run `go test` with all module dependencies downloaded and cached

I want to test my Go code in a CI environment which requires using Docker. How do I create a Docker image that has all the dependencies listed in go.mod already downloaded and compiled so that docker run $IMG go test uses the cached artifacts?
The desired properties of this image:
The image only uses go.mod to compile dependencies. I don't want to use the full source code because then any change to source code would invalidate the Docker layer that hold cached dependencies.
docker run $IMG go test ./... doesn't redownload or recompile dependencies listed in go.mod.
Avoid experimental Docker features.
Existing approaches
Parsing go.mod and using go get
From https://github.com/golang/go/issues/27719#issuecomment-578246826
This approach is close but it doesn't appear to use GOCACHE when I run go test. This also appears to choke on certain module paths, like gopkg.in/DataDog/dd-trace-go.v1:
FROM golang:1.13
WORKDIR /src
COPY go.mod ./
RUN set -eu \
&& go mod graph \
| cut -d '#' -f 1 \
| cut -d ' ' -f 2 \
| sort -u \
| sed -e 's#dd-trace-go.v1#&/ddtrace#' \
| xargs go get -v
docker run --mount /src:/src $IMG go test ./...
Using DOCKER_BUILDKIT with a mount cache
Originally described in https://github.com/golang/go/issues/27719#issuecomment-514747274. This only works for go build. I can't use it for go test because the cache mount is unmounted after the RUN command so it doesn't exist in the created Docker image.
This also depends on experimental docker features.
# syntax = docker/dockerfile:experimental
FROM golang:1.13 as go-builder
ARG VERSION
WORKDIR /src
COPY . /src/
# With a mount cache, Docker will cache the target directories for future
# invocations of this RUN layer. Meaning, once this command is run once, all
# successive calls will use the already downloaded and already compiled assets.
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
go build ./server
I often put COPY go.mod on the very begin of a Dockerfile, as it does not change that often.
FROM golang:1.14.3 as builder
WORKDIR /app
COPY go.mod .
COPY go.sum .
RUN go mod download
COPY . .
RUN go build -tags netgo -ldflags '-extldflags "-static"' -o app .
FROM centos:7
WORKDIR /root
COPY --from=builder /app/app .
So, if you go mod does not change, the line RUN go mod download only run for the first time.
In your dockerfile if you run go get ./... it will download all the dependencies to your docker image based on the go.mod and go.sum. You can also add the --insecure flag to the go get command if you are dealing with internal self signed repositories. Very similar to go mod download. Aside from that you can have a shell script that actually initiates the Go test ./... and reports it out to your CI environment if that allows, I know that Gitlab allows this.
FROM golang:1.15-alpine AS builder
RUN apk add --update git gcc musl-dev
RUN apk update && apk add bash
COPY . /app
RUN go version
WORKDIR /app
ENV CGO_ENABLED=0
RUN git config --global http.sslVerify false
RUN go get ./...
WORKDIR /app
RUN chmod +x ./unitTest.sh
RUN ./unitTest.sh
WORKDIR /app/cmd/svr
RUN go build -o app
RUN chmod 700 app
FROM alpine:latest
WORKDIR /root/
ARG build_stamp
ARG git_commit
ARG build_number
ENV BUILD_STAMP=$build_stamp
ENV GIT_COMMIT=$git_commit
ENV BUILD_NUMBER=$build_number
COPY --from=builder /app/cmd/svr .
EXPOSE 8000
CMD ["./app"]
and the script
#!/usr/bin/env bash
TESTS=$(go test -v -covermode=count -coverprofile=count.txt ./...)
echo "$TESTS"
if echo "$TESTS" | grep -q "FAIL" ; then
echo ""
echo "One or more Unit Tests for app have Failed. Build will now fail. Pipeline will also fail..."
echo ""
exit 1
else
echo ""
echo "All Unit Tests for application have passed!"
echo "Running Code Coverage..."
echo ""
COVERAGE=$(go tool cover -func=./count.txt)
echo "$COVERAGE"
exit 0
fi

Can't get Docker container to run on localhosts it says "connection reset"?

Can't get Docker container to run on localhosts it says "connection reset" when going to localhost:8080.
Here is what I do know so bear with me:
The code runs locally when I run it and I am able to see the http://localhost:8080 page
The Docker build command completes with no errors
Error when curling the server:
curl -X GET http://localhost:8080
curl: (52) Empty reply from server
docker run -d -p 8080:8080 --name goserver -it goserver
The Dockerfile:
FROM golang:1.9.2
ENV SRC_DIR=/go/src/
ENV GOBIN=/go/bin
WORKDIR $GOBIN
# Add the source code:
ADD . $SRC_DIR
RUN cd /go/src/;
RUN go get github.com/gorilla/mux;
CMD ["go","run","main.go"]
#ENTRYPOINT ["./main"]
EXPOSE 8080
Here is the go code:
package main
import (
"fmt"
"net/http"
"github.com/gorilla/mux"
)
func main() {
r := mux.NewRouter()
r.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "<h1>This is the homepage. Try /hello and /hello/Sammy\n</h1>")
})
r.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "<h1>Hello from Docker!\n</h1>")
})
r.HandleFunc("/hello/{name}", func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
title := vars["name"]
fmt.Fprintf(w, "<h1>Hello, %s!\n</h1>", title)
})
http.ListenAndServe(":8080", r)
}
You're starting your image in detached (-d) mode - this is why you don't see error messages. There are few issues with the Dockerfile, it should be fixed with #andre answer, but most probably you forgot to rebuild the image and didn't see the effect.
I'm submitting this answer to suggest some improvements of your Dockerfile:
# first stage - builds the binary from sources
FROM golang:1.12.14-alpine3.10 as build
# using build as current directory
WORKDIR /build
# Add the source code:
COPY main.go ./
# install build deps
RUN apk --update --no-cache add git
# downloading dependencies and
# building server binary
RUN go get github.com/gorilla/mux && \
go build -o server .
# second stage - using minimal image to run the server
FROM alpine:3.10
# using /app as current directory
WORKDIR /app
# copy server binary from `build` layer
COPY --from=build /build/server server
# binary to run
CMD "/app/server"
EXPOSE 8080
I've split your Dockerfile into two stages: build and run. Build stage is responsible for building the server binary, run stage is responsible for running it. See https://docs.docker.com/develop/develop-images/multistage-build/
Then I combined multiple RUNs into single one: go get github.com/gorilla/mux && go build -o server . to avoid creating redundant layers.
I fixed WORKDIRs and give them readable semantical names.
Don't forget to rebuild it with docker build . -t goserver and run it with
docker run -p 8080:8080 --name goserver goserver
If everything is fine, and you're ready to (and you need to) start in the detach mode, then add -d flag.
Also, you may want to check Dockerfile best practices.
your WORKDIR is wrong, based on how you are setting your CMD
change your WORKDIR to SRC_DIR instead of GOBIN and it will work
You could also run go install main.go on your Dockerfile
go install will create the executable and move it to the bin folder
here is an example of a working Dockerfile:
FROM golang:1
ENV SRC_DIR=/go/src/
ENV GOBIN=/go/bin
WORKDIR $SRC_DIR
# Add the source code:
ADD . $SRC_DIR
RUN go get github.com/gorilla/mux;
RUN go install main.go
WORKDIR $GOBIN
ENTRYPOINT ["./main"]
EXPOSE 8080
What was happening is: your CMD was failing because the WORKDIR was pointing to the bin folder.
A few side notes:
Don't do: RUN cd, as per: Docker : RUN cd ... does not work as expected

Golang cannot find package in Docker image

So, I am trying to dockerize a golang application with different directories containing supplementary code for my main file.
I am using gorilla/mux. The directory structure looks like this.
$GOPATH/src/github.com/user/server
|--- Dockerfile
|--- main.go
|--- routes/
handlers.go
|--- public/
index.gohtml
It works on my host machine with no problem. The problem is that when I try to deploy the docker image it does not run and exits shortly after creation. I have tried changing the WORKDIR command in my dockerfile to /go/src and dump all my files there, but still no luck. I have also tried the official documentation on docker hub. Doesn't work either.
My Dockerfile.
FROM golang:latest
WORKDIR /go/src/github.com/user/server
COPY . .
RUN go get -d github.com/gorilla/mux
EXPOSE 8000
CMD ["go","run","main.go"]
My golang main.go
package main
import (
"github.com/gorilla/mux"
"github.com/user/server/routes"
"log"
"net/http"
"time"
)
func main(){
//...
}
I get this error message when I check the logs of my docker image.
Error Message
main.go:5:2: cannot find package "github.com/user/server/routes" in any of:
/usr/local/go/src/github.com/user/server/routes (from $GOROOT)
/go/src/github.com/user/server/routes (from $GOPATH)
Try the following Docker file:
# GO Repo base repo
FROM golang:1.12.0-alpine3.9 as builder
RUN apk add git
# Add Maintainer Info
LABEL maintainer="<>"
RUN mkdir /app
ADD . /app
WORKDIR /app
COPY go.mod go.sum ./
# Download all the dependencies
RUN go mod download
COPY . .
# Build the Go app
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
# GO Repo base repo
FROM alpine:latest
RUN apk --no-cache add ca-certificates curl
RUN mkdir /app
WORKDIR /app/
# Copy the Pre-built binary file from the previous stage
COPY --from=builder /app/main .
# Expose port 8000
EXPOSE 8000
# Run Executable
CMD ["./main"]
Here, we are creating an intermediate docker builder container, copying the code into it, build the code inside the builder container and then copy the binary image to the actual docker.
This will help in both having all the dependencies in the final container and also, the size of the final image will be very small

Can't `go get` dependencies during docker build

I'm fairly new to both Docker and go, so this might be something obvious, but my google searches haven't found anything.
I'm trying to build a simple go program with docker, but I'm havign trouble with dependencies.
go file:
package main
import (
"fmt"
"log"
"html"
"net/http"
"github.com/gorilla/mux"
)
func hello(writer http.ResponseWriter, r *http.Request) {
path := mux.Vars(r)["rest"]
fmt.Fprintf(writer, "Hello, %q", html.EscapeString(path))
}
func main() {
router := mux.NewRouter().StrictSlash(true)
router.HandleFunc("/{rest:.*}", hello)
log.Println("Listening...")
log.Fatal(http.ListenAndServe(":8080", router))
}
Docker file:
FROM golang:latest
RUN mkdir /app
ADD ./HelloWorld.go /app/
WORKDIR /app
RUN go get ./*.go
RUN go build -o main .
CMD ["/app/main"]
Error:
Sending build context to Docker daemon 6.482MB
Step 1/7 : FROM golang:latest
---> 138bd936fa29
...
Step 5/7 : RUN go get ./*.go
---> Running in 1e29844961a2
HelloWorld.go:9:5: cannot find package "github.com/gorilla/mux" in any of:
/usr/local/go/src/github.com/gorilla/mux (from $GOROOT)
/go/src/github.com/gorilla/mux (from $GOPATH)
The command '/bin/sh -c go get ./*.go' returned a non-zero code: 1
You may use my Dockerfile as a base. First stage produces working image. That’s enough for many cases.
https://github.com/lisitsky/go-site-search-string/blob/light-docker/Dockerfile
If you want to shrink image size from ~800MB to about ~10-20MB use second stage too.
Just use $GOPATH everywhere in your path to build your images, e.g. $GOPATH/src/app instead of /app
Full example of multistage build:
FROM golang:alpine as builder
RUN apk add --update git
RUN mkdir -p $GOPATH/src/build
ADD . $GOPATH/src/build/
WORKDIR $GOPATH/src/build
RUN go get ./...
RUN go build -o main .
FROM alpine
RUN adduser -S -D -H -h /app appuser
USER appuser
COPY --from=builder /go/src/build/main /app/
WORKDIR /app
CMD ["./main"]

Resources