I'm making an app using docker and Postgres and gorm (go ORM library)
I have this Error (some kind of EOF?) while I want to build my docker image
My code is so simple and runs correctly without docker.
package main
import (
"fmt"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
func main() {
dsn := "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable TimeZone=Asia/Shanghai"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
fmt.Println("Error connecting to postgres", err)
}
fmt.Printf("%+v\n", db)
}
Also, it's my Dockerfile:
FROM golang:1.16.4-alpine3.13 AS build
RUN mkdir /app
COPY src/go.mod /app
COPY src/go.sum /app
WORKDIR /app
RUN go mod download
COPY src /app
RUN go build -o main .
FROM alpine AS final
COPY --from=build /app /app
ENTRYPOINT ["/app/main"]
This looks like a networking issue.
When you are running outside of Docker, it is likely that you don't see these errors because the affected modules are already in your local module cache from a previous build, whereas your Dockerfile is starting from a clean cache and redownloading the module source every time.
Related
I'm having difficulties setting up a golang dockerfile for a personal project.
project structure is:
Project
|
+-- log.go (contains main)
|
+-- go.mod
|
+-- hash
|
+-- hash.go
The app prints a random hash every 5 seconds and appends a timestamp to it.
File contents:
log.go
package main
import (
"fmt"
"github.com/postelniql/logger-output/hash"
"time"
)
func log() string {
dt := time.Now()
hash := hash.NewSHA1Hash()
return dt.Format("01-02-2006T15:04:05.000Z") + ": " + hash
}
func main() {
fmt.Println(log())
tick := time.Tick(5000 * time.Millisecond)
for range tick {
fmt.Println(log())
}
}
go.mod:
module github.com/postelniql/logger-output
go 1.19
hash.go:
package hash
import (
"crypto/sha1"
"fmt"
"math/rand"
"time"
)
func init() {
rand.Seed(time.Now().UnixNano())
}
func NewSHA1Hash(n ...int) string {
noRandomCharacters := 32
if len(n) > 0 {
noRandomCharacters = n[0]
}
randString := randomString(noRandomCharacters)
hash := sha1.New()
hash.Write([]byte(randString))
bs := hash.Sum(nil)
return fmt.Sprintf("%x", bs)
}
var characterRunes =
[]rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789")
// RandomString generates a random string of n length
func randomString(n int) string {
b := make([]rune, n)
for i := range b {
b[i] = characterRunes[rand.Intn(len(characterRunes))]
}
return string(b)
}
I scraped together the following Dockerfile:
FROM golang:1.19-alpine
WORKDIR /app
COPY go.mod ./
RUN apk add git
RUN go get github.com/postelniql/logger-output/hash
COPY *.go ./
RUN go build -o /logger-output-app
EXPOSE 8080
CMD [ "/logger-output-app" ]
However I keep getting this error (and similar sort of errors):
------
> [6/8] RUN go get github.com/postelniql/logger-output/hash:
#10 2.105 go: github.com/postelniql/logger-output/hash: no matching versions for
query "upgrade"
------
executor failed running [/bin/sh -c go get github.com/postelniql/logger-
output/hash]: exit code: 1
I've searched the web for hours trying to fix this, I genuinely don't understand what's wrong with it. I suspect I'm doing something wrong when it comes to dependency management in the Dockerfile.
I mention I'm a noob in go and am coding this as part of my learning process.
Please help me write a dockerfile that builds and runs.
Thanks!
This should work, explanation below.
#build stage
FROM golang:alpine AS builder
RUN apk add --no-cache git
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o /go/bin/app -v .
#final stage
FROM alpine:latest
RUN addgroup -S app && adduser -S app -G app
COPY --from=builder --chown=app /go/bin/app /app
USER app
ENTRYPOINT ["/app"]
Place the Dockerfile in your project so that you don't have to clone the project inside (makes no sense).
Use 2-stage build to have a clean final image.
Use a different user than root to run the final binary.
Don't expose any port since your app is not listening on any port.
Use ENTRYPOINT instead of CMD. This way you can later pass arguments on the docker run command line.
You don't need to go get github.com/postelniql/logger-output/hash: this is part of your local source tree and you have it locally. You do need to make sure you COPY it into your image.
The changes in your Dockerfile aren't complicated, but your Dockerfile isn't that large to start with:
FROM golang:1.19-alpine
WORKDIR /app
COPY go.mod go.sum ./ # add `go.sum`
RUN go mod download # add: downloads external dependencies
# RUN apk add git # delete
# RUN go get github.com/postelniql/logger-output/hash # delete
COPY *.go ./
COPY hash/ hash/ # add
RUN go build -o /logger-output-app
EXPOSE 8080
CMD [ "/logger-output-app" ]
The Dockerfile in #Mihai's answer should work too, since it also deletes the go get your-own-application line. The multi-stage build setup deletes the Go toolchain from the final build so you get a much smaller image out, but it's also more complex and a little harder to debug.
I have a local folder called images that contains bunch of folders and files within. When I run the container I get the following error:
the command I execute: docker run -t -i file-uploader -token=abcdefgh
panic: failed Walk: Failed to walk directory: *fs.PathError lstat ./images/: no such file or directory
goroutine 1 [running]:
main.main()
/src/main.go:57 +0x357
Here is the Dockerfile I created:
FROM golang:1.16
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY ./images/ images/
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT ["/bin/app"]
FROM scratch
COPY --from=0 /bin/app /bin/app
ENTRYPOINT ["/bin/app"]
And, here is the code in the program:
var (
token = flag.String("token", "", "user's token for application")
rootpath = flag.String("rootpath", "./images/","folder path to be uploaded")
)
func main() {
flag.Parse()
if *token == "" {
log.Fatal(Red + "please provide a client token => -token={$token}")
}
tokenSource := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: *token})
oauthClient := oauth2.NewClient(context.TODO(), tokenSource)
client := putio.NewClient(oauthClient)
paths := make(chan string)
var wg = new(sync.WaitGroup)
for i := 0; i < 20; i++ {
wg.Add(1)
go worker(paths, wg, client)
}
if err := filepath.Walk(*rootpath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return fmt.Errorf("Failed to walk directory: %T %w", err, err)
}
if !info.IsDir() {
paths <- path
}
return nil
}); err != nil {
panic(fmt.Errorf("failed Walk: %w", err))
}
close(paths)
wg.Wait()
}
}
If flag is not provided, its default value is the folder itself which is ./images/. When I run this normally like: go run main.go -token="abcde", it works properly. I did some changes on Dockerfile. Some of the changes I made and tried again and again.:
replacing COPY . . with COPY ./images/ /images. It should automatically creates a folder inside /src like /src/images and get the local folder from host and put into it. It didn't work.
I also did try COPY . ., believing that it will copy everything from host into docker container. It didn't work either.
I did put 2 COPY command together. Didn't work.
COPY . ./ didn't work either.
My structure of the project is as follows:
/file-uploader-cli
/images
Dockerfile
file-uploader-cli (binary)
go.mod with go.sum
main.go
How can I put /images folder into container and run it properly?
Extra question: /images folder is approx. 500 MB or something. Is it a good practice to put that folder into a container?
I guess it is possible to copy a folder like docker cp {$folder_name} ${container_id}:/{$path}, but it must be a running container or something? I did try this using image_id replacing the container_id but I got an error like No such container:path: 12312312312:/.
EDIT:
the problem is the scratch image which was done in order to reduce the size. However, when I delete the scratch thing, the size of the image became 1.1 GB. Any easier or convenient way to utilize the images folder without having too much size?
You don't need images when building your app, you need it when executing your app. Instead of adding images to the first image, add it to the final image:
FROM golang:1.16 AS builder
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
FROM scratch
WORKDIR /
copy images .
COPY --from=builder /bin/app /bin/app
ENTRYPOINT ["/bin/app"]
Or, if you want to provide images dynamically, you could mount it at runtime.
I also recommend removing docker containers automatically unless you actually want them sticking around. Otherwise you end up with lots and lots of Exited containers.
docker run --rm -it -v /my/path/to/images:/images:ro file-uploader -token=abcdefgh
I would also recommend you put your token in an environment variable so its not saved in bash history and docker runtime information.
So, you're saying don't dockerize the app?
I don't usually containerize Go programs unless containers are a good fit for deployment and operations (eg kubernetes). Most languages like C/C++, Java, Erlang, Python, Javascript, all require significant runtime components provided by the filesystem - compiled ones are usually dynamically linked with shared libraries from the operating system, VM based languages like Java or Erlang require the VM to be installed and configured (and it will likely also have runtime dependencies), and interpreted languages like Python, Ruby, or Javascript require the entire interpreter runtime, as well as any shared libraries the language's libraries are linked to.
Go is an exception to this though. Ignoring CGo (which I recommend avoiding whenever possible), Go binaries are statically linked and have minimal userspace runtime requirements. This is why a Go binary is one of the few things that can acutally work in a container built FROM scratch. The one exception to this, is that the Go program will need CA Certificates from the operating system to validate the certificates on HTTPS servers and other protocols secured with TLS.
I recommend that you don't dockerize the app unless docker is helping you distribute or operate it.
The problem is that the second build stage does not include the images directory. I don't think you can use the scratch base for this purpose, so we will have to change that. If you are concerned about image size, you should look into into alpine linux. I have converted your Dockerfile to use alpine.
FROM golang:1.16-alpine
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT ["/bin/app"]
FROM alpine:3.15.0
WORKDIR /opt/app
COPY --from=0 /bin/app app
COPY images .
ENTRYPOINT ["/opt/app/app"]
Note that I changed the app path in the second build stage to /opt/app. I did this because it would be odd to have an image folder under /bin. And /opt is a common place to store user applications.
As for whether or not you should containerize your code, that is up to you. One of Go's advantages is static compilation and easy cross-compiling. So you could distribute your binary (and images) as is.
So, I am trying to dockerize a golang application with different directories containing supplementary code for my main file.
I am using gorilla/mux. The directory structure looks like this.
$GOPATH/src/github.com/user/server
|--- Dockerfile
|--- main.go
|--- routes/
handlers.go
|--- public/
index.gohtml
It works on my host machine with no problem. The problem is that when I try to deploy the docker image it does not run and exits shortly after creation. I have tried changing the WORKDIR command in my dockerfile to /go/src and dump all my files there, but still no luck. I have also tried the official documentation on docker hub. Doesn't work either.
My Dockerfile.
FROM golang:latest
WORKDIR /go/src/github.com/user/server
COPY . .
RUN go get -d github.com/gorilla/mux
EXPOSE 8000
CMD ["go","run","main.go"]
My golang main.go
package main
import (
"github.com/gorilla/mux"
"github.com/user/server/routes"
"log"
"net/http"
"time"
)
func main(){
//...
}
I get this error message when I check the logs of my docker image.
Error Message
main.go:5:2: cannot find package "github.com/user/server/routes" in any of:
/usr/local/go/src/github.com/user/server/routes (from $GOROOT)
/go/src/github.com/user/server/routes (from $GOPATH)
Try the following Docker file:
# GO Repo base repo
FROM golang:1.12.0-alpine3.9 as builder
RUN apk add git
# Add Maintainer Info
LABEL maintainer="<>"
RUN mkdir /app
ADD . /app
WORKDIR /app
COPY go.mod go.sum ./
# Download all the dependencies
RUN go mod download
COPY . .
# Build the Go app
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
# GO Repo base repo
FROM alpine:latest
RUN apk --no-cache add ca-certificates curl
RUN mkdir /app
WORKDIR /app/
# Copy the Pre-built binary file from the previous stage
COPY --from=builder /app/main .
# Expose port 8000
EXPOSE 8000
# Run Executable
CMD ["./main"]
Here, we are creating an intermediate docker builder container, copying the code into it, build the code inside the builder container and then copy the binary image to the actual docker.
This will help in both having all the dependencies in the final container and also, the size of the final image will be very small
I am trying to create a docker container with a Dockerfile and a go file binary.
I have two files in my folder: Dockerfile and main, where the latter is a binary of my simple go file.
Contents of Dockerfile:
FROM golang:1.11-alpine
WORKDIR /app
COPY main /app/
RUN ["chmod", "+x", "/app/main"]
ENTRYPOINT ["./main"]
I tried following steps:
sudo docker build -t naive5cr .
sudo docker run -d -p 8080:8080 naive5cr
The error which i see in thru "docker logs " :
standard_init_linux.go:207: exec user process caused "no such file or directory"
my go file content [i think it is irrelevant to the problem]:
func main() {
http.HandleFunc("/", index)
http.ListenAndServe(port(), nil)
}
func port() string {
port := os.Getenv("PORT")
if len(port) == 0 {
port = "8080"
}
return ":" + port
}
the binary "main" runs as expected when run standalone. so there is no problem with the content of go file.
You need to compile with CGO_ENABLED=0 to prevent links to libc on Linux when networking is used in Go. Alpine ships with musl rather than libc, and attempts to find libc result in the no such file or directory error. You can verify this by running ldd main to see the dynamic links.
You can also build on an Alpine based host to link to musl instead of libc. The advantage of a completely statically compiled binary is the ability to run on scratch, without any libraries at all.
go compiles down to native code, so make sure to build your go code on the Docker image, instead of copying the binary to the docker image.
e.g.
FROM golang:1.11-alpine
WORKDIR /app
ADD . /app
RUN cd /app && go build -o goapp
ENTRYPOINT ./goapp
Also as a bonus, here is how to create really tiny Docker images with multistage Docker builds:
FROM golang:1.11-alpine AS build-env
ADD . /src
RUN cd /src && go build -o goapp
FROM alpine
WORKDIR /app
COPY --from=build-env /src/goapp /app/
ENTRYPOINT ./goapp
I'm fairly new to both Docker and go, so this might be something obvious, but my google searches haven't found anything.
I'm trying to build a simple go program with docker, but I'm havign trouble with dependencies.
go file:
package main
import (
"fmt"
"log"
"html"
"net/http"
"github.com/gorilla/mux"
)
func hello(writer http.ResponseWriter, r *http.Request) {
path := mux.Vars(r)["rest"]
fmt.Fprintf(writer, "Hello, %q", html.EscapeString(path))
}
func main() {
router := mux.NewRouter().StrictSlash(true)
router.HandleFunc("/{rest:.*}", hello)
log.Println("Listening...")
log.Fatal(http.ListenAndServe(":8080", router))
}
Docker file:
FROM golang:latest
RUN mkdir /app
ADD ./HelloWorld.go /app/
WORKDIR /app
RUN go get ./*.go
RUN go build -o main .
CMD ["/app/main"]
Error:
Sending build context to Docker daemon 6.482MB
Step 1/7 : FROM golang:latest
---> 138bd936fa29
...
Step 5/7 : RUN go get ./*.go
---> Running in 1e29844961a2
HelloWorld.go:9:5: cannot find package "github.com/gorilla/mux" in any of:
/usr/local/go/src/github.com/gorilla/mux (from $GOROOT)
/go/src/github.com/gorilla/mux (from $GOPATH)
The command '/bin/sh -c go get ./*.go' returned a non-zero code: 1
You may use my Dockerfile as a base. First stage produces working image. That’s enough for many cases.
https://github.com/lisitsky/go-site-search-string/blob/light-docker/Dockerfile
If you want to shrink image size from ~800MB to about ~10-20MB use second stage too.
Just use $GOPATH everywhere in your path to build your images, e.g. $GOPATH/src/app instead of /app
Full example of multistage build:
FROM golang:alpine as builder
RUN apk add --update git
RUN mkdir -p $GOPATH/src/build
ADD . $GOPATH/src/build/
WORKDIR $GOPATH/src/build
RUN go get ./...
RUN go build -o main .
FROM alpine
RUN adduser -S -D -H -h /app appuser
USER appuser
COPY --from=builder /go/src/build/main /app/
WORKDIR /app
CMD ["./main"]