How to deploy a web app with static files in docker? - docker

I built a web app with echo. Some source in server.go is
package main
import ...
type TemplateRenderer struct {
templates *template.Template
}
func (t *TemplateRenderer) Render(w io.Writer, name string, data interface{}, c echo.Context) error {
if viewContext, isMap := data.(map[string]interface{}); isMap {
viewContext["reverse"] = c.Echo().Reverse
}
return t.templates.ExecuteTemplate(w, name, data)
}
func main() {
e := echo.New()
e.Static("/static", "static")
renderer := &TemplateRenderer{
templates: template.Must(template.ParseGlob("public/views/*.html")),
}
e.Renderer = renderer
e.GET("/", func(c echo.Context) error {
return c.Render(http.StatusOK, "index.html", map[string]interface{}{})
})
e.Logger.Fatal(e.Start(":8080"))
}
Project tree
├── helper
│   └── string.go
└─── site
├── Dockerfile
├── go.mod
├── go.sum
├── server.go
├── public
│   └── views
│   └── index.html
└── static
I can run go run server.go to start server, everything works well. But got error if run it with docker.
Dockerfile
FROM golang:1.15.2-alpine3.12 AS builder
WORKDIR /app
COPY . .
WORKDIR /app/site
RUN CGO_ENABLED=0 GOOS=linux go build -o server
FROM alpine:3.12
COPY --from=builder /app/site /bin/.
ENTRYPOINT [ "server" ]
Built a docker image by
docker build -t gcr.io/${PROJECT_ID}/myapp -f site/Dockerfile .
Run app in docker
docker run --rm -p 8080:8080 gcr.io/${PROJECT_ID}/myapp
panic: html/template: pattern matches no files: `public/views/*.html`
goroutine 1 [running]:
html/template.Must(...)
/usr/local/go/src/html/template/template.go:372
main.main()
/app/site/server.go:76 +0x2af
It seems the public folder didn't been copied into image. But there wasn't any error when built the image. What's wrong?

Firstly, you are using a multi-stage Docker build which is perfect for only copying the compiled binary to the final image. In your Dockerfile, however, you are copying the entire build directory - the binary server as well as all the source.
Secondly, your main problem is when the image is run as a container, the default work directory is / - and thus any paths within your server are not find the html files in /bin/public.
If you ever need to debug a docker image - especially if it is an image based on a linux distro like alpine- simply:
docker run -it myimage /bin/sh
Anyway the 2 simple fixes to your docker:
FROM golang:1.15.2-alpine3.12 AS builder
WORKDIR /app
COPY . .
WORKDIR /app/site
RUN CGO_ENABLED=0 GOOS=linux go build -o server
FROM alpine:3.12
COPY --from=builder /app/site/server /bin
COPY --from=builder /app/site/public /public
ENTRYPOINT [ "/bin/server" ]

In Go 1.16 you can compile these files into the binary itself. So you'll need to upgrade the Go toolchain on your host system, and also the FROM line in your build stage in the Dockerfile. Go 1.16 adds the embed package and a new //go:embed directive to support this.
First, you need to tell the compiler to embed the template files, building a filesystem object:
import "embed"
// templateFiles contains the raw text of the template files.
//go:embed public/views/*.html
var templateFiles embed.FS
Then, when you go to use it, Go 1.16 also adds a corresponding ("html/template").ParseFS function:
renderer := &TemplateRenderer{
templates: template.Must(template.ParseFS(templateFiles)),
}
Now all of the files are embedded in the binary itself, and you shouldn't get "file not found" type errors. You might consider copying only the compiled binary and not anything else into the final image.
# Upgrade to Go 1.16
FROM golang:1.16-alpine3.12 AS builder
# Unchanged from original
WORKDIR /app
COPY . .
WORKDIR /app/site
RUN CGO_ENABLED=0 GOOS=linux go build -o server
FROM alpine:3.12
# Only copy the compiled binary and not the source tree
COPY --from=builder /app/site/server /bin
# Generally prefer CMD to ENTRYPOINT
CMD [ "server" ]

Related

'No required module provides package' when building Go docker image

My Dockerfile is below:
# syntax=docker/dockerfile:1
FROM golang:1.18-alpine
WORKDIR /app
COPY go.mod ./
COPY go.sum ./
RUN go mod download
COPY *.go ./
RUN go build -o /datapuller
EXPOSE 8080
CMD [ "/datapuller" ]
I tried to build with $ docker build --tag datapuller .
But got error:
main.go:13:2: no required module provides package gitlab.com/mycorp/mycompany/data/datapuller/dbutil; to add it:
go get gitlab.com/mycorp/mycompany/data/datapuller/dbutil
main.go:14:2: no required module provides package gitlab.com/mycorp/mycompany/data/datapuller/models; to add it:
go get gitlab.com/mycorp/mycompany/data/datapuller/models
How to solve this, I can run directly with go run main.go just fine.
My main.go's import is below. I think the imports caused this problem:
package main
import (
"encoding/json"
client "github.com/bozd4g/go-http-client"
"github.com/robfig/cron/v3"
"github.com/xuri/excelize/v2"
"gitlab.com/mycorp/mycompany/data/datapuller/dbutil"
"gitlab.com/mycorp/mycompany/data/datapuller/models"
"gorm.io/gorm"
)
func main() {
...
Because the associated package needs to be pulled when building.
Docker may be missing the necessary environment variables to pull these packages.
It is recommended that you use the go mod vendor command,then build image
FROM golang:1.18-alpine
ADD . /go/src/<project name>
WORKDIR /go/src/<project name>
RUN go build -mod=vendor -v -o /go/src/bin/main main.go
RUN rm -rf /go/src/<project name>
WORKDIR /go/src/bin
CMD ["/go/src/bin/main"]
When you copy your source code into the image, you only copy files in the current directory
COPY *.go ./ # just the current directory's *.go, not any subdirectories
It's usually more common to copy in the entire host source tree, maybe using a .dockerignore file to cause some of the source tree to be ignored
COPY ./ ./
Otherwise you need to copy the specific subdirectories you need into the image (each directory needs a separate COPY command)
COPY *.go ./
COPY dbutil/ dbutil/
COPY models/ models/

Why my docker file does not copy the HTML files?

My directory is:
-Dockerfile
app/
-main.go
media/
/css
/html
/img
/svg
Inside the html folder, I have subfolders to organise my HTML files, so the path to the HTML files is media/html/*/*.html
And I have my Dockerfile as follows:
FROM golang:alpine
# Set necessary environmet variables needed for our image
ENV GO111MODULE=on \
CGO_ENABLED=0 \
GOOS=linux \
GOARCH=amd64
# Copy the code into the container
COPY media .
# Move to working directory /build
WORKDIR /build
# Copy the code from /app to the build folder into the container
COPY app .
# Configure the build (go.mod and go.sum are already copied with prior step)
RUN go mod download
# Build the application
RUN go build -o main .
WORKDIR /app
# Copy binary from build to main folder
RUN cp /build/main .
# Export necessary port
EXPOSE 8080
# Command to run when starting the container
CMD ["/app/main"]
and my main.go is:
package main
import (
"net/http"
"github.com/gin-gonic/gin"
)
func main() {
// We create the instance for Gin
r := gin.Default()
// Path to the static files. /static is rendered in the HTML and /media is the link to the path to the images, svg, css.. the static files
r.StaticFS("/static", http.Dir("../media"))
// Path to the HTML templates. * is a wildcard
r.LoadHTMLGlob("../media/html/*/*.html")
r.NoRoute(renderHome)
// This get executed when the users gets into our website in the home domain ("/")
r.GET("/", renderHome)
r.Run(":8080")
}
func renderHome(c *gin.Context) {
c.HTML(http.StatusOK, "my-html.html", gin.H{})
}
Problem is, I can run without problem my app in Golang with go run main.go, I can build the Docker image without problems, but on the moment to run a Docker container from the image, I got the error:
panic: html/template: pattern matches no files: ../media/html/*/*.html
The path is correct (since is also proven because I can run it in plain go) and it seems that Docker is not coping the files correctly, or at least not in the right directory. What is failing? The full simple project can be found here
media is a bad choice for a Docker folder, because a typical Linux container already has a /media folder.
But that's not the root cause.
The root cause is that COPY media . copies the contents of media folder to /. You probably want COPY media/ /media/ if you want to preserve the media folder itself (or use WORKDIR /media).
As a debug tool, you can run your container with a shell as entrypoint to "look around" it without starting your app:
docker build . -t test
docker run -it --rm test sh
/app # ls /media
cdrom floppy usb
/app # ls -R /html
/html:
website
/html/website:
my-html.html
As you can see your media/html folder is located at /html.
Some more notes:
It's a good idea to move go mod download to before COPY app so that the downloaded modules can be cached:
FROM golang:alpine
WORKDIR /build
COPY app/go.mod app/go.sum ./
RUN go mod download
COPY app .
RUN go build -o main .
WORKDIR /app
RUN cp /build/main .
COPY media /media/
EXPOSE 8080
CMD ["/app/main"]
As a next step you can look into two-stage builds to not depend on the golang image for running the compiled app (is only needed for building really).

Docker golang busybox "no such file or directory" error

I'm building a multistage Docker image of a golang microservice and I want to make it very thin using busybox as base image to run the final executable.
The image is correctly built, but when I run it I get this error:
standard_init_linux.go:211: exec user process caused "no such file or directory"
I'm working on my Ubuntu laptop, so this error has nothing to do with the Windows OS as many other questions report.
This is my image.
# build stage
FROM golang:1.15.3 AS build-stage
RUN mkdir /build
ADD . /build/
WORKDIR /build
RUN go mod download
RUN go test ./...
RUN go build -o goapp .
# final stage
FROM busybox
WORKDIR /app
COPY --from=build-stage /build/goapp /app/
CMD ["./goapp"]
A very simplified version of my project folder could be:
project
├── Dockerfile
├── go.mod
├── go.sum
├── main.go
└── other-packages
You are building your app with CGO_ENABLED=1 (this is Go's default setting for builds) - and transplanting that executable to a docker image where there is no compatible library support (glibc, dns resolver etc.).
To ensure your build does not rely on any external libraries - statically binding all dependencies - and can then be deployed to a SCRATCH docker image or busybox - disable CGO during your build phase:
RUN CGO_ENABLED=0 go build -o goapp .
I guess there's a difference between the libc of the two images.
As introduced in description of busybox image, there are several libc variants, which is distinguished by image tags.
The libc of golang:1.15.3 is glibc (it is FROM the corresponding version of debian:buster), so you should use busybox:glibc on your final stage.
The default busybox:latest is using uclibc. They are incompatible. Check the sha256 digest of busybox:latest and busybox:uclibc.

Dockerfile: Error response from daemon: OCI runtime create failed: container_linux.go:349

I am following directory structure based on:
https://github.com/golang-standards/project-layout
I have created very simple app and basically I want to containerize it.
Basically I have two files there. server.go which is definition of http endpoint and main file which starts server called main.go under cmd/webserver.
Directory of this project looks like:
./
├── cmd
│   └── webserver
│   └── main.go
├── Dockerfile
├── go.mod
└── server.go
go.mod
module github.com/geborskimateusz/auth
go 1.15
a Dockerfile looks like this
FROM golang:alpine
# Set necessary environmet variables needed for our image
ENV GO111MODULE=on \
CGO_ENABLED=0 \
GOOS=linux \
GOARCH=amd64
# Move to working directory /build
WORKDIR /build
# Copy and download dependency using go mod
COPY go.mod .
RUN go mod download
# Copy the code into the container
COPY . .
# Build the application
RUN go build -o main .
# Move to /dist directory as the place for resulting binary folder
WORKDIR /dist
# Copy binary from build to main folder
RUN cp /build/main .
# Export necessary port
EXPOSE 3000
# Command to run when starting the container
CMD ["/dist/main"]:
Build is successful but the problem is that when I run
docker run -p 3000:3000 geborskimateusz/auth I got:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/dist/main\": permission denied": unknown.
ERRO[0000] error waiting for container: context canceled
What am I missing? I assume that maybe I need to cd in Dockerfile into cmd/webserver where main.go (executable file) is placed.
I actually fixed this by modyfing DockerFile to
FROM golang:alpine
WORKDIR /app
COPY go.mod .
RUN go mod download
COPY . .
RUN cd ./cmd/webserver/ && go build -o main . && cp main ../../ && cd ../../
CMD ["./main"]

COPY failed: Forbidden path outside the build context

I have a monorepo that has holds various Go services and libraries. The directory structure is like the following:
monorepo
services
service-a
- Dockerfile
go.mod
go.sum
This go.mod file resides in the root of the monorepo directory and the services use the dependencies stated in that file.
I build the Docker image with this command:
docker build -t some:tag ./services/service-a/
When I try to build my Docker image from the root of monorepo directory with the above docker command I get the following error:
COPY failed: Forbidden path outside the build context: ../../go.mod ()
Below is my Dockerfile
FROM golang:1.14.1-alpine3.11
RUN apk add --no-cache ca-certificates git
# Enable Go Modules
ENV GO111MODULE=on
# Set the Current Working Directory inside the container
WORKDIR /app
# Copy go mod and sum files
COPY ../../go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o service-a
ENTRYPOINT ["/app/service-a"]
Is there something I have to do to be able to add files into my Docker image that aren't in the current directory without having to have a separate go.mod and go.sum in each service within the monorepo?
Docker only allows adding files to the image from the context, which is by default the directory containing the Dockerfile. You can specify a different context when you build, but again, it won't let you include files outside that context:
docker build -f ./services/service-a/Dockerfile .
This should use the current directory as the context.
Alternatively, you can create a temp directory, copy all the artifacts there and use that as the build context. This can be automated by a makefile or build script.
You can build and manage your docker containers using docker-compose, then this problem can be solved with the help context directive, for example:
project_folder
├─── src
│ └── folder1
│ └── folder2
│ └── Dockerfile
├── docker-compose.yaml
└── copied_file.ext
docker-compose.yaml
version: '3'
services:
your_service_name:
build:
context: ./ #project_folder for this case
dockerfile: ./src/folder1/folder2/Dockefile
Dockerfile
FROM xxx
COPY copied_file.ext /target_folder/
build or rebuild services:
docker-compose build
run a one-off command on a service:
docker-compose run your_service_name <command> [arguments]

Resources