Currently, In my dockerfile i am using multiple COPY commands to copy directories from my repository.
COPY requirements.txt requirements.txt
COPY validation /opt/validation
COPY templates /opt/templates
COPY goss /opt/goss
COPY newman /opt/newman
COPY conftest.py /opt/validation/conftest.py
How can i achieve the same results as above using a single COPY command. Is there a way?
There is a little hack with the scratch image:
FROM scratch as tmp
COPY foo /opt/some/path/foo
COPY bar /usr/share/tmp/bar
FROM debian:buster
COPY --from=tmp / /
CMD bash -c "ls /opt/some/path /usr/share/tmp"
❯ docker build -t test . && docker run --rm test
/opt/some/path:
foo
/usr/share/tmp:
bar
scratch is a pseudo-image, it is much like an empty directory. The hack is to copy everything there as it should be in the final image, then merge root directories. The merge produces a single layer.
❯ docker inspect --format '{{.RootFS}}' test
{layers [
sha256:c2ddc1bc2645ab5d982c60434d8bbc6aecee1bd4e8eee0df7fd08c96df2d58bb
sha256:fd35279adf8471b9a168ec75e3ef830046d0d7944fe11570eef4d09e0edde936
] }
If you just want to copy things to the same folder /opt, maybe simply using next:
Folder structure:
.
├── conftest.py
├── Dockerfile
├── .dockerignore
├── goss
├── newman
├── requirements.txt
├── templates
└── validation
Dockerfile:
FROM alpine
COPY . /opt
#RUN mv /opt/conftest.py /opt/validation
RUN ls /opt
.dockerignore:
Dockerfile
Execution:
$ docker build -t abc:1 . --no-cache
Sending build context to Docker daemon 6.144kB
Step 1/3 : FROM alpine
---> 28f6e2705743
Step 2/3 : COPY . /opt
---> 8beb53be958c
Step 3/3 : RUN ls /opt
---> Running in cfc9228124fb
conftest.py
goss
newman
requirements.txt
templates
validation
Removing intermediate container cfc9228124fb
---> 4cdb9275d6f4
Successfully built 4cdb9275d6f4
Successfully tagged abc:1
Here, we use COPY . /opt to copy all things in current folder to /opt/ of container. We use .dockerignore to ignore the files/folders which won't want to copy to containers.
Additional, not sure the rules for COPY conftest.py /opt/validation/conftest.py correct or not, if it's correct, you may have to use RUN mv to move it to specified folder.
Related
I do
git clone https://github.com/openzipkin/zipkin.git
cd zipkin
The create a Dockerfile as below
FROM openjdk
RUN mkdir app
WORKDIR /app
COPY ./ .
ENTRYPOINT ["sleep", "1000000"]
then
docker build -t abc .
docker run abc
I then run docker exec -it CONTAINER_ID bash
pwd returns /app which is expected
but I ls and see that the files are not copied
only the directories and the xml file is copied into the /app directory
What is the reason? how to fix it?
Also I tried
FROM openjdk
RUN mkdir app
WORKDIR /app
COPY . /app
ENTRYPOINT ["sleep", "1000000"]
That repository contains a .dockerignore file which excludes everything except a set of things it selects.
That repository's docker directory also contains several build scripts for official images and you may find it easier to start your custom image FROM openzipkin/zipkin rather than trying to reinvent it.
I built a web app with echo. Some source in server.go is
package main
import ...
type TemplateRenderer struct {
templates *template.Template
}
func (t *TemplateRenderer) Render(w io.Writer, name string, data interface{}, c echo.Context) error {
if viewContext, isMap := data.(map[string]interface{}); isMap {
viewContext["reverse"] = c.Echo().Reverse
}
return t.templates.ExecuteTemplate(w, name, data)
}
func main() {
e := echo.New()
e.Static("/static", "static")
renderer := &TemplateRenderer{
templates: template.Must(template.ParseGlob("public/views/*.html")),
}
e.Renderer = renderer
e.GET("/", func(c echo.Context) error {
return c.Render(http.StatusOK, "index.html", map[string]interface{}{})
})
e.Logger.Fatal(e.Start(":8080"))
}
Project tree
├── helper
│ └── string.go
└─── site
├── Dockerfile
├── go.mod
├── go.sum
├── server.go
├── public
│ └── views
│ └── index.html
└── static
I can run go run server.go to start server, everything works well. But got error if run it with docker.
Dockerfile
FROM golang:1.15.2-alpine3.12 AS builder
WORKDIR /app
COPY . .
WORKDIR /app/site
RUN CGO_ENABLED=0 GOOS=linux go build -o server
FROM alpine:3.12
COPY --from=builder /app/site /bin/.
ENTRYPOINT [ "server" ]
Built a docker image by
docker build -t gcr.io/${PROJECT_ID}/myapp -f site/Dockerfile .
Run app in docker
docker run --rm -p 8080:8080 gcr.io/${PROJECT_ID}/myapp
panic: html/template: pattern matches no files: `public/views/*.html`
goroutine 1 [running]:
html/template.Must(...)
/usr/local/go/src/html/template/template.go:372
main.main()
/app/site/server.go:76 +0x2af
It seems the public folder didn't been copied into image. But there wasn't any error when built the image. What's wrong?
Firstly, you are using a multi-stage Docker build which is perfect for only copying the compiled binary to the final image. In your Dockerfile, however, you are copying the entire build directory - the binary server as well as all the source.
Secondly, your main problem is when the image is run as a container, the default work directory is / - and thus any paths within your server are not find the html files in /bin/public.
If you ever need to debug a docker image - especially if it is an image based on a linux distro like alpine- simply:
docker run -it myimage /bin/sh
Anyway the 2 simple fixes to your docker:
FROM golang:1.15.2-alpine3.12 AS builder
WORKDIR /app
COPY . .
WORKDIR /app/site
RUN CGO_ENABLED=0 GOOS=linux go build -o server
FROM alpine:3.12
COPY --from=builder /app/site/server /bin
COPY --from=builder /app/site/public /public
ENTRYPOINT [ "/bin/server" ]
In Go 1.16 you can compile these files into the binary itself. So you'll need to upgrade the Go toolchain on your host system, and also the FROM line in your build stage in the Dockerfile. Go 1.16 adds the embed package and a new //go:embed directive to support this.
First, you need to tell the compiler to embed the template files, building a filesystem object:
import "embed"
// templateFiles contains the raw text of the template files.
//go:embed public/views/*.html
var templateFiles embed.FS
Then, when you go to use it, Go 1.16 also adds a corresponding ("html/template").ParseFS function:
renderer := &TemplateRenderer{
templates: template.Must(template.ParseFS(templateFiles)),
}
Now all of the files are embedded in the binary itself, and you shouldn't get "file not found" type errors. You might consider copying only the compiled binary and not anything else into the final image.
# Upgrade to Go 1.16
FROM golang:1.16-alpine3.12 AS builder
# Unchanged from original
WORKDIR /app
COPY . .
WORKDIR /app/site
RUN CGO_ENABLED=0 GOOS=linux go build -o server
FROM alpine:3.12
# Only copy the compiled binary and not the source tree
COPY --from=builder /app/site/server /bin
# Generally prefer CMD to ENTRYPOINT
CMD [ "server" ]
I am following directory structure based on:
https://github.com/golang-standards/project-layout
I have created very simple app and basically I want to containerize it.
Basically I have two files there. server.go which is definition of http endpoint and main file which starts server called main.go under cmd/webserver.
Directory of this project looks like:
./
├── cmd
│ └── webserver
│ └── main.go
├── Dockerfile
├── go.mod
└── server.go
go.mod
module github.com/geborskimateusz/auth
go 1.15
a Dockerfile looks like this
FROM golang:alpine
# Set necessary environmet variables needed for our image
ENV GO111MODULE=on \
CGO_ENABLED=0 \
GOOS=linux \
GOARCH=amd64
# Move to working directory /build
WORKDIR /build
# Copy and download dependency using go mod
COPY go.mod .
RUN go mod download
# Copy the code into the container
COPY . .
# Build the application
RUN go build -o main .
# Move to /dist directory as the place for resulting binary folder
WORKDIR /dist
# Copy binary from build to main folder
RUN cp /build/main .
# Export necessary port
EXPOSE 3000
# Command to run when starting the container
CMD ["/dist/main"]:
Build is successful but the problem is that when I run
docker run -p 3000:3000 geborskimateusz/auth I got:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/dist/main\": permission denied": unknown.
ERRO[0000] error waiting for container: context canceled
What am I missing? I assume that maybe I need to cd in Dockerfile into cmd/webserver where main.go (executable file) is placed.
I actually fixed this by modyfing DockerFile to
FROM golang:alpine
WORKDIR /app
COPY go.mod .
RUN go mod download
COPY . .
RUN cd ./cmd/webserver/ && go build -o main . && cp main ../../ && cd ../../
CMD ["./main"]
I have a monorepo that has holds various Go services and libraries. The directory structure is like the following:
monorepo
services
service-a
- Dockerfile
go.mod
go.sum
This go.mod file resides in the root of the monorepo directory and the services use the dependencies stated in that file.
I build the Docker image with this command:
docker build -t some:tag ./services/service-a/
When I try to build my Docker image from the root of monorepo directory with the above docker command I get the following error:
COPY failed: Forbidden path outside the build context: ../../go.mod ()
Below is my Dockerfile
FROM golang:1.14.1-alpine3.11
RUN apk add --no-cache ca-certificates git
# Enable Go Modules
ENV GO111MODULE=on
# Set the Current Working Directory inside the container
WORKDIR /app
# Copy go mod and sum files
COPY ../../go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o service-a
ENTRYPOINT ["/app/service-a"]
Is there something I have to do to be able to add files into my Docker image that aren't in the current directory without having to have a separate go.mod and go.sum in each service within the monorepo?
Docker only allows adding files to the image from the context, which is by default the directory containing the Dockerfile. You can specify a different context when you build, but again, it won't let you include files outside that context:
docker build -f ./services/service-a/Dockerfile .
This should use the current directory as the context.
Alternatively, you can create a temp directory, copy all the artifacts there and use that as the build context. This can be automated by a makefile or build script.
You can build and manage your docker containers using docker-compose, then this problem can be solved with the help context directive, for example:
project_folder
├─── src
│ └── folder1
│ └── folder2
│ └── Dockerfile
├── docker-compose.yaml
└── copied_file.ext
docker-compose.yaml
version: '3'
services:
your_service_name:
build:
context: ./ #project_folder for this case
dockerfile: ./src/folder1/folder2/Dockefile
Dockerfile
FROM xxx
COPY copied_file.ext /target_folder/
build or rebuild services:
docker-compose build
run a one-off command on a service:
docker-compose run your_service_name <command> [arguments]
So, I am trying to dockerize a golang application with different directories containing supplementary code for my main file.
I am using gorilla/mux. The directory structure looks like this.
$GOPATH/src/github.com/user/server
|--- Dockerfile
|--- main.go
|--- routes/
handlers.go
|--- public/
index.gohtml
It works on my host machine with no problem. The problem is that when I try to deploy the docker image it does not run and exits shortly after creation. I have tried changing the WORKDIR command in my dockerfile to /go/src and dump all my files there, but still no luck. I have also tried the official documentation on docker hub. Doesn't work either.
My Dockerfile.
FROM golang:latest
WORKDIR /go/src/github.com/user/server
COPY . .
RUN go get -d github.com/gorilla/mux
EXPOSE 8000
CMD ["go","run","main.go"]
My golang main.go
package main
import (
"github.com/gorilla/mux"
"github.com/user/server/routes"
"log"
"net/http"
"time"
)
func main(){
//...
}
I get this error message when I check the logs of my docker image.
Error Message
main.go:5:2: cannot find package "github.com/user/server/routes" in any of:
/usr/local/go/src/github.com/user/server/routes (from $GOROOT)
/go/src/github.com/user/server/routes (from $GOPATH)
Try the following Docker file:
# GO Repo base repo
FROM golang:1.12.0-alpine3.9 as builder
RUN apk add git
# Add Maintainer Info
LABEL maintainer="<>"
RUN mkdir /app
ADD . /app
WORKDIR /app
COPY go.mod go.sum ./
# Download all the dependencies
RUN go mod download
COPY . .
# Build the Go app
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
# GO Repo base repo
FROM alpine:latest
RUN apk --no-cache add ca-certificates curl
RUN mkdir /app
WORKDIR /app/
# Copy the Pre-built binary file from the previous stage
COPY --from=builder /app/main .
# Expose port 8000
EXPOSE 8000
# Run Executable
CMD ["./main"]
Here, we are creating an intermediate docker builder container, copying the code into it, build the code inside the builder container and then copy the binary image to the actual docker.
This will help in both having all the dependencies in the final container and also, the size of the final image will be very small