I have created a docker file in which I have installed golang dep tool which will be used to install the dependencies required by golang project. I have been able to install the tool. But unable to install the dependencies using that tool. I am not sure how to configure dep tool to be able to run dep command in docker image which will install all the dependencies required by golang project
I am using below command to run dep tool and it works in local machine
# initialize the project and install dependencies
RUN dep init
I am always getting an error:
init failed: unable to determine the import path for the root project
/go: /go is not within any GOPATH/src
Now I do not know if I need to set path to o binary files or how I can achieve that. There are tutorials to build a docker file to build golang project but nothing is there on internet to install dependencies using golang dep tool.
Here is an example of Dockerfile with dep:
FROM golang:latest
LABEL version="1.0"
RUN mkdir /go/src/app
RUN go get -u github.com/golang/dep/cmd/dep
ADD ./main.go /go/src/app
COPY ./Gopkg.toml /go/src/app
WORKDIR /go/src/app
RUN dep ensure
RUN go test -v
RUN go build
CMD ["./app"]
You need to change the directory to that of your project. Also, in order to get dependencies, you will usually already have a Gopkg.toml and Gopkg.lock - dep init is ONLY used when you're moving from a project which was using another vendoring tool, no vendoring at all or you're starting a project from scratch.
To sum up, I'd do something like this:
FROM golang:latest
RUN go get -u github.com/golang/dep/cmd/dep \
&& mkdir /go/src/github.com/you \
&& git clone https://github.com/you/yourproject /go/src/github.com/you/yourproject
WORKDIR /go/src/github.com/you/yourproject
RUN dep ensure -v
&& go build
CMD ["./yourproject"]
Related
Using the official golang docker image, I can use the protoc command to generate the x.pb.go and x_grpc.pb.go files. The problem is that it uses the latest versions, while I want to generate those using whichever version that is part of the go.mod file.
I tried to start from the golang image, then get my project's go.mod file, get the dependencies and try to generate from there. Here is my dockerfile:
FROM golang:1.15
WORKDIR /app
RUN apt-get update
RUN apt install -y protobuf-compiler
COPY go.* .
RUN go mod download
RUN go get all
RUN export PATH="$PATH:$(go env GOPATH)/bin"
RUN mkdir /api
Then I try to bind the volume of the .proto file and the /pb folder to output them, and use the protoc command again (I'm trying directly from the docker right now). Something like this:
protoc --proto_path=/api --go_out=/pb --go-grpc_out=/pb /api/x.proto
I'm getting this error though:
protoc-gen-go: program not found or is not executable
--go_out: protoc-gen-go: Plugin failed with status code 1.
My go.sum file has google.golang.org/protobuf v1.25.0 in it, so how come it is not found?
go.mod & go.sum are used for versioning when building go programs. This is not what you need here. You want the protoc compiler to use the correct plugin versions when running it against your .proto file(s).
To install the desired protoc-gen-go (and protoc-gen-go-grpc if using gRPC) plugins, install them directly. Update your Dockerfile like so:
FROM golang:1.15
WORKDIR /app
RUN apt-get update
RUN apt install -y protobuf-compiler
RUN GO111MODULE=on \
go get google.golang.org/protobuf/cmd/protoc-gen-go#v1.25.0 \
google.golang.org/grpc/cmd/protoc-gen-go-grpc#v1.1.0
# export is redundant here `/go/bin` is already in `golang` image's path
# (and actual any env change here is lost once the command completes)
# RUN export PATH="$PATH:$(go env GOPATH)/bin"
RUN mkdir /api
If you want the latest versions of either plugin, either use #latest - or drop the # suffix
I'm trying to use the lilliput library for Go. It is only made to run on Linux and OS X.
On my Linux (Debian 10.3) host machine as well as my WSL2 setup (Ubuntu 20.04.1), I have no problems running and building my code (excerpt below).
// main.go
package main
import (
"github.com/discordapp/lilliput"
)
func main() {
...
decoder, err := lilliput.NewDecoder(data)
...
}
However, when I try to put it in a Docker container, with the following configuration, it fails to build.
# Dockerfile v1
FROM golang:1.14.4-alpine AS build
RUN apk add build-base
WORKDIR /src
ENV CGO_ENABLED=1
COPY go.mod .
COPY go.sum .
RUN go mod download
COPY . .
RUN go build -o /out/api .
ENTRYPOINT ["/out/api"]
EXPOSE 8080
I already tried adjusting the Dockerfile with different approaches, for example:
FROM alpine:edge AS build
RUN apk update
RUN apk upgrade
RUN apk add --update go=1.15.3-r0 gcc=10.2.0-r5 g++=10.2.0-r5
WORKDIR /app
RUN go env
ENV GOPATH /app
ADD . /app/src
WORKDIR /app/src
RUN go get -d -v
RUN CGO_ENABLED=1 GOOS=linux go build -o /app/bin/server
FROM alpine:edge
WORKDIR /app
RUN cd /app
COPY --from=build /app/bin/server /app/bin/server
CMD ["bin/server"]
Both result in the following build log:
https://pastebin.com/zMEbEac3
For completeness, the go env of the host machine.
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/kingofdog/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/kingofdog/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go-1.11"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.11/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/kingofdog/{PROJECT FOLDER}/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build589460337=/tmp/go-build -gno-record-gcc-switches"
I already searched online for this error, but all I could find dealt with errors in the way others imported C libraries in their Go project. Yet, in my case I'm quite sure that it is not a mistake regarding the source code but rather a configuration mistake of the docker container, as the code itself works perfectly fine outside Docker and I couldn't find a similar issue on the lilliput repository.
The alpine docker image is a minimalistic Linux version - using musl-libc instead of glibc - and is typically used for building tiny images.
To get the more featureful glibc - and resolve your missing CGO dependencies - use the non-alpine version of the golang Docker image to build your asset:
#FROM golang:1.14.4-alpine AS build
#RUN apk add build-base
FROM golang:1.14.4 AS build
Did you build the dependencies?
You have to run the script to build the dependencies on Linux.
Script: https://github.com/discord/lilliput/blob/master/deps/build-deps-linux.sh
Their documentation mentions:
Building Dependencies
Go does not provide any mechanism for arbitrary building of dependencies, e.g. invoking make or cmake. In order to make lilliput usable as a standard Go package, prebuilt static libraries have been provided for all of lilliput's dependencies on Linux and OSX. In order to automate this process, lilliput ships with build scripts alongside compressed archives of the sources of its dependencies. These build scripts are provided for OSX and Linux.
In case it still fails, then issue might be linked to glibc-musl because alpine images have musl libc instead of glibc (GNU's libc). So, you can try it with maybe Ubuntu/ CentOS/etc. minimal images or find a way to get glibc on alpine.
Everytime I build the container I have to wait for apk add docker to finish which takes a long time.
Since everytime it downloads the same thing, can I somehow force Docker to cache apk's downloads for development purposes?
Here's my Dockerfile:
FROM golang:1.13.5-alpine
WORKDIR /go/src/app
COPY src .
RUN go get -d -v ./...
RUN go install -v ./...
RUN apk add --update docker
CMD ["app"]
BTW, I am using this part volumes: - /var/run/docker.sock:/var/run/docker.sock in my docker-compose.yml to use sibling containers, if that matters.
EDIT: I've found google to copy docker.tgz in Chromium:
# add docker client -- do not install docker via apk -- it will try to install
# docker engine which takes a lot of space as well (we don't need it, we need
# only the small client to communicate with the host's docker server)
ADD build/docker/docker.tgz /
What is that docker.tgz? How can I get it?
Reorder your Dockerfile and it should work.
FROM golang:1.13.5-alpine
RUN apk add --update docker
WORKDIR /go/src/app
COPY src .
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
As you are copying before installation, so whenever you change something in src the cache will invalidate for docker installtion.
Whenever you have a COPY command, if any of the files involve change, it causes every command after that to get re-run. If you move your RUN apk add ... command to the start of the file before it COPYs anything, it will get cached across runs.
A fairly generic recipe for most Dockerfiles to accommodate this pattern looks like:
FROM some-base-image
# Install OS-level dependencies
RUN apk add or apt-get install ...
WORKDIR /app
# Install language-level dependencies
COPY requirements.txt requirements.lock ./
RUN something install -r requirements.txt
# Install the rest of the application
COPY main.app ./
COPY src src/
# Set up standard run-time metadata
EXPOSE 12345
CMD ["/app/main.app"]
(Go and Java applications need the additional step of compiling the application, which often lends itself to a multi-stage build, but this same pattern can be repeated in both stages.)
You can download Docker x86_64 binaries for mac, linux, windows and unzip/untar and make it executable.
Whenever you are installing any packages in Docker container those should go at the beginning of Dockerfile, so it won’t ask you again to install same packages and COPY command part must be at the end of Dockerfile.
I want to put my code inside a docker container, I have created dockerfile and when I run, I got an error
internal/server/handlers.go:16:2: cannot find package "github.com/lib/pq" in any of:
/usr/local/go/src/github.com/lib/pq (from $GOROOT)
/go/src/github.com/lib/pq (from $GOPATH)
but when I launch my code locally without docker by typing go run main.go everything is fine
Make sure you installed all your package inside container. Because your docker container is a different machine with your current computer. You need to make sure that all dependencies package installed in your docker image. For an Dockefile example, install my package at Dockerfile as you can see:
FROM golang:latest
# Create working folder
RUN mkdir /app
COPY . /app
RUN apt -y update && apt -y install git
RUN go get github.com/go-sql-driver/mysql
RUN go get github.com/gosimple/slug
RUN go get github.com/gin-gonic/gin
RUN go get gopkg.in/russross/blackfriday.v2
RUN go get github.com/gin-gonic/contrib/sessions
WORKDIR /app
Now you run docker run -it -p 8080:8080 your_docker_image_name go run main.go should work.
This question already has answers here:
Build Docker with Go app: cannot find package
(6 answers)
Closed 3 years ago.
I have a project with the path /Users/me/Documents/dev/grafana/src/github.com/grafana/grafana. This project uses several other projects, for example:
/Users/me/Documents/dev/grafana/src/github.com/BurntSushi/toml
/Users/me/Documents/dev/grafana/src/github.com/Unknwon/com
I can build everything fine on my machine, but when I try to build within Docker, I get a bunch of cannot find package errors.
go install -v ./pkg/cmd/grafana-server
pkg/login/ldap_settings.go:7:2: cannot find package "github.com/BurntSushi/toml" in any of:
/usr/local/go/src/github.com/BurntSushi/toml (from $GOROOT)
/go/src/github.com/BurntSushi/toml (from $GOPATH)
pkg/services/notifications/codes.go:9:2: cannot find package "github.com/Unknwon/com" in any of:
/usr/local/go/src/github.com/Unknwon/com (from $GOROOT)
/go/src/github.com/Unknwon/com (from $GOPATH)
When I build myself, I have $GOPATH=/Users/me/Documents/dev/grafana/ -- in my Dockerfile I have:
FROM golang:latest AS build
RUN go version
ENV SRC_DIR=/go/src/github.com/grafana/grafana/
ENV GIT_SSL_NO_VERIFY=1
COPY . $SRC_DIR
WORKDIR $SRC_DIR
[... dependency installations ...]
# Building of Grafana
RUN npm run build
RUN go run build.go setup
RUN go run build.go build
I can't figure out why this step (Wich starts in the RUN go run build.go setup step) keeps reporting that it can't access the packages.
I've looked around for similar questions, but almost everything related doesn't specify building in Docker (and the ones that do aren't super helpful for this scenario).
Can you try the below and see if that works.
Step 1: Install go dep (go get -u github.com/golang/dep/cmd/dep)
Step 2: Run command dep init within your project. This will create a Gopkg.toml and Gopkg.lock
Step 3: Change your docker file
FROM golang:latest AS build
RUN go version
WORKDIR /go/src/github.com/grafana/grafana/
ENV GIT_SSL_NO_VERIFY=1
COPY . .
RUN go get -u github.com/golang/dep/cmd/dep && \
dep ensure && \
npm run build && \
RUN go run build.go setup && \
RUN go run build.go build && \
How dep works is when you run command dep ensure it pulls all the dependencies and puts them inside the vendor directory and can be easily accessed by your code.