I've seen many examples how to get an access to another build within one Dockerfile like here:
https://docs.docker.com/v17.09/engine/userguide/eng-image/multistage-build/#use-multi-stage-builds
FROM golang:1.7.3 as builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
Here is used COPY --FROM=builder
But can I get an access somehow, if I split the build into 2 Dockerfiles, from one to another?
When I do that it says: "failed to build: invalid from flag value client: pull access denied. repository does not exist or may require 'docker login'" How to get around it?
The Dockerfile documentation for COPY says, in part:
Optionally COPY accepts a flag --from=<name|index> that can be used to set the source location.... In case a build stage with a specified name can’t be found an image with the same name is attempted to be used instead.
So if you build your first half
docker build -t gena/app-builder -f Dockerfile.builder .
Then in the second half Dockerfile you should be able to
COPY --from=gena/app-builder /go/src/github.com/alexellis/href-counter/app .
Related
I'm able to run a docker image of a web app when using golang:1.13 base, but not when using scratch. The working Dockerfile is:
FROM golang:1.13 AS builder
WORKDIR /app
COPY . .
RUN go build -o server
FROM golang:1.13
COPY --from=builder /app/server /app/server
COPY --from=builder /app/credentials/service-account.json /app/credentials/service-account.json
ENTRYPOINT ["/app/server"]
But when I change the final image base to scratch (line 6) like this:
FROM golang:1.13 AS builder
WORKDIR /app
COPY . .
RUN go build -o server
FROM scratch # <-- CHANGED
COPY --from=builder /app/server /app/server
COPY --from=builder /app/credentials/service-account.json /app/credentials/service-account.json
ENTRYPOINT ["/app/server"]
I get a standard_init_linux.go:211: exec user process caused "no such file or directory" error.
To build the docker image, I use docker build -t myimage ., and to run the image, I use docker run --rm -p 8080:8080 myimage:latest.
The app is a Go based web API that uses Gin framework, and GCP Service Account to access GCP services (the JSON file that I copy on build.)
Provided you are not using CGO (as #jakub mentioned), try disabling CGO in your build.
So change this line in your Dockerfile:
#RUN go build -o server
RUN CGO_ENABLED=0 go build -o server
I’ve the following docker which works ok, I was able to run it and build it successfully!
FROM golang:1.13.6 AS build-env
ENV GO111MODULE=on
ENV GOOS=linux
ENV CGO_ENABLED=0
RUN mkdir -p /go/src/github.company.corp/deng/fst-cl
WORKDIR /go/src/github.company.corp/deng/fsr-clie
COPY ./ ./
# build the code
RUN go build -v -o ./fsr ./src/cmd/main.go
Now I want to change the image to use lighter docker image such as go alpine
So I change the from and added alpine version and also added git ,however the build is failing for
So go lib which doesn’t happen before the change, any idea what could be missing ?
FROM golang:1.13.6-alpine AS build-env
ENV GO111MODULE=on
ENV GOOS=linux
ENV CGO_ENABLED=0
## git is required to fetch go dependencies
RUN apk add --no-cache ca-certificates git
RUN apk add --no-cache gcc musl-dev
RUN mkdir -p /go/src/github.company.corp/deng/fst-cl
WORKDIR /go/src/github.company.corp/deng/fsr-clie
COPY ./ ./
# build the code
RUN go build -v -o ./fsr ./src/cmd/main.go
The error is for specifid repo which resides on our company git repo, but I don’t understand why its happen on golang:1.13.6-alpine and works ok on golang:1.13.6 ????
Btw I try to use different version of go alpine without success…
This is the error:
get "github.company.corp/deng/logger-ut": found meta tag get.metaImport{Prefix:"github.company.corp/deng/logger-ut", VCS:"git", RepoRoot:"https://github.company.corp/deng/logger-ut.git"} at //github.company.corp/deng/logger-ut?go-get=1
go: github.company.corp/deng/logger-ut#v1.0.0: reading github.company.corp/deng/logger-ut/go.mod at revision v1.0.0: unknown revision v1.0.0
If you want a lighter image and wish to use apline, you can use example below. Your final app image should be something like 7MB on scratch. Adjust it as it fits!
# STAGE 1: prepare
FROM golang:1.13.1-alpine3.10 as prepare
WORKDIR /source
COPY go.mod .
COPY go.sum .
RUN go mod download
# STAGE 2: build
FROM prepare AS build
COPY . .
RUN CGO_ENABLED=0 go build -ldflags "-s -w" -o bin/app -v your/app.go
# STAGE 3: run
FROM scratch as run
COPY --from=build /source/bin/app /app
ENTRYPOINT ["/app"]
I'm beginner with Docker, and I'm trying to build an image in two stages, as explained here: https://docs.docker.com/develop/develop-images/multistage-build/
You can selectively copy artifacts from one stage to another
Looking at the examples given there, I had thought that one could build some files during a first stage, and then make them available for the next one:
FROM golang:1.7.3 AS builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
(Example taken from the above-linked page)
Isn't that what the COPY app.go . and the COPY --from=builder /go/src/github.com/alexellis/href-counter/app . are supposed to do?
I probably have a complete misunderstanding of what is going on, because when I try to do something similar (see below), it seems that the COPY command from the first stage is not able to see the files that have just been built (I can confirm that they have been actually built using a RUN ls step, but then I get a lstat <the file>: no such file or directory error).
And indeed, most other information I can gather regarding COPY (except the examples in the above link) rather suggest that COPY is actually meant to copy files from the directory where the docker build command was launched, not from within the build environment.
Here is my Dockerfile:
FROM haskell:8.6.5 as haskell
RUN git clone https://gitlab+deploy-token-75:sakyTxfe-PxPHDwqsoGm#gitlab.pasteur.fr/bli/bioinfo_utils.git
WORKDIR bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
RUN stack --resolver ghc-8.6.5 build && \
stack --resolver ghc-8.6.5 install --local-bin-path .
RUN pwd; echo "---"; ls
COPY remove-duplicates-from-sorted-fastq .
FROM python:3.7-buster
RUN python3.7 -m pip install snakemake
RUN mkdir -p /opt/bin
COPY --from=haskell /bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell/remove-duplicates-from-sorted-fastq /opt/bin/remove-duplicates-from-sorted-fastq
CMD ["/bin/bash"]
And here is how the build ends when I run docker build . from the directory containing the Dockerfile:
Step 5/11 : RUN pwd; echo "---"; ls
---> Running in 28ff49fe9150
/bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
---
LICENSE
Setup.hs
install.sh
remove-duplicates-from-sorted-fastq
remove-duplicates-from-sorted-fastq.cabal
src
stack.yaml
---> f551efc6bba2
Removing intermediate container 28ff49fe9150
Step 6/11 : COPY remove-duplicates-from-sorted-fastq .
lstat remove-duplicates-from-sorted-fastq: no such file or directory
How am I supposed to proceed to have the built file available for the next stage?
Well, apparently, I was mislead by the COPY step used in the first stage in the doc example. In my case, this is actually useless, and I can just COPY --from=haskell in my second stage, without any COPY in the first stage.
The following Dockerfile builds without issues:
FROM haskell:8.6.5 as haskell
RUN git clone https://gitlab+deploy-token-75:sakyTxfe-PxPHDwqsoGm#gitlab.pasteur.fr/bli/bioinfo_utils.git
WORKDIR bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
RUN stack --resolver ghc-8.6.5 build && \
stack --resolver ghc-8.6.5 install --local-bin-path .
FROM python:3.7-buster
RUN python3.7 -m pip install snakemake
RUN mkdir -p /opt/bin
COPY --from=haskell /bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell/remove-duplicates-from-sorted-fastq /opt/bin/remove-duplicates-from-sorted-fastq
CMD ["/bin/bash"]
I have a Dockerfile with the following:
FROM i386/alpine
WORKDIR /tmp/src/mybuild
ADD . /tmp/src/mybuild
FROM travisci/travis-build
My goal is to end up with a 32 bit image which contains the stuff from both images. After running sudo docker build --rm --tag travis-32 ., the "image" is built, but when I run sudo docker run -it travis-32 /bin/bash, I end up in a bash terminal and typing uname -m gives me x86_64, which is clearly not 32 bit as I was expecting.
How can I make this work?
what's wrong with the Dockerfile:
The Dokcerfile mentioned in the question is a multi-stage build file. Every stage starts with FROM, where one is supposed to copy the artifacts from the first build stage into the second, in the goal to achieve a smaller docker image to use in production at the end.
in the mentioned Dokcerfile:
the first stage copies some files to an i386/alpine image:
FROM i386/alpine
WORKDIR /tmp/src/mybuild
ADD . /tmp/src/mybuild
then everything done is ignored, and another image is taken instead:
FROM travisci/travis-build
so the end result is an exact copy of travisci/travis-build.
Regarding the 32b,64b question:
Usually what is compiled under 32b works only under 32b, and in order for it to run under 64b, you need to compile it under 64b (except for some languages like go, where you can define the target platform), so one need to be careful.
Example:
Take a look, and notice where the artifacts are moved from stage to another using the COPY --from= statement:
FROM golang:1.7.3 as builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
More info can be found in the official docs: docker multistage-builds
hope this helps.
I have my Dockerfile in the root of directory with src/myapp folder, myapp contains myapp.go with main package.
Dockerfile looks like following:
FROM golang:1.9.2
ADD . /
RUN go build myapp;
ENTRYPOINT ["/go/bin/myapp"]
I get following error:
can't load package: package myapp: cannot find package "myapp" in any of:
/usr/local/go/src/myapp (from $GOROOT)
/go/src/myapp (from $GOPATH)
What am I doing wrong? Can I log ls command after docker has done ADD?
You are copying all the files to Image root directory, Didn't installed any dependencies, Trying to Build it and then run the binary from /go/bin/app. The binary doesn't exists in that directory and it's generating errors.
I would recommend using a Dockerfile like this,
FROM golang:1.9.2
ADD . /go/src/myapp
WORKDIR /go/src/myapp
RUN go get myapp
RUN go install
ENTRYPOINT ["/go/bin/myapp"]
This'll do the following.
Copy project files to /go/src/myapp.
Set Working directory to /go/src/myapp.
Install dependencies, I used go get but replace it with which ever dependency management tool you are using.
Install/build the binary.
Set entry point.
You can run ls or any other command using docker exec.
Example:
docker exec <image name/hash> ls
You can also enter the shell in the generated image to understand it well using
docker run --rm -it <image hash/name> /bin/sh
After experiments I've come to this way of building Golang apps.
This way has several advantages:
dependencies are installed on build stage
if you need you may uncomment test options
build first fully-functional image about 800 MB
copies your program to an fresh empty image and produces very small image about 10 MB
Dockerfile:
# Two-stage build:
# first FROM prepares a binary file in full environment ~780MB
# second FROM takes only binary file ~10MB
FROM golang:1.9 AS builder
RUN go version
COPY . "/go/src/github.com/your-login/your-project"
WORKDIR "/go/src/github.com/your-login/your-project"
#RUN go get -v -t .
RUN set -x && \
#go get github.com/2tvenom/go-test-teamcity && \
go get github.com/golang/dep/cmd/dep && \
dep ensure -v
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /your-app
CMD ["/your-app"]
EXPOSE 8000
#########
# second stage to obtain a very small image
FROM scratch
COPY --from=builder /your-app .
EXPOSE 8000
CMD ["/your-app"]
For go 1.11 , you can use go module, the following is example
FROM alpine AS base
RUN apk add --no-cache curl wget
FROM golang:1.11 AS go-builder
WORKDIR /go/app
COPY . /go/app
RUN GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /go/app/main /go/app/cmd/myapp/main.go
FROM base
COPY --from=go-builder /go/app/main /main
CMD ["/main"]
The official docs suggests the following Dockerfile:
FROM golang:1.8
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
Please, visit https://hub.docker.com/_/golang for more info
myapp needs to be in /go/src/myapp as suggested, or in /usr/local/go/src/myapp. You can add it in ADD section.
If the objective is to create a container that simply runs your binary, I would take different approach.
First build the binary for linux:
GOOS=linux CGO_ENABLED=0 go build -a -installsuffix cgo
Then build a lightweight docker image from scratch:
FROM scratch
COPY myApp
CMD ["/myApp"]