I have a Dockerfile with the following:
FROM i386/alpine
WORKDIR /tmp/src/mybuild
ADD . /tmp/src/mybuild
FROM travisci/travis-build
My goal is to end up with a 32 bit image which contains the stuff from both images. After running sudo docker build --rm --tag travis-32 ., the "image" is built, but when I run sudo docker run -it travis-32 /bin/bash, I end up in a bash terminal and typing uname -m gives me x86_64, which is clearly not 32 bit as I was expecting.
How can I make this work?
what's wrong with the Dockerfile:
The Dokcerfile mentioned in the question is a multi-stage build file. Every stage starts with FROM, where one is supposed to copy the artifacts from the first build stage into the second, in the goal to achieve a smaller docker image to use in production at the end.
in the mentioned Dokcerfile:
the first stage copies some files to an i386/alpine image:
FROM i386/alpine
WORKDIR /tmp/src/mybuild
ADD . /tmp/src/mybuild
then everything done is ignored, and another image is taken instead:
FROM travisci/travis-build
so the end result is an exact copy of travisci/travis-build.
Regarding the 32b,64b question:
Usually what is compiled under 32b works only under 32b, and in order for it to run under 64b, you need to compile it under 64b (except for some languages like go, where you can define the target platform), so one need to be careful.
Example:
Take a look, and notice where the artifacts are moved from stage to another using the COPY --from= statement:
FROM golang:1.7.3 as builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
More info can be found in the official docs: docker multistage-builds
hope this helps.
Related
Please see the command below:
docker build -t iansbasecontainer:v1 -f DotnetDebug.Dockerfile .
It creates one container as shown below:
DotnetDebug.Dockerfile looks like this:
FROM microsoft/aspnetcore:2.0 AS base
# Install the SSHD server
RUN apt-get update \
&& apt-get install -y --no-install-recommends openssh-server \
&& mkdir -p /run/sshd \
&& echo "root:Docker!" | chpasswd
#Copy settings file. See elsewhere to find them.
COPY sshd_config /etc/ssh/sshd_config
COPY authorized_keys root/.ssh/authorized_keys
# Install Visual Studio Remote Debugger
RUN apt-get install zip unzip
RUN curl -sSL https://aka.ms/getvsdbgsh | bash /dev/stdin -v latest -l ~/vsdbg
EXPOSE 2222
I then run this command:
docker build -t iansimageusesbasecontainer:v1 -f DebugASP.Dockerfile .
However, two images appear:
DebugASP.Dockerfile looks like this:
FROM iansbasecontainer:v1 AS base
WORKDIR /app
MAINTAINER Vladimir Vladimir#akopyan.me
FROM microsoft/aspnetcore-build:2.0 AS build
WORKDIR /src
COPY ./DebugSample .
RUN dotnet restore
FROM build AS publish
RUN dotnet publish -c Debug -o /app
FROM base AS final
COPY --from=publish /app /app
COPY ./StartSSHAndApp.sh /app
EXPOSE 5000
CMD /app/StartSSHAndApp.sh
#If you wish to only have SSH running and start
#your service when you start debugging
#then use just the SSH server, you don't need the script
#CMD ["/usr/sbin/sshd", "-D"]
Why do two images appear? Please note I am relatively new to Docker so this may be a simple answer. I have spent the last few hours Googling it.
Also why is the repository and tag set to: .
Why do two images appear?
As mentioned here:
When using multi-stage builds, each stage produces a new image. That image is stored in the local image cache and will be used on subsequent builds (as part of the caching mechanism). You can run each build-stage (and/or tag the stage, if desired).
Read more about multi-stage builds here.
Docker produces intermediate(aka <none>:<none>) images for each layer, which are later used for final image. You can actually see them if execute docker images -a command.
But what you see is called dangling image. It happens, because some intermediate image is no longer used by final image. In case of multi-stage builds -- images for previous stages are not used in final image, so they become dangling.
Dangling images are useless and use your space, so it's recommended to regularly get rid of them(it's called pruning). You can do that with command:
docker image prune
I'm beginner with Docker, and I'm trying to build an image in two stages, as explained here: https://docs.docker.com/develop/develop-images/multistage-build/
You can selectively copy artifacts from one stage to another
Looking at the examples given there, I had thought that one could build some files during a first stage, and then make them available for the next one:
FROM golang:1.7.3 AS builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
(Example taken from the above-linked page)
Isn't that what the COPY app.go . and the COPY --from=builder /go/src/github.com/alexellis/href-counter/app . are supposed to do?
I probably have a complete misunderstanding of what is going on, because when I try to do something similar (see below), it seems that the COPY command from the first stage is not able to see the files that have just been built (I can confirm that they have been actually built using a RUN ls step, but then I get a lstat <the file>: no such file or directory error).
And indeed, most other information I can gather regarding COPY (except the examples in the above link) rather suggest that COPY is actually meant to copy files from the directory where the docker build command was launched, not from within the build environment.
Here is my Dockerfile:
FROM haskell:8.6.5 as haskell
RUN git clone https://gitlab+deploy-token-75:sakyTxfe-PxPHDwqsoGm#gitlab.pasteur.fr/bli/bioinfo_utils.git
WORKDIR bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
RUN stack --resolver ghc-8.6.5 build && \
stack --resolver ghc-8.6.5 install --local-bin-path .
RUN pwd; echo "---"; ls
COPY remove-duplicates-from-sorted-fastq .
FROM python:3.7-buster
RUN python3.7 -m pip install snakemake
RUN mkdir -p /opt/bin
COPY --from=haskell /bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell/remove-duplicates-from-sorted-fastq /opt/bin/remove-duplicates-from-sorted-fastq
CMD ["/bin/bash"]
And here is how the build ends when I run docker build . from the directory containing the Dockerfile:
Step 5/11 : RUN pwd; echo "---"; ls
---> Running in 28ff49fe9150
/bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
---
LICENSE
Setup.hs
install.sh
remove-duplicates-from-sorted-fastq
remove-duplicates-from-sorted-fastq.cabal
src
stack.yaml
---> f551efc6bba2
Removing intermediate container 28ff49fe9150
Step 6/11 : COPY remove-duplicates-from-sorted-fastq .
lstat remove-duplicates-from-sorted-fastq: no such file or directory
How am I supposed to proceed to have the built file available for the next stage?
Well, apparently, I was mislead by the COPY step used in the first stage in the doc example. In my case, this is actually useless, and I can just COPY --from=haskell in my second stage, without any COPY in the first stage.
The following Dockerfile builds without issues:
FROM haskell:8.6.5 as haskell
RUN git clone https://gitlab+deploy-token-75:sakyTxfe-PxPHDwqsoGm#gitlab.pasteur.fr/bli/bioinfo_utils.git
WORKDIR bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
RUN stack --resolver ghc-8.6.5 build && \
stack --resolver ghc-8.6.5 install --local-bin-path .
FROM python:3.7-buster
RUN python3.7 -m pip install snakemake
RUN mkdir -p /opt/bin
COPY --from=haskell /bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell/remove-duplicates-from-sorted-fastq /opt/bin/remove-duplicates-from-sorted-fastq
CMD ["/bin/bash"]
I've seen many examples how to get an access to another build within one Dockerfile like here:
https://docs.docker.com/v17.09/engine/userguide/eng-image/multistage-build/#use-multi-stage-builds
FROM golang:1.7.3 as builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
Here is used COPY --FROM=builder
But can I get an access somehow, if I split the build into 2 Dockerfiles, from one to another?
When I do that it says: "failed to build: invalid from flag value client: pull access denied. repository does not exist or may require 'docker login'" How to get around it?
The Dockerfile documentation for COPY says, in part:
Optionally COPY accepts a flag --from=<name|index> that can be used to set the source location.... In case a build stage with a specified name can’t be found an image with the same name is attempted to be used instead.
So if you build your first half
docker build -t gena/app-builder -f Dockerfile.builder .
Then in the second half Dockerfile you should be able to
COPY --from=gena/app-builder /go/src/github.com/alexellis/href-counter/app .
I have a Dockerfile that builds a golang project (that listens to the Twitter stream and lists the tweets by some filter) from the latest golang docker image, right now 1.10.3, like so:
FROM golang:1.10.3
COPY . /destination/
WORKDIR /destination/
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
CMD ["./main"]
and when I run the image that is created by the docker build command, it runs without a problem. The problem is that the image is about 900MB in size.
When I try to build the docker with the multistage build, like so:
FROM golang:1.10.3-alpine3.7 as builder
...
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
FROM busybox
COPY --from=builder /go/src/github.com/mygithubname/ /
WORKDIR /path-to-the-main-file/
CMD ["./main"]
The docker image is built successfully and it runs, but it just doesn't show me tweets when they are being posted, and it also doesn't show any errors, that is the most confusing part. Even when I ssh into the running container and ping google it shows everything as it should be.
Anyone has some hints how I could debug this problem, or to get some more logs out of the running Docker image?
Few things
1) Did the container exit? If you do a docker ps does it seem to be running.
2) Did you check docker log {container id}
3) Does /go/src/github.com/mygithubname/ actually reflect the location of the build in the first stage of the docker container?
Sample docker file where the copy reflects the work dir of the build environment:
FROM golang AS build-env
ADD . /src/project
WORKDIR /src/project
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
# final stage
FROM busybox
WORKDIR /app
COPY --from=build-env /src/project/ .
CMD ["./main"]
I have my Dockerfile in the root of directory with src/myapp folder, myapp contains myapp.go with main package.
Dockerfile looks like following:
FROM golang:1.9.2
ADD . /
RUN go build myapp;
ENTRYPOINT ["/go/bin/myapp"]
I get following error:
can't load package: package myapp: cannot find package "myapp" in any of:
/usr/local/go/src/myapp (from $GOROOT)
/go/src/myapp (from $GOPATH)
What am I doing wrong? Can I log ls command after docker has done ADD?
You are copying all the files to Image root directory, Didn't installed any dependencies, Trying to Build it and then run the binary from /go/bin/app. The binary doesn't exists in that directory and it's generating errors.
I would recommend using a Dockerfile like this,
FROM golang:1.9.2
ADD . /go/src/myapp
WORKDIR /go/src/myapp
RUN go get myapp
RUN go install
ENTRYPOINT ["/go/bin/myapp"]
This'll do the following.
Copy project files to /go/src/myapp.
Set Working directory to /go/src/myapp.
Install dependencies, I used go get but replace it with which ever dependency management tool you are using.
Install/build the binary.
Set entry point.
You can run ls or any other command using docker exec.
Example:
docker exec <image name/hash> ls
You can also enter the shell in the generated image to understand it well using
docker run --rm -it <image hash/name> /bin/sh
After experiments I've come to this way of building Golang apps.
This way has several advantages:
dependencies are installed on build stage
if you need you may uncomment test options
build first fully-functional image about 800 MB
copies your program to an fresh empty image and produces very small image about 10 MB
Dockerfile:
# Two-stage build:
# first FROM prepares a binary file in full environment ~780MB
# second FROM takes only binary file ~10MB
FROM golang:1.9 AS builder
RUN go version
COPY . "/go/src/github.com/your-login/your-project"
WORKDIR "/go/src/github.com/your-login/your-project"
#RUN go get -v -t .
RUN set -x && \
#go get github.com/2tvenom/go-test-teamcity && \
go get github.com/golang/dep/cmd/dep && \
dep ensure -v
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /your-app
CMD ["/your-app"]
EXPOSE 8000
#########
# second stage to obtain a very small image
FROM scratch
COPY --from=builder /your-app .
EXPOSE 8000
CMD ["/your-app"]
For go 1.11 , you can use go module, the following is example
FROM alpine AS base
RUN apk add --no-cache curl wget
FROM golang:1.11 AS go-builder
WORKDIR /go/app
COPY . /go/app
RUN GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /go/app/main /go/app/cmd/myapp/main.go
FROM base
COPY --from=go-builder /go/app/main /main
CMD ["/main"]
The official docs suggests the following Dockerfile:
FROM golang:1.8
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
Please, visit https://hub.docker.com/_/golang for more info
myapp needs to be in /go/src/myapp as suggested, or in /usr/local/go/src/myapp. You can add it in ADD section.
If the objective is to create a container that simply runs your binary, I would take different approach.
First build the binary for linux:
GOOS=linux CGO_ENABLED=0 go build -a -installsuffix cgo
Then build a lightweight docker image from scratch:
FROM scratch
COPY myApp
CMD ["/myApp"]