I have two docker images. Image x just copies some files to the image, Image y copies other files and install some needed libs.
I want to build third dockerfile=z that will be based on those two images.
example for z:
From x
.
.
From y
.
.
How do I build dockerfile z that can use those files from x and y?
Contents below are from the Docker official site.
Use multi-stage builds
With multi-stage builds, you use multiple FROM statements in your Dockerfile. Each FROM instruction can use a different base, and each of them begins a new stage of the build. You can selectively copy artifacts from one stage to another, leaving behind everything you don’t want in the final image. To show how this works, let’s adapt the Dockerfile from the previous section to use multi-stage builds.
Dockerfile:
FROM golang:1.7.3
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=0 /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
You only need the single Dockerfile. You don’t need a separate build script, either. Just run docker build.
$ docker build -t alexellis2/href-counter:latest .
The end result is the same tiny production image as before, with a significant reduction in complexity. You don’t need to create any intermediate images and you don’t need to extract any artifacts to your local system at all.
How does it work? The second FROM instruction starts a new build stage with the alpine:latest image as its base. The COPY --from=0 line copies just the built artifact from the previous stage into this new stage. The Go SDK and any intermediate artifacts are left behind, and not saved in the final image.
Source: Docker Multi Stage Builds
Try this:
FROM x as image1
FROM y as image2
FROM centos
COPY --from=image1 <source_path> <destinantion_path>
COPY --from=image2 <source_path> <destinantion_path>
Reference
Related
I've been operating under the assumption that the benefit of a multi-stage build (and using build_kit) is a smaller final image. My experience hasn't shown that.
Consider this:
FROM debian:latest AS builder
# Do stuff
FROM alpine:latest AS runner
WORKDIR /
COPY --from=builder /work/myapp.zip .
RUN unzip myapp.zip -d /myapp
Then:
docker build -t myapp .
docker tag....
docker push ...
Then:
docker pull our-private-repo/myapp:latest
And we are getting back 4 layers with over 600MB of stuff in it combined. What I think I should be getting is just alpine linux plus our application. It feels like it's pushing everything. Is it pushing everything?
If your builder can skip zipping the output, then you can copy the output directly COPY --from=builder /work/myapp (instead of unzipping), you can get an alpine base with your program only.
I want to use go mod graph inside the docker container to generate dependency graph in go repository. I only use that command and not use the other go functionalities. I have try the other tools like godegraph, gomod, dept and nothing can do better than go mod graph . Currently I installed all the go functionalities and add about 400MB of my docker image size.
Question : Can I reduce the size of the docker container by installing spesific command in golang ? Or can I get the binary of go mod graph so its can reduce the size of the container?
If you don't need Go when executing your image (docker run), but only need it for building your image (docker build), then, as commented, use a multi-stage build.
Here is an example: "Create lean Docker images using the Builder Pattern" from Mike Kaperys:
Each FROM instruction defines a new base image for the following instructions, and begins a new build stage.
Before multi-stage builds it was possible to to achieve the same effect, but the process required 2 Dockerfiles.
Using multi-stage builds allows you to selectively copy files between build stages — this is the basis of the builder pattern.
See kaperys/blog/docker-builder-pattern
FROM golang:1.16
COPY . /go/src/github.com/kaperys/blog/docker-builder-pattern
WORKDIR /go/src/github.com/kaperys/blog/docker-builder-pattern
RUN go get && CGO_ENABLED=0 GOOS=linux go build -o server .
FROM scratch
LABEL maintainer="Mike Kaperys <mike#kaperys.io>"
COPY --from=0 /go/src/github.com/kaperys/blog/docker-builder-pattern/server /opt/kaperys/vision/server
COPY --from=0 /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ADD html/ /opt/kaperys/vision/html
EXPOSE 8080
WORKDIR /opt/kaperys/vision
ENTRYPOINT [ "./server" ]
That way, you start from an image where Go is already installed (included go mod), then you build your actual image from any other one you want (including, in this example, "scratch").
I'm building a multistage docker image where we have a base image that will be extended to build the actual final image. However I can't seem to copy files that are coming from the parent image into the child.
Base image
FROM debian:10.9-slim AS base
# PHP Version
ONBUILD ARG PHP_VERSION
ONBUILD ENV PHP_VERSION=${PHP_VERSION:-7.4}
# More stuff.
...
# Copy PHP settings into the image
COPY ./php/pool.d/*.conf /build-php/pool.d/
COPY ./php/conf.d/*.ini /build-php/conf.d/
# Onbuild copy the settings to the correct PHP version
ONBUILD COPY /build-php/pool.d/*.conf /etc/php/${PHP_VERSION}/fpm/pool.d/
ONBUILD COPY /build-php/conf.d/*.ini /etc/php/${PHP_VERSION}/fpm/conf.d/
ONBUILD COPY /build-php/conf.d/*.ini /etc/php/${PHP_VERSION}/cli/conf.d/
Specific image
ARG PHP_VERSION=8.0
FROM abcdef.amazonaws.com/php-base:v1.14.0 AS base
When building the secondary image I get the following error
$ docker build -t php8.0:base .
...
=> ERROR [7/1] COPY /build-php/pool.d/*.conf /etc/php/8.0/fpm/pool.d/ 0.0s
------
> [7/1] COPY /build-php/pool.d/*.conf /etc/php/8.0/fpm/pool.d/:
------
lstat /var/lib/docker/tmp/buildkit-mount474178567/build-php/pool.d: no such file or directory
I checked the parent image and the files are copied fine into /build-php. Also when building the child image (with the onbuild copy disabled) the files are present at /build-php. I don't understand why the onbuild copy can't find the files.
COPY always copies files from the host system (more specifically from the Docker build context); if it's run ONBUILD then it forces the derived image's build context (source tree) to have a specific layout.
In this setup, the files are already in the base image (in /build-php) and so you need to RUN cp /build-php/... /etc/php/... to move the files within the image.
Using ONBUILD, in general, can make it hard to reason about what steps are happening when, and can lead to confusing side effects. Rather than having one base image, which eventually will install and configure some version of PHP, the setup might be easier if you have a fully-configured base image for each version of the interpreter you use.
# For simplicity, use the Docker Hub base image
# (There can be good reasons to build your own)
ARG PHP_VERSION=7.4
FROM php:${PHP_VERSION}
ARG PHP_VERSION
# Copy PHP settings into the image
COPY ./php/pool.d/*.conf /etc/php/${PHP_VERSION}/fpm/pool.d/
COPY ./php/conf.d/*.ini /etc/php/${PHP_VERSION}/fpm/conf.d/
COPY ./php/conf.d/*.ini /etc/php/${PHP_VERSION}/cli/conf.d/
docker build \
--build-arg PHP_VERSION=8.0 \
-t abcdef.amazonaws.com/php-80-base:v1.14.0 \
.
I am new to Docker and got confused in couple of things. I have few queries as well. Please find them below. I would be happy if these queries got resolved.
Question 1:
In multi-stage builds, how does docker identify the artifacts of a stage? Because I saw in many articles, people say COPY --from=0 src dest will copy the artifacts from the last build stage. I mean where have we explicitly defined that you need to copy the artifacts from this folder? What defines the build output/build artifact of a stage?
# Stage 1
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /build
COPY . .
RUN dotnet restore
RUN dotnet publish -c Release -o /app
# Stage 2
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS final
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "Core3Api.dll"]
Q2 - Does COPY --from=build src dest copy files from a previous build or the image?
Q3 - Normally copy has two parameters, one is the source and the
other is a destination. Above we have three parameters,
--from=build src dest. How does this copy works?
Q4 - Do we need to set WORKDIR in every build stage? Because I
think every build stage executes in a new isolated context? OR it
uses the same context?
Q5 - Where does Docker stores previous build's image when it moves to
next stage?
All my questions are interrelated, that's why I clubbed all into one.
Docker builds images; each image consists of a filesystem plus some additional metadata. Within the context of a multi-stage build (Q5) each FROM line starts a new image build, and the result of these image builds are totally normal images; you can see them (probably with a <none> name) in docker images output. Similarly, because each build stage stars FROM the context of some previous image, (Q4) you do need to repeat WORKDIR in every stage where it matters (but since each stage is in an isolated build context, different stages don't necessarily need the same working directory).
The Dockerfile COPY command (Q3) can take options, in much the same way as you can cp -r from to in a normal shell. The --from option (Q2) names an image; it can be an earlier build stage FROM ... AS name in the same Dockerfile, or it can be an arbitrary other image. (Q1) It copies from whatever the filesystem result of the image build is.
FROM ... AS build
WORKDIR /build
RUN ...
COPY ...
# An image is created that's effectively a snapshot here
FROM ...
WORKDIR /app
COPY src dest # from the build context
COPY --from=build /build/dir dest2 # from the "snapshot" point
COPY --from=0 /build/dir dest3 # from the first image in this Dockerfile
COPY --from=busybox:latest /bin/busybox dest4 # from another image
I have a Dockerfile with the following:
FROM i386/alpine
WORKDIR /tmp/src/mybuild
ADD . /tmp/src/mybuild
FROM travisci/travis-build
My goal is to end up with a 32 bit image which contains the stuff from both images. After running sudo docker build --rm --tag travis-32 ., the "image" is built, but when I run sudo docker run -it travis-32 /bin/bash, I end up in a bash terminal and typing uname -m gives me x86_64, which is clearly not 32 bit as I was expecting.
How can I make this work?
what's wrong with the Dockerfile:
The Dokcerfile mentioned in the question is a multi-stage build file. Every stage starts with FROM, where one is supposed to copy the artifacts from the first build stage into the second, in the goal to achieve a smaller docker image to use in production at the end.
in the mentioned Dokcerfile:
the first stage copies some files to an i386/alpine image:
FROM i386/alpine
WORKDIR /tmp/src/mybuild
ADD . /tmp/src/mybuild
then everything done is ignored, and another image is taken instead:
FROM travisci/travis-build
so the end result is an exact copy of travisci/travis-build.
Regarding the 32b,64b question:
Usually what is compiled under 32b works only under 32b, and in order for it to run under 64b, you need to compile it under 64b (except for some languages like go, where you can define the target platform), so one need to be careful.
Example:
Take a look, and notice where the artifacts are moved from stage to another using the COPY --from= statement:
FROM golang:1.7.3 as builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
More info can be found in the official docs: docker multistage-builds
hope this helps.