This question already has answers here:
Docker: Combine multiple images
(4 answers)
Closed 2 months ago.
Here's a hypothetical scenario: I have Image A, with the task command installed on it and Image B with fzf. Both images are built from alpine. I understand that I can do a multi-stage build like this to produce a single image with both commands on it:
FROM alpine as stage1
RUN apk add task
FROM stage1 as final
RUN apk add fzf
CMD ash
But if Image A and Image B are already built, is there a way to merge the two?
I found this cool project called docker-merge but it throws errors.
It looks like this can be accomplished using something like COPY from=ImageA / /.
Here's an example Dockerfile that worked for me, combining an image with the task on it named tw and another, separate image with ripgrep and fzf on it name ripgres-fzf:
FROM tw
COPY --from=ripgrep-fzf / /
After building this Dockerfile and then running the container, the image contains all three commands (task, fzf, and ripgrep). The / / was a brute force copy to ensure configuration files get carried over. Probably a better way to do that. See https://levelup.gitconnected.com/docker-multi-stage-builds-and-copy-from-other-images-3bf1a2b095e0 for a tutorial.
Related
This question already has answers here:
How to include files outside of Docker's build context?
(19 answers)
Closed 1 year ago.
I keep my image on a fork repo of the official docker ubuntu images, the Dockerfile I am working on is in another repo I am trying to mention the relative path to the image like this:
FROM scratch
ADD ./../../../Dev-Ops-docker-brew-ubuntu-core/bionic/ubuntu-bionic-core-cloudimg-amd64-root.tar.gz /
but it doesn´t work, I get
=> ERROR [1/4] ADD ./../../../Dev-Ops-docker-brew-ubuntu-core/bionic/ubuntu-bionic-core-cloudimg-amd64-root.tar. 0.0s
------
> [1/4] ADD ./../../../Dev-Ops-docker-brew-ubuntu-core/bionic/ubuntu-bionic-core-cloudimg-amd64-root.tar.gz /:
------
failed to compute cache key: "/Dev-Ops-docker-brew-ubuntu-core/bionic/ubuntu-bionic-core-cloudimg-amd64-root.tar.gz" not found: not found
P
Is there a way to specify the path of the image or to reference the image I don´t want it to add it also in this git repo?
Researching more it seems for security reasons you are not allowed to go upper to parent in folders, so the best way is too run the build on the folder that is common parent for the two repos like this, and specify the Dockerfile in the subfolder (Powershell in my case notice the backslashes):
docker image build --tag "eduardflorinescu/test" --file ".\DevOps-Tools\docker\frontend\Dockerfile" .
and in this case the ADD section will be:
FROM scratch
ADD Dev-Ops-docker-brew-ubuntu-core/bionic/ubuntu-bionic-core-cloudimg-amd64-root.tar.gz /
This approach has the advantage if there are more repos you need to add files from to the image.
One probable caveat mentioned in comments by Matt :
the main issue you can run into then is build context size... everything
in the context is sent as a tar to the docker daemon so you can get
slow build starts if there's some large data. .dockerignore can help
with this though
I have a big tar/executable (over 30GB) I COPY/ADD it but this is used only for the installation. Once the application is installed I don't need it anymore.
How can I do? I am trying to use it but:
Everytime I run a build, it takes minutes to define the build context.
I'd like to share this image, if I create a tar with docker save, Is the final version or each layer included in it?
I found some solutions that said I can use RUN wget tar ... && rm tar but I don't want to create webserver for that.
Why isn't possible to mount a volume during build process?! It would be very useful.
Use Docker's multi-stage builds. This mechanism allows you to drop intermediate artifacts and therefore achieve a lightweight image.
Example:
FROM alpine:latest as build
# copy large file
# build
FROM alpine:latest as output
# copy necessary files built in the previous stage
COPY --from=build app /app
Anything built in the build stage will not be included in the final image, unless you explicitly COPY them.
Docs: https://docs.docker.com/develop/develop-images/multistage-build/
This is solvable using 2 different context.
Please follow these steps as mentioned below.
Objective is to create a
docker image that will have you large-build file.
docker image that will have you real codebase/executables.
For this you have to create 2 folders (Build & CodeBase) as follow.
Application<br/>
|---> BUILD <br/>
|======|--->Large-File<br/>
|======|--->Dockerfile<br/>
|--->CodeBase<br/>
|======|--->SRC+Other stuff<br/>
|======|--->Dockerfile<br/>
Build & Codebase both folders will have individual Dockerfile and arrange files accordingly.
Dockerfile(Build)
FROM **Base-Image**
COPY Large-File /tmp/Large-File
Build this and tag it with some name like (base-build-app-image)
#>cd Application <==Application root folder as mentioned above==>
#>docker build -t base-build-app-image BUILD <==path of your build-folder==>
Dockerfile(Codebase)
FROM base-build-app-image
RUN *****
CMD *****
RUN rm -f **/tmp/Large-File**
RUN rm -f **Remove installation files that is not required**
ENTRYPOINT *****
Build this-code-base and base-build-app-image is already in your local docker-repository and your large iso file is not in the current-buid-context
#>cd Application <==Application root folder as mentioned above==>
#>docker build CodeBase <==path of your code-base==>
This time since the context size is only your code base and since this doesn't include that Large file - it will definitely reduce your build time.
You can also take an advance of using docker-compose to do both operations together so you will not have to execute 2 separate commands.
If you need help on preparing this docker-compose file then do let me know in comments.
If anything is not clear then leave a comment or come over chat to fix this issue.
I see that dockerfiles usually have a line beginning with "from" keywork, for example:
FROM composer/composer:1.1-alpine AS composer
As far as I know, dockerfiles are a set of commands that help to build and run many containers in docker.
As the example above, docker uses a image named composer/composer:1.1-alpine from docker hub. The As composer just make an alias, so we can use it more convenient.
When I looked for the image, I found the link enter link description here and then enter link description here.
The thing I dont really understand is that:
I guess docker will use the image to build something, but how exactly does it use the image? Does docker run the image or just prepare to use it when in need. Sometimes I dont see the dockerfiles use the image in following line (like this example, there are no lines using the keyword "composer" except the first line). It makes me confused.
Any help would be appreciated.
Thanks.
DockerFiles describes layers: Each command creates it's own layer. For example:
RUN touch test.txt
RUN cp test.txt foo.txt
would create two layers - the first one with the file test.txt, the second one without test.txt but with foo.txt
Each layer adds something to a container. When we walk the layers "up" we find that the very first layer is the empty layer, e.g. it contains only the linux (or windows) kernel itself - nothing else. But that's not really useful - we need a lot of tools (e.g. bash) to be able to run an app. So common base images like alpine add suc tools and core os functions.
It would be annoying as hell if we had to do this setup in every container so there a lots of base images, which do exactly this kind of setup.
If you want to see what a base image does, just search the name on hub.docker.com - there you will find the Dockerfile, describing the build process.
Aditionally, containers can be extenend, e.g. you use the elasticsearch container as a base image, and add your own functionality - that's the second use case for base images.
For your second question: You have to decide if you have to replicate the steps in your base image or not. If you inherit from a general OS image like apline - probably not, since linux normally ships with these tools. If you inherit from a more specialized container, it depends - if your machine matches the environment in the container, you don't need to, but if not you will have to apply these steps to your machine, too. E.g. if you don't have elasticsearch installed, you have to install it.
As for multiple froms in one Dockerfile: Please look up the documentation for Multi Stage builds. Essentially, they encapsulate multiple containers in a single dockerfile. Which can be very useful if you need a different set to build an app and to run the app. The first Container is responsible to build your app, while the second one takes the compiled source code and just runs it.
Watch for COPY --from= lines, these are copying files from one container to another.
The FROM instruction initializes a new build stage and sets the Base Image for subsequent instructions. As such, a valid Dockerfile must start with a FROM instruction. The image can be any valid image – it is especially easy to start by pulling an image from the Public Repositories.
FROM can appear multiple times within a single Dockerfile to create multiple images or use one build stage as a dependency for another. Simply make a note of the last image ID output by the commit before each new FROM instruction. Each FROM instruction clears any state created by previous instructions.
Optionally a name can be given to a new build stage by adding AS name to the FROM instruction. The name can be used in subsequent FROM and COPY --from= instructions to refer to the image built in this stage.
The tag or digest values are optional. If you omit either of them, the builder assumes a latest tag by default. The builder returns an error if it cannot find the tag value.
Taken from : https://docs.docker.com/engine/reference/builder/#from
I am containerising a codebase that serves multiple applications. I have created three images;
app-base:
FROM ubuntu
RUN apt-get install package
COPY ./app-code /code-dir
...
app-foo:
FROM app-base:latest
RUN foo-specific-setup.sh
and app-buzz which is very similar to app-foo.
This works currently, except I want to be able to build versions of app-foo and app-buzz for specific code branches and versions. It's easy to do that for app-base and tag appropriately, but app-foo and app-buzz can't dynamically select that tag, they are always pinned to app-base:latest.
Ultimately I want this build process automated by Jenkins. I could just dynamically re-write the Dockerfile, or not have three images and just have two nearly-but-not-quite identical Dockerfiles for each app that would need to be kept in sync manually (later increasing to 4 or 5). Each of those solutions has obvious drawbacks however.
I've seen lots of discussions in the past about things such as an INCLUDE statement, or dynamic tags. None seemed to come to anything.
Does anyone have a working, clean(ish) solution to this problem? As long as it means Dockerfile code can be shared across images, I'd be happy. If it also means that the shared layers of images don't need to be rebuilt for each app, then even better.
You could still use build args to do this.
Dockerfile:
FROM ubuntu
ARG APP_NAME
RUN echo $APP_NAME-specific-setup.sh >> /root/test
ENTRYPOINT cat /root/test
Build:
docker build --build-arg APP_NAME=foo -t foo .
Run:
$ docker run --rm foo
foo-specific-setup.sh
In your case you could run the correct script in the RUN using the argument you just set before. You would have one Dockerfile per app-base variant and run the correct set-up based on the build argument.
FROM ubuntu
RUN apt-get install package
COPY ./app-code /code-dir
ARG APP_NAME
RUN $APP_NAME-specific-setup.sh
Any layers before setting the ARG would not need to be rebuilt when creating other versions.
You can then push the built images to separate docker repositories for each app.
If your apps need different ENTRYPOINT instructions, you can have an APP_NAME-entrypoint.sh per app and rename it to entrypoint.sh within your APP_NAME-specific-setup.sh (or pass it through as an argument to run).
According to the Docker documentation, to build your own image, you must always specify a base image using the FROM instruction.
Obviously, there are lots of images to choose from in the Docker index, but what if I wanted to build my own? Is that possible?
The image base is built off Ubuntu if I understand correctly, and I want to experiment with a Debian image. Plus, I want to really understand how Docker works, and the base image is still a blackbox for me.
Edit: official documentation on creating a base image
You can take a look at how the base images are created and go from there.
You can find them here: https://github.com/dotcloud/docker/tree/master/contrib.
There is mkimage-busybox.sh, mkimage-unittest.sh, mkimage-debian.sh
Quoting Solomon Hykes:
You can easily create a new container from any tarball with "docker import". For example:
debootstrap raring ./rootfs
tar -C ./rootfs -c . | docker import - flimm/mybase
(credit to fatherlinux) Get information from https://developers.redhat.com/blog/2014/05/15/practical-introduction-to-docker-containers/ , which explains better
Create the tar files for your file system, simply could be
tar --numeric-owner --exclude=/proc --exclude=/sys -cvf centos6-base.tar /
Transfer the tar file to other docker system if not installed locally and import it
cat centos6-base.tar | docker import - centos6-base
Now you can verify by running it.
docker run -i -t centos6-base cat /etc/redhat-release
The scripts from dotcloud combine first two steps together which make me confused and looks complicated in the beginning.
The docker official guideline using debootstrap also tries to make clean file system.
You can judge by yourself how to do step 1.
To start building your own image from scratch, you can use the scratch image.
Using the scratch “image” signals to the build process that you want the next command in the Dockerfile to be the first filesystem layer in your image.
FROM scratch
ADD hello /
CMD ["/hello"]
http://docs.docker.com/engine/articles/baseimages/#creating-a-simple-base-image-using-scratch
If you want to make your own base image I would first take a look at
Official Images, specifically stackbrew inside that repo.
Otherwise there are some great references for minimal OS images in the docker repo itself.
For example here is a script for making a minimal arch image and there are more here.