What exaxtly a Docker image may contain? As it is mantioned that image can have all possible dependecies require for a microservice, so how do an image intract with databases?
A docker image just contains the software and its dependencies. Then you use that image to create one or multiple containers using that image. When creating the container you can inject configuration and mount up external persistent storage if needed.
Think of running containers as virtual machines with their own ip addresses except that the container itself do not run an actual OS. The processes will actually run on the host OS, but they are completely isolated by the kernel.
I docker image mainly contains files. Often they contain file structure from an an OS. For example ubuntu, centos, alpine linux etc.. These files are only there to support the application and provide the most common tools.
Related
I know that containers are a form of isolation between the app and the host (the managed running process). I also know that container images are basically the package for the runtime environment (hopefully I got that correct). What's confusing to me is when they say that a Docker image doesn't retain state. So if I create a Docker image with a database (like PostgreSQL), wouldn't all the data get wiped out when I stop the container and restart? Why would I use a database in a Docker container?
It's also difficult for me to grasp LXC. On another question page I see:
LinuX Containers (LXC) is an operating system-level virtualization
method for running multiple isolated Linux systems (containers) on a
single control host (LXC host)
What does that exactly mean? Does it mean I can have multiple versions of Linux running on the same host as long as the host support LXC? What else is there to it?
LXC and Docker, Both are completely different. But we say both are container holders.
There are two types of Containers,
1.Application Containers: Whose main motto is to provide application dependencies. These are Docker Containers (Light Weight Containers). They run as a process in your host and gets all the things done you want. They literally don't need any OS Image/ Boot Up thing. They come and they go in a matter of seconds. You cannot run multiple process/services inside a docker container. If you want, you can do run multiple process inside a docker container, but it is laborious. Here, resources (CPU, Disk, Memory, RAM) will be shared.
2.System Containers: These are fat Containers, means they are heavy, they need OS Images
to launch themselves, at the same time they are not as heavy as Virtual Machines, They are very similar to VM's but differ in architecture a bit.
In this, Let us say Ubuntu as a Host Machine, if you have LXC installed and configured in your ubuntu host, You can run a Centos Container, a Ubuntu(with Differnet Version), a RHEL, a Fedora and any linux flavour on top of a Ubuntu Host. You can also run multiple process inside an LXC contianer. Here also resoucre sharing will be done.
So, If you have a huge application running in one LXC Container, it requires more resources, simultaneously if you have another application running inside another LXC container which require less resources. The Container with less requirement will share the resources with the container with more resource requirement.
Answering Your Question:
So if I create a Docker image with a database (like PostgreSQL), wouldn't all the data get wiped out when I stop the container and restart?
You won't create a database docker image with some data to it(This is not recommended).
You run/create a container from an image and you attach/mount data to it.
So, when you stop/restart a container, data will never gets lost if you attach that data to a volume as this volume resides somewhere other than the docker container (May be a NFS Server or Host itself).
Does it mean I can have multiple versions of Linux running on the same host as long as the host support LXC? What else is there to it?
Yes, You can do this. We are running LXC Containers in our production.
In every docker tutorial, one of the main advantages of the docker is that docker container use host OS. But if that is true, I don't understand why I need to include OS in the image. For example here is image of centOS. I understand that if I want to run centOS in container I must pull this image but where then host OS come? It would be best if someone can point me to some link to read about that because I cannot find appropriate one.
What Docker uses of the host is actually only the OS's kernel.
What you include in the Docker container is not the actual OS (i.e., the kernel), but rather all the files that make up a specific distribution, such as Ubuntu or Fedora, or whatever…
This is also the reason why you can't run Linux containers on Windows and vice-versa (without a VM), because Linux software of course doesn't work with the Windows kernel, and Windows software doesn't work with the Linux kernel.
So, all Docker containers running on a given host share the host OS's kernel.
It actually shares the kernel & required libraries to boot the image from host OS. That's why those images are really small & not like traditional ISO files. It primarily utilizes union file system, cgroups and namespaces to manage the images and containers.
You can give a quick read to below -
https://kjanshair.github.io/2017/07/04/Docker-Containers-vs-System-Processes/
How is Docker different from a normal virtual machine?
I am trying to understand Docker concepts but one thing I can not catch:
As I understand image (consequently - a container) can be instantiated from different linux distributives, such as Ubuntu, CentOS and others.
Let's say on host machine I run standard Ubuntu 14.04,
What happens if I use container that is not instantiated from same distributive?
Not 14.04?
Not Ubuntu (or any other Debian-based)?
What disadvantages of using different base-images of images you use? (Let's say I use Image A that uses Ubuntu as a base image, Image B that used Debian as base image and Image C that uses CentOS as base image)?
Bonus question: How can I tell what base image used for an image if developer didn't specified it in a Docker hub description?
Thank you in advance!
Docker does not use LXC (not since Docker 0.9) but libcontainer (now runc), a built-in execution driver which manipulates namespaces, control groups, capabilities, apparmor profiles, network interfaces and firewalling rules – all in a consistent and predictable way, and without depending on LXC or any other userland package.
A docker image represents a set of files winch will run as a container in their own memory and disk and user space, while accessing the host kernel.
This differs from a VM, which does not access the host kernel but includes its own hardware/software stack through its hypervisor.
A container has just to set limits (disk, memory, cpu) in the host. An actual VM has to build an entire new host.
That docker image (group of files) can be anything, as long as:
it does not depends on host libraries (since it is isolated in its own disk space, it does not have access to hosts files, unless volumes are mounted)
it does only system calls: see "What is meant by shared kernel in Docker?"
That means an image can be anything: another linux distro, or even a single executable file. Any executable compile in go (https://golang.org/) for instance, could be packaged in its own docker image without any linux distro:
FROM scratch
COPY my_go_exe /
ENTRYPOINT /my_go_exe
scratch is the "empty" image, and a go executable is statically linked, so it is self-contained and only depends on system calls to the kernel.
The main thing shared between the host OS and docker container is the kernel. The main risk of running docker containers from different distributions/versions is that they may depend on kernel functionality not present on the host system, for example if the container expects a newer kernel than the host has installed.
In theory, the Linux kernel is backwards compatible. As long as the host kernel is newer than the container kernel it should work.
From an operational perspective, each time you start depending on a different base image that is another dependency that you need to monitor for updates and security issues. Standardizing on one distribution reduces the workload for your ops team when the next big vulnerability is discovered.
Docker uses LXC, which is an operating-system-level virtualization method for running multiple isolated Linux systems (containers) on a control host using a single Linux kernel.
You can compare this to a VM on your machine, where you start another Linux distro, which does not have to be the same as your host OS. So it does not matter, if your host os is the same as the base image of your container.
In Docker, the container is built from layers. Each step (command) in your Dockerfile represent one layer, which are applied one after the other. The first step ist to apply the base OS layer, which is indicated by FROM.
So to answer your bonus question, you can have a look inside the Dockerfile of the container you're using (it's the third tab on DockerHub) and see in the first statement, which is the base image (os).
Consider the following scenario:
There are three independent web applications A, B and C that require an apache server to be run on.
A linux server runs web application A and may act as a Docker host. It is required that the applications B and C are isolated from the linux server and each other. They are therefore realized as two Docker containers, initially created from the same image.
My question updates, and security updates in special.
Do the two Docker containers require a full OS installation image? Can they share the host's apache binary, so security updates of the host's apache could be propagated to the containers automatically?
If this doesn't work: Would I need to install updates for apache on both containers independently, or could I benefit from the fact, that they are based on the same image, and somehow simplify the update process?
Docker will not share binaries from the host server that would be missing the point. What docker does have is a layered file system which means two docker images that share a common base will be more space efficient when sharing the same host.
As for patching. If you update the base image, you'll need to rebuild the container images that derive from it. Not a big deal, if you're using a registry to store and distribute images. to update the host server you perform a "pull" operation to update the local images, followed by a container restart.
I am running a Docker daemon on my GUEST OS which is CentOS. I want to install software services on top of that in an isolated manner and I do not need another OS image inside my Docker container.
I want to have a Docker container with just the additional binaries and libraries for the software application I am going to install.
Is there a "whiteglove/blank" base image in Docker I can use ? I want a very lean container that uses as a starting point what my GUEST OS has to offer. Is that possible ?
What you're asking for isn't possible out-of-the-box with Docker. Each Docker image has its own root filesystem, which needs to have some sort of OS installed.
Your options are:
Use a minimal base image, such as the BusyBox image. This will give you the absolute minimum you need to get a container running.
Use the CentOS base image, in which case your container will be running the same or very similar OS.
The reason Docker images are like this is because they're meant to be portable. Any Docker image is meant to run anywhere Docker is running, regardless of the operating system. This means that the Docker image must contain an entire root filesystem and OS installation.
What you can do if you need stuff from the host OS is share a directory using Docker volumes. However, this is generally meant to be used for mounting data directories, and it still necessitates the Docker image having an OS.
That said, if you have a statically-linked binary that has absolutely no dependencies, it becomes easy to create a very minimal image. This is called a "microcontainer", and Go in particular is well-suited to producing these. Here is some further reading on microcontainers and how to produce them.
One other option you could look into if all you want is the resource management part of containers is using lxc-execute, as described in this answer. But you lose out on all the other nice Docker features as well. Unfortunately, what you're trying to do is just not what Docker is built for.
As I understood docker, when you use a base image, you really do not install an additional OS.
Its just a directory structure sort of thing with preinstalled programs or we can say a file system of an actual base image OS.
In most cases [click this link for the exception], docker itself [the docker engine] runs on a linux VM when used on mac and windows.
If you are confused with virtualization, there is no virtualization inside Docker Container. Containers run in user space on top of the host operating system's kernel. So, the containers and the host OS would share the same kernel.
So, to sumarize:
Consider the host OS to be windows or mac.
Docker when installed, is inside a linux VM running on these host OS.[use this resource for more info]
The base linux images inside the docker container then use this linux VM machine as host OS and not the native windows or mac.
On linux, The base linux images inside the docker container direclty uses the host OS which is linux itself without any virtualization.
The base image inside Docker Container is just the snapshot of that linux distributions programs and tool.
The base image make use of the host kernel (which in all three cases, is linux).
Hence, there is no virtualisation inside a container but docker can use a single parent linux virtual machine to run itself [the docker engine] inside it.
Conclusion:
When you install a base image inside docker, there is no additional OS that is installed inside the container but just the copy of filesystem with minimal programs and tools is created.
From Docker's best practices:
Whenever possible, use current Official Repositories as the basis for your image. We recommend the Debian image since it’s very tightly controlled and kept extremely minimal (currently under 100 mb), while still being a full distribution.
What you're asking for is completely against the idea of using Docker Containers. You don't want to have any dependability on your GUEST OS. If you do your Docker wont be portable.
When you create a container, you want it to run on any machine that runs Docker. Be it CentoOS, Ubuntu, Mac, or Microsoft Azure :)
Ideally there are no advantages of your base container OS having to do anything with your Host OS.
For any container, you need to have at least a root file system. That is why you need to use a base image that have the root file system. Your idea is not completely against the container paradigm of usage; as opposed to VMs, we want container to be minimal without much of repetitive elements that it can leverage from the underlayer OS.
Following the links of Rohan Singh, I found some related info, that doesn't generally contradict, but relates to the core ide of the question:
The base image for all Docker images is the scratch image. It has essentially nothing in it. This may sound useless, but you can actually use it to create the smallest possible image for your application, if you can compile your application to a static binary with zero dependencies like you can with Go or C.