When using Docker for Windows, the containers run side-by-side in a hyper-v linux VM on Windows.
So when launching a container in ubuntu, is any virtualization solution like hyper-v needed or are the containers just running as processes inside ubuntu?
Source for my first statement - How docker desktop runs linux containers on Windows machine
First, why hyper-v?
The reason for docker on windows using hyper-v VM just because: for a linux container, it had to share the linux kernel of host. But on windows, we do not have linux kernel, so docker set a hyper-v VM for you, then let your container to share the kernel.
Second, why not VM on linux?
But on linux, the host already has a linux kernel, so container can share this kernel without using any VM.
In fact, from next diagram you can see when you start a new container, it will auto new a process containerd-shim, it will run as a process which you can use ps aux | grep docker to see it on linux host.
And, finally, what is container?
Docker uses a technology called namespaces to provide the isolated workspace called the container. When you run a container, Docker creates a set of namespaces for that container, then every process in container will run in a separated namespace. See official docementation.
"Containers" is a concept that combines (primarily) two features implemented in the Linux kernel - control groups and namespaces. You need the VM on top of Windows because Windows does not implement these two features.
Therefore, when you run containers natively on Linux, each container will simply run as separate processes constrained by control groups and namespaces.
Related
So I have a use case where I need to detect inside of a WSL2 VM whether the Docker setup is Docker for Windows w/ WSL integration vs Docker just running inside of the WSL VM (say installed directly via apt or dnf). The networking situation between these two use cases is different because with Docker for Windows WSL integration you cannot reach containers by their IP from the WSL VM. This poses some problems for some dev-tooling that we have and wasn't previously an issue with devs running on Linux-native machines but we've recently run into it now that some devs are using Windows machines with WSL and Docker for Windows.
Any thoughts on how I can do this? Look for specific env vars, mount points etc?
Looks like I can just stat /mnt/wsl/docker-desktop and see if that exists.
From what I understand, the container includes all dependencies to run, but all containers running on the same platform whether it's a VM, or bare-metal will share the underlying kernel.
I believe I read somewhere that in order to run linux containers on windows, the Docker client spins up a linux based VM, and runs the container in that.
But now I see that docker for windows runs linux containers natively (ie, without hyper-v).
My question is: How can an image that was built to run on linux run on a system that has a windows kernel?
This is the original source that my question arose from:
https://www.hanselman.com/blog/DockerAndLinuxContainersOnWindowsWithOrWithoutHyperVVirtualMachines.aspx
With the latest version of Windows 10 (or 10 Server) and the beta of
Docker for Windows, there's native Linux Container support on Windows.
That means there's no Virtual Machine or Hyper-V involved (unless you
want), so Linux Containers run on Windows itself using Windows 10's
built in container support.
I saw some similar questions, but they explained how a linux container runs on a windows platform by utilising a vm/hyper-v
How docker desktop runs linux containers on Windows machine
Does "Docker On Windows" launch a linux virtual machine?
Perhaps I didn't understand their answers, but from what I understood, it still seems like the linux container is sitting on-top of the windows kernel.
this is the magic of LCOW (https://github.com/linuxkit/lcow)
you are right to run a container the base KERNEL should be same , since container is just an abstraction , so to run a linux container on windows there are two options
either use moby linux on hyperv and run containers there
use lcow to run light weight linux vm for each container. (lcow)
https://learn.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/linux-containers
with WSL in windows in future we might be able to get a third method don't know if already someone is working on it .
When using Docker for Windows, you must choose to either use Windows or Linux containers, but you can't use both at the same time. What is the technical reason(s) for this? It's a little counter-intuitive, since each container has its own isolated operating system.
Linux containers on Docker for Windows are not handled by windows itself, but they are using Hyper-V Linux VM - MobyLinuxVM. Hence the necessity to switch between Linux and Windows.
However starting from Windows 1709 and Docker in edge version you can try out linux containers on windows - see: https://blog.docker.com/2017/09/docker-windows-server-1709/
Update
As #v.karbovnichy brought up, technically on Docker for Windows you "can" run linux and windows containers simultanously - you can use docker-machine command to create additional linux-based virtual machine that will run your linux containers. Then, you can connect them into one swarm and, with a dose of good will, you will run linux and windows containers simultaneously on same machine.
docker client command itself can connect to both linux or windows docker-server and "manage" both of them - check docker login - it's widely used in server configurations.
However as stated above, true running linux and windows containers next to each other is in preview state.
Docker ecosystem on your Windows machine contains several components.
One is Docker command line: the docker command that you use for everything-management. The second one is Docker daemon - A self-sufficient runtime for containers, the core.
Docker daemons for Linux Containers and Windows Containers are different, but they listen for connections from the docker client on the same pipe. So one needs to be stopped for other to be started. This is the technical reason that you asked for.
However, you can observe that containers started for ex. in MobyLinuxVM is still running and available for connections when you switch to Windows containers. The only thing here is that you cannot manage them because the Docker daemon for Windows does not know how to manage Linux containers in MobyLinuxVM.
UPDATE: As described in this post,
Docker for Windows 18.02 now supports Linux and Windows containers running side-by-side via LCOW, using a single Docker daemon.
So actually now you can use one docker daemon to manage both worlds, it's just about using the new --platform flag in docker pull.
Is it possible that I run docker without any host OS. I mean run it natively. It would be a performance boost that way I believe if possible.
Suppose I have a tool which runs on linux kernel. I create a docker container with some extra dependencies. Now I share that container with other person who has linux to run that container.
But I want to run that container without host OS. as it will be double layer of OS with container.
Docker itself is not a VM, so there is no double layer of OS. Docker is a tool to run applications with settings that isolate them from other applications running on the same OS kernel. Docker does include a VM with Docker for Windows and Docker for Mac to run the Linux kernel so you can run Linux containers. There is an option to run native Windows containers with Server 2016, but if you are looking for minimal and efficiency, I would suggest looking elsewhere.
The closest things to what you are looking for are:
Unikernels: these are applications compiled into a kernel with everything else removed, designed to run inside of a VM for a very specialized task, often security related. These are still early in their development stage, but Docker does use some of their technology inside their project.
LinuxKit (part of the Moby Project): this is how Docker creates their VMs for Docker for Windows and Docker for Mac. It is a container based Linux operating system that you can custom compile with only the containers you want to run. Most of the focus of this is still designed for VMs, but bare metal is an option.
Scratch base image: if you statically compile your application to remove all of the library dependencies, you can have a container without any shell or other OS tools. This is often seen in Go binaries shipped as Docker containers to do a single task with a very small attack surface. As a Docker container, it still requires the underlying Linux OS to run the binary.
So I have read this in many places that docker is faster and more efficient because it uses containers over VMs but when I downloaded docker on my mac I realized that it uses virtual box to run the containers. I believe on a linux machine docker doesn't need virtual box and can run on Linux Kernel. Is this correct ?
Back to original question. Is docker still faster/efficient because it uses a single VM to run multiple containers as opposed to Vargrant's new VM for every environment ?
I believe on a linux machine docker doesn't need virtual box and can run on Linux Kernel. Is this correct ?
Yes, hence the need for a VirtualBox Linux VM (using a TinyCore distribution)
Is docker still faster/efficient because it uses a single VM to run multiple containers as opposed to Vargrant's new VM for every environment ?
Yes, because of the lack of Hypervisor simulating the hardware and OS: here you can launch multiple containers all using directly the kernel (through direct system calls), without having to simulate an OS.
(Note: May 2018, gVisor is another option: a container, simulating an OS!)
See more at "How is Docker different from a normal virtual machine?".
Of course, remember that Vagrant can use a docker provider.
That means you don't have to always provision a full-fledged VM with Vagrant, but rather images and containers.
Vagrant.configure("2") do |config|
config.vm.provider "docker" do |d|
d.image = "foo/bar"
end
end
See Vagrant docker provisioner.