Scenario:
I need to build a web-app, from which I can run/sop/delete/etc. containers in a cluster. So I installed Kubernetes and tested the API from the console. Everything seems working and looks fine.
Following the Docs, they write about Docker, but do I need it necessarily?
I mean I had to disable Hyper-V to make Minikube work, and after a reboot, Docker (which usually starts at startup) says that "something went wrong.. bla bla" .. but I can create deployments and proxys on Minikube.
This is somehow confusing.
Can someone explain this please for dummies?
Technically, you need a container runtime which respects CRI (Container Runtime Interface).
That is why you have CRI-O, which provides an integration path between OCI conformant runtimes and the kubelet.
See "CRI-O, the Project to Run Containers without Docker, Reaches 1.0" by Susan Hall.
The project “opens the door for plugging alternative container runtimes in the kubelet more easily, instead of relying on the default docker runtime.
Those new runtimes may include virtual machines-based ones, such as runv and Clear Containers, or standard Linux containers runtimes like rkt,” Red Hat senior engineer Antonio Murdaca wrote on the Project Atomic blog.
But in your case, your issue is to make Minikube work with HyperV: see "Minikube on Windows 10 with Hyper-V" from Jock Reed.
The trick is to create a new (External) Virtual network switch, named "Primary Virtual Switch", and to start Minikube with:
minikube start --vm-driver hyperv --hyperv-virtual-switch "Primary Virtual Switch"
If you're running minikube on OSX or Linux, you can configure Docker to use the minikube environment by running
eval $(minikube docker-env)
When you do this, you don't need a separate Docker desktop application. You're still technically using both Docker and Kubernetes, but you won't be using the setup with the "whale" icon on OSX.
Fundamentally Kubernetes takes responsibility for launching and managing Docker containers. (Or potentially other things, but almost always Docker containers.) It's a more complicated and more powerful tool along the same lines as Docker Compose and Docker Swarm. Minikube is very small Kubernetes cluster that runs inside a virtual machine; if you run kubectl get nodes you will see the single VM node, and that includes a copy of Docker.
As mentioned above, you need any runtime container. Kubernetes and docker are part of ecosystem. Both have different responsibilities, kubernetes handle cluster health and docker run application containers.
Related
While playing around with Docker and orchestration (kubernetes) I had to install and use minikube to create a simple sandbox environment. At the beginning I thought that minikube installs some kind of VM and run the "minified" kubernetes environment inside the same, however, after the installation listing my local Docker running containers I found minikube running as a container!!
Why minikube itself run as a Docker container? and how can it runs other containers?
Experimental Docker support looks to have been added in minikube 1.7.0, and started becoming the default runtime in minikube 1.9.0. As I'm writing this, current is 1.15.1.
The minikube documentation on the "docker" driver notes, particularly on a native-Linux host, there is not an intermediate virtual machine: if you can run Kubernetes in a container, it can use the entire host system's resources without special configuration or partitioning. The previous minikube-on-VirtualBox installation required preallocating memory and disk to the VM, and it was easy to get those settings wrong. Even on non-Linux hosts, if you're running Docker Desktop, sharing its hidden Linux VM can improve resource utilization, and you don't need to decide to allocate exactly 2 GB RAM to Docker Desktop and exactly 4 GB to the minikube VM.
For a long time it's been possible, but discouraged, to run a separate Docker daemon inside a Docker container; similarly, it's possible, but usually discouraged, to run a multi-process init system in a container. If you do both of these things then you can have the core Kubernetes components (etcd, apiserver, kubelet, ...) inside a single container pretending to be a Kubernetes node. It also helps here that Kubernetes already knows how to pull Docker images, which minimizes some of the confusing issues with running Docker in Docker.
First of all, I am not an expert in container orchestration tools.
I've just installed microk8s according to the guide:
https://microk8s.io/docs/
And if I run microk8s kubectl get nodes, I see, that my node is actually running containerd engine.
My application build process is set up to generate docker file and automatically create docker images, so I would like microk8s also use docker.
I used minikube before, and now I decided to try microk8s. Now I am a bit confused, maybe it was a bad idea to stick with docker from the beginning?
Is it possible to set a docker engine for microk8s?
I've never used contained before, and I don't know how to prepare contained images for my app. That's why I am asking.
To run Nvidia GPU enabled containers, I had to switch from containerd to docker in microk8s. Here's how I did that:
Edit /var/snap/microk8s/current/args/kubelet
Change --container-runtime=docker from remote. Then, execute the following commands.
microk8s stop
microk8s start
You don't need specifically docker to run pods using docker images on kubernetes.Any OCI standard runtime such as containerd, docker, CRI-O etc as OCI runtime can run docker images because they all follow same OCI standard.
microk8s does not offer the ability to choose from different OCI runtimes
First of all, I am not an expert in container orchestration tools.
MicroK8s is just a single snap package that can be installed on Ubuntu, as well as other Linux distributions. MicroK8s is easy to install and has a small disk and memory footprint, making it a good entry point for those interested in exploring K8s.
As you know, the container needs a runtime engine; while the Docker is the most common container runtime used in a Pod, Pods can use other container runtime engines, such as CoreOS rkt, etc, etc if desired. For a container itself it makes no difference. That's the whole idea of that approach.
You can easily run your containers on microk8s.
Hope that helps.
When I run my docker container using Docker Desktop for Windows I am able to connect to it using
docker run -p 5051:5000 my_app
http://0.0.0.0:5051
However when I open another terminal and do this
minikube docker-env | Invoke-Expression
and build and run the same container using the same run command as above
I cannot connect to the running instance.
Should I be running and testing the containers using Docker Desktop, then using minikube to store the images only (for Kubernetes)? Or can you run them and test them as well through minikube?
That's because on your second attempt, the container is not running on the host but on the minikube VM. You'll be able to access it using the minikube VM IP.
To get the minikube ip you can run minikube ip
Why ?
Invoking minikube docker-env sets all the docker env variable on your host to match the minikube environment. This means that when you run a container after that, it is run with the docker daemon on the minikube VM.
I asked you if there are any specific reasons to use Docker Desktop and Minikube together on a single machine as these are two competitive solutions which basically enable you to perform similar tasks and achieve same goals.
This article nicely explains differences between these two tools.
Docker-for-windows uses Type-1 hypervisor, such as Hyper-V, which are
better compared to Type-2 hypervisors, such as VirtualBox, while
Minikube supports both hypervisors. Unfortunately, there are a couple
of limitations in which technology you are using, since you cannot
have Type-1 or Type-2 hypervisors running at the same time on your
machine
If you use Docker Desktop and Minikube at the same time I assume you're using Type-1 hypervisor, such as mentioned Hyper-V, but keep in mind that even if they use the same hypervisor, both tools create their own instances of virtual machine. Basically you are not supposed to use those two tools together expecting that they will work as a kind of hybrid that lets you manage single container environment.
First check what hypervisor you are using exactly. If you're using Hyper-V, simple Get-VM command in Powershell (more details in this article) should tell you what you currently have.
#mario no, I didn't know minikube had a docker daemon until recently
which is why I have both
Yes, Minikube has built in docker environment (in fact it sets everything up, but yes, it also sets up container runtime) so basically you don't need to install docker additionally, and as #Marc ABOUCHACRA already suggested in his answer, Minikube runs the whole environment (single node k8s cluster with docker runtime) on a separate VM. Linux version has an option --vm-driver=none which allows you to use your host container runtime and set-up k8s components on it, but this is not the case with Windows version - here you can only use one of two currently supported hypervisors: Hyper-V or VirtualBox (ref).
I wouldn't say that Docker Destkop runs everything on your host. It also uses Type-1 hypervisor to run the container runtime environment. Please check the Get-VM command on your computer and it should be clear what VMs you have and created by which tool.
I'm testing the side-by-side Windows/Linux container experimental feature in Docker for Windows and all is going well. I can create Linux containers while the system is set to use Windows containers. I see my ReplicaSets, Services, Deployments, etc in the Kubernetes dashboard and all status indicators are green. The issue, though, is that my external service endpoints don't seem to resolve to anything when Docker is set to Windows container mode. The interesting thing, however, is that if I create all of my Kubernetes objects in Linux mode and then switch to Windows mode, I can still access all services and the Linux containers behind them.
Most of my Googling took me to errors with services and Kubernetes but this doesn't seem to be suffering from any errors that I can report. Is there a configuration somewhere which must be set in order for this to work? Or is this just a hazard of running the experimental features?
Docker Desktop 2.0.0.3
Docker Engine 18.09.2
Kubernetes 1.10.11
just to confirm your thoughts about experimental features:
Experimental features are not appropriate for production environments or workloads. They are meant to be sandbox experiments for new ideas. Some experimental features may become incorporated into upcoming stable releases, but others may be modified or pulled from subsequent Edge releases, and never released on Stable.
Please consider additional steps to resolve this issue:
The Kubernetes client command, kubectl, is included and configured to connect to the local Kubernetes server. If you have kubectl already installed and pointing to some other environment, such as minikube or a GKE cluster, be sure to change context so that kubectl is pointing to docker-for-desktop
> kubectl config get-contexts
> kubectl config use-context docker-for-desktop
If you installed kubectl by another method, and experience conflicts, remove it.
To enable Kubernetes support and install a standalone instance of Kubernetes running as a Docker container, select Enable Kubernetes and click the Apply and restart button.
By default, Kubernetes containers are hidden from commands like docker service ls, because managing them manually is not supported. To make them visible, select Show system containers (advanced) and click Apply and restart. Most users do not need this option.
Please verify also System requirements.
Most tutorials I've seen for developing with Kubernetes locally use Minikube. In the latest Edge release of Docker for Windows, you can also enable Kubernetes. I'm trying to understand the differences between the two and which I should use.
Minikube lets you choose the version of Kubernetes you want, can Docker for Windows do that? I don't see a way to configure it.
Minikube has CLI commands to enable the dashboard, heapster, ingress and other addons. I'm not sure why because my undertstanding is that these are simply executing kubectl apply -f http://....
With Minikube I can do a minikube ip to get the cluster IP address for ingress, how can I do this with Docker for Windows?
Is there anything else different that I should care about.
I feel like you largely understand the space, and mostly have answers to your questions already. You might find Docker for Mac vs. Docker Toolbox an informative read, even if it's about the Mac equivalent rather than Windows and about Docker packaged as a VM rather than Kubernetes specifically.
In fact you are stuck with the specific version of Kubernetes the Docker Edge desktop distribution publishes.
is answered in the question.
I believe NodePort-type Services are published on your host's IP address; there isn't an intermediate VM address like there is with Docker Toolbox.
Docker Toolbox and minikube always use a full-blown virtual machine with an off-the-shelf hypervisor. The Docker desktop application might use a lighter-weight virtualization engine if one is available.
Kubernetes can involve some significant background work. If you're using Kubernetes-in-Docker it's hard to "turn off" Kubernetes and still have Docker available; but if you have a separate minikube VM you can just stop it.
Here is a part from Docker documentation:
Kubernetes is only available in Docker for Windows 18.02 CE Edge.
Kubernetes support is not included in Docker for Windows 18.02 CE
Stable. To find out more about Stable and Edge channels and how to
switch between them, see General configuration.
Docker for Windows 18.02 CE Edge includes a standalone Kubernetes
server and client, as well as Docker CLI integration. The Kubernetes
server runs locally within your Docker instance, is not configurable,
and is a single-node cluster.
The Kubernetes server runs within a Docker container on your local
system, and is only for local testing. When Kubernetes support is
enabled, you can deploy your workloads, in parallel, on Kubernetes,
Swarm, and as standalone containers. Enabling or disabling the
Kubernetes server does not affect your other workloads.
See Docker for Windows > Getting started to enable Kubernetes and
begin testing the deployment of your workloads on Kubernetes.
If you have enough CPU and RAM resources, you can easily have both, minikube and docker-for-desktop on the same machine and switch between them by selecting the context, e.g.:
> kubectl config get-contexts
> kubectl config use-context docker-for-desktop
There is no place at the moment to choose the Kubernetes version for docker-for-desktop. It is preselected in the Docker distribution package.
Unfortunately, version of docker inside the minikube is a step behind compared to available on the docker website, and some features could be missing, but if you are interested in Kubernetes environment, it doesn't really matter in most cases.
Some of the features of Kubernetes rely on external resources and could be unavailable for you in case of local installation. For example, you need an ability to create Cloud Load Balancer to use some types of the Ingress or Service objects.
But other resources can be easily started inside the minikube, or docker-for-desktop using kubectl create/apply.
To expose your cluster resources externally, you can use Webhook Relay Ingress controller.
Here is a link to the installation manual.