Issue:
I want my web-server to have either a Kubernetes config and a Docker-Compose config.
I run them on a MacBook (no native containers support), so at the moment I
run docker-compose on Docker Desktop
run kubectl on a Minikube VM
(I stick to Minikube because of the convenient commands like minikube ip & minikube service, that I rely on in my scripts)
This makes slow and inconvenient to switch from one setup to the other (suspend Minikube, start Docker Desktop, rebuild the images, re-run the services; and vice-versa).
They won't share built images or containers, and they run in different environments.
Attempt
I tried running docker-compose on Docker from the Minikube VM.
Works fine, but I cannot connect to the service, because there is no tunnelling or port-forwarding from the Host machine to the Minikube machine
(while compose takes care of the inner forwarding from $(minikube ip):8081 to docker-container-webserver:8080.
Question
How to open the connection from the Host machine (browser) to the container (web-server) running inside Minikube?
I tried minikube tunnel but it didn't seem to help (it is designed to work with Kubernetes LoadBalancer services).
Related
I am working on a web application with all the infrastructure based on Kubernetes. In my local environment, I am using Skaffold.
I have two computers (Desktop and Laptop) with 8Gb of RAM each. By starting minikube (virtualbox driver) and skaffold dev the Deskop is freezing.
So I decided to use the Laptop for coding and the Desktop for running minikube and everything related.
I successfully managed to set up kubeconfig on the laptop to have a context with the minikube server.
Actually, The issue is skaffold.
When I run skaffold dev, it fails because minikube of the Deskop doesn't see the images build by skaffold on my laptop. kubectl get po returns ImagePullBackOff.
That is because skaffold uses the local docker to build the image.
The question is how to make skaffold use the docker install in my Desktop?
I changed the docker context of my laptop so that it's linked to the Desktop context but it's still not working, skaffold is still using the default docker context installed in my laptop.
How to make the images build by Skaffold being available on my Desktop?
Is it possible for Skaffold to use a remote docker context? If yes, how?
Minikube uses its own Docker installation to power its cluster. This daemon runs in Minikube's VM (or container, if using the docker driver) and is completely independent from the host's Docker daemon (your Desktop). You can access to Minikube's daemon by setting the environment returned by minikube docker-env.
While playing around with Docker and orchestration (kubernetes) I had to install and use minikube to create a simple sandbox environment. At the beginning I thought that minikube installs some kind of VM and run the "minified" kubernetes environment inside the same, however, after the installation listing my local Docker running containers I found minikube running as a container!!
Why minikube itself run as a Docker container? and how can it runs other containers?
Experimental Docker support looks to have been added in minikube 1.7.0, and started becoming the default runtime in minikube 1.9.0. As I'm writing this, current is 1.15.1.
The minikube documentation on the "docker" driver notes, particularly on a native-Linux host, there is not an intermediate virtual machine: if you can run Kubernetes in a container, it can use the entire host system's resources without special configuration or partitioning. The previous minikube-on-VirtualBox installation required preallocating memory and disk to the VM, and it was easy to get those settings wrong. Even on non-Linux hosts, if you're running Docker Desktop, sharing its hidden Linux VM can improve resource utilization, and you don't need to decide to allocate exactly 2 GB RAM to Docker Desktop and exactly 4 GB to the minikube VM.
For a long time it's been possible, but discouraged, to run a separate Docker daemon inside a Docker container; similarly, it's possible, but usually discouraged, to run a multi-process init system in a container. If you do both of these things then you can have the core Kubernetes components (etcd, apiserver, kubelet, ...) inside a single container pretending to be a Kubernetes node. It also helps here that Kubernetes already knows how to pull Docker images, which minimizes some of the confusing issues with running Docker in Docker.
When I run my docker container using Docker Desktop for Windows I am able to connect to it using
docker run -p 5051:5000 my_app
http://0.0.0.0:5051
However when I open another terminal and do this
minikube docker-env | Invoke-Expression
and build and run the same container using the same run command as above
I cannot connect to the running instance.
Should I be running and testing the containers using Docker Desktop, then using minikube to store the images only (for Kubernetes)? Or can you run them and test them as well through minikube?
That's because on your second attempt, the container is not running on the host but on the minikube VM. You'll be able to access it using the minikube VM IP.
To get the minikube ip you can run minikube ip
Why ?
Invoking minikube docker-env sets all the docker env variable on your host to match the minikube environment. This means that when you run a container after that, it is run with the docker daemon on the minikube VM.
I asked you if there are any specific reasons to use Docker Desktop and Minikube together on a single machine as these are two competitive solutions which basically enable you to perform similar tasks and achieve same goals.
This article nicely explains differences between these two tools.
Docker-for-windows uses Type-1 hypervisor, such as Hyper-V, which are
better compared to Type-2 hypervisors, such as VirtualBox, while
Minikube supports both hypervisors. Unfortunately, there are a couple
of limitations in which technology you are using, since you cannot
have Type-1 or Type-2 hypervisors running at the same time on your
machine
If you use Docker Desktop and Minikube at the same time I assume you're using Type-1 hypervisor, such as mentioned Hyper-V, but keep in mind that even if they use the same hypervisor, both tools create their own instances of virtual machine. Basically you are not supposed to use those two tools together expecting that they will work as a kind of hybrid that lets you manage single container environment.
First check what hypervisor you are using exactly. If you're using Hyper-V, simple Get-VM command in Powershell (more details in this article) should tell you what you currently have.
#mario no, I didn't know minikube had a docker daemon until recently
which is why I have both
Yes, Minikube has built in docker environment (in fact it sets everything up, but yes, it also sets up container runtime) so basically you don't need to install docker additionally, and as #Marc ABOUCHACRA already suggested in his answer, Minikube runs the whole environment (single node k8s cluster with docker runtime) on a separate VM. Linux version has an option --vm-driver=none which allows you to use your host container runtime and set-up k8s components on it, but this is not the case with Windows version - here you can only use one of two currently supported hypervisors: Hyper-V or VirtualBox (ref).
I wouldn't say that Docker Destkop runs everything on your host. It also uses Type-1 hypervisor to run the container runtime environment. Please check the Get-VM command on your computer and it should be clear what VMs you have and created by which tool.
I have been working with docker to run my scripts on chrome-node and firefox -node and debug with the selenium-hub image where it runs smoothly, but when I use the same with k8s the whole system slows down. Why is this happening, any idea. I am using minikubes for kubernetes and docker toolbox and docker compose for docker.
Thanks,
There would definitely be an additional overhead when you start Kubernetes using minikube locally, compared to just starting a Docker container on the host.
In order to have a Kubernetes cluster, minikube creates a VM on the machine where the Kubernetes components will run in addition to the Docker container.
Anyway, minikube is not a production way for running Kubernetes. It is mostly meant for local development and testing. Therefore, you shouldn't evaluated kubernetes performance based on a minikube installation.
Looking at Rancher, what is the performance like? I guess my main question, is everything deployed in Rancher docker in docker? After reading http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ I trying to stay away from that idea. It looks like the Rancher CI pipeline with Docker/Jenkins is docker in docker, but what about the rest? If i setup a docker-compose or deploy something from their catalog, is it all docker in docker? I've read through their documentation and this simple question has still just flown over my head. Any guidance would be much appreciated.
Thank you
Rancher itself is not deployed with Docker in Docker (DinD). The main components of Rancher, rancher/server and rancher/agent are both normal containers. The server, in a normal deployment, runs the orchestration piece and a few other key services for the catalog, Docker Machine provisioning, websocket-proxy and MySQL. All of these can be broken out if desired, but for simplicity of getting started, its all in one. We use s6 to manage the orchestration and database processes.
The rancher/agent container is privileged and requires the user to bind mount the hosts Docker socket. We package a Docker binary in the container and use it to communicate with the host on startup. It is similar to the way a Mac talks to Boot2docker, the binary is just a client talking to a remote Docker daemon. Once the agent is bootstrapped, it communicates back to the Rancher server container over a websocket connection. When containers and stacks are deployed Rancher server sends events to the agents which then call the hosts Docker daemon for deployment. The deployed containers are running as normal Docker containers on the host, just as if the user typed docker run .... In fact, a neat feature of Rancher is that if you do type docker run ... on the host, the resulting container will show up in the Rancher UI.
The Jenkins entry in the Rancher catalog, when using the Swarm plugin is doing a host bind mount of the Docker socket as well. We have some early experiments that used DinD to test out some concepts with Jenkins, but those were not released.