I have a problem to understand the kubernetes workflow:
So as I understand the flow:
You have a master which contains etcd, api-server, controller manager and scheduler.
You have nodes which contain pods (wich contain containers), kubelet and a proxy.
The proxy is working as a basic proxy to make it possible for a service to communicate with other nodes.
When a pod dies, the controller manager will see this (it 'reads' the replication controller which describes how many pods there normally are).
unclear:
The controller manager will inform the API-server (I'm not right about this).
The API-server will tell the scheduler to search a new place for the pod.
After the scheduler has found a good place, the API will inform kubelet to create a new pod.
I'm not sure about the last scenario? Can you tell me the right proces is a clear way?
Which component is creating the pod and container? Is it kubelet?
So it's the kubelet that actually creates the pods and talks to the docker daemon. If you do a docker ps -a on your nodes (as in not master) in your cluster, you'll see the containers in your pod running. So the workflow is run a kubectl command, that goes to the API server, which passes it to the controller, say that command was to spawn a pod, the controller relays that to the API server which then goes to the scheduler and tells it to spawn the pod. Then the kubelet is told to spawn said pod.
I suggest reading the Borg paper that Kubernetes is based on to better understand things in further detail. http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43438.pdf
Related
I am currently working with a Hyperledger Fabric v1.4 deployment over k8s. The chaincode containers that are generated are basically create by the container running within the peer pods and k8s as such has no knowledge and control of the chaincode containers. In such a scenario where there is a Docker container running along with k8s and k8s has no knowledge of a particular docker container, is it possible for the Docker container to in some way gain access to the k8s master API and gain access to the whole k8s cluster consequently?
My intention with asking this question is to figure out if there is a way to use an container external to any pods in k8s, to cause any undesirable impact to the k8s cluster by gaining unauthorized access to k8s. The chaincode container that I talked about, is created using a trusted template image and the only possible malicious component in the container is a single golang, java or nodejs script that is provided by the user. So my real question here is, "Is it possible using these user scripts gain unauthorized access to the k8s cluster?" And I am primarily focusing on a manager k8s service like Azure Kubernetes Service.
Your question totally changed the meaning so I'll try to rewrite the answer.
You have to remember that the pod you are running the code on by default is limited to just the namespace it's running on. If you didn't gave it any higher privileges. Also the code is not running as root.
You can read about Pod Security Policies and Configure a Security Context for a Pod or Container.
TLDR.
As long as you don't give it any special privileges or rights it should be fairly save for your cluster.
Does Kubernetes develop the code for creating Pod object or it is just a part of container engine like Docker or cri-o?
Pod is kubernetes specific abstraction for which kubernetes has implemented code. Pod is a logical grouping of containers. A pod can have one or more than one containers. When a user asks Kubernetes to create a pod with two containers in it Kubernetes API server takes the request and kubernetes code instructs kubelet to actually start the containers from the docker image.
Kubelet which is part of kubernetes uses docker or cri-o or containerd via a concept called Container runtime interface(CRI) to actually invoke lifecycle operations (start, stop etc)on the container.
How does kubernetes pod gets IP instead of container instead of it as CNI plugin works at container level?
How all containers of same pod share same network stack?
Containers use a feature from the kernel called virtual network interface, the virtual network Interface( lets name it veth0) is created and then assigned to a namespace, when a container is created, it is also assigned to a namespace, when multiple containers are created within the same namespace, only a single network interface veth0 will be created.
A POD is just the term used to specify a set of resources and features, one of them is the namespace and the containers running in it.
When you say the POD get an IP, what actually get the ip is the veth0 interface, container apps see the veth0 the same way applications outside a container see a single physical network card on a server.
CNI is just the technical specification on how it should work to make multiple network plugins work without changes to the platform. The process above should be the same to all network plugins.
There is a nice explanation in this blog post
its the kubeproxy that makes everything work. one pod has one proxy which translates all the ports over one IP for the remaining containers. only in specific cases it is said that you want to have multiple containers in the same pod. its not preferred but its possible. this is why they call it "tightly" coupled. please refer to: https://kubernetes.io/docs/concepts/cluster-administration/proxies/
Firstly, let's dig deeper into the CNI aspect. In production systems, workload/pod (workload can be thought of as one or many containerized applications used to fulfill a certain function) network isolation is a first class security requirement. Moreover, depending on how the infrastructure is set up, the routing plane might also need to be a attribute of either the workload (kubectl proxy), or the host-level proxy (kube proxy), or the central routing plane (apiserver proxy) that the host-level proxy exposes a gateway for.
For both service discovery, and actually sending requests from a workload/pod, you don't want individual application developers to talk to the apiserver proxy, since it may incur overhead. Instead you want them to communicate with other applications via either the kubectl or kube proxy, with those layers being responsible for knowing when and how to communicate with the apiserver plane.
Therefore, when spinning up a new workload, the kubelet can be passed --network-plugin=cni and a path to a configuration telling kubelet how to set up the virtual network interface for this workload/pod.
For example, if you dont want your application containers in pod to be able to talk to host-level kube proxy directly, since you want to do some infrastructure specific monitoring, your CNI and workload configuration would be:
monitoring at outermost container
outermost container creates virtual network interface for every other container in pod
outermost container is on bridge interface (also a private virtual network interface) that can talk to kube proxy on host
The IP that the pod gets is to allow other workloads to send bytes to this pod, via its bridge interface - since fundamentally, other people should be talking to the pod, not individual work units inside the pod.
There is a special container called 'pause container' that holds the network namespace for the pod. It does not do anything and its container process just goes into sleep.
Kubernetes creates one pause container for each pod, to acquire the respective pod's IP address and set up the network namespace for all other containers that are part of specific pod. All containers in a pod can reach each other using localhost.
This means that your 'application' container can die, and come back to life, and all of the network setup will still be intact.
I previously created a Flask server that spawns Docker containers using the Docker Python SDK. When a client hits a specific endpoint, the server would generate a container. It would maintain queues, and it would be able to kill containers that didn't respond to requests.
I want to migrate towards Kubernetes, but I am starting to think my current server won't be able to "spawn" jobs as pods automatically like in docker.
docker.from_env().containers.run('alpine', 'echo hello world')
Is Docker Swarm a better solution for this, or is there a hidden practice that is done in Kubernetes? Would the Kubernetes Python API be a logical solution for automatically generating pods and jobs, where the Flask server is a pod that manages other pods within the cluster?
'Kubectl run' is much like 'docker run' in that it will create a Pod with a container based on a docker image (e.g. How do i run curl command from within a Kubernetes pod). See https://kubernetes.io/docs/reference/kubectl/docker-cli-to-kubectl/ for more comparison. But what you run with k8s are Pods/Jobs that contain containers rather than running containers directly so this will add an extra layer of complexity for you.
Kubernetes is more about orchestrating services rather than running short-lived jobs. It has some features and can be used to run jobs but that isn't its central focus. If you're going in that direction you may want to look at knative (and knative build) or kubeless as what you describe sounds rather like the serverless concept. Or if you are thinking more about Jobs then perhaps brigade (https://brigade.sh). (For more see https://www.quora.com/Is-Kubernetes-suited-for-long-running-batch-jobs) If you are looking to run web app workloads that serve requests then note that you don't need to kill containers that fail to respond on k8s as k8s will monitor and restart them for you.
I don't know swarm well enough to compare. I suspect it would be a bit easier for you as it is aimed more centrally at docker (the k8s API is intended to support other runtimes) but perhaps somebody else can comment on that. Whether using swarm instead helps you will I guess depend on your motivations.
This question pertains to the Kubernetes tutorial on Google's CodeLabs found here: https://codelabs.developers.google.com/codelabs/cloud-compute-kubernetes/index.html?index=..%2F..%2Fgcp-next#15
I'm new to both Docker and Kubernetes and am confused over their use of the term "pods" which seems to contradict itself.
From that tutorial:
A Kubernetes pod is a group of containers, tied together for the purposes of administration and networking. It can contain one or more containers. All containers within a single pod will share the same networking interface, IP address, disk, etc. All containers within the same pod instance will live and die together. It's especially useful when you have, for example, a container that runs the application, and another container that periodically polls logs/metrics from the application container.
That is in-line with my understanding of how Kubernetes pods relate to containers, however they then go on to say:
Optional interlude: Look at your pod running in a Docker container on the VM
If you ssh to that machine (find the node the pod is running on by using kubectl describe pod | grep Node), you can then ssh into the machine with gcloud compute ssh . Finally, run sudo docker ps to see the actual pod
My problems with the above quote:
. "Look at your pod running in a Docker container" appears to be
backwards. Shouldn't it say "Look at your Docker container running
on the VM"?
"...run sudo docker ps to see the actual pod" doesn't make sense, since "docker ps" lists docker containers, not pods.
So am I way off base here or is the tutorial incorrect?
As mentioned above pod can run more than one container, but in fact to make it simple running more than one container in a pod is an exception and definitely not the common use. you may look at a pod as a container++ that's the easy way to look at it.
If you starting with kubernetes I have wrote the blog below that explain the main 3 entities you need to be familiar with to get started with kubernetes, which are pods, deployments and services.
here it is
http://codefresh.io/blog/kubernetes-snowboarding-everything-intro-kubernetes/
feedback welcome!
One nuance that most people don't know about docker running Kubenretes is that it is running a outdated version. I found that if I went to Google's cloud based solution for Kubernetes everything was quite easy to setup. Here is my sample code of how I set up Kubernetes with Docker.
I had to use the command line utility for Docker though to properly get everything to work. I think this should point you in the right direction.
(I've started learning Kubernetes and have some experience with Docker).
I think the important side of pods is that it may have container(s) which are not from Docker, but from some other implementation.
In this aspect the phrase in problem 1. is fully valid: output confirms that pod is in Docker, not anywhere else.
And reg. problem 2 - the phrase means that further details about the pod you should inspect from docker command. Theoretically different command may be needed in other cases.