I have Rancher running behind this reverse proxy https://github.com/jwilder/nginx-proxy and I can start any containers and workloads I want. But because it hosts the containers on a managed network the reverse proxy cant get the IP address of the container and wont forward to the application. I still learning how to use Rancher but from the docs there were a couple of labels that I thought would be useful but never got to try them because it doesnt allow me to add labels on the workload. Im using rancher 2.1.1
You can add labels and annotations on a workload through the Rancher UI under 'Show Advanced Options', but depending on what you are doing you might want to use the Ingress Controller that rancher can deploy for you to route traffic through services. You dont want to have to route traffic to workloads directly by cluster IP.
https://rancher.com/docs/rancher/v2.x/en/k8s-in-rancher/load-balancers-and-ingress/ingress/
https://kubernetes.io/docs/concepts/services-networking/ingress/
Related
I am setting up 2 VPC on GCP, I setup kubeadm on each, let's call them kubemaster and kubenode1. So I ran kubeadm on kubemaster and kubenode1 which :
kubeadm init on kubemaster
kubeadm join on kubenode1
When I was trying to kubectl apply -f (a deployment which contains a pod with simple webapps inside) and kubectl apply -f (a NodePort type of Service which target the deployment port)
After that I simply access the webapps from my browser (on my local machine not on GCP), it just does not work as what I tried on minikube (I setup minikube with same kubectl apply as above too). I dig some search and there are a lot of people saying regarding Ingress and network layer (flannel in kubernetes website example)
My question is what are these Ingress and flannel ? Which one is necessary or both are not necessary at all if I just want my webapp run ? How does each other works against others ? Because from my understanding the layering is as per below :
Traffic -> Services -> Deployments/Pods
Where are these ingress and flannel suits to ? If its not about them both, why my apps does not work as intended (i open all port in GCP setting so its not security issue I suppose), I tried setting up Kubernetes Dashboard-UI, run kubectl proxy and still my browser cannot access both services (my webapp inside the deployment and also Dashboard API), may be I am a little bit lost here.
The flannel and the Ingress are completely different things.
flannel is a CNI or Container Network Interface plugin which task is networking between containers. As coreOS says:
each container is assigned an IP address that can be used to
communicate with other containers on the same host. For communicating
over a network, containers are tied to the IP addresses of the host
machines and must rely on port-mapping to reach the desired container.
This makes it difficult for applications running inside containers to
advertise their external IP and port as that information is not
available to them.
flannel solves the problem by giving each container an IP that can be
used for container-to-container communication. It uses packet
encapsulation to create a virtual overlay network that spans the whole
cluster. More specifically, flannel gives each host an IP subnet (/24
by default) from which the Docker daemon is able to allocate IPs to
the individual containers.
The Kubernetes supports some other CNI plugins: Calico, weave, etc. They vary according to functionality ( e.g. supporting features like NetworkPolicy for restricting resources )
The Ingress is a Kubernetes object which is usually operate at the application layer of the network stack (HTTP) and allow you to expose your Service externally, it also provides a features such as HTTP requests routing, cookie-based session affinity, HTTPS traffic termination and so on. (just like a web server Nginx or Apache)
I want to add few more points along with exiting answers.
After that I simply access the webapps from my browser (on my local
machine not on GCP), it just does not work as what I tried on minikube
Did you open the security rules/firewall rules for the NodePort? On which instance did you open and which instance are you hitting to access your app?
My question is what are these Ingress and flannel?
I recommend you to read offical docs. But anyway, since you asked the question, I would like to tell few words.
Flannel is a overrelay network for containers which the subnet for the container can span across multiple nodes(Which is opposite to native docker networking-host n/w, NAT, etc). Each containers gets it own IP every time it spawn. The flannel is more like control plain for container network which is internal to K8s
Highly recommend you to read How Flannel N/W works
Ingress is smart router for the load balancer(Or simple for now, we can say it exposes the application to out side of K8s). It works at application level. Once you hit the "Ingress" enpoint, it will forward to service(which depends on ingress rules) and then to app pod.
A blog on Ingress - https://medium.com/#cashisclay/kubernetes-ingress-82aa960f658e
I see you were talking about ClusterIP. Generally, the the ClusterIP is the IP for the K8s service which is nothing but a magic of "IP Tables Rules". Kube-Proxy is responsible to write ip table rules in every node once you define "Service". These ip table rules or ClusterIP points to actual pod IP(The IP assigned by flannel daemon). I hope you can understand, how flannel and "Ingress" fit into the picture or work together or responsible for application traffic.(Please correct if I'm wrong..!!)
Can you paste ingress controller yaml content? What are the rules you defined?
Since you are using GCP, why don't you try GKE? I mean it is easy to deploy, besides you can access your application with LoadBalancer instead of depending on Ingress(Anyway, its none of my business :-) )
Said short, flannel or pod-to-pod networking layer in general, is what enables pods to talk to each other in Kubernetes. Ingress Controller on the other hand is what takes Ingress objects and turns them into rules for receiving and forwarding (mostly) HTTP(S) traffic to the backing services, over pod-to-pod network.
As you can see, technically, you need only the first one (pod-to-pod networking) as you can directly expose your service somewhere with NodePort or LoadBalancer service, it is very convenient though to use Ingress if you expose multiple services (pretty much like you do with vhosts on classic web server installations.
I have a container which runs a chatbot using python, exposed port 5000 where the bot is running. Now when i deploy this container on kubernetes, I have few questions
Do i need to run nginx container in the pod where my app container is
running ? If yes why do i need to ? since kubernetes does load
balancing
If i run nginx container on port 80, do I need to run my
app container also on 80 or (can i use a different port like 5000)
what role does gunicorn play here ?
I am a little confused because most of the examples i see online everyone pretty much have nginx container in their pods along with the app containers
As you mentioned Kubernetes can handle its own load balancing, so the answer to your first question is no, you don't need to run nginx especially in the pod where your application is.
Typically, services and pods have IPs that are routable by the cluster network and all traffic which ends at an edge router will be dropped. So in Kubernetes there is a collection of rules that allows inbound connections to reach a cluster services. Which we call Ingress:
An API object that manages external access to the services in a
cluster, typically HTTP.
Confusing part is that Ingress on it's own does not do much. You will have to create an Ingress controller which is a daemon deployed as a Pod. Its job is to read the Ingress Resource information and to process that accordingly. Actually any system capable of reverse proxying can be ingress controller. You can read more about Ingress and Ingress controller in practical approach in this article. Also I do not know about your environment so please remember that you should use type LoadBalancer if you are on the Cloud and type NodePort if in bare-metal environment.
Going to your second question you can run your application on any port you want, just remember to adjust that port in all other configuration files.
About ports and how to expose services you should check the sources in documentation on how the Kubernetes model works in comparison to containers model. You can find a an instructive article here.
Unfortunately I do not have experience with gunicorn so I won't be able to tell you what role does it play in here. Hope this helps.
I'm trying to setup a microservice deployment (deployment file at https://github.com/mojlighetsministeriet/groups/blob/master/docker-compose.example.yml) with several services that will use HTTP (hopefully HTTPS later on) to communicate internally without being exposed outside the network. I later on will add a proxy service that will expose specific features. I want to do this specifically with docker swarm mode and I like the possibility to define the deployment in a docker-compose.yml so I can initiate with:
$ docker stack deploy my-platform -c docker-compose.example.yml
I want the API urls internally to be like GET http://identity-provider/public-key and GET http://groups/b0c44674-58e0-4a8a-87e0-e1de35088964 . I have done this with Kubernetes setups before and that works great but now I want to get this working with docker swarm mode.
The DNS parts works without any problems, but docker swarm mode won't allow me to have each service listening on port 80 (will later be 443). It keeps complaining about port conflicts even though each service has it's unique domain name like identity-provider or groups and so on.
Should I use a specific network driver to get this working? I currently use overlay.
Using domain names without random ports would make calling in between the services much more simple to remember than e.g. http://identity-provider:1234 and http://groups:1235, the ports only adds complexity to the setup.
I'm fine with using any super cutting edge version of docker-ce if that helps somehow.
This should be possible right?
Docker Swarm routes incoming requests based on the published port, you can't have two applications with the same port number in a single Swarm.
I want to run multiple service such as GItlab, racktables on same host with https enabled in different containers. How can I achieve this?
You achieve this by running a reverse proxy (nginx or apache) that forwards traffic to the different containers using different virtualhosts.
gitlab.foo.bar -> gitlab container
racktables.foo.bar -> racktables container
etc
The reverse proxy container will map port 80 and 443 to the host. All the other containers will not need port mapping as all traffic goes through the revers proxy.
I think the quickest way to get this working is to use jwilder/nginx-proxy. It's at least extremely newbie friendly as it automates almost everything for you. You can also learn a lot by looking at the generated config files in the container. Even getting TLS to work is not that complicated and you get setup with A+ rating from ssllabs by default.
I've used this for my hobby projects for almost a year and it works great (with Let's Encrypt).
You can of course also manually configure everything, but it's a lot of work with so many pitfalls.
The really bad way to do this is to run the reverse proxy on the host and map lots of ports from all the containers to the host. Please don't do that.
I'm trying to figure out whether Docker Swarm or Kubernetes are a good choice for my use case.
Basically, I want to build a small cluster of forward proxies (via squid, nginx or a custom nodejs script), and be able to deploy/start/stop/purge them all together.
I should be able to access the proxy cluster via a single IP address, manager should be able to load-balance the request to a node, and each proxy node must use a unique outgoing IP address.
I'm wondering:
Are Docker Swarm and/or Kubernetes the right way to go about it?
If so, should I set-up Docker Swarm and/or Kubernetes and its worker nodes (running the proxy) on a single dedicated server or separate virtual servers?
Is it also possible for all the cluster nodes to share a file system storage for caching, common config etc.
Any other tips to get this working.
Thanks!
Docker running in swarm mode should work well for this
Run docker on a single dedicated server; I see no need for virtual servers. You could also run the swarm across multiple dedicated servers.
https://docs.docker.com/engine/swarm/secrets/ work well for some settings and configurations. If you require significant storage, simply add a database service to your cluster
Docker swarm mode fits your requirements quite well; requests are automatically balanced across your swarm and each service instance can be configured to have a unique address. You should check out the swarm mode tutorial: https://docs.docker.com/engine/swarm/swarm-tutorial/