I have 1 master and 2 worker on my k8s cluster.It's bare metal and I can't use any of cloud providers. I just can use DNS load balancer. I want to expose valid ports (like 80 and 443) on my nodes because of that I can't use NodePort. What is the best solution?
My only solution was to install Nginx on all of my nodes and proxy ports to my ClusterIp services.I don't know that this is a good solution or not.
Following things that you are doing right :
Cluster IP service - If you don't want to expose your services to be invoked form outside the cluster, CLusterIP is right way instead of NodePort or LoadBalancer.
Following things that you can do:
Create an Ingress Controller and and Ingress resource for your cluster which will listen on port 80 and 443 and proxy the requests to your services according to routes mentioned in the ingress.
You can create inginx-ingress controller using link: https://kubernetes.github.io/ingress-nginx/deploy/
Then create an Ingress resource using link https://kubernetes.io/docs/concepts/services-networking/ingress/
I found the solution. I need to edit /etc/kubernetes/manifests/kube-apiserver.yaml and edit service-node-port-range to 80 to any number that I want. Then declare my ingress service as nodePort.
Related
i have a handful of dockerized microservices, each is listening for http requests on a certain port, and i have these deployments formalized as kubernetes yaml files
however, i can't figure out a working strategy to expose my deployments on the interwebs (in terms of kubernetes services)
each deployment has multiple replicas, and so i assume each deployment should have a matching load balancer service to expose it to the outside
now i can't figure out a strategy to sanely expose these microservices to the internet... here's what i'm thinking:
the whole cluster is exposed on a domain name, and services are subdomains
say the cluster is available at k8s.mydomain.com
each loadbalancer service (which exposes a corresponding microservice) should be accessible by a subdomain
auth-server.k8s.mydomain.com
profile-server.k8s.mydomain.com
questions-board.k8s.mydomain.com
so requests to each subdomain would be load balanced to the replicas of the matching deployment
so how do i actually achieve this setup? is this desirable?
can i expose each load balancer as a subdomain? is this done automatically?
or do i need an ingress controller?
am i barking up the wrong tree?
i'm looking for general advice on how to expose a single app which is a mosaic of microservices
each service is exposed on the same ip/domain, but each gets its own port
perhaps the whole cluster is accessible at k8s.mydomain.com again
can i map each port to a different load balancer?
k8s.mydomain.com:8000 maps to auth-server-loadbalancer
k8s.mydomain.com:8001 maps to profile-server-loadbalancer
is this possible? it seems less robust and less desirable than strategy 1 above
each service is exposed on its own ip/domain?
perhaps each service specifies a static ip, and my domain has A records pointing each subdomain at each of these ip's in a manual way?
how do i know which static ip's to use? in production? in local dev?
maybe i'm conceptualizing this wrong? can a whole kubernetes cluster map to one ip/domain?
what's the simplest way to expose a bunch of microservies in kubernetes? on the other hand, what's the most robust/ideal way to expose microservices in production? do i need a different strategy for local development in minikube? (i was just going to edit /etc/hosts a lot)
thanks for any advice, cheers
I think the first option is by far the best.
Your Ingress might look like this:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: auth-server.k8s.mydomain.com
http:
paths:
- backend:
serviceName: service1
servicePort: 80
- host: profile-server.k8s.mydomain.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80
- host: questions-board.k8s.mydomain.com
http:
paths:
- backend:
serviceName: service3
servicePort: 80
You can read more about it on Kubernetes docs regarding Ingress and Name based virtual hosting.
You can also use many Ingress Controllers depending where you will end up setting your cluster. You mentioned that you will be testing this on Minikube so I think nginx ingress will be a good choice here.
If you are thinking about managing your traffic you could consider istio .
Here is a nice guide Setting up HTTP(S) Load Balancing with Ingress and another once Configuring Domain Names with Static IP Addresses.
The first method is typically the format that everyone follows ie each microservice gets its own subdomain. You can achieve the same using Kubernetes ingress (for example Nginx Ingress https://kubernetes.github.io/ingress-nginx/)
They need not be in the same domain also ie you can have both *.example.com and *.example2.com
The second method doesn't scale up as you would have a limited number of available ports and running on non-standard ports comes with its own issues.
Use an ingress:
https://kubernetes.io/docs/concepts/services-networking/ingress/#types-of-ingress
With an ingress, you can assign subdomains to different services, or you can serve all the services under different context roots with some url rewriting.
I don't suggest exposing services using different ports. Nonstandard ports have other problems.
I have a server that is orchestrated using k8s it's service looks like below
➜ installations ✗ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
oxd-server ClusterIP 10.96.124.25 <none> 8444/TCP,8443/TCP 3h32m
and it's pod.
➜ helm git:(helm-rc1) ✗ kubectl get po
NAME READY STATUS RESTARTS AGE
sam-test-oxd-server-6b8f456cb6-5gwwd 1/1 Running 0 3h2m
Now, I have a docker image with an env variable that requires the URL of this server.
I have 2 questions from here.
How can the docker image get the URL or access the URL?
How can I access the same URL in my terminal so I make some curl commands through it?
I hope I am clear on the explanation.
If your docker container is outside the kubernetes cluster, then it's not possible to access you ClusterIP service.
As you could guess by its name, ClusterIP type services are only accessible from within the cluster.
By within the cluster I mean any resource managed by Kubernetes.
A standalone docker container running inside a VM which is part of your K8S cluster is not a resource managed by K8S.
So, in order to achieve what you want, you'll have those possibilities :
Set a hostPort inside your pod. This is not recommanded and is listed as a bad practice in the doc. Keep this usage for very specific case.
Switch your service to NodePort instead of ClusterIP. This way, you'll be able to access it using a node IP + the node port.
Use a LoadBalancer type of service, but this solution needs some configuration and is not straightforward.
Use an Ingress along with an IngressController but just like the load balancer, this solution needs some configuration and is not that straightforward.
Depending on what you do and if this is critical or not, you'll have to choose one of these solutions.
1 & 2 for debug/dev
3 & 4 for prod, but you'll have to work with your k8s admin
You can use the name of the service oxd-server from any other pod in the same namespace to access it i.e., if the service is backed by pods that are serving HTTPS, you can access the service at https://oxd-server:8443/.
If the client pod that wants to access this service is in a different namespace, then you can use oxd-server.<namespace> name. In your case that would be oxd-server.default since your service is in default namespace.
To access this service from outside the cluster(from your terminal) for local debugging, you can use port forwarding.
Then you can use the URL localhost:8443 to make any requests and request would be port forwarded to the service.
kubectl port-forward svc/oxd-server 8443:8443
If you want to access this service from outside the cluster for production use, you can make the service as type: NodePort or type: LoadBalancer. See service types here.
I need to expose dashboard service from multiple external ips. In order to expose a service, I used metallb bare metal loadbalancer.
I just manage to expose dashboard service from single external IP. I wonder if it is possible to expose same service from multiple external IPS?
i think you do it using ingress and expose that service as the LoadBalancer so it will be accessible form both load balancer or from both end point.
Ingress will create one Load Balancer and your service also create one load balancer so it will be accessible from both ip
I have a statfulset application which has a server running on port 1000 and has 3 replicas.
Now, I want to expose the application so I have used type: NodePort.
But, I also want 2 replicas to communicate with each other at the same port.
When I do nslookup in case of NodePort type application it gives only one dns name <svc_name>.<namespace>.svc.cluster.local (individual pods don't get a dns) and the application is exposed.
When I do clusterIP: None I get node specific DNS <statfulset>.<svc_name>.<namespace>.svc.cluster.local but application is not exposed. But both do not work together.
How can I achieve both, expose the same port for inter replica communication and expose same port externally?
LoadBalancer: Exposes the service externally using a cloud provider’s load balancer. NodePort and ClusterIP services, to which the external load balancer will route, are automatically created.
I have a container which runs a chatbot using python, exposed port 5000 where the bot is running. Now when i deploy this container on kubernetes, I have few questions
Do i need to run nginx container in the pod where my app container is
running ? If yes why do i need to ? since kubernetes does load
balancing
If i run nginx container on port 80, do I need to run my
app container also on 80 or (can i use a different port like 5000)
what role does gunicorn play here ?
I am a little confused because most of the examples i see online everyone pretty much have nginx container in their pods along with the app containers
As you mentioned Kubernetes can handle its own load balancing, so the answer to your first question is no, you don't need to run nginx especially in the pod where your application is.
Typically, services and pods have IPs that are routable by the cluster network and all traffic which ends at an edge router will be dropped. So in Kubernetes there is a collection of rules that allows inbound connections to reach a cluster services. Which we call Ingress:
An API object that manages external access to the services in a
cluster, typically HTTP.
Confusing part is that Ingress on it's own does not do much. You will have to create an Ingress controller which is a daemon deployed as a Pod. Its job is to read the Ingress Resource information and to process that accordingly. Actually any system capable of reverse proxying can be ingress controller. You can read more about Ingress and Ingress controller in practical approach in this article. Also I do not know about your environment so please remember that you should use type LoadBalancer if you are on the Cloud and type NodePort if in bare-metal environment.
Going to your second question you can run your application on any port you want, just remember to adjust that port in all other configuration files.
About ports and how to expose services you should check the sources in documentation on how the Kubernetes model works in comparison to containers model. You can find a an instructive article here.
Unfortunately I do not have experience with gunicorn so I won't be able to tell you what role does it play in here. Hope this helps.