I'm aiming to deploy a small test application to GCE. Every guide I've read seems to point to using a LoadBalancer service to expose the pod to the internet. Unfortunately, this comes with a high associated cost and I'd like to be able to expose the containers without creating a load balancer (or using HAProxy / nginx to roll our own).
Is it possible to do so? If so, what are the steps I need to take and possible other associated costs?
Thanks!
The NGINX ingress controller found at https://github.com/kubernetes/ingress/tree/master/controllers/nginx should satisfy your cost saving requirement. I would not consider this "rolling your own" as this lives beside the GLBC ingress controller.
There should be sufficient documentation to satisfy your installation requirements and if there's not please open an issue on https://github.com/kubernetes/ingress
You can do that by choosing a NodePort as the service type.
apiVersion: v1
kind: Service
metadata:
name: myapp-servoce
labels:
name: myapp
context: mycontext
spec:
type: NodePort
ports:
# the port that this service should serve on
- port: 8080
# label keys and values that must match in order to receive traffic for this service
selector:
name: myapp
context: mycontext
This would expose that service on port 8080 of each node of the cluster. Now all of your nodes would have externally accessible IP address and you can use the same for testing
Related
Problem
I have two pods A and B running in a cluster on minikube, both have external IPs www.service-a.com and www.service-b.com. Both external IPs are accessible from outside.
I need A to be able to call B with it's external IP rather than its cluster DNS, that is A needs to use www.service-b.com rather than b.svc.cluster.local (which does work but I can't use it).
I set A to use hostNetwork: true and dnsPolicy: ClusterFirstWithHostNet. If I spin up a NodeJS docker container manually, it indeed can connect and find it. However, A is still unable to connect to service-b.com. Am I using hostNetwork wrong? How can I configure my pod to connect to b in that way?
A's Deployment YAML
...
spec:
replicas: 1
selector:
matchLabels:
app: a-app
template:
metadata:
labels:
app: a-app
spec:
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
containers:
...
B's service YAML
...
spec:
externalTrafficPolicy: Cluster
type: LoadBalancer
ports:
- port: ...
targetPort: ...
protocol: TCP
name: http
...
Background:
I'm using Minio (a local S3-like solution) and I need to presign the URLs to get and put objects. Minio's pods are running in the same cluster as my authentication pod which would generate the presigned urls. The presigned urls would be used from outside the cluster. Hence I can't sign the url with the cluster dns names like minio.svc.cluster.local because this URL would not be accessible from outside the cluster and replacing the host with my-minio.com and keeping the signature does not work because I guess minio signs the entire host and path. Hence I need to have my authentication pod connect to Minio's public facing my-minio.com instead which does not seem to work.
Regarding hostNetwork, it looks like you misunderstood it. Setting it to true means that Pod will have access to the host where it's running. In case of minikube it's VM and not your host where actual containers are running.
Also, i'm not sure how you expose your services to external world, but i suggest you to try Ingress for that.
As Grigoriy suggested, I used an ingress with nginx.ingress.kubernetes.io/upstream-vhost annotation to forward all requests into the cluster with Host: service-b to resolve my issue. Previously I had nginx.ingress.kubernetes.io/rewrite-target: /$1 which stripped the path from the request that caused a serious of issues, so I removed that. The details of how I got it working are here:
NGINX controller Kubernetes: need to change Host header within ingress
i have a handful of dockerized microservices, each is listening for http requests on a certain port, and i have these deployments formalized as kubernetes yaml files
however, i can't figure out a working strategy to expose my deployments on the interwebs (in terms of kubernetes services)
each deployment has multiple replicas, and so i assume each deployment should have a matching load balancer service to expose it to the outside
now i can't figure out a strategy to sanely expose these microservices to the internet... here's what i'm thinking:
the whole cluster is exposed on a domain name, and services are subdomains
say the cluster is available at k8s.mydomain.com
each loadbalancer service (which exposes a corresponding microservice) should be accessible by a subdomain
auth-server.k8s.mydomain.com
profile-server.k8s.mydomain.com
questions-board.k8s.mydomain.com
so requests to each subdomain would be load balanced to the replicas of the matching deployment
so how do i actually achieve this setup? is this desirable?
can i expose each load balancer as a subdomain? is this done automatically?
or do i need an ingress controller?
am i barking up the wrong tree?
i'm looking for general advice on how to expose a single app which is a mosaic of microservices
each service is exposed on the same ip/domain, but each gets its own port
perhaps the whole cluster is accessible at k8s.mydomain.com again
can i map each port to a different load balancer?
k8s.mydomain.com:8000 maps to auth-server-loadbalancer
k8s.mydomain.com:8001 maps to profile-server-loadbalancer
is this possible? it seems less robust and less desirable than strategy 1 above
each service is exposed on its own ip/domain?
perhaps each service specifies a static ip, and my domain has A records pointing each subdomain at each of these ip's in a manual way?
how do i know which static ip's to use? in production? in local dev?
maybe i'm conceptualizing this wrong? can a whole kubernetes cluster map to one ip/domain?
what's the simplest way to expose a bunch of microservies in kubernetes? on the other hand, what's the most robust/ideal way to expose microservices in production? do i need a different strategy for local development in minikube? (i was just going to edit /etc/hosts a lot)
thanks for any advice, cheers
I think the first option is by far the best.
Your Ingress might look like this:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: auth-server.k8s.mydomain.com
http:
paths:
- backend:
serviceName: service1
servicePort: 80
- host: profile-server.k8s.mydomain.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80
- host: questions-board.k8s.mydomain.com
http:
paths:
- backend:
serviceName: service3
servicePort: 80
You can read more about it on Kubernetes docs regarding Ingress and Name based virtual hosting.
You can also use many Ingress Controllers depending where you will end up setting your cluster. You mentioned that you will be testing this on Minikube so I think nginx ingress will be a good choice here.
If you are thinking about managing your traffic you could consider istio .
Here is a nice guide Setting up HTTP(S) Load Balancing with Ingress and another once Configuring Domain Names with Static IP Addresses.
The first method is typically the format that everyone follows ie each microservice gets its own subdomain. You can achieve the same using Kubernetes ingress (for example Nginx Ingress https://kubernetes.github.io/ingress-nginx/)
They need not be in the same domain also ie you can have both *.example.com and *.example2.com
The second method doesn't scale up as you would have a limited number of available ports and running on non-standard ports comes with its own issues.
Use an ingress:
https://kubernetes.io/docs/concepts/services-networking/ingress/#types-of-ingress
With an ingress, you can assign subdomains to different services, or you can serve all the services under different context roots with some url rewriting.
I don't suggest exposing services using different ports. Nonstandard ports have other problems.
I followed this guide to create a Windows Server container on Azure Kubernetes Service and this guide to secure it with an ingress-controller. I was successful and the web frontend of the container can now be reached via https through the ingress-controller. However, it can also be reached via the external IP adress of the service itself, which is not secure.
Now I already read something about a ClusterIP which, if I understand it correctly, is a type of service that has no external IP adress, but I wasn't able to find specific documentation on how to create one. Also I noticed that my service already has the type load-balancer. Can one service have multiple types or do I have to create an additional service?
You can have only one service type if you want to expose your application to the internet you can use the service type Load Balancer.
Service ClusterIp will not expose the application to the internet.
apiVersion: v1
kind: Service
metadata:
name: my-service
labels:
run: my-service
spec:
ports:
- port: 80
protocol: TCP
selector:
run: my-service
Example if service having clusterIP. If you have not mentation any type in service it will create the default service as ClusterIp assign default IP to the pod.
I just tried setting up kubernetes on my bare server,
Previously I had successfully create my docker compose
There are several apps :
Apps A (docker image name : a-service)
Apps B (docker image name : b-service)
Inside Application A and B there are configs (actually there are apps A,B,C,D,etc lots of em)
The config file is something like this
IPFORSERVICEA=http://a-service:port-number/path/to/something
IPFORSERVICEB=http://b-service:port-number/path/to/something
At least above config work in docker compose (the config is inside app level, which require to access another apps). Is there any chance for me to access another Kubernetes Service from another service ? As I am planning to create 1 app inside 1 deployment, and 1 service for each deployment.
Something like:
App -> Deployment -> Service(i.e: NodePort,ClusterIP)
Thanks !
Is there any chance for me to access another Kubernetes Service from
another service ?
Yes, you just need to specify DNS name of service (type: ClusterIP works fine for this) you need to connect to as:
<service_name>.<namespace>.svc.cluster.local
In this case such domain name will be correctly resolved into internal IP address of the service you need to connect to using built-in DNS.
For example:
nginx-service.web.svc.cluster.local
where nginx-service - name of your service and web - is apps's namespace, so service yml definition can look like:
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: web
spec:
ports:
- name: http
protocol: TCP
port: 80
selector:
app: nginx
type: ClusterIP
See official docs to get more information.
Use Kubernetes service discovery.
Service discovery is the process of figuring out how to connect to a
service. While there is a service discovery option based on
environment variables available, the DNS-based service discovery is
preferable. Note that DNS is a cluster add-on so make sure your
Kubernetes distribution provides for one or install it yourself.
Service dicovery by example
I need to deploy a Docker running HAProxy which I already have working on on premise dockers into IBM Cloud (Bluemix) Kubernetes service. I am a bit lost on how to expose por 80 and 443. In plain simple docker that is very straightforward but seems complicated in Kubernetes, or at least in IBM Cloud.
I don't need load balancing, virtual hosts, or any extra configuration, as HAProxy will take care of it. Just need to replicate (move) my on premise running HAProxy exposing ports 80 and 443 into bluemix. (For multiple reasons I want to use HAproxy, so the request here is very specific: Simplest way to expose HAProxy ports 443 and 80 to a permanent IP address in IBM Cloud Kubernetes service.
could I have a basic example yaml kubectl file for that? Thanks
NodePort
To keep the same image running in both environments then you can define a Deployment for the HAProxy containers and a Service to access them via a NodePort on the NodeIP or clusterIP. A NodePort is similar in concept to running docker run -p n:n.
The IP:NodePort would need to be accessable externally and HAProxy will take over from there. Here's a sample HAProxy setup that uses an AWS ELB to get external users to a Node. Most people don't recommend running services via NodePort because Kubernetes offers alternate methods that provide more integration.
LoadBalancer
A LoadBalancer is specifically for automatic configuration of a cloud providers load balancer service. I don't believe IBM Clouds load balancer has any support in Kubernetes, maybe IBM have added something in? If they have you could use this instead of a NodePort to get to your Service.
Ingress
If you are running Docker locally and Kubernetes externally you've kind of thrown consistency out the window already so you could setup Ingress with an Ingress Controller based on HAProxy, there's a few available:
https://github.com/appscode/voyager
https://github.com/jcmoraisjr/haproxy-ingress
This gives you the standard Kubernetes abstraction of how to manage ingress for a service but using HAProxy underneath. This will not be your HAProxy image though, it's likely you can configure the same things for the HAProxy Ingress as you do in your HAProxy image.
Voyagers docco is pretty good:
apiVersion: voyager.appscode.com/v1beta1
kind: Ingress
metadata:
name: test-ingress
namespace: default
spec:
rules:
- host: appscode.example.com
http:
paths:
- path: '/test'
backend:
serviceName: test-service
servicePort: '80'
backendRules:
- 'acl add_url capture.req.uri -m beg /test-second'
- 'http-response set-header X-Added-From-Proxy added-from-proxy if add_url'
If you are fine with running this HAProsy on each node that is supposed to expose port 80/443 then consider running DaemonSet with hostNetwork: true. That will allow you to create pods that open 80 and 443 directly on node network. If you have a loadbalancer support in your cluster, you can instead use a Service of LoadBalancer type. It will forward from high node ports like ie. 32080 to your backing haproxy pods, and also automaticaly configure LB in front of it to give you an external IP and forward 80/443 from that IP to your high node ports (again, assuming your kube deployment supports use of LB services)
IBM Cloud has built-in solutions for load balancer and Ingress. The docs include sample YAMLs for both.
Load Balancer: https://console.bluemix.net/docs/containers/cs_loadbalancer.html#loadbalancer
Ingress: https://console.bluemix.net/docs/containers/cs_ingress.html#ingress
If you need tls termination or want to use a route rather than an IP address for accessing your HAProxy, then Ingress would be the best choice. If those options don't matter, then I'd suggest starting with the provided load balancer to see if that meets your needs.
Note, both load balancer and Ingress required a paid cluster. For lite clusters, only NodePort is available.
Here's a sample YAML that deploys IBM Liberty and exposes it via a load balancer service.
#If you are not logged into the US-South https://api.ng.bluemix.net
region, change the image registry location to match your region.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: ibmliberty-deployment
spec:
replicas: 3
template:
metadata:
labels:
app: ibmliberty
spec:
containers:
- name: ibmliberty
image: registry.ng.bluemix.net/ibmliberty
---
apiVersion: v1
kind: Service
metadata:
name: ibmliberty-loadbalancer
spec:
type: LoadBalancer
selector:
app: ibmliberty
ports:
- protocol: TCP
port: 9080