How to serve static contents in a kubernetes application - docker

I have a small java webapp comprising of three microservices - api-service,book-service and db-service all of which are deployed on a kubernetes cluster locally using minikube.
I am planning to keep separate UIs for api-service and book-service , with the common static files served from a separate pod, probably an nginx:alpine image.
I was able to create a front end that serves the static files from nginx:alpine referring to this tutorial.
I would like to use ingress-nginx controller for routing requests to the two services.
The below diagram crudely shows where I am now.
I am confused as to where I should place the pod that serves the static content, and how to connect it to the ingress resource.I guess that keeping a front end pod before ingress defeats the purpose of ingress-nginx controller. What is the best practice to serve static files. Appreciate any help. Thanks.

Looks like you are confusing the fact that users, browsing online, will trigger standard requests to both "download" your static content, and use your 2 APIs (book and api). It's not the NGINX service serving the static content that is accessing your APIs, but the users browsers/applications, and they do that exactly the same for both static content and APIs (former has more/specific headers and data, like auth...).
On your diagram you'll want to put your new static-service at the exact same level as your book-service and api-service, ie behind the ingress. But your static-service won't have a link with the db-service like the other 2. Then just complete your ingress rules, with the static-service at the end as in this example:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: your-global-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /book-service
backend:
serviceName: book-service
servicePort: 80
- path: /api-service
backend:
serviceName: api-service
servicePort: 80
- path: /
backend:
serviceName: static-service
servicePort: 80
You'll have to adjust your services names and ports, and pick the paths you want your users to access your APIs, in the example above you'd have:
foo.bar.com/book-service for your book-service
foo.bar.com/api-service for the api-service
foo.bar.com/ ie everything else going to the static-service

You should have, 3 distinct pods I guess :
- static
- book-service
- api-service
The static pod will most likely not scale at the same speed than the two other.
Creating the services for each of your deployment. Then use the ingres to route the traffic on the proper endpoint.
Is it something like that you are trying to achieve?

Related

Kubernetes Ingress With Dynamic routing?

I have a Jenkins build process that deploys our code to several Kubernetes environments. Currently, it works fantastic with known/core branches (develop, release, master) and such, BUT we would like to make all of our feature branches and bug fix branches as well. In order to do this, we currently have it deploying an ingress with branch subdomains, like "{branchName}.domain.tld". The problem here is that there is a proliferation of ingresses and such for each branch.
I would LIKE to do something like "branch.domain.tld/{branchName}" and have the ingress dynamically route based on the "branchName" path. Unfortunately I only seem to get a single ingress per subdomain/host assignment (branch.domain.tld/bugfix_nasty doesn't route because branch.domain.tld/feat_cool already is defined).
What I would like to see is a single ingress that looks something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: dynamic_ingress
annotations:
{...stuff here}
spec:
rules:
- host: branch.domain.tld
http:
paths:
- path: /{branchName}/*
backend:
serviceName: {branchName}
servicePort: 80
Is there anyway to get to such a zenlike ingress?
All the stuff I see about this type of thing is advice on using regex for the path to capture and route to a known, defined service. I want this to route to variable service names.
As far as I know Ingress objects don't allow what you're trying to do. But there could be a workaround if you can modify the Ingress object every time you deploy a new branch.
Whenever a new branch is deployed update your "dynamic_ingress" to include an extra path for the new branch. This can be achieved using a small utility that can read and write YAMLs which can then be applied to update the existing Ingress OR you can use kubectl patch to update the existing Ingress at deploy time.
This way the Ingress definition has static paths but neither do you have to set the paths manually nor do you need multiple Ingress objects.
So you would have something like this.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: dynamic_ingress
annotations:
{...stuff here}
spec:
rules:
- host: branch.domain.tld
http:
paths:
- path: /bug_fix_nasty/
pathType: Prefix
backend:
serviceName: bug_fix_nasty
servicePort: 80
- path: /feat_cool/
pathType: Prefix
backend:
serviceName: feat_cool
servicePort: 80
Check out all the examples in the Ingress doc and see which suits you best.

How to setup load balancer with ingress nginx controller

I am new to microservices. I have few apps to deploy as microservices.
I need a API and Load balancer. For API gateway I come to know about Ingress Nginx. But I am not sure hot setup load balancing. However I could configure it for API gateway as below.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: example.dev
http:
paths:
- path: /api/users/?(.*)
backend:
serviceName: auth-srv
servicePort: 3000
- path: /api/orders/?(.*)
backend:
serviceName: order-srv
servicePort: 3000
- path: /?(.*)
backend:
serviceName: client-srv
servicePort: 3000
I also have one confusion:
Load balancer sits before API Gateway i.e nginx controller,
How would it I configure Ingress-Nginx for load balancing?
Because load balancer will redirect request to nginx controller
So, Load balancer -> API Gateway > /api/orders
Now, /api/orders -> order-srv -> pods
Clearly Load balancer should decide to which pod it should route the request?
How can I achieve that?
Well, this is something like -
You got a request from external world and that has been at first level intercepted at load balancer layer and API Gateway should be one of the Microservices where you will keep all of the URL mappings say - /api/orders/{orderId} brings this to your API Gateway and in that API Gateway you can have a logic to redirect this to some order service behind the scenes via Fully Qualified Domain Name of order service (FQDN) :portNumber/{uri}
So its good idea to simply route traffic to frontend and API Gateway via your ingress rules , i.e.
- path: /?(.*) will take it to client service or frontend
- path: /api/?(.*) this shall take it to API Gateway service which has routing map to all of the services appearing behind the scenes

Kubernetes Ingress does not accommodate CSS files for Visual Studio Generated ASP.NET MVC Hello World WebApp

I'm following these instructions https://learn.microsoft.com/en-us/azure/aks/ingress-basic using Docker for Desktop/Kubernetes on Windows 10 again. The tutorial works great and now I want to enhance it by adding my C#/ASP.NETCore3 MVC WebApp called KubernetesHelloWorldASPNET (available on docker hub here). This is basically the template created by Visual Studio 2019 for a C# ASP.NET/MVC Core WebApp.
Here is the hello-world-ingress.yaml:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-world-ingress
namespace: ingress-basic
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
- backend:
serviceName: aks-helloworld-one
servicePort: 80
path: /hello-world-one(/|$)(.*)
- backend:
serviceName: aks-helloworld-two
servicePort: 80
path: /hello-world-two(/|$)(.*)
- backend:
serviceName: kuberneteshelloaspnet
servicePort: 80
path: /kuberneteshelloaspnet(/|$)(.*)
- backend:
serviceName: todolistclient
servicePort: 80
path: /todolistclient(/|$)(.*)
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-world-ingress-static
namespace: ingress-basic
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /static/$2
spec:
rules:
- http:
paths:
- backend:
serviceName: aks-helloworld-one
servicePort: 80
path: /static(/|$)(.*)
This is working with the above ingress tutorial in the sense that I can see the web page for KubernetesHelloWorldASPNET from the browser (http://localhost/kuberneteshelloaspnet) via ingress. However, it is not formatted properly. I believe the problem is that ingress is not allowing the browser to get the CSS and other static files generated by Visual Studio.
How can I make my little HelloWorld program display properly with the CSS?
The web page looks fine when run directly with Visual Studio (no docker, no kubernetes, no ingress).
Thu Sep 09 2020 Update
As per Matt's comments, I used the developer tools in the browser and confirmed that I'm getting 404 errors for for bootstrap.min.css, boostrap.bundle.min.js, site.js, jquery.min.js.
I found a similar post here. However, it looks like the solution precludes having ingress accommodate multiple web sites each having their own CSS and JS directories. They are creating rules for paths like "/css" and "/js" and that makes me think that all the applications must share the same css and js directories?
I'm following the tutorial and creating a path that has wild cards in it. So I have
path: /kuberneteshelloaspnet(/|$)(.*)
This should accommodate (in my example using docker for desktop kubernetes) http://localhost/kuberneteshelloaspnet/lib/jquery/dist/jquery.min.js -- correct? Well it does not seem to be working!
Now that developer tools network display in Chrome seems to indicate that the js and css files have only a file name and no path. But when click on "Inspect" I see the the HTML and the HTML is referencing child directories like "lib" and "css" and this is consistent with the Visual Studio generated project directory.
So apparently my ingress rules are allowing access to the home MVC controller at http://localhost:80/kuberneteshelloaspnet but is not allowing access to http://localhost:80/kuberneteshelloaspnet/lib/jquery/dist/jquery.min.js (for example).
How do I modify my ingress rules to accommodate these child directories of static files?
Sat Sep 12 2020 Update: Progess!
This version of hello-world-ingress.yaml works for a single web-app:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-world-ingress
namespace: ingress-basic
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- backend:
serviceName: kuberneteshelloaspnet
servicePort: 80
path: /
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-world-ingress-static
namespace: ingress-basic
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- backend:
serviceName: kuberneteshelloaspnet
servicePort: 80
path: /lib(/|$)(.*)
It was just a guess and I don't know why it works. The web page seems to be working because it is formatted properly and I don't see any more 404 errors in the chrome developer tools panel.
Please help me understand the contents of this hello-world-ingress.yaml file. I was expecting to see kind: service and later kind: deployment and instead I just see kind: Ingress twice. Is this file defining a service or a deployment or both
Is it possible to enhance my solution so I host multiple web sites where each has a separate path and each has has separate sub-paths like /js and /css and /lib so each web site can have its own copy of (for example) jquery.min.js because they need different versions of jquery?
Among other things, I deleted the rewrite rules. When would I want to use rewrite rules? Could rewrite rules help me accommodate multiple web sites?
Fri Sep 18 2020: Progress?
I may have found the solution of using the function UsePathBase in this blog. The symptoms described in this blog match mine. However, as I describe here, I cannot make this work! Calling the function UseBasePath seems to have no affect.
Thanks
Siegfried
I had been using the Microsoft Ingress Example that demonstrates the rewrite rule.
The problem with using the rewrite rule is that it does not accommodate multiple web-apps that have different copies of static files like jquery.min.js.
The solution for C#/VB.NET ASP.NET Core programmers is to use UseBasePath.
I wish the above tutorial/example would explain these different options and their perils and merits. Obviously UseBasePath won't work with java or python webapps (although there might be counterparts).
I did not realize that since the rewrite rule executes first to remove the root segment of the path, then there is nothing for the UseBasePath to remove and it seems to have no affect.
The solution is to remove the rewrite rule after having added a call to UseBasePath.
In chrome check what path is missing. Hit the URL in chrome developer mode(press F12) then you will be to find. Add that path & this will be resolved.
I faced the same issue.

How to do Dynamic Proxying to Kubernetes Pods?

I'm wanting to create a service that can do some kind of dynamic proxying back to Kubernetes Pods. Basically I'll have hundreds of K8s Pods that are running the same application that map to a random port on the host (like 10456). However, each Pod is unique and I want traffic directed at a specific pod based on hostname. So when a request comes in for abc123.app.com, I'll have a proxy layer that does a lookup in a database to find what host and port that domain is running on (like 10.0.0.5:10456), then forward the request there. Is there a service that supports this? I've worked with Nginx a lot before, but I'm not clear if it could support this lookup functionality.
Has anyone built something like this before? what's the best way to build a proxy layer that can do lookups like that? How would I update the database when a pod moves from one host to another?
Thanks in advance!
EDIT:
I should have put this in there the first time, but the types of traffic going to these pods are RPC traffic and Peer to Peer traffic
You're describing something very similar to what kubernetes ingress definitions do for http traffic.
An ingress definition configures an ingress controller to point requests for a hostname at a service. The service selects endpoints (pods) via label selectors. When pods move, kubernetes updates the service automatically.
The work on your end just becomes pushing out config changes from your database via one of the API clients to kubernetes rather than directing a proxy. If your environment was extremely dynamic requiring reconfiguration all the time or you need to make dynamic decisions about where traffic should go, you might want to continue looking at a custom proxy or istio, openresty.
It sounds like you have unique deployments going to kubernetes already, so in addition to that include a service and ingress definition.
A simple example including a label on the a pod, a service that use the label. Then an ingress definition using the service.
apiVersion: v1
kind: Pod
metadata:
name: my-app
labels:
app: host-abc123
spec:
containers:
- name: host-abc123
image: me/my-app:1.2.1
ports:
- containerPort: 10456
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: host-abc123
spec:
rules:
- host: abc123.bar.com
http:
paths:
- backend:
serviceName: host-abc123
servicePort: 80
apiVersion: v1
kind: Service
metadata:
name: host-abc123
spec:
ports:
- protocol: TCP
port: 80
targetPort: 10456
The single ingress definition could include all hosts but I'm not sure how kubernetes and the ingress controllers would go replacing that regularly.
There are nginx based ingress controllers too. You end up with a nginx server config per ingress/host definition.

Nginx Ingress Looking for Service in all Custom Namespaces

I have three namespaces dev, test and staging. test and staging have no pods in them. In dev I have nginx, ingress and a frontend service. For all requests to the nginx it's forwarded to the frontend service.
But the issue is nginx in dev trying to find frontend service in test and staging namespaces also. It's doing round robin between the 3 namespaces. So sometimes the page is loading and sometimes it's 503 error.
Here is the ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: frontend-ingress
namespace: dev
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: frontend
servicePort: 80
And here is the log of nginx:
I0327 07:54:50.867120 1 command.go:76] change in configuration detected. Reloading...
W0327 07:54:50.867339 1 controller.go:841] service test/frontend does not have any active endpoints
W0327 07:54:50.867370 1 controller.go:841] service staging/frontend does not have any active endpoints
W0327 07:54:50.868198 1 controller.go:777] upstream test-frontend-80 does not have any active endpoints. Using default backend
W0327 07:54:50.868219 1 controller.go:777] upstream staging-frontend-80 does not have any active endpoints. Using default backend
Specify --force-namespace-isolation=true argument when deploying nginx pod. And update image to quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.12.0
I'd like to elaborate on #Narayan-Prusty's answer.
I had to add --force-namespace-isolation=true and set image to quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.12.0 but also had to add --watch-namespace=$(POD_NAMESPACE).

Resources