nginx ingress rate limiting - docker

i am not able to understand one point in the rate-limiting of Nginx ingress
i was referring to one article regarding rate limiting with nginx ingress : https://medium.com/titansoft-engineering/rate-limiting-for-your-kubernetes-applications-with-nginx-ingress-2e32721f7f57#:~:text=When%20we%20use%20NGINX%20ingress,configure%20rate%20limits%20with%20annotations.&text=As%20an%20example%20above%2C%20the,qps)%20on%20the%20Hello%20service.
in limitation section at last
It applies to the whole ingress and is not able to configure
exceptions, eg. when you want to exclude a health check path /healthz
from your service.
if i am creating two ingresses with different names, one has path /hello1 and another /hello2 both pointing to the same service backend.
Now if i am adding rate limiting to only one ingress or path /hello1 will it affect another? if the same host or domain is there ???
ingress 1 : example.com/hello1 - rate-limit set
ingress 2 : example.com/hello2 no rate limiting
Thanks in advance

Rate limit will be applied only to that ingress where you specified it. What is basically nginx-ingress doing in the background - it merges rules into 1 huge config, however they applies to different objects.
e.g 2 different ingresses for same host and diff path.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test1
annotations:
kubernetes.io/ingress.class: 'nginx'
nginx.ingress.kubernetes.io/limit-rps: '5'
spec:
rules:
- host: example.com
http:
paths:
- path: /path1
backend:
serviceName: service1
servicePort: 8080
and
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test2
annotations:
kubernetes.io/ingress.class: 'nginx'
nginx.ingress.kubernetes.io/limit-rps: '10'
spec:
rules:
- host: example.com
http:
paths:
- path: /path2
backend:
serviceName: service1
servicePort: 8080

Related

Many different hosts in Ingress configuration file

I am trying to automate the hosts in Ingress Controller and I'm facing the problem of generating many hosts into one file. What I mean is, I have this ingress.yaml:
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-host
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/default-backend: a
spec:
rules:
- host: a.example.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: a
port:
number: 80
- host: b.example.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: b
port:
number: 80
...
- host: x.example.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: x
port:
number: 80
...
in this example I have multiple instances: a, b, all the way to x and I anticipate a lot more. Right now I am programmatically regenerating the whole ingress.yaml to add/remove certain hosts. This is prone to errors and hard to maintain, as I must constantly be aware about ingress.yaml to be broken for one reason or another.
What would really help me is to put every host into a separate file and (maybe) just tell ingress.yaml to scan the whole directory where those files are to be stored. This way, I can just add/remove a single file and reload Ingress
Is there an option for that? I found somewhere that IngressSpec could be somehow defined, but I do not see any usefull link with a valid example. Maybe someone found a solution to that already and can point me to the right direction?
Hostname wildcards
Hosts can be precise matches (for example “foo.bar.com”) or a wildcard (for example “*.foo.com”).
Or use template system, such as helm or Kustomize
As #jordanm suggested in the comment, I went with multiple Ingress objects on one IngressController, being sure I get rid of nginx.ingress.kubernetes.io/default-backend annotation:
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-x-host
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: x.example.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: x
port:
number: 80
I generate a unique file for each of hosts, replacing x with my unique name.
I also have to make sure that metadata.name is unique. If metadata.name is the same for every object, then it just gets replaced as I apply the new configuration. This works perfectly.

Nginx Redirection of Jenkins on Kubernetes

I am trying to deploy Jenkins on Kubernetes. I have deployed it with ClusterIP along with Nginx Ingress Controller on AKS.
When I access the IP of the Ingress-Controller, the Jenkins login URL (http://ExternalIP/login?from=%2F) comes up. However the UI of the Jenkins page isn't coming up and there is a some sort of redirection happening and keeps growing (http://ExternalIP/login?from=%2F%3Ffrom%3D%252F%253Ffrom%253D%25252F%25253F). I am very new to Ingress controller and annotations. I am not able to figure on what's causing this redirection.
Below are my configuration files. Can anyone please help on what's going wrong ?
ClusterIP-Service.yml
kind: Service
apiVersion: v1
metadata:
name: jenkins-nodeport-svc
namespace: jenkins
labels:
env: poc
app: myapp_jenkins
spec:
ports:
- name: "http"
port: 80
targetPort: 8080
type: ClusterIP
selector:
app: myapp_jenkins
Ingress.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: jenkins-ingress
namespace: jenkins
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/cors-allow-headers: Authorization, origin, accept
nginx.ingress.kubernetes.io/cors-allow-methods: GET, OPTIONS
nginx.ingress.kubernetes.io/enable-cors: "true"
spec:
rules:
- http:
paths:
- backend:
serviceName: jenkins-nodeport-svc
servicePort: 80
path: /(.*)
There's something in your ingress:
path: /(.*)
is a regular expression with a single capturing group that match everything. For example with following url: http://ExternalIP/login?from=myurl your capturing group $1 (the first and only one) would match login?from/myurl.
Now the problem is that nginx.ingress.kubernetes.io/rewrite-target: /$2 annotation is rewriting your url with a non existing capturing group.
You don't need rewriting, you just need to plain forward every request to the service.
Here you can find Rewrite Examples if you interested on it.
But in your case you can set:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: jenkins-ingress
namespace: jenkins
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/cors-allow-headers: Authorization, origin, accept
nginx.ingress.kubernetes.io/cors-allow-methods: GET, OPTIONS
nginx.ingress.kubernetes.io/enable-cors: "true"
spec:
rules:
- http:
paths:
- backend:
serviceName: jenkins-nodeport-svc
servicePort: 80
path: /
and you're good to go.

Docker nginx reverse proxy on Kubernetes

I have a couple of applications, which runs in Docker containers (all on the same VM).
In front of them, I have an nginx container as a reverse proxy.
Now I want to migrate that to Kubernetes.
When I start them by docker-composer locally it works like expected.
On Kubernetes not.
nginx.conf
http {
server {
location / {
proxy_pass http://app0:80;
}
location /app1/ {
proxy_pass http://app1:80;
rewrite ^/app1(.*)$ $1 break;
}
location /app2/ {
proxy_pass http://app2:80;
rewrite ^/app2(.*)$ $1 break;
}
}
}
edit: nginx.conf is not used on kubernetes. I have to use ingress-controller for that:
deployment.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: app0
spec:
replicas: 1
template:
metadata:
labels:
app: app0
spec:
nodeSelector:
"beta.kubernetes.io/os": linux
containers:
- name: app0
image: appscontainerregistry1.azurecr.io/app0:latest
imagePullPolicy: Always
ports:
- containerPort: 80
name: nginx
---
#the other apps
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-nginx
annotations:
# use the shared ingress-nginx
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: apps-url.com
http:
paths:
- path: /
backend:
serviceName: app0
servicePort: 80
- path: /app1
backend:
serviceName: app1
servicePort: 80
- path: /app2
backend:
serviceName: app2
servicePort: 80
---
apiVersion: v1
kind: Service
metadata:
name: loadbalancer
spec:
type: LoadBalancer
ports:
- port: 80
selector:
app: ingress-nginx
I get the response on / (app0). Unfortunately, the subroutes are not working. What I´m doing wrong?
EDIT
I figured out. Ich missed installing the ingress controller. Like on this page (https://kubernetes.io/docs/concepts/services-networking/ingress/) described, the ingress doesn't work if no controller is installed.
I used ingress-nginx as a controller (https://kubernetes.github.io/ingress-nginx/deploy/) because it was the best-described install guide which I was able to find and I didn´t want to use HELM.
I have one more question. How I can change my ingress that subdomains are working.
For example, k8url.com/app1/subroute shows me every time the start page of my app1.
And if I use a domain name proxying, it rewrites every time the domain name by the IP.
you have created deployment successfully but with that service should be there. nginx ngress on kubernetes manage traffic based on the service.
so flow goes like
nginx-ingress > service > deployment pod.
you are missing to create the service for both applications and add the proper route based on that in kubernetes ingress.
Add this :
apiVersion: v1
kind: Service
metadata:
name: loadbalancer
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 80
selector:
app: ingress-nginx
Because you didnot route for Service Load balancer to targetPort to 80

Paths in Kubernetes Ingress Specs

I have the following ingress resource for one of my apps
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name:""
annotations:
ingress.kubernetes.io..
spec:
rules:
- host: my-app
http:
paths:
- path: /path/to/service
backend:
serviceName: my-service
servicePort: 80
This works as expected and I can access my service at http://my-app/path/to/service. However the service in my app takes query parameters that dont seem to correctly get redirected for eg:
http://my-app/path/to/service/more/paths
This brings me back to http://my-app/path/to/service
How can I maintain this path structure ?
I believe you need to use wildcards on your path:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name:""
annotations:
ingress.kubernetes.io..
spec:
rules:
- host: my-app
http:
paths:
- path: /path/to/service/*
backend:
serviceName: my-service
servicePort: 80
More information here. Seems like it's hard to find any docs with wildcard examples. Not that this is specific to nginx, it may not work with other ingress controllers.

kubernetes unhealthy ingress backend

I followed the load balancer tutorial: https://cloud.google.com/container-engine/docs/tutorials/http-balancer which is working fine when I use the Nginx image, when I try and use my own application image though the backend switches to unhealthy.
My application redirects on / (returns a 302) but I added a livenessProbe in the pod definition:
livenessProbe:
httpGet:
path: /ping
port: 4001
httpHeaders:
- name: X-health-check
value: kubernetes-healthcheck
- name: X-Forwarded-Proto
value: https
- name: Host
value: foo.bar.com
My ingress looks like:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: foo
spec:
backend:
serviceName: foo
servicePort: 80
rules:
- host: foo.bar.com
Service configuration is:
kind: Service
apiVersion: v1
metadata:
name: foo
spec:
type: NodePort
selector:
app: foo
ports:
- port: 80
targetPort: 4001
Backends health in ingress describe ing looks like:
backends: {"k8s-be-32180--5117658971cfc555":"UNHEALTHY"}
and the rules on the ingress look like:
Rules:
Host Path Backends
---- ---- --------
* * foo:80 (10.0.0.7:4001,10.0.1.6:4001)
Any pointers greatly received, I've been trying to work this out for hours with no luck.
Update
I have added the readinessProbe to my deployment but something still appears to hit / and the ingress is still unhealthy. My probe looks like:
readinessProbe:
httpGet:
path: /ping
port: 4001
httpHeaders:
- name: X-health-check
value: kubernetes-healthcheck
- name: X-Forwarded-Proto
value: https
- name: Host
value: foo.com
I changed my service to:
kind: Service
apiVersion: v1
metadata:
name: foo
spec:
type: NodePort
selector:
app: foo
ports:
- port: 4001
targetPort: 4001
Update2
After I removed the custom headers from the readinessProbe it started working! Many thanks.
You need to add a readinessProbe (just copy your livenessProbe).
It's explained in the GCE L7 Ingress Docs.
Health checks
Currently, all service backends must satisfy either of the following requirements to pass the HTTP health checks sent to it from the GCE loadbalancer: 1. Respond with a 200 on '/'. The content does not matter. 2. Expose an arbitrary url as a readiness probe on the pods backing the Service.
Also make sure that the readinessProbe is pointing to the same port that you expose to the Ingress. In your case that's fine since you have only one port, if you add another one you may run into trouble.
I thought it's worth noting that this is a quite important limitation in the documentation:
Changes to a Pod's readinessProbe do not affect the Ingress after it is created.
After adding my readinessProbe I basically deleted my ingress (kubectl delete ingress <name>) and then applied my yaml file again to re-create it and shortly after everything was working again.
I was having the same issue. Followed Tex's tip but continued to see that message. It turns out I had to wait a few minutes before ingress to validate the service health. If someone is going through the same and done all the steps like readinessProbe and linvenessProbe, just ensure your ingress is pointing to a service that is either a NodePort, and wait a few minutes until the yellow warning icon turns into a green one. Also, check the log on StackDriver to get a better idea of what's going on.
I was also having exactly the same issue, after updating my ingress readinessProbe.
I can see Ingress status labeled Some backend services are in UNKNOWN state status in yellow.
I waited for more than 30 min, yet the changes were not reflected.
After more than 24 hours the changes reflected and status turned green.
I didn't get any official documentation for this but seems like a bug in GCP Ingress resource.
If you don't want to change your pod spec, or rely on the magic of GKE pulling out your readinessProbe, you can also configure a BackendConfig like this to explicitly configure the health check.
This is also helpful if you want to use a script for your readinessProbe, which isn't supported by GKE ingress health checks.
Note that the BackendConfig needs to be explicitly referenced in your Service definition.
---
apiVersion: v1
kind: Service
metadata:
name: my-service
namespace: my-namespace
annotations:
cloud.google.com/neg: '{"ingress":true}'
# This points GKE Ingress to the BackendConfig below
cloud.google.com/backend-config: '{"default": "my-backendconfig"}'
spec:
type: ClusterIP
ports:
- name: health
port: 1234
protocol: TCP
targetPort: 1234
- name: http
...
selector:
...
---
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: my-backendconfig
namespace: my-namespace
spec:
healthCheck:
checkIntervalSec: 15
port: 1234
type: HTTP
requestPath: /healthz
Everyone of these answers helped me.
In addition, the http probes need to return a 200 status. Stupidly, mine was returning a 301. So I just added a simple "ping" endpoint and all was well/healthy.

Resources