Kubernetes Nginx Ingress Controller expose Nginx Webserver - docker

I basically want to access the Nginx-hello page externally by URL. I've made a (working) A-record for a subdomain to my v-server running kubernetes and Nginx ingress: vps.my-domain.com
I installed Kubernetes via kubeadm on CoreOS as a single-node cluster using these tutorials: https://kubernetes.io/docs/setup/independent/install-kubeadm/, https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/, and nginx-ingress using https://kubernetes.github.io/ingress-nginx/deploy/#bare-metal.
I also added the following entry to the /etc/hosts file:
31.214.xxx.xxx vps.my-domain.com
(xxx was replaced with the last three digits of the server IP)
I used the following file to create the deployment, service, and ingress:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- name: http
containerPort: 80
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
run: my-nginx
spec:
type: ClusterIP
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
selector:
run: my-nginx
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-nginx
annotations:
kubernetes.io/ingress.class: "nginx"
ingress.kubernetes.io/ssl-redirect: "False"
spec:
rules:
- host: vps.my-domain.com
http:
paths:
- backend:
serviceName: my-nginx
servicePort: 80
Output of describe ing:
core#vps ~/k8 $ kubectl describe ing
Name: my-nginx
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
vps.my-domain.com
my-nginx:80 (<none>)
Annotations:
kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"extensions/v1beta1",...}
kubernetes.io/ingress.class: nginx
ingress.kubernetes.io/ssl-redirect: False
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal UPDATE 49m (x2 over 56m) nginx-ingress-controller Ingress default/my-nginx
While I can curl the Nginx hello page using the nodeip and port 80 it doesn't work from outside the VM. Failed to connect to vps.my-domain.com port 80: Connection refused
Did I forgot something or is the configuration just wrong? Any help or tips would be appreciated!
Thank you
EDIT:
Visiting "vps.my-domain.com:30519` gives me the nginx welcome page. But in the config I specified port :80.
I got the port from the output of get services:
core#vps ~/k8 $ kubectl get services --all-namespaces | grep "my-nginx"
default my-nginx ClusterIP 10.107.5.14 <none> 80/TCP 1h
I also got it to work on port :80 by adding
externalIPs:
- 31.214.xxx.xxx
to the my-nginx service. But this is not how it's supposed to work, right? In the tutorials and examples for kubernetes and ingress-nginx, it worked always without externalIPs. Also the ingress rules doesn't work now (e.g. if I set the path to /test).

So apparently I was missing one part: the load balancer. I'm not sure why this wasn't mentioned in those instructions as a requirement. But i followed this tutorial: https://kubernetes.github.io/ingress-nginx/deploy/baremetal/#a-pure-software-solution-metallb and now everything works.
Since metallb requires multiple ip addresses, you have to list your single ip-adress with the subnet \32: 31.214.xxx.xxx\32

Related

Minikube exposed service url returns connection refused but kubectl port-forward working fine

I did this setup to test kubernetes with minikube 'Set up Ingress on Minikube' and everything worked fine.
Then I tried to do the same with my own app and am having a problem after configuring all the steps.
The steps that I did to setup my app and kubernetes are:
Create an app that works on port 5000
Containarized the app in a docker image and upload to the minikube image registry
Created a deployment for kubernetes with my container
Run kubectl port-forward pod/app 5000 and everyting works fine
Created a service with type Nodeport to expose the deployment
Run kubectl port-forward service/app-service 5000 and everyting works fine
Created an ingress to expose the service
Run curl app.info and it returns 502 bad gateway
Tryied again kubectl port-forward service/app-service 5000 and it still works fine
Check minikube service app-service --url and tried the result URL and it returns Connection refused, the equivalent url in the demo setup that I did previously works fine so it looks like something is wrong in this step even when doing the port-forwarding works correctly.
kind: Deployment
metadata:
namespace: echo-app
name: app
labels:
app: echo
tier: services
spec:
replicas: 1
selector:
matchLabels:
tier: services
template:
metadata:
labels:
tier: services
spec:
containers:
- name: echo-api
image: echo/api:v1.0.0b39c8f9a
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5000
apiVersion: v1
kind: Service
metadata:
name: app-service
namespace: echo-app
spec:
type: NodePort
selector:
tier: services
ports:
- protocol: TCP
port: 5000
targetPort: 5000
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: api-ingress
namespace: echo-app
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: echo.info
http:
- paths:
path: /
pathType: Prefix
backend:
service:
name: api-service
port:
number: 5000

Cannot access ingress on local minikube

I am trying to setup a jenkins server inside of a kubernetes container with minikube. Following up the documentation for this, I have the following kubernetes configuration.
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins
spec:
replicas: 1
selector:
matchLabels:
app: jenkins
template:
metadata:
labels:
app: jenkins
spec:
containers:
- name: jenkins
image: jenkins/jenkins:lts
ports:
- name: http-port
containerPort: 8080
- name: jnlp-port
containerPort: 50000
---
apiVersion: v1
kind: Service
metadata:
name: jenkins-service
spec:
selector:
app: jenkins
ports:
- protocol: TCP
port: 8080
targetPort: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: jenkins-ingress
annotations:
nginx.ingress.kubernetes.io/default-backend: ingress-nginx-controller
kubernetes.io/ingress.class: nginx
## tells ingress to check for regex in the config file
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- http:
paths:
- path: /?(.*)
pathType: Prefix
backend:
service:
name: jenkins-service
port:
number: 8080
I am trying to create a deployment (pod), service for it and an ingress to expose it.
I got the ip address of the ingress with kubectl get ingress and it is the following:
NAME CLASS HOSTS ADDRESS PORTS AGE
jenkins-ingress <none> * 80 5s
jenkins-ingress <none> * 192.168.49.2 80 5s
When I try to open 192.168.49.2 in the browser, I get timeout. The same happens when I try to ping this ip address from the terminal.
The addon for ingress is enabled in minikube, and when I describe the ingress, I see the following:
Name: jenkins-ingress
Namespace: default
Address: 192.168.49.2
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
*
/?(.*) jenkins-service:8080 (172.17.0.4:8080)
Annotations: kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/default-backend: ingress-nginx-controller
nginx.ingress.kubernetes.io/use-regex: true
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 6m15s (x2 over 6m20s) nginx-ingress-controller Scheduled for sync
Can someone help me what I am doing wrong and how can I access the jenkins instance through the browser?

Ingress configuration issue in Docker kubernetes cluster

I am recently new to Kubernetes and Docker in general and am experiencing issues.
I am running a single local Kubernetes cluster via Docker and am using skaffold to control the build up and teardown of objects within the cluster. When I run skaffold dev the build seems successful, yet when I attempt to make a request to my cluster via Postman the request hangs. I am using an ingress-nginx controller and I feel the bug lies somewhere here. My request handling logic is simple and so I feel the issue is not in the route handling but the configuration of my cluster, specifically with the ingress controller. I will post below my skaffold yaml config and my ingress yaml config.
Any help is greatly appreciated as I have struggled with this bug for sometime.
ingress yaml config :
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
rules:
- host: ticketing.dev
http:
paths:
- path: /api/users/?(.*)
pathType: Prefix
backend:
service:
name: auth-srv
port:
number: 3000
Note that I have a redirect in my /etc/hosts file from ticketing.dev to 127.0.0.1
Auth service yaml config :
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-depl
spec:
replicas: 1
selector:
matchLabels:
app: auth
template:
metadata:
labels:
app: auth
spec:
containers:
- name: auth
image: conorl47/auth
---
kind: Service
metadata:
name: auth-srv
spec:
selector:
app: auth
ports:
- name: auth
protocol: TCP
port: 3000
targetPort: 3000
skaffold yaml config :
apiVersion: skaffold/v2alpha3
kind: Config
deploy:
kubectl:
manifests:
- ./infra/k8s/*
build:
local:
push: false
artifacts:
- image: conorl47/auth
context: auth
docker:
dockerfile: Dockerfile
sync:
manual:
- src: 'src/**/*.ts'
dest: .
For installing the ingress nginx controller I followed the installation instructions at https://kubernetes.github.io/ingress-nginx/deploy/ , namely the Docker desktop installation instruction.
After running that command I see the following two Docker containers running in Docker desktop
The two services created in the ingress-nginx namespace are :
❯ k get services -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.103.6.146 <pending> 80:30036/TCP,443:30465/TCP 22m
ingress-nginx-controller-admission ClusterIP 10.108.8.26 <none> 443/TCP 22m
When I kubectl describe both of these services I see the following :
❯ kubectl describe service ingress-nginx-controller -n ingress-nginx
Name: ingress-nginx-controller
Namespace: ingress-nginx
Labels: app.kubernetes.io/component=controller
app.kubernetes.io/instance=ingress-nginx
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/version=1.0.0
helm.sh/chart=ingress-nginx-4.0.1
Annotations: <none>
Selector: app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
Type: LoadBalancer
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.103.6.146
IPs: 10.103.6.146
Port: http 80/TCP
TargetPort: http/TCP
NodePort: http 30036/TCP
Endpoints: 10.1.0.10:80
Port: https 443/TCP
TargetPort: https/TCP
NodePort: https 30465/TCP
Endpoints: 10.1.0.10:443
Session Affinity: None
External Traffic Policy: Local
HealthCheck NodePort: 32485
Events: <none>
and :
❯ kubectl describe service ingress-nginx-controller-admission -n ingress-nginx
Name: ingress-nginx-controller-admission
Namespace: ingress-nginx
Labels: app.kubernetes.io/component=controller
app.kubernetes.io/instance=ingress-nginx
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=ingress-nginx
app.kubernetes.io/version=1.0.0
helm.sh/chart=ingress-nginx-4.0.1
Annotations: <none>
Selector: app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.108.8.26
IPs: 10.108.8.26
Port: https-webhook 443/TCP
TargetPort: webhook/TCP
Endpoints: 10.1.0.10:8443
Session Affinity: None
Events: <none>
As it seems, you have made the ingress service of type LoadBalancer, this will usually provision an external loadbalancer from your cloud provider of choice. That's also why It's still pending. Its waiting for the loadbalancer to be ready, but it will never happen.
If you want to have that ingress service reachable outside your cluster, you need to use type NodePort.
Since their docs are not great on this point, and it seems to be by default like this. You could download the content of https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.0/deploy/static/provider/cloud/deploy.yaml and modify it before applying. Or you use helm, then you probably can configure this.
You could also do it in this dirty fashion.
kubectl apply --dry-run=client -o yaml -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.0/deploy/static/provider/cloud/deploy.yaml \
| sed s/LoadBalancer/NodePort/g \
| kubectl apply -f -
You could also edit in place.
kubectl edit svc ingress-nginx-controller-admission -n ingress-nginx

Enable Ingress controller on Docker Desktop with WLS2

Currently, I'm using Docker Desktop with WSL2 integration. I found that Docker Desktop automatically had created a cluster for me. It means I don't have to install and use Minikube or Kind to create cluster.
The problem is that, how could I enable Ingress Controller if I use "built-in" cluster from Docker Desktop?
I tried to create an Ingress to check if this work or not, but as my guess, it didn't work.
The YAML file I created as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp
spec:
minReadySeconds: 30
selector:
matchLabels:
app: webapp
replicas: 1
template:
metadata:
labels:
app: webapp
spec:
containers:
- name: webapp
image: nodejs-helloworld:v1
---
apiVersion: v1
kind: Service
metadata:
name: webapp-service
spec:
selector:
app: webapp
ports:
- name: http
port: 3000
nodePort: 30090 # only for NotPort > 30,000
type: NodePort #ClusterIP inside cluster
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: webapp-ingress
spec:
defaultBackend:
service:
name: webapp-service
port:
number: 3000
rules:
- host: ingress.local
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: webapp-service
port:
number: 3000
I tried to access ingress.local/ but it was not successful. (I added ingress.local to point to 127.0.0.1 in host file. And the webapp worked fine at kubernetes.docker.internal:30090 )
Could you please help me to know the root cause?
Thank you.
Finally I found the way to fix. I have to deploy ingress Nginx by command:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.41.2/deploy/static/provider/cloud/deploy.yaml
(Follows the instruction at https://kubernetes.github.io/ingress-nginx/deploy/#docker-for-mac. It works just fine for Docker for Windows)
Now I can access http://ingress.local successfully.
You have to install an ingress-nginx controller on your cluster, so that your nodes will have an opened port 80/443.
Using helm (v3 - see documentation):
helm install --namespace kube-system nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx
Using kubectl (see documentation):
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.43.0/deploy/static/provider/cloud/deploy.yaml
Then manually adding your ingresses' hostnames to /etc/hosts:
127.0.0.1 ingress.local
127.0.0.1 my.other.service.local
# ...
Then if you make a request on http://ingress.local:
the DNS resolution will route to your cluster node
then the ingress controller will serve the request on port 80
then ingress will route the request to the configured backend service
and the service will route to an available pod
The newest version of Docker Desktop for Windows already adds a hosts file entry: 127.0.0.1 kubernetes.docker.internal.
You had to do use kubernetes.docker.internal URL as a hostname in Ingress definition if you want to point to 127.0.0.1. This should be in the docs on this page kubernetes.github.io/ingress-nginx/deploy but there is no Docker Desktop for Windows section there.
Your files should look like this:
apiVersion: v1
kind: Service
metadata:
name: webapp-service
spec:
type: NodePort
selector:
app: webapp
ports:
- name: http
protocol: TCP
port: 3000
nodePort: 30090
Your Ingress file should look like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: webapp-ingress
spec:
rules:
- host: kubernetes.docker.internal
http:
paths:
- path: /
backend:
serviceName: webapp-service
servicePort: http
Then you are able to connect to app using http://kubernetes.docker.internal/.
Example you can see here: wsl2-docker-for-desktop.
I used the Docker-Desktop version to install the nginx-ingress controller
https://kubernetes.github.io/ingress-nginx/deploy/#docker-desktop
curl http://kubernetes.docker.internal/
Offcourse I've not installed any workload yet but the default ingress controller works just fine.
With Kustomize you can simply use
helmCharts:
- name: ingress-nginx
releaseName: ingress-nginx
repo: https://kubernetes.github.io/ingress-nginx
This is just to point out that Amel Mahmuzićs comment is still valid with a recent (I used the ingress-nginx Helm Chart 4.4.2) ingress deployment.
I could not get this to work for far too long (I tried to follow the Strapi fodadvisor example with Docker Desktop build in Kubernetes instead of minikube) and always received a 404 from the ingress.
However, after using this yaml with the added annotation
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: main-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: foodadvisor.backend
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: foodadvisor-backend
port:
number: 1337
- host: foodadvisor.client
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: foodadvisor-frontend
port:
number: 3000
it worked immediately. The K82 docs mention, that this annotation is deprecated.

Why I can't access my web service from the kubernetes cluster?

I'm trying to execute an application inside a kubernetes cluster.
I used to launch the application with docker-compose without problems, but when I create
my kubernetes deployment files, I am not able to access the service inside the cluster even after exposing them. here is my deployment file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 3
selector:
matchLabels:
app: myapp
# type: LoadBalancer
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: jksun12/vdsaipro
# command: ["/run.sh"]
ports:
- containerPort: 80
- containerPort: 3306
# volumeMounts:
# - name: myapp-pv-claim
# mountPath: /var/lib/mysql
# volumes:
# - name: myapp-pv-claim
# persistentVolumeClaim:
# claimName: myapp-pv-claim
---
apiVersion: apps/v1
kind: PersistentVolumeClaim
metadata:
name: myapp-pv-claim
labels:
app: myapp
spec:
accesModes:
- ReadWriteMany
resources:
requests:
storage: 4Gi
Here is the result of
kubectl describe service myapp-service
:
Name: myapp-service
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=myapp
Type: NodePort
IP: 10.109.12.113
Port: port-1 80/TCP
TargetPort: 80/TCP
NodePort: port-1 31892/TCP
Endpoints: 172.18.0.5:80,172.18.0.8:80,172.18.0.9:80
Port: port-2 3306/TCP
TargetPort: 3306/TCP
NodePort: port-2 32393/TCP
Endpoints: 172.18.0.5:3306,172.18.0.8:3306,172.18.0.9:3306
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
And here are the errors that I get when I try to access them:
curl 172.17.0.2:32393
curl: (1) Received HTTP/0.9 when not allowed
And here is the next result when I try to access the other port
curl 172.17.0.2:31892
curl: (7) Failed to connect to 172.17.0.2 port 31892: Connection refused
curl: (7) Failed to connect to 172.17.0.2 port 31892: Connection refused
I'm running ubuntu server 20.04.1 LTS. The manip is on top of minikube.
Thanks for your help.
If you are accessing the service from inside the cluster use ClusterIP as the IP. So curl should be 10.109.12.113:80 and 10.109.12.113:3306
In case accessing it from outside the cluster then use NODEIP and NODEPORT. So curl should be on <NODEIP>:32393 and <NODEIP>:31892
From inside the cluster I would also use POD IPs directly to understand if the issue is at service level or pod level.
You need to make sure that the application is listening on port 80 and port 3306. Only mentioning containerPort as 80 and 3306 does not make the application listen on those ports.
Also make sure that the application code inside the pod is listening on 0.0.0.0 instead of 127.0.0.1

Resources