I've set up a Kubernetes ingress with minikube, on a virtual machine of CentOS 7.6.
It finally works well in that machine, described as below:
Name: my-ingress
Namespace: default
Address: 172.17.0.2
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
localhost
/route1/?(.*) service1 (172.18.0.4:80)
/route2/?(.*) service2 (172.18.0.4:80)
Annotations: kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
And I made my /etc/hosts as follow
172.17.0.2 localhost
172.17.0.2 0.0.0.0
Which works fine on the virtual machine, that I can successfully access my API through curl localhost/route1/api/values.
But here I would like to access this on other machine for developing. My thought was to see the same successful result through curl 192.168.2.21/route1/api/values on other machine, with 192.168.2.21 the IP address of the virtual machine with Kubernetes. But it failed with message "empty reply from server".
Is there other method that I can make this happen, accessing the result of ingress on other machine?
What I tried was to install local-dev-with-docker-for-mac-kubernetes, but didn't help.
And also saw some other suggestions to work around services, but for I would have to work with a lot of services, afraid that may be hard to manage if I have to avoid any port duplicated. So am looking for result workaround Ingress mainly.
Your config specifying host as localhost, so only incoming traffic with localhost got handled. You can verify this with curl 172.17.0.2/route1/api/values from the same machine. Should get the same empty reply message.
To fix this, you can omit the host setting so ingress controller will handle all incoming HTTP traffics. https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-rules
UPDATE
minimal ingress example:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: test-ingress
spec:
backend:
serviceName: testsvc
servicePort: 80
Related
I'm learning and trying to setup minikube in an ubuntu server to manage and deploy applications.
I'm using nginx proxy manager application to manage the proxy's in the server.
I've followed this tutorial to setup ingress with NGINX Ingress Controller, and everyhing works fine, when I run curl *minikube_ip*:service_port I get
Hello, world!
Version: 1.0.0
Hostname: web-746c8679d4-zhtll
Now, the problem is, I'm trying to expose this to the outside world by adding a proxy host in nginx proxy manager that proxies domain_name.com to the *minikube_ip*:service_port but it just keeps giving me 504 Gateway Time-out.
here's the ingress yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-world.info
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web
port:
number: 8080
- path: /v2
pathType: Prefix
backend:
service:
name: web2
port:
number: 8080
When I run kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 2d12h
web NodePort 10.104.186.135 <none> 8080:31806/TCP 2d12h
In my hosts file
*minikube_ip* hello-world.info
I suspect it might be related to the minikube docker container not being in the same network as the nginx proxy manager container, but I really don't know how to solve this, help pls
A NodePort type service can be accessed via the IP Address of the node(s) running in your cluster. Kubernetes will route the request to the proper pod based on the IP and port provided.
If you want to use a host name defined in an Ingress resource instead of the IP Address and port, change your service to type=ClusterIP.
Try running the following command to change your service to type ClusterIP:
kubectl patch svc web -p '{"spec": {"type": "ClusterIP"}}'
Wait for an EXTERNAL-IP to be assigned to the service, you can watch using kubectl get svc --watch
Update your hosts file with the EXTERNAL-IP value instead of the minikube_ip
Finally, try visiting hello-world.info in the browser
In this application, nodejs pods are running inside kubernetes, and mongodb itself sitting outside at host as localhost.
This indeed not good design, but its only for dev environment. In production a separte mongodb server will be there, as such option to have a non loopback ip in endpoint, so will not be a problem in Production.
Have considered following options for dev environment
Use localhost connect string to connect to mongodb, but it will refer to pod's own localhost not host's localhost
Use headless service and provide localhost ip and port in endpoint. However endpoint doesn't allow loopback
Suggest if there is a way to access mongodb database at host's localhost from inside cluster (pod / nodejs application).
I'm running on docker for windows, and for me just using host.docker.internal instead of localhost seems to work fine.
For example, my mongodb connection string looks like this:
mongodb://host.docker.internal:27017/mydb
As an aside, my hosts file includes the following lines (which I didn't add, I guess the docker desktop installation did that):
# Added by Docker Desktop
192.168.1.164 host.docker.internal
192.168.1.164 gateway.docker.internal
127.0.0.1 is a localhost(lo0) interface IP address. Hosts, nodes and pods have their own localhost interfaces and they are not connected to each other.
Your mongodb is running on the Host machine and cannot be accessible using the localhost (or it's IP range) from inside a cluster pod or from inside vm.
In your case, create a headless service and Endpoint for it inside the cluster:
Your mongodb-service.yaml file should look like this:
apiVersion: v1
kind: Service
metadata:
name: mongodb-service
spec:
clusterIP: None
ports:
- protocol: TCP
port: <multipass-port-you-are-using>
targetPort: <multipass-port-you-are-using>
selector:
name: example
type: ClusterIP
---
apiVersion: v1
kind: Endpoints
metadata:
name: mongodb-service
subsets:
- addresses:
- ip: 10.62.176.1
ports:
- port: <multipass-port-you-are-using>
I have add IP you've mentioned in comment section.
After creating service and endpoint you can use mongodb-service name and port <multipass-port-you-are-using> inside any pod of this cluster as a destination point.
Take a look: mysql-localhost, mongodb-localhost.
If you are using minikube to deploy a local kubernetes, you can reach your local environment using the variable host.minikube.internal.
I can add one more solution with Ingress and external-service, which may help some of you.
I deploy my complete system locally with a special Kustomize overlay.
When I want to replace one of the deployments with a service running locally in my IDE, I do the following:
I add an ExternalName service which forwards to host.docker.internal:
kind: Service
apiVersion: v1
metadata:
name: backend-ide
spec:
type: ExternalName
externalName: host.docker.internal
and reconfigured my ingress to forward certain request from my web-app to this external-service:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: backend-ingress
spec:
ingressClassName: nginx
rules:
- host: url.used.by.webapp.com
http:
paths:
- path: /customerportal/api(/|$)(.*)
pathType: Prefix
backend:
service:
name: backend-ide
port:
number: 8080
The same way, I can access all other ports on my host.
I can't connect to my app running with nginx ingress (Docker Desktop win 10).
The nginx-ingress controller pod is running, the app is healthy, and I have created an ingress. However, when I try to connect to my app on localhost, I get "connection refused".
I see this error in the log:
[14:13:13.028][VpnKit ][Info ] vpnkit.exe: Connected Ethernet interface f6:16:36:bc:f9:c6
[14:13:13.028][VpnKit ][Info ] vpnkit.exe: UDP interface connected on 10.96.181.150
[14:13:22.320][GoBackendProcess ][Info ] Adding vpnkit-k8s-controller tcp forward from 0.0.0.0:80 to 10.96.47.183:80
[14:13:22.323][ApiProxy ][Error ] time="2019-12-09T14:13:22-05:00" msg="Port 443 for service ingress-nginx is already opened by another service"
I think port 443 is used by another app, possibly zscaler security or skype.
Excerpt from netstat -a -b:
[svchost.exe]
TCP 0.0.0.0:443 0.0.0.0:0 LISTENING 16012
[com.docker.backend.exe]
TCP 0.0.0.0:443 0.0.0.0:0 LISTENING 8220
I don't know how to make the ingress work. Please help!
My ingress:
$ kubectl describe ing kbvalues-deployment-dev-ingress
Name: kbvalues-deployment-dev-ingress
Namespace: default
Address: localhost
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
localhost
/ kbvalues-deployment-dev-frontend:28000 (10.1.0.174:8080)
Annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/cors-allow-headers: X-Forwarded-For, X-app123-XPTO
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 42m nginx-ingress-controller Ingress default/kbvalues-deployment-dev-ingress
Normal UPDATE 6s (x5 over 42m) nginx-ingress-controller Ingress default/kbvalues-deployment-dev-ingress
My service:
$ kubectl describe svc kbvalues-deployment-dev-frontend
Name: kbvalues-deployment-dev-frontend
Namespace: default
Labels: chart=tomcat-sidecar-war-1.0.4
environment=dev
name=kbvalues-frontend-dev
release=kbvalues-test
tier=frontend
Annotations: <none>
Selector: app=kbvalues-dev
Type: ClusterIP
IP: 10.98.89.94
Port: <unset> 28000/TCP
TargetPort: 8080/TCP
Endpoints: 10.1.0.174:8080
Session Affinity: None
Events: <none>
I am trying to access the app at: http://localhost:28000/health. I verified that the /health URL is accessible locally within the web server container.
I appreciate any help you can offer.
Edit:
I tried altering the ingress-nginx service to remove HTTPS, as suggested here:
https://stackoverflow.com/a/56303330/166850
This got rid of the 443 error in the logs, but didn't fix my setup (still getting connection refused).
Edit 2: Here is the Ingress YAML definition (kubectl get -o yaml):
$ kubectl get ing -o yaml
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
creationTimestamp: "2019-12-09T18:47:33Z"
generation: 5
name: kbvalues-deployment-dev-ingress
namespace: default
resourceVersion: "20414"
selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/kbvalues-deployment-dev-ingress
uid: 5c34bf7f-1ab4-11ea-80e4-00155d169409
spec:
rules:
- host: localhost
http:
paths:
- backend:
serviceName: kbvalues-deployment-dev-frontend
servicePort: 28000
path: /
status:
loadBalancer:
ingress:
- hostname: localhost
kind: List
metadata:
resourceVersion: ""
selfLink: ""
Edit 3: Output of kubectl get svc -A (ingress line only):
ingress-nginx ingress-nginx LoadBalancer 10.96.47.183 localhost 80:30470/TCP 21h
Edit 4: I tried to get the VM's IP address from windows HyperV, but it seems like the VM doesn't have an IP?
PS C:\> (Get-VMNetworkAdapter -VMName DockerDesktopVM)
Name IsManagementOs VMName SwitchName MacAddress Status IPAddresses
---- -------------- ------ ---------- ---------- ------ -----------
Network Adapter False DockerDesktopVM DockerNAT 00155D169409 {Ok} {}
Edit 5:
Output of netstat -a -n -o -b for port 80:
TCP 0.0.0.0:80 0.0.0.0:0 LISTENING 4
Can not obtain ownership information
I have managed to create Ingress resource in Kubernetes on Docker in Windows.
Steps to reproduce:
Enable Hyper-V
Install Docker for Windows and enable Kubernetes
Connect kubectl
Enable Ingress
Create deployment
Create service
Create ingress resource
Add host into local hosts file
Test
Enable Hyper-V
From Powershell with administrator access run below command:
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All
System could ask you to reboot your machine.
Install Docker for Windows and enable Kubernetes
Install Docker application with all the default options and enable Kubernetes
Connect kubectl
Install kubectl .
Enable Ingress
Run this commands:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/cloud-generic.yaml
Edit: Make sure no other service is using port 80
Restart your machine. From a cmd prompt running as admin, do:
net stop http
Stop the listed services using services.msc
Use: netstat -a -n -o -b and check for other processes listening on port 80.
Create deployment
Below is simple deployment with pods that will reply to requests:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello
spec:
selector:
matchLabels:
app: hello
version: 2.0.0
replicas: 3
template:
metadata:
labels:
app: hello
version: 2.0.0
spec:
containers:
- name: hello
image: "gcr.io/google-samples/hello-app:2.0"
env:
- name: "PORT"
value: "50001"
Apply it by running command:
$ kubectl apply -f file_name.yaml
Create service
For pods to be able for you to communicate with them you need to create a service.
Example below:
apiVersion: v1
kind: Service
metadata:
name: hello-service
spec:
type: NodePort
selector:
app: hello
version: 2.0.0
ports:
- name: http
protocol: TCP
port: 80
targetPort: 50001
Apply this service definition by running command:
$ kubectl apply -f file_name.yaml
Create Ingress resource
Below is simple Ingress resource using service created above:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-ingress
spec:
rules:
- host: hello-test.internal
http:
paths:
- path: /
backend:
serviceName: hello-service
servicePort: http
Take a look at:
spec:
rules:
- host: hello-test.internal
hello-test.internal will be used as the hostname to connect to your pods.
Apply your Ingress resource by invoking command:
$ kubectl apply -f file_name.yaml
Add host into local hosts file
I found this Github link that will allow you to connect to your Ingress resource by hostname.
To achieve that add a line 127.0.0.1 hello-test.internal to your C:\Windows\System32\drivers\etc\hosts file and save it.
You will need Administrator privileges to do that.
Edit: The newest version of Docker Desktop for Windows already adds a hosts file entry:
127.0.0.1 kubernetes.docker.internal
Test
Display the information about Ingress resources by invoking command:
kubectl get ingress
It should show:
NAME HOSTS ADDRESS PORTS AGE
hello-ingress hello-test.internal localhost 80 6m2s
Now you can access your Ingress resource by opening your web browser and typing
http://kubernetes.docker.internal/
The browser should output:
Hello, world!
Version: 2.0.0
Hostname: hello-84d554cbdf-2lr76
Hostname: hello-84d554cbdf-2lr76 is the name of the pod that replied.
If this solution is not working please check connections with the command:
netstat -a -n -o
(with Administrator privileges) if something is not using port 80.
On Windows the Kubernetes cluster is running in a VM. Try to access ingress on that VM-s IP address instead of localhost.
i was facing similar issue while deploying ingress-nginx controller using the manual steps mentioned for bareMetal node at ingress-nginx-deploy however was facing an issue , however referred to the link Github link mentioned by #RMorrisey which leads to other threads where they have mentioned to install ingress-nginx using the steps mentioned for mac and it worked without making cny changes to host file , etc
The problem is that your service has a type of ClusterIP, which isn't accessible externally. You need it to be of type NodePort, which is what is done in Dawid Kruk's instructions.
We access the container through cluster IP and even we deploy web application containers can be accessed.The issue with how can we access container from outside the host.
Tried with giving external IP to containers.
You can create a service and bind it to a node port, from outside your cluster if you try to access that service using node_ip:port.
apiVersion: v1
kind: Service
metadata:
name: api-server
spec:
ports:
- port: 80
name: http
targetPort: api-http
nodePort: 30004
- port: 443
name: https
targetPort: api-http
type: LoadBalancer
selector:
run: api-server
if you do kubectl get service you can get the external ip.
The best approach would be to expose your pods with ClusterIP type services, and then use an Ingress resource along with Ingress Controller to expose HTTP and/or HTTPS routes so you can access your app outside of the cluster.
For testing purposes it's ok to use NodePort or LoadBalancer type services. Whether you are running on your own infrastructure or using a managed solution, you can use NodePort, while using LoadBalancer requires cloud provider's load balancer.
Source: Official docs
I'm trying to expose a Kubernetes pod on a single node bare metal cluster without a domain.
In my understanding I've the these options:
Expose using NodePort
Expose using an Ingress controller
Expose using ClusterIP and manually set an external IP
As I mentioned already, I only have a single node cluster. This means that the master is master and node at the same time directlly running on a fedora host system.
The simplest solution is to use a NodePort. But the limitation here is (if I'm right), that the service port will be automatically selected from a given port range.
The next better solution is to use an ingress controller. But for this I need a public domain which I haven't. So the ingress controller also doesn't fit to me.
What for other options do I have? I just want to expose my service directly on port 9090.
Why not Option 3 ？ you can setup externalIPs to your node ip.
apiVersion: v1
kind: Service
...
spec:
externalIPs:
- your node ip
Also with NodePort, the service port can be specified.
You can set a custom port range for NodePort by adding this option to your apiserver settings (/etc/kubernetes/manifests/kube-apiserver.yaml):
--service-node-port-range portRange
Default: 30000-32767
A port range to reserve for services with NodePort visibility. Example:
'30000-32767'.
Inclusive at both ends of the range.
This is the part from Kubernetes documentation related to Services:
If you want a specific port number, you can specify a value in the
nodePort field, and the system will allocate you that port or else the
API transaction will fail (i.e. you need to take care about possible
port collisions yourself). The value you specify must be in the
configured range for node ports.
Example for this answer was taken from the article Hosting Your Own Kubernetes NodePort Load Balancer:
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
name: nginx
spec:
type: NodePort
ports:
- port: 80
nodePort: 30080
name: http
- port: 443
nodePort: 30443
name: https
selector:
name: nginx