Can't access minikube service using NodePort from host on Mac - docker

Question

I'm trying to deploy a single web application to Minikube on my Mac, and then access it in the browser. I'm trying to use the simplest of setups, but it's not working, I just get a "connection refused" error and I can't figure out why.
This is what I'm trying:
$ minikube start --insecure-registry=docker.example.com:5000
😄 minikube v1.12.3 on Darwin 10.14.6
✨ Using the docker driver based on existing profile
👍 Starting control plane node minikube in cluster minikube
🔄 Restarting existing docker container for "minikube" ...
🐳 Preparing Kubernetes v1.18.3 on Docker 19.03.8 ...
🔎 Verifying Kubernetes components...
🌟 Enabled addons: default-storageclass, storage-provisioner
🏄 Done! kubectl is now configured to use "minikube"
$ eval $(minikube -p minikube docker-env)
$ docker build -t web-test .
Sending build context to Docker daemon 16.66MB
Step 1/3 : FROM docker.example.com/library/openjdk:11-jdk-slim
11-jdk-slim: Pulling from library/openjdk
bf5952930446: Pull complete
092c9b8e633f: Pull complete
0b793152b850: Pull complete
7900923f09cb: Pull complete
Digest: sha256:b5d8f95b23481a9d9d7e73c108368de74abb9833c3fae80e6bdfa750663d1b97
Status: Downloaded newer image for docker.example.com/library/openjdk:11-jdk-slim
---> de8b1b4806af
Step 2/3 : COPY target/web-test-0.0.1-SNAPSHOT.jar app.jar
---> 6838e3db240a
Step 3/3 : ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","app.jar"]
---> Running in 550bf762bf2d
Removing intermediate container 550bf762bf2d
---> ce1468d1ff10
Successfully built ce1468d1ff10
Successfully tagged web-test:latest
$ kubectl apply -f web-test-service.yaml
service/web-test unchanged
$ kubectl apply -f web-test-deployment.yaml
deployment.apps/web-test configured
$ kubectl get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web-test-6bb45ffc54-8mxbc 1/1 Running 0 16m 172.18.0.2 minikube <none> <none>
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 16m
web-test NodePort 10.102.19.201 <none> 8080:31317/TCP 16m
$ minikube ip
127.0.0.1
$ curl http://127.0.0.1:31317
curl: (7) Failed to connect to 127.0.0.1 port 31317: Connection refused
$ kubectl logs web-test-6bb45ffc54-8mxbc
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.3.RELEASE)
2020-08-26 14:45:32.692 INFO 1 --- [ main] com.example.web.WebTestApplication : Starting WebTestApplication v0.0.1-SNAPSHOT on web-test-6bb45ffc54-8mxbc with PID 1 (/app.jar started by root in /)
2020-08-26 14:45:32.695 INFO 1 --- [ main] com.example.web.WebTestApplication : No active profile set, falling back to default profiles: default
2020-08-26 14:45:34.041 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2020-08-26 14:45:34.053 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2020-08-26 14:45:34.053 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.37]
2020-08-26 14:45:34.135 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2020-08-26 14:45:34.135 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1355 ms
2020-08-26 14:45:34.587 INFO 1 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
2020-08-26 14:45:34.797 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2020-08-26 14:45:34.810 INFO 1 --- [ main] com.example.web.WebTestApplication : Started WebTestApplication in 2.808 seconds (JVM running for 3.426)
$ minikube ssh
docker#minikube:~$ curl 10.102.19.201:8080
Up and Running
docker#minikube:~$
As you can see, the web app is up and running, and I can access it from inside the cluster by doing a minikube ssh, but from outside the cluster, it won't connect. These are my service and deployment manifests:
web-test-service.yaml:
apiVersion: v1
kind: Service
metadata:
labels:
app: web-test
name: web-test
spec:
type: NodePort
ports:
- nodePort: 31317
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: web-test
web-test-deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: web-test
name: web-test
spec:
replicas: 1
selector:
matchLabels:
app: web-test
strategy: {}
template:
metadata:
labels:
app: web-test
spec:
containers:
- image: web-test
imagePullPolicy: Never
name: web-test
ports:
- containerPort: 8080
restartPolicy: Always
status: {}
Anyone have any idea what I'm doing wrong? Or perhaps how I could try to diagnose the issue further? I have allow tried deploying an ingress, but that doesn't work either.

Answer 1

You are mostly facing this issue when you use minikube ip which returns 127.0.0.1. It should work if you use internal ip from kubectl get node -o wide instead of 127.0.0.1.
A much easier approach from the official reference docs is you can get the url using minikube service web-test --url and use it in browser or if you use minikube service web-test it will open the url in browser directly.
Your deployment yamls and everything else looks good and hopefully should not have any issue when deploying to a remote cluster.

Answer 2

It seems that is related to the default docker driver used when you start the minikube. To avoid these problems you can force a specific driver (e.g. "virtualbox"). To do so, follow the next steps:
Remove old minikube with:
minikube delete
Start minikube with virtualbox driver:
minikube start --memory=4096 --driver=virtualbox
Run minikube ip. You'll see an output like 192.168.99.100.
Then, create again the Pods and the service and it should work properly.
I've found this info in this issue: https://github.com/kubernetes/minikube/issues/7344#issuecomment-703225254

Answer 3

You can export an Service from minikube with minikube service web-test
https://kubernetes.io/docs/tutorials/hello-minikube/#create-a-service
Edit:
If you have a deployment, you can export that deployment with the following kubectl command.
minikube kubectl -- expose deployment your-deployment --port 80 --type=LoadBalancer

Answer 4

Just in case you have not already stumbled across a broader concept for accessing a nodeport service that applies in general vs proprietary minikube constructs:
$ k get service -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP
default nginx LoadBalancer 10.43.228.207 172.22.0.240 80:30467/TCP 11h
$ kubectl port-forward --address 0.0.0.0 service/nginx 8082:80
Then from a different host on my network, I do: curl [the host running minikube]:8082
Forwarding from 0.0.0.0:8082 -> 80
Handling connection for 8082
Then you can connect from a different host as well.

Answer 5

docker-desktop UI for Mac and Windows provides an easier alternative compared to minikube, which you could simply activate the Kubernetes feature on your docker-desktop UI:
once it is setup you can right click on the docker desktop icon > Kubernetes
To verify now that your deployement/service works properly:
kubectl apply -f /file.yaml

Answer 6

One checkpoint we should keep in mind for ports.
targetPort: 80
Belongs to the port which we have exposed in our(Dockerfile or Docker-compose file). If the port is
mismatched you won’t be able to access it.

Answer 7

The answer is never use Minikube. It does not allow you to use Nodeport connections. You will always get ECONNREFUSED with minikube no matter what. Just use the docker desktop context, kill minikube, and then re-apply your services. Minikube is only there to further confuse people who are learning Kubernetes.