azure-aks docker container python-django url not working - docker

I have created Docker that has debian + python-django that runs on 8000 port. But after deploying into azure-aks, url path is not working under 8000 port. Keeping important detials below.
Step 1:
Dockerfile :
EXPOSE 8000
RUN /usr/local/bin/python3 manage.py migrate
CMD [ "python3", "manage.py", "runserver", "0.0.0.0:8000" ]
Step 2:
After building docker image, pushing it to azure registry.
Step 3:
myfile.yaml : this is to deploy azure registry file into aks cluster.
apiVersion: apps/v1
kind: Deployment
metadata:
name: myops
spec:
replicas: 1
selector:
matchLabels:
app: myops
template:
metadata:
labels:
app: myops
spec:
containers:
- name: myops
image: quantumregistry.azurecr.io/myops:v1.0
ports:
- containerPort: 8000
---
# [START service]
apiVersion: v1
kind: Service
metadata:
name: myops-python
spec:
type: LoadBalancer
ports:
- port: 8000
targetPort: 8888
selector:
app: myops
# [END service]
Deploy into aks : kubectl apply -f myops.yaml
Step 4: check sevice
kubectl get service myops-python --watch
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
myops-python LoadBalancer <cluster-ip> <external-ip> 8000:30778/TCP 37m
Note: i have masked IP to not to expose to public.
step 5: i see container is running alright
kubectl get pods
NAME READY STATUS RESTARTS AGE
myops-5bbd459745-cz2vc 1/1 Running 0 19m
step 6: I see container log and it shows that python is running under host 0.0.0.0:8000 port.
kubectl logs -f myops-5bbd459745-cz2vc
Watching for file changes with StatReloader
Performing system checks...
WARNING:param.main: pandas could not register all extension types imports failed with the following error: cannot import name 'ABCIndexClass' from 'pandas.core.dtypes.generic' (/usr/local/lib/python3.9/site-packages/pandas/core/dtypes/generic.py)
System check identified no issues (0 silenced).
September 19, 2021 - 06:47:57
Django version 3.2.5, using settings 'myops_project.settings'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
The issue is that when I open this in browser http://:8000/myops_app, it is not working and timing out.

The Service myops-python is set up to receive requests on port 8000 but then it will send the request to the pod on target port 8888.
ports:
- port: 8000
targetPort: 8888
The container myops in the Pod myops, however, is not listening on port 8888. Rather it is listening on port 8000.
Dockerfile:
EXPOSE 8000
RUN /usr/local/bin/python3 manage.py migrate CMD [ "python3", "manage.py", "runserver", "0.0.0.0:8000" ]
Please set spec.ports[0].targetPort to 8000 manually or remove targetPort from spec.ports[0] in the Service myops-python. By default and for convenience, the targetPort is set to the same value as the port field. For more information please see Defining a Service.
Tip: You can use kubectl edit service <service-name> -n <namepsace> to edit your Service manifest.

Related

Access a web service in kubernetes pod from local browser using NodePort yields Connection refused

What do I need to do in order to get my local browser to and request a resource to a web service running inside Minikube instance running locally on my machine?
I am getting a Connection refused when trying to kubectl port-forward.
My workflow is:
Creating Dockerfile with web service on
Start minikube in docker
Build docker image
Import image locally into Minikube
Created a deployment with one container and a NodePort service
Applied deployment/service
Ran kubectl port-forward (to hopefully forward requests to my container)
Open browser to 127.0.0.1:31000
Port Configuration Summary
Dockerfile:
Expose: 80
uvicorn: 80
Deployment
NodePort Service:
Port: 80
Target Port: 80
Node Port: 31000
Kubectl Command: 8500:31000
Browser: 127.0.0.1:8500
Setup and run through
dev.dockerfile (Step 1)
FROM python:3.11-buster # Some Debian Python image... I built my own
COPY ../sources/api/ /app/
RUN pip install --no-cache-dir --upgrade -r /app/requirements.txt
ENV PYTHONPATH=/app/
EXPOSE 80
CMD ["uvicorn", "app.main:app", "--proxy-headers", "--host", "0.0.0.0", "--port", "80"]
Build Sequence (Steps 2 to 4)
# 2 - start minikube
minikube start --bootstrapper=kubeadm --vm-driver=docker
minikube docker-env
## 3 - build image
docker build -f ../../service1/deploy/dev.dockerfile ../../service1 -t acme-app.service1:latest
## 4 - load image into minikube
minikube image load acme-app.service1:latest
Deployment (Step 5 and 6)
deployment.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: acme-service-1-deployment
namespace: acme-app-dev
labels:
app: service-1
spec:
replicas: 1
selector:
matchLabels:
app: service-1
template:
metadata:
labels:
app: service-1
spec:
containers:
- name: service1-container
image: docker.io/library/acme-app.service1:latest
imagePullPolicy: Never
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: service-1-service
namespace: acme-app-dev
spec:
type: NodePort
selector:
app: service-1
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 31000
Deploy
kubectl apply -f deployment.yaml
kubectl port forward (Step 7)
Find Pod
kubectl get pods -n acme-app-dev
NAME READY STATUS RESTARTS AGE
acme-service-1-deployment-76748d7ff6-llcsr 1/1 Running 0 11s
Port Forward to pod
port-forward acme-service-1-deployment-76748d7ff6-llcsr 8500:31000 -n acme-app-dev
Forwarding from 127.0.0.1:8500 -> 31000
Forwarding from [::1]:8500 -> 31000
Test in Browser (Step 8)
Open favorite browser and navigate to 127.0.0.1:31000.
The console running the port forward now outputs:
E0123 14:54:16.208010 25932 portforward.go:406] an error occurred forwarding 8500 -> 31000: error forwarding port 31000 to pod d4c0fa6cb16ce02335a05cad904fbf2ab7818e2073d7c7ded8ad05f193aa37e7, uid : exit status 1: 2023/01/23 14:54:16 socat[39370] E connect(5, AF=2 127.0.0.1:31000, 16): Connection refused
E0123 14:54:16.213268 25932 portforward.go:234] lost connection to pod
What have I looked at?
I've tried looking through the docs on kubernetes website as well as issues on here (yes there are similar). This is pretty similar - although no marked answer and still an issue by the looks of it. I couldn't see a solution for my issue here.
NodePort exposed Port connection refused
I am running Minikube on Windows and I'm just setting out on a kubernetes journey.
The image itself works in docker from a docker compose. I can see the pod is up and running in minikube from the logs (minikube dashboard).
You got your wires crossed:
The pod is listening on port 80
The NodePort service is listening on port 31000 on the node, but its underlying ClusterIP service is listening on port 80 as well.
You are trying to port-forward to port 31000 on the Pod. This will not work.
Call one of the following instead:
kubectl port-forward -n acme-app-dev deploy/acme-service-1-deployment 8500:80
or kubectl port-forward -n acme-app-dev service/service-1-service 8500:80
or use minikube service -n acme-app-dev service-1-service and use the provided URL.

Deployment pod cannot connect ClusterIP service

I try to expose my server IP by using Ingress.
The server is an Express.js app. It listens at http://localhost:5000 locally when without docker.
Here are my Kubernetes config files:
server-deployment.yaml
apiVersion: v1
apiVersion: apps/v1
kind: Deployment
metadata:
name: server-deployment
spec:
replicas: 1
selector:
matchLabels:
component: server
template:
metadata:
labels:
component: server
spec:
containers:
- name: server
image: hongbomiao/hongbomiao-server:latest
ports:
- containerPort: 5000
env:
- name: NODE_ENV
value: development
server-cluster-ip-service.yaml
kind: Service
metadata:
name: server-cluster-ip-service
spec:
type: ClusterIP
selector:
component: server
ports:
- port: 5000
targetPort: 5000
ingress-service.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: server-cluster-ip-service
port:
number: 5000
I got my minikube IP by
➜ minikube ip
192.168.64.12
When I open 192.168.64.12 in my browser, I got 502 Bad Gateway.
I got some debug idea after reading https://cloud.google.com/kubernetes-engine/docs/how-to/exposing-apps#kubectl-apply. Here is what I have tried:
➜ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 5h34m
server-cluster-ip-service ClusterIP 10.102.5.161 <none> 5000/TCP 4h39m
➜ kubectl get pods
NAME READY STATUS RESTARTS AGE
server-deployment-bc6777445-pj59f 1/1 Running 0 4h39m
➜ kubectl exec -it server-deployment-bc6777445-pj59f -- sh
/app # apk add --no-cache curl
...
/app # curl 10.102.5.161:5000
curl: (28) Failed to connect to 10.102.5.161 port 5000: Operation timed out
It seems my deployment pod has issue connecting ClusterIP service now. Any help will be nice!
It turns out the issue is caused by my VPN.
I didn't change anything for the Kubernetes config in my question.
Also, letting the Express.js server explicitly listen at 0.0.0.0 is not necessary neither.
(Note #David Maze's comment under the question about 0.0.0.0 is still valuable)
const app = express()
.use(bodyParser.json())
.use(express.static(path.join(__dirname, '../dist')))
app.listen(5000); // This just works. No need explicitly change to app.listen(5000, '0.0.0.0');
At the time of writing, I was in China. To get rid of the VPN while still using Kubenetes / minikube, I found a way and posted it at GitHub here.
After turning off the VPN with this workaround solution, everything just works.
Copy my solution using minikube in China here:
Step 1 - Download the Aliyun version minikube
curl -Lo minikube https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v1.14.2/minikube-darwin-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
Note: You can find if there is a new version to replace v1.14.2 in the command above at https://github.com/AliyunContainerService/minikube/wiki#%E5%AE%89%E8%A3%85minikube
Step 2 - Start the minikube
minikube start --image-mirror-country cn \
--driver=hyperkit \
--iso-url=https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/iso/minikube-v1.15.0.iso \
--registry-mirror=https://xxxxxxxx.mirror.aliyuncs.com
Note 1: You can find latest minikube version at https://github.com/kubernetes/minikube/blob/master/CHANGELOG.md, then replace v1.15.0 in the command above to newer version.
However, Aliyun's minikube version is a little behind. To verify if a new version exists, you can replace the version in the URL of https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/iso/minikube-v1.15.0.iso to different new versions, such as v1.15.1, and then open it in the browser.
Note 2: For the xxxxxxxx in the command above, you can find yours at
https://cr.console.aliyun.com/cn-hangzhou/instances/mirrors
(Need register an Aliyun account first)
Note 3: You can pass more parameters to this Aliyun version minikube start, check at https://github.com/AliyunContainerService/minikube/wiki#%E5%90%AF%E5%8A%A8
In my case, I am using the driver hyperkit on macOS, and Aliyun's iso-url, registry-mirror to speed up.

Unable to connect to minikube service in host

I've gone through a fair few stackoverflow posts, none are working... So here's my issue:
I've got a simple node app running on broadcast 0.0.0.0 at port 5000, it's got a simple single endpoint at /.
I've got two k8s objects, here's my Deployment object:
### pf deployment
apiVersion: apps/v1
kind: Deployment
metadata:
# Unique key of the Deployment instance
name: pf-deployment
spec:
# 3 Pods should exist at all times.
replicas: 1
selector:
matchLabels:
app: public-facing
template:
metadata:
labels:
# Apply this label to pods and default
# the Deployment label selector to this value
app: public-facing
spec:
containers:
- name: public-facing
# Run this image
image: pf:ale8k
ports:
- containerPort: 5000
Next, here is my Service object:
### pf service
apiVersion: v1
kind: Service
metadata:
name: pf-service
labels:
run: pf-service-label
spec:
type: NodePort ### may be ommited as it is a default type
selector:
name: public-facing ### should match your labels defined for your angular pods
ports:
- protocol: TCP
targetPort: 5000 ### port your app listens on
port: 5000 ### port on which you want to expose it within your cluster
Finally, a very simple dockerfile:
### generic docker file
FROM node:12
WORKDIR /usr/src/app
COPY . .
RUN npm i
EXPOSE 5000
CMD ["npm", "run", "start"]
I have my image in the minikubes local docker registry, so that's not the issue...
When I try:
curl $(minikube service pf-service --url)
I get:
curl: (7) Failed to connect to 192.168.99.101 port 31753: Connection refused
When I try:
minikube service pf-service
I get a little further output:
Most likely you need to configure your SUID sandbox correctly
I have the hello-minikube image running, this works perfectly fine. So I presume it isn't my nacl?
I'm very new to kubernetes, so apologies in advance if it's very simple.
Thanks!
Service has got selector name: public-facing but pod has got label app: public-facing. They need to be same for Endpoints of the service to be populated with pod IPs.
If you execute below command
kubectl describe svc pf-service
You will see that Endpoints has got no IPs which is the cause of connection refused error.
Change the selector in service as below to make it work.
### pf service
apiVersion: v1
kind: Service
metadata:
name: pf-service
labels:
run: pf-service-label
spec:
type: NodePort ### may be ommited as it is a default type
selector:
app: public-facing ### should match your labels defined for your angular pods
ports:
- protocol: TCP
targetPort: 5000 ### port your app listens on
port: 5000 ### port on which you want to expose it within your cluster

404 when I curl a kubernetes tomcat service

Hello I am trying to deploy a simple tomcat service. Below are the details:
1.minikube version: v1.8.1
2.OS: mac
3.The deployment.yaml file (I am in the directory of the yaml file)
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deployment
spec:
selector:
matchLabels:
app: tomcat
replicas: 1
template:
metadata:
labels:
app: tomcat
spec:
containers:
- name: tomcat
image: tomcat:9.0
ports:
- containerPort: 8080
4.Commands used to deploy and expose the service
kubectl apply -f deployment.yaml
kubectl expose deployment tomcat-deployment --type=NodePort
minikube service tomcat-deployment --url
curl [URL]
I get a 404 when I curl the URL.
I am unsure if there's an issue with the deployment.yaml file or some minikube settings.
Tomcat image comes with the default preinstalled apps (ROOT, manager...) inside webapps.dist folder, to avoid them to be loaded by default at container startup (https://github.com/docker-library/tomcat/issues/183). You can for example simply rename webapps.dist to webapps and (via e.g. kubectl exec <pod_name> -- bash), and after apps are deployed, the 404 should no longer occur.
You should set target-Port which in your case would be 8080.
All of this is nicely explained on Set up Ingress on Minikube with the NGINX Ingress Controller
Create a Deployment using the following command:
kubectl run web --image=gcr.io/google-samples/hello-app:1.0 --port=8080
Output:
deployment.apps/web created
Expose the Deployment:
kubectl expose deployment web --target-port=8080 --type=NodePort
Output:
service/web exposed
Verify the Service is created and is available on a node port:
kubectl get service web
Output:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
web NodePort 10.104.133.249 <none> 8080:31637/TCP 12m
Visit the service via NodePort:
minikube service web --url
Output:
http://172.17.0.15:31637
If that does not help check the logs of the tomcat pod to see it started on port 8080
Look for a line
09-Mar-2020 13:36:00.157 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
You can also check docker logs by first locating the docker container docker ps and fetching the logs from it with docker logs <container-name>

Remote debug container in kubernetes using intellij

I try to remote debug the application in attached mode with host: 192.168.99.100 and port 5005, but it tells me that it is unable to open the debugger port. The IP is 192.268.99.100 (the cluster is hosted locally via minikube).
Output of kubectl describe service catalogservice
Name: catalogservice
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=catalogservice
Type: NodePort
IP: 10.98.238.198
Port: web 31003/TCP
TargetPort: 8080/TCP
NodePort: web 31003/TCP
Endpoints: 172.17.0.6:8080
Port: debug 5005/TCP
TargetPort: 5005/TCP
NodePort: debug 32003/TCP
Endpoints: 172.17.0.6:5005
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
This is the pods service.yml:
apiVersion: v1
kind: Service
metadata:
name: catalogservice
spec:
type: NodePort
selector:
app: catalogservice
ports:
- name: web
protocol: TCP
port: 31003
nodePort: 31003
targetPort: 8080
- name: debug
protocol: TCP
port: 5005
nodePort: 32003
targetPort: 5005
And in here I expose the containers port
spec:
containers:
- name: catalogservice
image: elps/myimage
ports:
- containerPort: 8080
name: app
- containerPort: 5005
name: debug
The way I build the image:
FROM openjdk:11
VOLUME /tmp
EXPOSE 8082
ADD /target/catalogservice-0.0.1-SNAPSHOT.jar catalogservice-0.0.1-SNAPSHOT.jar
ENTRYPOINT ["java", "-agentlib:jdwp=transport=dt_socket,address=5005,server=y,suspend=n", "-jar", "catalogservice-0.0.1-SNAPSHOT.jar"]
When I execute nmap -p 5005 192.168.99.100 I receive
PORT STATE SERVICE
5005/tcp closed avt-profile-2
When I execute nmap -p 32003 192.168.99.100 I receive
PORT STATE SERVICE
32003/tcp closed unknown
When I execute nmap -p 31003 192.168.99.100 I receive
PORT STATE SERVICE
31003/tcp open unknown
When I execute kubectl get services I receive
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
catalogservice NodePort 10.108.195.102 <none> 31003:31003/TCP,5005:32003/TCP 14m
minikube service customerservice --url returns
http://192.168.99.100:32004
As an alternative to using a NodePort in a Service you could also use kubectl port-forward to access the debug port in your Pod.
kubectl port-forward allows using resource name, such as a pod name, to select a matching pod to port forward to since Kubernetes v1.10.
You need to expose the debug port in the Deployment yaml for the Pod
spec:
containers:
...
ports:
...
- containerPort: 5005
Then get the name of your Pod via
kubectl get pods
and then add a port-forwarding to that Pod
kubectl port-forward podname 5005:5005
In IntelliJ you will be able to connect to
Host: localhost
Port: 5005
Alternatively, you can use the Cloud Code Intellij plugin.
Also, if you use Fabric8, it provides the fabric8:debug goal.
There was a slip in the yaml you first posted as:
- containerPort: 5050
name: debug
Should be:
- containerPort: 5005
name: debug
You also need to use the external port of 32003 when configuring the IntelliJ debugger. With those changes it should work.
You may also want to think about how to make it more flexible. In the past when I've done this I've used a different form for the docker start command that allows you to turn remote debug on and off by an environment variable called REMOTE_DEBUG, which for you would be:
CMD if [ "x$REMOTE_DEBUG" = "xfalse" ] ; then java $JAVA_OPTS -jar catalogservice-0.0.1-SNAPSHOT.jar ; else java $JAVA_OPTS -agentlib:jdwp=transport=dt_socket,address=5005,server=y,suspend=n -jar catalogservice-0.0.1-SNAPSHOT.jar ; fi
You'll probably find you want to set the env var $JAVA_OPTS to limit jvm memory use to avoid issues in k8s.

Resources