I'm trying to run a docker container on Google Container-optimized VM in GCE.
Here is my dockerfile. I built a container image and push it to gcr.io.
FROM nginx:1.9
COPY config /etc/nginx
And I wrote a container manifest file.
version: v1beta2
containers:
- name: test
image: gcr.io/myproject/test
ports:
- name: http
hostPort: 80
containerPort: 80
- name: https
hostPort: 443
containerPort: 443
I deployed to GCE with the manifest file, but port binding was not as I had expected. Why did the host port 80 and 443 redirect to google_containers/pause instead of myproject/test?
local$ gcloud compute instance create test \
--image container-vm \
--metadata-from-file google-container-manifest=container.yaml \
--zone us-central1-b \
--machine-type f1-micro \
--tags http-server,https-server
local$ gcloud compute ssh --zone us-central1-b test
test$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
818828ccc2c6 gcr.io/myproject/test:latest "nginx -g 'daemon of 23 seconds ago Up 22 seconds k8s_test.9de3822_7f9f8ecace94a22b2bea59ee14f3bcd0-test_df40d10c4dfa4
f40d10c4dfa4 gcr.io/google_containers/pause:0.8.0 "/pause" 32 seconds ago Up 31 seconds 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp k8s_POD.c6ce2a78_7f9f8ecace94a22b2bea59ee14f3bcd0-test_default_7f9f8ecace94a22b2bea59ee14f3bcd0-test_64d51838
I had updated version of the manifest v1beta2 to v1 (v1beta3) and tried it again. The result of port binding seems to be same as previous one, but the container can communicate with external network through port 80 and 443.
version:1
kind: Pod
spec:
restartPolicy: Always
dnsPolicy: Default
containers:
- name: test
image: gcr.io/myproject/test
imagePullPolicy: Always
ports:
- name: http
hostPort: 80
containerPort: 80
protocol: TCP
- name: https
hostPort: 443
containerPort: 443
protocol: TCP
Related
What do I need to do in order to get my local browser to and request a resource to a web service running inside Minikube instance running locally on my machine?
I am getting a Connection refused when trying to kubectl port-forward.
My workflow is:
Creating Dockerfile with web service on
Start minikube in docker
Build docker image
Import image locally into Minikube
Created a deployment with one container and a NodePort service
Applied deployment/service
Ran kubectl port-forward (to hopefully forward requests to my container)
Open browser to 127.0.0.1:31000
Port Configuration Summary
Dockerfile:
Expose: 80
uvicorn: 80
Deployment
NodePort Service:
Port: 80
Target Port: 80
Node Port: 31000
Kubectl Command: 8500:31000
Browser: 127.0.0.1:8500
Setup and run through
dev.dockerfile (Step 1)
FROM python:3.11-buster # Some Debian Python image... I built my own
COPY ../sources/api/ /app/
RUN pip install --no-cache-dir --upgrade -r /app/requirements.txt
ENV PYTHONPATH=/app/
EXPOSE 80
CMD ["uvicorn", "app.main:app", "--proxy-headers", "--host", "0.0.0.0", "--port", "80"]
Build Sequence (Steps 2 to 4)
# 2 - start minikube
minikube start --bootstrapper=kubeadm --vm-driver=docker
minikube docker-env
## 3 - build image
docker build -f ../../service1/deploy/dev.dockerfile ../../service1 -t acme-app.service1:latest
## 4 - load image into minikube
minikube image load acme-app.service1:latest
Deployment (Step 5 and 6)
deployment.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: acme-service-1-deployment
namespace: acme-app-dev
labels:
app: service-1
spec:
replicas: 1
selector:
matchLabels:
app: service-1
template:
metadata:
labels:
app: service-1
spec:
containers:
- name: service1-container
image: docker.io/library/acme-app.service1:latest
imagePullPolicy: Never
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: service-1-service
namespace: acme-app-dev
spec:
type: NodePort
selector:
app: service-1
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 31000
Deploy
kubectl apply -f deployment.yaml
kubectl port forward (Step 7)
Find Pod
kubectl get pods -n acme-app-dev
NAME READY STATUS RESTARTS AGE
acme-service-1-deployment-76748d7ff6-llcsr 1/1 Running 0 11s
Port Forward to pod
port-forward acme-service-1-deployment-76748d7ff6-llcsr 8500:31000 -n acme-app-dev
Forwarding from 127.0.0.1:8500 -> 31000
Forwarding from [::1]:8500 -> 31000
Test in Browser (Step 8)
Open favorite browser and navigate to 127.0.0.1:31000.
The console running the port forward now outputs:
E0123 14:54:16.208010 25932 portforward.go:406] an error occurred forwarding 8500 -> 31000: error forwarding port 31000 to pod d4c0fa6cb16ce02335a05cad904fbf2ab7818e2073d7c7ded8ad05f193aa37e7, uid : exit status 1: 2023/01/23 14:54:16 socat[39370] E connect(5, AF=2 127.0.0.1:31000, 16): Connection refused
E0123 14:54:16.213268 25932 portforward.go:234] lost connection to pod
What have I looked at?
I've tried looking through the docs on kubernetes website as well as issues on here (yes there are similar). This is pretty similar - although no marked answer and still an issue by the looks of it. I couldn't see a solution for my issue here.
NodePort exposed Port connection refused
I am running Minikube on Windows and I'm just setting out on a kubernetes journey.
The image itself works in docker from a docker compose. I can see the pod is up and running in minikube from the logs (minikube dashboard).
You got your wires crossed:
The pod is listening on port 80
The NodePort service is listening on port 31000 on the node, but its underlying ClusterIP service is listening on port 80 as well.
You are trying to port-forward to port 31000 on the Pod. This will not work.
Call one of the following instead:
kubectl port-forward -n acme-app-dev deploy/acme-service-1-deployment 8500:80
or kubectl port-forward -n acme-app-dev service/service-1-service 8500:80
or use minikube service -n acme-app-dev service-1-service and use the provided URL.
I am using WSL2 on Windows.
I made a flask service in minikube in WSL2 and a docker container in WSL2 separately.
I want to make a request to flask service in minikube from container in WSL2.
Steps to create a flask service
flask_service.py (only last line, service is running on /rss)
if __name__ == '__main__':
app.run(debug=False, host='0.0.0.0', port=8001)
Dockerfile
FROM python:3
COPY flask_service.py ./
WORKDIR .
RUN apt-get update
RUN apt install nano
RUN pip install numpy pandas Flask connectorx sqlalchemy pymysql jsonpickle
EXPOSE 8001
ENTRYPOINT ["python"]
CMD ["flask_service.py"]
minikube setting
minikube start --mount --mount-string="/home/sjw/kube:/home/sjw/kube"
kubectl proxy --address 0.0.0.0 --port 30001
minikube tunnel
getdb service menifest
apiVersion: apps/v1
kind: Deployment
metadata:
name: getdbdp
spec:
replicas: 1
selector:
matchLabels:
app: getdb
template:
metadata:
labels:
app: getdb
spec:
containers:
- name: getdb
image: "desg2022/01getdb:v02"
env:
- name: "PORT"
value: "8001"
---
apiVersion: v1
kind: Service
metadata:
name: getdb-lb
spec:
type: LoadBalancer
selector:
app: getdb
ports:
- protocol: TCP
port: 8080
targetPort: 8001
First, local access(from windows) to the flask service was possible with the address below.
http://localhost:30001/api/v1/namespaces/default/services/http:getdb-lb:8080/proxy/rss
Second, when connecting in the same minikube
http://localhost:8001/rss
My question. I created a docker container in wsl2 as follows.
docker-compose.yaml (image is ubunut with only installed python and pip )
version: '2.3'
services:
master:
container_name: gputest1
image : desg2022/ubuntu:v01
stdin_open: true # docker run -i
tty: true # docker run -t
ports:
- 8080:8888
command:
"/bin/bash"
extra_hosts:
- "host.docker.internal:host-gateway"
ipc: 'host'
Inside this container I want to access getdb in minikube, what address should i put in?
minikube start fails with error libmachine: Error dialing TCP: dial tcp 10.43.239.243:49167: connect: no route to host when run in the below setup:
k8s cluster (with containerd as container runtime) with 2 pods: one with docker client container, second with docker daemon container.
dind daemon resources:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: dind
spec:
selector:
matchLabels:
app: dind
serviceName: "dind"
template:
metadata:
labels:
app: dind
spec:
containers:
- name: dind-daemon
image: docker:20.10.17-dind
securityContext:
privileged: true
env:
- name: DOCKER_TLS_CERTDIR
value: ""
apiVersion: v1
kind: Service
metadata:
name: dind
spec:
selector:
app: dind
type: ClusterIP
ports:
- name: daemon
protocol: TCP
port: 2375
targetPort: 2375
dind client resources:
apiVersion: v1
kind: Pod
metadata:
name: "docker-client"
labels:
app: "docker-client"
spec:
containers:
- name: docker-client
image: "docker:latest"
env:
- name: DOCKER_HOST
value: "tcp://dind:2375"
minikube start runs inside docker client container
How to debug this issue and what might be the reason for it? 10.43.239.243 is ip of ClusterIP dind service. The error happens after lines in minikube log:
I0804 09:46:35.049413 222 main.go:134] libmachine: About to run SSH command:
sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname
I tried to make the same experiment when both containers run without kubernetes (using docker daemon). In that case, both were using the same docker network, daemon container started with dind network alias and minikube start succeeded.
Below are the used commands:
docker daemon container:
docker run --name dind -d --privileged --network dind --network-alias dind -e DOCKER_TLS_CERTDIR="" docker:dind
docker client container:
docker run --name dind-client -it --network dind -e DOCKER_HOST="tcp://dind:2375"docker sh
/ # wget https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
/ # mv minikube-linux-amd64 minikube
/ # chmod +x minikube
/ # ./minikube start --force
...
* Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
/ # ./minikube kubectl -- run --image=hello-world
/ # ./minikube kubectl -- logs pod/hello
Hello from Docker!
I'm trying to run a simple node server on port 8080, but with the following config any attempt at hitting the subdomain results in a 502 Bad Gateway error. If I go the node I can see there doesn't appear to be any ports open on the container itself. So, assuming I've checked everything correctly, is there anything else I need to do in the config to open the port for the node server?
Edit: If I ssh into the pod and curl localhost on 8080 I'm able to hit the node server.
Dockerfile
FROM node:12.18.1
WORKDIR /app
COPY ["package.json", "package-lock.json*", "./"]
RUN npm install
COPY . .
RUN npm run build
EXPOSE 8080
CMD [ "node", "server.js" ]
k8s deployment
spec:
containers:
- name: test
image: test_image
ports:
- name: http
protocol: TCP
containerPort: 8080
service yaml
apiVersion: v1
kind: Service
metadata:
name: test-service
spec:
ports:
- name: http
port: 80
targetPort: 8080
protocol: TCP
- name: https
port: 443
targetPort: 8080
protocol: TCP
selector:
app: test-deployment
type: NodePort
externalTrafficPolicy: Cluster
Ingress
spec:
rules:
- host: dev.test.com
http:
paths:
- backend:
serviceName: test-service
servicePort: 80
path: /
This wound up being on the application side. The server needed to be bound to 0.0.0.0 instead of 127.0.0.1.
Hope doing good all.
Env: centos 7.3.1611, kubernetes : 1.5, docker 1.12
Problem 1 : Extended jboss docker not working but docker image created successfully
POD gets an error see below, step 7.
Problem 2 : Once problem #1 fixed then i wish to upload to docker hub: https://hub.docker.com/
how can i upload steps please if possible.
1) pull
docker pull jboss/wildfly
2) vi Dockerfile
FROM jboss/wildfly
RUN /opt/jboss/wildfly/bin/add-user.sh admin admin123$ --silent
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]
3) Extend docker image
docker build --tag=nbasetty/wildfly-server .
4) [root#centos7 custom-jboss]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nbasetty/wildfly-server latest c1fbb87faffd 43 minutes ago 583.8 MB
docker.io/httpd latest e0645af13ada 2 weeks ago 177.5 MB
5)vi jboss-wildfly-rc-service-custom.yaml
apiVersion: v1
kind: Service
metadata:
name: wildfly-service
spec:
externalIPs:
- 10.0.2.15
selector:
app: wildfly-rc-pod
ports:
- name: web
port: 8080
#- name: admin-console
# port: 9990
type: LoadBalancer
---
apiVersion: v1
kind: ReplicationController
metadata:
name: wildfly-rc
spec:
replicas: 2
template:
metadata:
labels:
app: wildfly-rc-pod
spec:
containers:
- name: wildfly
image: nbasetty/wildfly-server
ports:
- containerPort: 8080
#- containerPort: 9990
6) kubectl create -f jboss-wildfly-rc-service-custom.yaml
7) [root#centos7 jboss]# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-pvc-pod 1/1 Running 6 2d
wildfly-rc-d0k3h 0/1 ImagePullBackOff 0 23m
wildfly-rc-hgsfj 0/1 ImagePullBackOff 0 23m
[root#centos7 jboss]# kubectl logs wildfly-rc-d0k3h
Error from server (BadRequest): container "wildfly" in pod
"wildfly-rc-d0k3h" is waiting to start:
trying and failing to pull image
Glad you have found a way to make it working. here is step I followed.
I labeled node-01 as 'dbserver: mysql'
create the docker image in node-01
created this pod, it worked.
apiVersion: v1 kind: ReplicationController metadata: name: wildfly-rc spec: replicas: 2 template:
metadata:
labels:
app: wildfly-rc-pod
spec:
containers:
- name: wildfly
image: nbasetty/wildfly-server
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
nodeSelector:
dbserver: mysql
Re-creating the issue:
docker pull jboss/wildfly
mkdir jw
cd jw
echo 'FROM jboss/wildfly
RUN /opt/jboss/wildfly/bin/add-user.sh admin admin123$ --silent
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]' | tee Dockerfile
docker build --tag=docker.io/surajd/wildfly-server .
See the images available:
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/surajd/wildfly-server latest 10e96902ea12 11 seconds ago 583.8 MB
Create a config that works:
echo '
apiVersion: v1
kind: Service
metadata:
name: wildfly
spec:
selector:
app: wildfly
ports:
- name: web
port: 8080
type: LoadBalancer
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: wildfly
spec:
replicas: 2
template:
metadata:
labels:
app: wildfly
spec:
containers:
- name: wildfly
image: docker.io/surajd/wildfly-server
imagePullPolicy: Never
ports:
- containerPort: 8080
' | tee config.yaml
kubectl create -f config.yaml
Notice the field imagePullPolicy: Never, this helps you use the image available on the node(the image we built using docker build). This works on single node cluster but may or may not work on multiple node cluster. So not recommended to put that value, but since we are doing experiment on single node cluster we can set it to Never. Always set it to imagePullPolicy: Always. So that whenever the pod is scheduled the image will be pulled from registry. Read about imagePullPolicy and some config related tips.
Now to pull the image from registry the image should be on registry, so to answer your question of pushing it to docker hub run command:
docker push docker.io/surajd/wildfly-server
So in the above example replace surajd with your docker registry username.
Here are steps I used to do setup of single node cluster on CentOS:
My machine version:
$ cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core)
Here is what I have done:
Setup single node k8s cluster on CentOS as follows (src1 & src2):
yum update -y
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
setenforce 0
yum install -y docker kubelet kubeadm kubectl kubernetes-cni
systemctl enable docker && systemctl start docker
systemctl enable kubelet && systemctl start kubelet
sysctl net.bridge.bridge-nf-call-iptables=1
sysctl net.bridge.bridge-nf-call-ip6tables=1
kubeadm init
cp /etc/kubernetes/admin.conf $HOME/
chown $(id -u):$(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf
kubectl taint nodes --all node-role.kubernetes.io/master-
Now k8s version:
# kubectl version
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae", GitTreeState:"clean", BuildDate:"2017-05-19T18:44:27Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae", GitTreeState:"clean", BuildDate:"2017-05-19T18:33:17Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}