I am new to kubernetes. I have an issue in the pods. When I run the command
kubectl get pods
Result:
NAME READY STATUS RESTARTS AGE
mysql-apim-db-1viwg 1/1 Running 1 20h
mysql-govdb-qioee 1/1 Running 1 20h
mysql-userdb-l8q8c 1/1 Running 0 20h
wso2am-default-813fy 0/1 ImagePullBackOff 0 20h
Due to an issue of "wso2am-default-813fy" node, I need to restart it. Any suggestion?
In case of not having the yaml file:
kubectl get pod PODNAME -n NAMESPACE -o yaml | kubectl replace --force -f -
Usually in case of "ImagePullBackOff" it's retried after few seconds/minutes. In case you want to try again manually you can delete the old pod and recreate the pod. The one line command to delete and recreate the pod would be:
kubectl replace --force -f <yml_file_describing_pod>
$ kubectl replace --force -f <resource-file>
if all goes well, you should see something like:
<resource-type> <resource-name> deleted
<resource-type> <resource-name> replaced
details of this can be found in the Kubernetes documentation, "manage-deployment" and kubectl-cheatsheet pages at the time of writing.
If the Pod is part of a Deployment or Service, deleting it will restart the Pod and, potentially, place it onto another node:
$ kubectl delete po $POD_NAME
replace it if it's an individual Pod:
$ kubectl get po -n $namespace $POD_NAME -o yaml | kubectl replace -f -
Try with deleting pod it will try to pull image again.
kubectl delete pod <pod_name> -n <namespace_name>
First try to see what's wrong with the pod:
kubectl logs -p <your_pod>
In my case it was a problem with the YAML file.
So, I needed to correct the configuration file and replace it:
kubectl replace --force -f <yml_file_describing_pod>
Most probably the issue of ImagePullBackOff is due to either the image not being present or issue with the pod YAML file.
What I will do is this
kubectl get pod -n $namespace $POD_NAME --export > pod.yaml | kubectl -f apply -
I would also see the pod.yaml to see the why the earlier pod didn't work
There is also possibility that the pull policy is not defined or kubernetes is configured to pull from the hub but fails due network issues. Try setting up a local secure registry and execute a pull . It would work.
Related
I have deployed pods using kubectl apply command and I can see pods running:
$kubectl describe pod test-pod -n sample | grep -i container
Containers:
Container ID: containerd://ce6cd9XXXXXX69538XXX
ContainersReady True
Can I say that it's using contained runtime technology? How do I verify the runtime used by containers.
I am also getting some errors like below in pod:
kubectl logs test-pod -n sample
'docker.images' is not supported: Cannot fetch data: Get http://1.28/images/json: dial unix /var/run/docker.sock: connect: no such file or directory.
Is it because I am not using docker runtime?
As i already mentioned in a comment the command is
kubectl get nodes -o wide
It will returns the container runtime for each node.
I am trying to debug a pod with the status "ImagePullBackOff".
The pod is in the namespace minio-operator, but when I try to to describe the pod, it is apparently not found.
Why does that happen?
[psr-admin#zon-psr-2-u001 ~]$ kubectl get all -n minio-operator
NAME READY STATUS RESTARTS AGE
pod/minio-operator-5dd99dd858-n6fdj 0/1 ImagepullBackoff 0 7d
NAME READY. UP-TO-DATE AVAILABLE AGE
deployment.apps/minio-operator 0 1 0 7d
NAME DESIRED CURRENT READY AGE
replicaset.apps/minio-operator-5dd99dd858 1 1 0 7d
[psr-admin#zon-psr-2-u001 ~]$ kubectl describe pod minio-operator-5dd99dd858-n6fdj
Error from server (NotFound): pods "minio-operator-5dd99dd858-n6fdj" not found
Error from server (NotFound): pods "minio-operator-5dd99dd858-n6fdj" not found
You've not specified the namespace in your describe pod command.
You did kubectl get all -n minio-operator, which gets all resources in the minio-operator namespace, but your kubectl describe has no namespace, so it's looking in the default namespace for a pod that isn't there.
kubectl describe pod -n minio-operator <pod name>
Should work OK.
Most resources in kubernetes are namespaced, so will require the -n <namespace> argument unless you switch namespaces.
I'm going through this tutorial
Setting up Jenkins on Container Engine
https://cloud.google.com/solutions/jenkins-on-container-engine-tutorial
and failing on "Creating the Jenkins deployment and services" step
I got this error at one point:
jenkins- 0/1 rpc error: code = 2 desc = failed to start container "": Error response from daemon: {"message":"linux spec user: unable to find user jenkins: no matching entries in passwd file"}
And I get these results for the following commands:
> kubectl apply -f jenkins/k8s/
deployment "jenkins" configured
service "jenkins-ui" configured
service "jenkins-discovery" configured
> get pods --namespace jenkins
NAME READY STATUS RESTARTS AGE
jenkins-<some id> 0/1 CrashLoopBackOff 5 10m
I get it that it is looking for jenkins user in the passwd file, but I'm still not sure why this error took place and what the correct way to fix it is. Any insight would be highly appreciated.
Edit: output of running "kubectl get pods --namespace jenkins"
The very first time running it:
> kubectl get pods --namespace jenkins
NAME READY STATUS RESTARTS AGE
jenkins-1937056428-fp7vr 0/1 ContainerCreating 0 16s
Second time running it:
> kubectl get pods --namespace jenkins
NAME READY STATUS RESTARTS AGE
jenkins-1937056428-fp7vr 0/1 rpc error: code = 2 desc = failed to start container "10a8ab7e3eb0ad153fd6055d86336b1cdfe9642b6993684a7e01fefbeca7a566": Error response from
daemon: {"message":"linux spec user: unable to find user jenkins: no matching entries in passwd file"} 1 39s
Third and after:
> kubectl get pods --namespace jenkins
NAME READY STATUS RESTARTS AGE
jenkins-1937056428-fp7vr 0/1 CrashLoopBackOff 270 22h
It appears that the persistent disk volume for the jenkins is not properly setup. Try running the following commands to reconfigure disk volumes and rerun jenkins pod,
kubectl delete -f jenkins/k8s/
gcloud compute disks delete jenkins-home
gcloud compute images delete jenkins-home-image
gcloud config set compute/zone us-east1-d
gcloud compute images create jenkins-home-image --source-uri https://storage.googleapis.com/solutions-public-assets/jenkins-cd/jenkins-home-v3.tar.gz
gcloud compute disks create jenkins-home --image jenkins-home-image --zone us-east1-d
kubectl apply -f jenkins/k8s/
I basically did one step wrong:
Provision a Kubernetes cluster using Container Engine.
gcloud container clusters create jenkins-cd \
--network jenkins \
--scopes "https://www.googleapis.com/auth/projecthosting,storage-rw"
Here make sure the options --network and --scopes actually get passed in. I guess I copied the command without fixing it up and the options got dropped.
Im running OpenShift.
OpenShift Master: v3.3.1.7
Kubernetes Master: v1.3.0+52492b4
But am having problems trying to run a build in Jenkins (running in a pod). This is not a problem with the java code that I'm trying to build, but is a problem in the Kubernetes/Openshift setup.
The builds fail with:
Caused by: java.net.UnknownHostException: kubernetes.default: Name does not resolve
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
at java.net.InetAddress.getAllByName(InetAddress.java:1192)
at java.net.InetAddress.getAllByName(InetAddress.java:1126)
at okhttp3.Dns$1.lookup(Dns.java:39)
...
Does anyone know how to fix this?
First confirm that DNS is actually working with:
› kubectl run -i -t busybox --image=busybox --restart=Never
Waiting for pod default/busybox to be running, status is Pending, pod ready: false
If you don't see a command prompt, try pressing enter.
/ # nslookup kubernetes.default
Server: 192.168.60.10
Address 1: 192.168.60.10 kube-dns.kube-system.svc.cluster.local
Name: kubernetes.default
Address 1: 192.168.60.1 kubernetes.default.svc.cluster.local
If that doesn't work check if the DNS pods are running:
kubectl get pods --namespace=kube-system -l k8s-app=kube-dns
Will respond sth like:
NAME READY STATUS RESTARTS AGE
kube-dns-v14-3u5zi 3/3 Running 36 166d
Finally checking the related logs is worth a try:
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kube-dns
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c healthz
Full instructions can be found on kubernetes.io
Please check on manage jenkins -> Configure Global Security -> Agent port is 50000 and Fixed
I'm newbie of the Kubernetes while I'm using Google Cloud Container. I just follow the tutorials as belows:
https://cloud.google.com/container-engine/docs/tutorials/http-balancer
http://kubernetes.io/docs/hellonode/#create-your-pod
In these tutorials, I'll get the replicacontroller after I run the "kubectl run" but there is no replicacontrollers so that I cannot run the command of "kubectl expose rc" in order to open a port.
Here is my result of the commands:
ChangMatthews-MacBook-Pro:frontend changmatthew$ kubectl run nginx --image=nginx --port=80
deployment "nginx" created
ChangMatthews-MacBook-Pro:frontend changmatthew$ kubectl expose rc nginx --target-port=80 --type=NodePort
Error from server: replicationcontrollers "nginx" not found
Here is my result when I run "kubectl get rc,svc,ingress,deployments,pods":
ChangMatthews-MacBook-Pro:frontend changmatthew$ kubectl get rc,svc,ingress,deployments,pods
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.3.240.1 <none> 443/TCP 12m
NAME RULE BACKEND ADDRESS AGE
basic-ingress - nginx:80 107.178.247.247 12m
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
nginx 1 1 1 1 11m
NAME READY STATUS RESTARTS AGE
nginx-198147104-zgo7m 1/1 Running 0 11m
One of my solution is to create yaml file which define the replicacontroller. But is there any way to create replicacontroller via kubectl run command like above tutorials?
Thanks,
Now that kubectl run creates a deployment, you specify that the type being exposed in a deployment rather than a replication controller:
kubectl expose deployment nginx --target-port=80 --type=NodePort
The team might still be updating the docs to reflect 1.2. Note the output you got:
$ kubectl run nginx --image=nginx --port=80
deployment "nginx" created
kubectl run now creates a deployemtn+replica-set.
To view these you can do kubectl get deployment, and get rs respectively.
Deployments are essentially a nicer way to perform rolling update server side, but there's a little more to it. See docs: http://kubernetes.io/docs/user-guide/deployments/
In version 1.15.0, it works as follows.
root#k8smaster ~]# kubectl run guestbook --image=coolguy/k8s_guestbook:1.0 --port=8080 --generator=run/v1
kubectl run --generator=run/v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create
instead.
***replicationcontroller/guestbook created***
In version 1.19.0:
[root#k8smaster ~]# kubectl run guestbook --image=dmsong2008/k8s_guestbook:1.0 --port=8080 --generator=run/v1
***Flag --generator has been deprecated, has no effect and will be removed in the future.***
pod/guestbook created