On Docker Desktop(Win, open file sharing already), create pv/pvc and bound successfully, but start pod failed shows:
Warning Failed 4s (x4 over 34s) kubelet, docker-desktop Error: stat /c/cannot-found: no such file or directory
Have already created the "cannot-found" folder on my root which path is C:\cannot-found
Here are my pv/pvc and pod yaml:
apiVersion: v1
kind: PersistentVolume
metadata:
name: global-volume
labels:
pv_pvc_label: nfs
spec:
accessModes:
- ReadWriteMany
capacity:
storage: 1Gi
hostPath:
path: "/c/cannot-found"
persistentVolumeReclaimPolicy: Delete
storageClassName: standard
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: global-volume
spec:
storageClassName: standard
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
selector:
matchLabels:
pv_pvc_label: nfs
volumeName: global-volume
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: alpine
namespace: default
spec:
selector:
matchLabels:
component: centos
module: centos
replicas: 1
template:
metadata:
labels:
component: centos
module: centos
spec:
volumes:
- name: nfs
persistentVolumeClaim:
claimName: global-volume
containers:
- image: centos
command:
- /bin/sh
- "-c"
- "touch /root/test.txt; echo \"test mount\">/root/test.txt; cp /root/test.txt /tmp/test.txt; sleep 60m"
imagePullPolicy: IfNotPresent
name: alpine
volumeMounts:
- name: nfs
mountPath: /tmp
subPath: mountPath
restartPolicy: Always
My pv/pvc status:
leih#CNleih01 MINGW64 /c/dev/SAW/x-workspace/deployers/common (leih/workspace)
$ kubectl create -f global-volume-tmp.yaml
persistentvolume/global-volume created
persistentvolumeclaim/global-volume created
leih#CNleih01 MINGW64 /c/dev/SAW/x-workspace/deployers/common (leih/workspace)
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
global-volume Pending global-volume 0 standard 7s
leih#CNleih01 MINGW64 /c/dev/SAW/x-workspace/deployers/common (leih/workspace)
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
global-volume Bound global-volume 1Gi RWX standard 10s
leih#CNleih01 MINGW64 /c/dev/SAW/x-workspace/deployers/common (leih/workspace)
$ kubectl scale deployment alpine --replicas=1
deployment.apps/alpine scaled
leih#CNleih01 MINGW64 /c/dev/SAW/x-workspace/deployers/common (leih/workspace)
$ kubectl get po |grep alpine
alpine-6559ddcb88-n262l 0/1 CreateContainerConfigError 0 14s
leih#CNleih01 MINGW64 /c/dev/SAW/x-workspace/deployers/common (leih/workspace)
$ kubectl describe po alpine-6559ddcb88-n262l |grep Error
Reason: CreateContainerConfigError
Warning Failed 4s (x4 over 34s) kubelet, docker-desktop Error: stat /c/cannot-found: no such file or directory
File sharing:
k8s version: v1.16.6-beta.0
docker desktop version(win): Docker Desktop Community 2.3.0.2
docker version: v19.03.8
How can I solve it?
Update:
After rollback docker desktop from v2.3.0.2 to v2.2.0.5, it works fine.
Related
Net core application image and I am trying to create deployment in local kubernetes.
I created docker image as below.
docker tag microservicestest:dev microservicestest .
docker build -t microservicestest .
docker run -d -p 8080:80 --name myapp microservicetest
Then I created deployment as below.
kubectl run microservicestest-deployment --image=microservicestest:latest --port 80 --replicas=3
kubectl expose deployment microservicestest-deployment --type=NodePort
then when I see Kubectl get pods I see below error
Below is the output when I run docker images
Below is the output
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
creationTimestamp: "2020-09-22T04:29:14Z"
generation: 1
labels:
run: microservicestest-deployment
name: microservicestest-deployment
namespace: default
resourceVersion: "17282"
selfLink: /apis/apps/v1/namespaces/default/deployments/microservicestest-deployment
uid: bf75410a-d332-4016-9757-50d534114599
spec:
progressDeadlineSeconds: 600
replicas: 3
revisionHistoryLimit: 10
selector:
matchLabels:
run: microservicestest-deployment
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
run: microservicestest-deployment
spec:
containers:
- image: microservicestest:latest
imagePullPolicy: Always
name: microservicestest-deployment
ports:
- containerPort: 80
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
status:
conditions:
- lastTransitionTime: "2020-09-22T04:29:14Z"
lastUpdateTime: "2020-09-22T04:29:14Z"
message: Deployment does not have minimum availability.
reason: MinimumReplicasUnavailable
status: "False"
type: Available
- lastTransitionTime: "2020-09-22T04:29:14Z"
lastUpdateTime: "2020-09-22T04:29:14Z"
message: ReplicaSet "microservicestest-deployment-5c67d587b9" is progressing.
reason: ReplicaSetUpdated
status: "True"
type: Progressing
observedGeneration: 1
replicas: 3
unavailableReplicas: 3
updatedReplicas: 3
I am not able to understand why my pods are not able to pull the image from local. Can someone help me to identify the issue What I am making here. Any help would be appreciated. Thank you
if you are using minikube you first need to build the images in the docker hosted in the minikube machine doing this in your bash session eval $(minikube docker-env) for windows check here
then you need to tell Kubernetes your image pull policy to be Never or IfNotPresent to look for local images
spec:
containers:
- image: my-image:my-tag
name: my-app
imagePullPolicy: Never
check here the official documentation
By default, the kubelet tries to pull each image from the specified registry. However, if the imagePullPolicy property of the container is set to IfNotPresent or Never, then a local image is used (preferentially or exclusively, respectively).
as you are not using yaml file you can create the resources like this
kubectl run microservicestest-deployment --image=microservicestest:latest --image-pull-policy=Never --port 80 --replicas=3
kubectl expose deployment microservicestest-deployment --type=NodePort
I tried running simple DaemonSet on kube cluster - the Idea was that other kube pods would connect to that containers docker daemon (dockerd) and execute commands on it. (The other pods are Jenkins slaves and would have just env DOCKER_HOST point to 'tcp://localhost:2375'); In short the config looks like this:
dind.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: dind
spec:
selector:
matchLabels:
name: dind
template:
metadata:
labels:
name: dind
spec:
# tolerations:
# - key: node-role.kubernetes.io/master
# effect: NoSchedule
containers:
- name: dind
image: docker:18.05-dind
resources:
limits:
memory: 2000Mi
requests:
cpu: 100m
memory: 500Mi
volumeMounts:
- name: dind-storage
mountPath: /var/lib/docker
volumes:
- name: dind-storage
emptyDir: {}
Error message when running
mount: mounting none on /sys/kernel/security failed: Permission denied
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: mounting none on /tmp failed: Permission denied
I took the idea from medium post that didn't describe it fully: https://medium.com/hootsuite-engineering/building-docker-images-inside-kubernetes-42c6af855f25 describing docker of docker, docker in docker and Kaniko
found the solution
apiVersion: v1
kind: Pod
metadata:
name: dind
spec:
containers:
- name: jenkins-slave
image: gcr.io/<my-project>/myimg # it has docker installed on it
command: ['docker', 'run', '-p', '80:80', 'httpd:latest']
resources:
requests:
cpu: 10m
memory: 256Mi
env:
- name: DOCKER_HOST
value: tcp://localhost:2375
- name: dind-daemon
image: docker:18.05-dind
resources:
requests:
cpu: 20m
memory: 512Mi
securityContext:
privileged: true
volumeMounts:
- name: docker-graph-storage
mountPath: /var/lib/docker
volumes:
- name: docker-graph-storage
emptyDir: {}
i am using nfs for volume in kubernetes pod using deployment.
Below are details of all files.
Filename :- nfs-server.yaml
kind: Service
apiVersion: v1
metadata:
name: nfs-service
spec:
selector:
role: nfs
ports:
# Open the ports required by the NFS server
# Port 2049 for TCP
- name: tcp-2049
port: 2049
protocol: TCP
# Port 111 for UDP
- name: udp-111
port: 111
protocol: UDP
---
kind: Pod
apiVersion: v1
metadata:
name: nfs-server-pod
labels:
role: nfs
spec:
containers:
- name: nfs-server-container
image: cpuguy83/nfs-server
securityContext:
privileged: true
args:
# Pass the paths to share to the Docker image
- /exports
Both service and pod are running. Below is the output.
Now i have to use this in my web-server. Below is the details of the deployment file for web.
Filename :- deployment.yaml
apiVersion: v1
kind: Service
metadata:
name: apache-service
spec:
selector:
app: apache
ports:
- protocol: TCP
port: 80
targetPort: 80
type: NodePort
---
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: apache-deployment
spec:
selector:
matchLabels:
app: apache
replicas: 1 # tells deployment to run 2 pods matching the template
template:
metadata:
labels:
app: apache
spec:
volumes:
- name: nfs-volume
nfs:
server: 10.99.56.195
path: /exports
containers:
- name: apache
image: mobingi/ubuntu-apache2-php7:7.2
ports:
- containerPort: 80
volumeMounts:
- name: nfs-volume
mountPath: /var/www/html
when i am running this file without volume everything works fine. But when i am running this with nfs than pod give following error.
kubectl describe pod apache-deployment-577ffcf9bd-p8s75
Give following output:-
Name: apache-deployment-577ffcf9bd-p8s75
Namespace: default
Priority: 0
Node: worker-node2/10.160.0.4
Start Time: Tue, 09 Jul 2019 09:53:39 +0000
Labels: app=apache
pod-template-hash=577ffcf9bd
Annotations: <none>
Status: Pending
IP:
Controlled By: ReplicaSet/apache-deployment-577ffcf9bd
Containers:
apache:
Container ID:
Image: mobingi/ubuntu-apache2-php7:7.2
Image ID:
Port: 80/TCP
Host Port: 0/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-p9qdb (ro)
/var/www/html from nfs-volume (rw)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
nfs-volume:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 10.244.1.50
Path: /exports
ReadOnly: false
default-token-p9qdb:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-p9qdb
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 6m21s default-scheduler Successfully assigned default/apache-deployment-577ffcf9bd-p8s75 to worker-node2
Warning FailedMount 4m16s kubelet, worker-node2 MountVolume.SetUp failed for volume "nfs-volume" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume --scope -- mount -t nfs 10.244.1.50:/exports /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume
Output: Running scope as unit: run-r3a55a8a3287448a59f7e4dbefa0333af.scope
mount.nfs: Connection timed out
Warning FailedMount 2m10s kubelet, worker-node2 MountVolume.SetUp failed for volume "nfs-volume" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume --scope -- mount -t nfs 10.244.1.50:/exports /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume
Output: Running scope as unit: run-r5fe7befa141d4f989e14b291afa43208.scope
mount.nfs: Connection timed out
Warning FailedMount 2m3s (x2 over 4m18s) kubelet, worker-node2 Unable to mount volumes for pod "apache-deployment-577ffcf9bd-p8s75_default(29114119-5815-442a-bb97-03fa491206a4)": timeout expired waiting for volumes to attach or mount for pod "default"/"apache-deployment-577ffcf9bd-p8s75". list of unmounted volumes=[nfs-volume]. list of unattached volumes=[nfs-volume default-token-p9qdb]
Warning FailedMount 4s kubelet, worker-node2 MountVolume.SetUp failed for volume "nfs-volume" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume --scope -- mount -t nfs 10.244.1.50:/exports /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume
Output: Running scope as unit: run-rd30c104228ae43df933839b6da469107.scope
mount.nfs: Connection timed out
Can Anyone please help to solve this problem.
Make sure there is no firewall between the nodes
Make sure nfs-utils installed on cluster nodes
Here is a blog posts about the docker image that you are using for nfs server, you need to do some tweaks for ports to be used by nfs server.
https://medium.com/#aronasorman/creating-an-nfs-server-within-kubernetes-e6d4d542bbb9
I try to start RStudio in docker container via kubernetes. All objects are created, but when I try to open rstudio using such commands in Ubuntu 18:
kubectl create -f rstudio-ing.yml
IP=$(minikube ip)
xdg-open http://$IP/rstudio/
there is error: #RStudio initialization error: unable connect to service.
Usual docker command works fine:
docker run -d -p 8787:8787 -e PASSWORD=123 -v /home/aabor/r-projects:/home/rstudio aabor/rstudio
The same intended operation in kubernetes fails.
rstudio-ing.yml file creates all objects well. RStudio is accessible if I do not mount any folder. But if I add folder mounts it produces an error. Any suggestions?
The content of the rstudio-ing.yml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: r-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /rstudio/
backend:
serviceName: rstudio
servicePort: 8787
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rstudio
spec:
replicas: 1
selector:
matchLabels:
service: rstudio
template:
metadata:
labels:
service: rstudio
language: R
spec:
containers:
- name: rstudio
image: aabor/rstudio
env:
- name: PASSWORD
value: "123"
volumeMounts:
- name: home-dir
mountPath: /home/rstudio/
volumes:
- name: home-dir
hostPath:
#RStudio initialization error: unable connect to service
path: /home/aabor/r-projects
---
apiVersion: v1
kind: Service
metadata:
name: rstudio
spec:
ports:
- port: 8787
selector:
service: rstudio
This is pod description:
Name: rstudio-689c4fd6c8-fgt7w
Namespace: default
Node: minikube/10.0.2.15
Start Time: Fri, 23 Nov 2018 21:42:35 +0300
Labels: language=R
pod-template-hash=2457098274
service=rstudio
Annotations: <none>
Status: Running
IP: 172.17.0.9
Controlled By: ReplicaSet/rstudio-689c4fd6c8
Containers:
rstudio:
Container ID: docker://a6bdcbfdf8dc5489a4c1fa6f23fb782bc3d58dd75d50823cd370c43bd3bffa3c
Image: aabor/rstudio
Image ID: docker-pullable://aabor/rstudio#sha256:2326e5daa3c4293da2909f7e8fd15fdcab88b4eb54f891b4a3cb536395e5572f
Port: <none>
Host Port: <none>
State: Running
Started: Fri, 23 Nov 2018 21:42:39 +0300
Ready: True
Restart Count: 0
Environment:
PASSWORD: 123
Mounts:
/home/rstudio/ from home-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-mrkd8 (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
home-dir:
Type: HostPath (bare host directory volume)
Path: /home/aabor/r-projects
HostPathType:
default-token-mrkd8:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-mrkd8
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 10s default-scheduler Successfully assigned rstudio-689c4fd6c8-fgt7w to minikube
Normal SuccessfulMountVolume 10s kubelet, minikube MountVolume.SetUp succeeded for volume "home-dir"
Normal SuccessfulMountVolume 10s kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-mrkd8"
Normal Pulling 9s kubelet, minikube pulling image "aabor/rstudio"
Normal Pulled 7s kubelet, minikube Successfully pulled image "aabor/rstudio"
Normal Created 7s kubelet, minikube Created container
Normal Started 6s kubelet, minikube Started container
You have created a service of type ClusterIP that can only be possible to access in the cluster not the outside. So to make it available outside of the cluster, change the service type LoadBalancer.
apiVersion: v1
kind: Service
metadata:
name: rstudio
spec:
ports:
- port: 8787
selector:
service: rstudio
type: LoadBalancer
In that case, the loadbalancer type service don't need the ingress and use the url as:
$ minikube service rstudio --url
I want to use ceph rbd with kubernetes.
I have a kubernetes 1.9.2 and ceph 12.2.5 cluster and on my k8s nodes I have installed ceph-common pag.
[root#docker09 manifest]# ceph auth get-key client.admin|base64
QVFEcmxmcGFmZXlZQ2hBQVFJWkExR0pXcS9RcXV4QmgvV3ZFWkE9PQ==
[root#docker09 manifest]# cat ceph-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: ceph-secret
data:
key: QVFEcmxmcGFmZXlZQ2hBQVFJWkExR0pXcS9RcXV4QmgvV3ZFWkE9PQ==
kubectl create -f ceph-secret.yaml
Then:
[root#docker09 manifest]# cat ceph-pv.yaml |grep -v "#"
apiVersion: v1
kind: PersistentVolume
metadata:
name: ceph-pv
spec:
capacity:
storage: 2Gi
accessModes:
- ReadWriteOnce
rbd:
monitors:
- 10.211.121.61:6789
- 10.211.121.62:6789
- 10.211.121.63:6789
pool: rbd
image: ceph-image
user: admin
secretRef:
name: ceph-secret
fsType: ext4
readOnly: false
persistentVolumeReclaimPolicy: Recycle
[root#docker09 manifest]# rbd info ceph-image
rbd image 'ceph-image':
size 2048 MB in 512 objects
order 22 (4096 kB objects)
block_name_prefix: rbd_data.341d374b0dc51
format: 2
features: layering
flags:
create_timestamp: Fri Jun 15 15:58:04 2018
[root#docker09 manifest]# cat task-claim.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: ceph-claim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
[root#docker09 manifest]# kubectl get pv,pvc
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv/ceph-pv 2Gi RWO Recycle Bound default/ceph-claim 54m
pv/host 10Gi RWO Retain Bound default/hostv 24d
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc/ceph-claim Bound ceph-pv 2Gi RWO 53m
pvc/hostv Bound host 10Gi RWO 24d
I create a pod use this pvc .
[root#docker09 manifest]# cat ceph-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: ceph-pod2
spec:
containers:
- name: ceph-busybox
image: busybox
command: ["sleep", "60000"]
volumeMounts:
- name: ceph-vol1
mountPath: /usr/share/busybox
readOnly: false
volumes:
- name: ceph-vol1
persistentVolumeClaim:
claimName: ceph-claim
[root#docker09 manifest]# kubectl get pod ceph-pod2 -o wide
NAME READY STATUS RESTARTS AGE IP NODE
ceph-pod2 0/1 ContainerCreating 0 14m <none> docker10
The pod is still in ContainerCreating status.
[root#docker09 manifest]# kubectl describe pod ceph-pod2
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 15m default-scheduler Successfully assigned ceph-pod2 to docker10
Normal SuccessfulMountVolume 15m kubelet, docker10 MountVolume.SetUp succeeded for volume "default-token-85rc7"
Warning FailedMount 1m (x6 over 12m) kubelet, docker10 Unable to mount volumes for pod "ceph-pod2_default(56af9345-7073-11e8-aeb6-1c98ec29cbec)": timeout expired waiting for volumes to attach/mount for pod "default"/"ceph-pod2". list of unattached/unmounted volumes=[ceph-vol1]
I don't know why this happening, need your help... Best regards.
There's no need to reinvent a wheel here. There's already project called ROOK, which deploys ceph on kubernetes and it's super easy to run.
https://rook.io/
rbd -v (included in ceph-common) should return the same version as your cluster. You should also check the messages of kubelet.