I have setup a new Kubernetes v1.5 cluster.
I locally created a new docker image using :
# MAIN IMAGE
FROM gcr.io/google_containers/nginx-slim
I created it using the command :
docker build -t myapp:1 .
I can see that the image is avaiable and running :
docker ps | grep app
d6fc0508e56b myapp:1 "nginx -g 'daemon ..." 31 seconds ago Up 30 seconds 0.0.0.0:32354->80/tcp
Now I am trying to use the same image in a kubernetes deployment.
kubectl run app-deployment --image myapp:1 --replicas=1 --port=80
But doing so does not start the pod and I get the ERROR :
19s 3s 2 {kubelet 10.0.0.17} spec.containers{app-deployment} Normal Pulling pulling image "myapp:1"
18s 2s 2 {kubelet 10.0.0.17} spec.containers{app-deployment} Warning Failed Failed to pull image "myapp:1": unauthorized: authentication required
18s 2s 2 {kubelet 10.0.0.17} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "app-deployment" with ErrImagePull: "unauthorized: authentication required"
The files /root/.docker/config.json & /var/lib/kubelet/.dockercfg are currently empty. Is there something Ive missed in setting up Kubernetes ?
Since you are building a custom docker image, you have to build it in every node of your cluster that the scheduler could put the pod into.
Furthermore, you need yo specify in your PodSpec an imagePullPolicy of ifNotPresent to indicate the kubelet not to try to download your image if it is already present.
This should make your image work, but I strongly suggest you to push your image in a docker registry and let the nodes pull it from there.
Related
I am trying to deploy Windows Container image on the following software stack
Windows 10 Pro + Docker Desktop + Embedded Kubernetes in docker desktop
Due to some reason 'embedded kubernetes' does not recognize 'local images' no matter whatever --image-pull-policy was set
Docker images
PS C:\WINDOWS\system32> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
myimg final 90c09acbfc59 15 hours ago 5.45GB
Kubectl run
PS C:\WINDOWS\system32> kubectl run --image=myimg:final tskuberun
Pod output
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 25s default-scheduler Successfully assigned default/tskuberun to docker-desktop
Normal BackOff 23s (x2 over 24s) kubelet Back-off pulling image "myimg:final"
Warning Failed 23s (x2 over 24s) kubelet Error: ImagePullBackOff
Normal Pulling 9s (x2 over 25s) kubelet Pulling image "myimg:final"
Warning Failed 8s (x2 over 25s) kubelet Failed to pull image "myimg:final": rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.65.5:53: no such host
Warning Failed 8s (x2 over 25s) kubelet Error: ErrImagePull
However, when I execute docker run it pulled the local image. Following worked as expected
PS C:\WINDOWS\system32> docker run myimg:final
I googled for the answer but most of the links were related to Unix flavors and Minikube.
Only few links were related to Docker desktop + embedded kubernetes, but unfortunately none resolved the issue
I am struggling to get rid of this issue. Any help is highly appreciated
EDIT
On further investigation, I observed that 'Docker desktop' refers to local images in case had I selected option "Switch to Linux Containers"
Kubectl run for Linux image
PS C:\WINDOWS\system32> kubectl run --image=wphp --image-pull-policy=IfNotPresent lntest
PS C:\WINDOWS\system32> kubectl describe pod/lntest
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 40s default-scheduler Successfully assigned default/lntest to docker-desktop
Normal Pulled 2s (x4 over 39s) kubelet Container image "wphp" already present on machine
Normal Created 2s (x4 over 39s) kubelet Created container lntest
Normal Started 2s (x4 over 39s) kubelet Started container lntest
It appears that this issue occurs only for 'Windows containers' ie Docker desktop does NOT refer local images had I selected option 'Switch to Windows Containers'
Allthough imagePullPolicy: never should do the trick for you, there could be some certificate related issues.
Personally I avoided using locally built Docker images because of those issues.
You can try to integrate docker push to docker hub in your workflow or build a docker registry in your kubernetes cluster e.g. using https://www.linuxtechi.com/setup-private-docker-registry-kubernetes/
The VM used by Docker Desktop is unable to access the internet. You'll have to sort out that networking.
Docker Image: -
docker images | grep -i "gcc"
gcc-docker latest 84c4359e6fc9 21 mites ago 1.37GB
docker run -it gcc-docker:latest
hello,world
Kubernetes pod deployed:-
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned default/hello-world to master-node
Normal Pulling 4s kubelet, master-node Pulling image "gcc-docker:latest"
Warning Failed 0s kubelet, master-node Failed to pull image "gcc-docker:latest": rpc error: code = Unknown desc = Erroresponse from daemon: pull access denied for gcc-docker, repository does not exist or may require 'docker login': denied: requested acce to the resource is denied
Warning Failed 0s kubelet, master-node Error: ErrImagePull
Normal BackOff 0s kubelet, master-node Back-off pulling image "gcc-docker:latest"
Warning Failed 0s kubelet, master-node Error: ImagePullBackOff
-->yaml files used to deploy pod
apiVersion: v1
kind: Pod
metadata:
name: hello-world
labels:
type: hello-world
spec:
containers:
- name: hello-world
image: gcc-docker:latest
command: ['sh', '-c', 'echo Hello Kubernetes! && sleep 60']
ports:
- containerPort: 80
I tried pulling gcc-docker and got the same error.You may have this image present on your system already and now its not on dockerhub.
if you know the repository for this image, try to use the same and for authentication create secrets of docker type and use them as image pull secrets.
Also, one more thing you are running the container on the master node, and I assume it's minikube or some local setup.
Minikube uses a dedicated VM to run Kubernetes which is not the same as the machine on which you have installed minikube.
So images available on your laptop will not be available to minikube.
I tried to deploy 'deployments' in kubernetes which is pull docker image from private registry (I don't know who did this setup) but during "docker pull images" through kubernetes i'm getting following error
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 85s default-scheduler Successfully assigned default/trusted-enc-assettag1-deployment-8467b74958-6fbp7 to k8s-node
Normal BackOff 24s (x2 over 61s) kubelet, k8s-node Back-off pulling image "10.105.168.81:5000/simplehttpserverenc:enc_v1"
Warning Failed 24s (x2 over 61s) kubelet, k8s-node Error: ImagePullBackOff
Normal Pulling 12s (x3 over 82s) kubelet, k8s-node Pulling image "10.105.168.81:5000/simplehttpserverenc:enc_v1"
Warning Failed 0s (x3 over 62s) kubelet, k8s-node Failed to pull image "10.105.168.81:5000/simplehttpserverenc:enc_v1": rpc error: code = Unknown desc = Error response from daemon: Get https://10.105.168.81:5000/v2/: net/http: TLS handshake timeout
Warning Failed 0s (x3 over 62s) kubelet, k8s-node Error: ErrImagePull
[root#k8s-master ~]# docker pull 10.105.168.81:5000/simplehttpserverenc:enc_v1
ImagePullBackOff and net/http: TLS handshake timeout error.
Initially this "net/http: TLS handshake timeout" error is observed in docker pull as well. I referred some answers and
configured certificate(/etc/docker/certs.d//ca.crt ) and
proxy (/etc/systemd/system/docker.service.d/proxy.conf)
after that able to perform docker pull from private image.
[root#k8s-master ~]# docker pull 10.105.168.81:5000/simplehttpserverenc:enc_v1
enc_v1: Pulling from simplehttpserverenc
54fec2fa59d0: Pull complete
cd3f35d84cab: Pull complete
a0afc8e92ef0: Pull complete
9691f23efdb7: Pull complete
6512e60b314b: Pull complete
a8ac6632d329: Pull complete
68f4c4e0aa8c: Pull complete
Digest: sha256:0358708cd11e96f6cf6f22b29d46a8eec50d7107597b866e1616a73a198fe797
Status: Downloaded newer image for 10.105.168.81:5000/simplehttpserverenc:enc_v1
10.105.168.81:5000/simplehttpserverenc:enc_v1
[root#k8s-master ~]#
But still unable to perform this docker pull through kubernetes. How to solve this ?
If you use docker as container engine in your k8s, AFAIK it's the same with Understand the configuration. Because the image pulling is conducted by the container engine and it depends the proprietary configuration of each one on the certificates. How about pulling the same image on the worker node in your k8s ? Is it possible to pull the one without errors ?
As your dockerconfigjson is not working properly. Try this method :
kubectl create secret docker-registry regcred --docker-server=10.105.168.81:5000 --docker-username=<your-name> --docker-password=<your-pword>
And in Kubernetes manifest :
apiVersion: v1
kind: Pod
metadata:
name: private-reg
spec:
containers:
- name: private-reg-container
image: 10.105.168.81:5000/simplehttpserverenc:enc_v1
imagePullSecrets:
- name: regcred
I had encounted this many times, when I forgot to configure these secrets. Also if you have any othernamespace, you will have to generate secrets for each of these namespaces separately passing -n <your-ns> to above kubectl create secret
Edit : As you can not pull the image from worker node.
Make sure you copied docker-registry ca.crt to /etc/docker/certs.d/ca.crt
and then try docker pull .
I'm new to openshift and I have a showstopper:
On my Computer I created a Dockerimage called restservice and I successfully tested it:
docker run -d -p 8080:8080 restservice
Then I created an app in Openshift-Online with the image:
oc new-app restservice
I can see the deployment-pod is starting and after the creating of the running pod is failing.
with
oc describe pod restservice-2-50n0h
I get the following Error:
...
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
1m 1m 1 {default-scheduler } Normal Scheduled Successfully assigned restservice-2-50n0h to ip-172-31-54-238.us-west-2.compute.internal
41s 41s 1 {kubelet ip-172-31-54-238.us-west-2.compute.internal} spec.containers{restservice} Normal Pulling pulling image "restservice:latest"
39s 39s 1 {kubelet ip-172-31-54-238.us-west-2.compute.internal} spec.containers{restservice} Warning Failed Failed to pull image "restservice:latest": unauthorized: authentication required
39s 39s 1 {kubelet ip-172-31-54-238.us-west-2.compute.internal} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "restservice" with ErrImagePull: "unauthorized: authentication required"
55s 9s 2 {kubelet ip-172-31-54-238.us-west-2.compute.internal} Warning FailedSync Error syncing pod, skipping: failed to "SetupNetwork" for "restservice-2-50n0h_wgbeckmann" with SetupNetworkError: "Failed to setup network for pod \"restservice-2-50n0h_wgbeckmann(06f892b4-7568-11e7-914e-0a69cdf75e6f)\" using network plugins \"cni\": CNI request failed with status 400: 'Failed to execute iptables-restore: exit status 1 (iptables-restore: line 3 failed\n)\n'; Skipping pod"
I have no Idea what Authentication is needet.
The missing step is to pusch the Image to openshift online.
So these are the Steps:
Build a Image on the local Computer
docker build -t restservice .
Tag it with registry/username/image-name
docker tag restservice registry.starter-us-west2.openshift.com/myusername/myrestservice
Get your Secret for the Login into the Openshift Registry
oc whoami -t
sr3grwkegr3kjrk42k2jrg34kb5k43g5k4jg53 (sr3... is the output)
Log into Openshift Registry
docker login -u name#mail.com -p sr3grwkegr3kjrk42k2jrg34kb5k43g5k4jg53 https://registry.starter-us-west-2.openshift.com
Push the Image to the Registry
docker push registry.starter-us-west-2.openshift.com/myusername/myrestservice
Create the new App with the Image
oc new-app myrestservice
Thats all ....
Started working on Docker and kubernetes lately.
I ran into a problem which I don't actually understand fully.
The thing is when I execute my svc.yaml(service) and rc.yaml(replication controller) pods get created but its status is terminated.
I tried checking the possible reason for failure by using the command
docker ps -a
954c3ee817f9 localhost:5000/HelloService
"/bin/sh -c ./startSe" 2 minutes ago Exited (127) 2 minutes
ago
k8s_HelloService.523e3b04_HelloService-64789_default_40e92b63-707a-11e7-9b96-080027f96241_195f2fee
then tried running
docker run -i -t localhost:5000/HelloService
/bin/sh: ./startService.sh: not found
what is the possible reason I am getting these errors.
Docker File:
FROM alpine:3.2
VOLUME /tmp
ADD HelloService-0.0.1-SNAPSHOT.jar app.jar
VOLUME /etc
ADD /etc/ /etc/
ADD startService.sh /startService.sh
RUN chmod 700 /startService.sh
ENTRYPOINT ./startService.sh
startService.sh
#!/bin/sh
touch /app.jar
java -Djava.security.egd=file:/dev/./urandom -Xms256m -Xmx256m -jar /app.jar
Also I would like to know if there any specific way I can access the logs from kubernetes for the terminated pods?
Update :
on running below command
kubectl describe pods HelloService-522qw
24s 24s 1 {default-scheduler } Normal Scheduled Successfully
assigned HelloService-522qw to ssantosh.centos7 17s 17s 1 {kubelet
ssantosh.centos7} spec.containers{HelloService} Normal Created Created
container with docker id b550557f4c17; Security:[seccomp=unconfined]
17s 17s 1 {kubelet
ssantosh.centos7} spec.containers{HelloService} Normal Started Started
container with docker id b550557f4c17 18s 16s 2 {kubelet
ssantosh.centos7} spec.containers{HelloService} Normal Pulling pulling
image "localhost:5000/HelloService" 18s 16s 2 {kubelet
ssantosh.centos7} spec.containers{HelloService} Normal Pulled Successfully
pulled image "localhost:5000/HelloService" 15s 15s 1 {kubelet
ssantosh.centos7} spec.containers{HelloService} Normal Created Created
container with docker id d30b10211b1b; Security:[seccomp=unconfined]
14s 14s 1 {kubelet
ssantosh.centos7} spec.containers{HelloService} Normal Started Started
container with docker id d30b10211b1b 12s 11s 2 {kubelet
ssantosh.centos7} spec.containers{HelloService} Warning BackOff Back-off
restarting failed docker container 12s 11s 2 {kubelet
ssantosh.centos7} Warning FailedSync Error syncing pod, skipping:
failed to "StartContainer" for "HelloService" with CrashLoopBackOff:
"Back-off 10s restarting failed container=HelloService
pod=HelloService-522qw_default(1e951b45-7116-11e7-9b96-080027f96241)"
The issue was there is no java as a part of alpine image.
So modified the
FROM alpine:3.2
to
FROM anapsix/alpine-java
you need jdk on the machine also need to update the Dockerfile, remove the . infront of startService.sh command. like below
ENTRYPOINT /startService.sh
this will fix this error message.
/bin/sh: ./startService.sh: not found