I have a problem to deploy docker image via kubernetes.
One issue is that, we cannot use any docker image registry service e.g. docker hub or any cloud services. But, yes I have docker images as .tar file.
However, it always fails with following message
Warning Failed 1s kubelet, dell20
Failed to pull image "test:latest": rpc
error: code = Unknown
desc = failed to resolve image "docker.io/library/test:latest":
failed to do request: Head https://registry-1.docker.io/v2/library/test/manifests/latest: dial tcp i/o timeout
I also change deployment description with IfNotPresent or Never. In this case it will fail anyway with ErrImageNeverPull.
My guess is: kubernetes tries to use Docker Hub anyway, since it https://registry-1.docker.io in order to pull the image. I just want to use tar docker image in local disk, rather than pulling from some services.
And yes the image is in docker:
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
test latest 9f4916a0780c 6 days ago 1.72GB
Can anyone give me any advices on this problem?
I was successful with using local image with Kubernetes cluster. I provided the explanation with example below:
The only prerequisite is that you need to make sure you have access to upload this image directly to nodes.
Create the image
Pull the default nginx image from docker registry with below command:
$ docker pull nginx:1.17.5
Nginx image is used only for demonstration purposes.
Tag this image with new name as nginx-local with command:
$ docker tag nginx:1.17.5 nginx-local:1.17.5
Save this image as nginx-local.tar executing command:
$ docker save nginx-local:1.17.5 > nginx-local.tar
Link to documentation: docker save
File nginx-local.tar is used as your image.
Copy the image to all of the nodes
The problem with this technique is that you need to ensure all of the nodes have this image.
Lack of image will result in failed pod creation.
To copy it you can use scp. It's secure way to transer files between machines.
Example command for scp:
$ scp /path/to/your/file/nginx-local.tar user#ip_adddress:/where/you/want/it/nginx-local.tar
If image is already on the node, you will need to load it into local docker image repository with command:
$ docker load -i nginx-local.tar
To ensure that image is loaded invoke command
$ docker images | grep nginx-local
Link to documentation: docker load:
It should show something like that:
docker images | grep nginx
nginx-local 1.17.5 540a289bab6c 3 weeks ago 126MB
Creating deployment with local image
The last part is to create deployment with use of nginx-local image.
Please note that:
The image version is explicitly typed inside yaml file.
ImagePullPolicy is set to Never. ImagePullPolicy
Without this options the pod creation will fail.
Below is example deployment which uses exactly that image:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-local
namespace: default
spec:
selector:
matchLabels:
run: nginx-local
replicas: 5
template:
metadata:
labels:
run: nginx-local
spec:
containers:
- image: nginx-local:1.17.5
imagePullPolicy: Never
name: nginx-local
ports:
- containerPort: 80
Create this deployment with command:
$ kubectl create -f local-test.yaml
The result was that pods were created successfully as shown below:
NAME READY STATUS RESTARTS AGE
nginx-local-84ddb99b55-7vpvd 1/1 Running 0 2m15s
nginx-local-84ddb99b55-fgb2n 1/1 Running 0 2m15s
nginx-local-84ddb99b55-jlpz8 1/1 Running 0 2m15s
nginx-local-84ddb99b55-kzgw5 1/1 Running 0 2m15s
nginx-local-84ddb99b55-mc7rw 1/1 Running 0 2m15s
This operation was successful but I would recommend you to use local docker repository. It will easier management process with images and will be inside your infrastructure.
Link to documentation about it: Local Docker Registry
Related
I'm trying to run some docker images on Kubernetes.
docker images
master* $ docker images [15:16:49]
REPOSITORY TAG IMAGE ID CREATED SIZE
usm latest 4dd5245393bf About an hour ago 158MB
kuard latest 497961f486c7 4 days ago 22.9MB
docker container
master* $ docker ps [15:21:40]
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a46850d28303 usm "/docker-entrypoint.…" About an hour ago Up About an hour 0.0.0.0:6061->80/tcp, :::6061->80/tcp usm
88471e086486 gcr.io/k8s-minikube/kicbase:v0.0.32 "/usr/local/bin/entr…" 2 days ago Up 2 hours 127.0.0.1:49157->22/tcp, 127.0.0.1:49156->2376/tcp, 127.0.0.1:49155->5000/tcp, 127.0.0.1:49154->8443/tcp, 127.0.0.1:49153->32443/tcp minikube
Dockerfile
FROM nginx
COPY ./dist /usr/share/nginx/html
EXPOSE 80
kube version
master* $ minikube version [15:37:13]
minikube version: v1.26.0
commit: f4b412861bb746be73053c9f6d2895f12cf78565
When I run kubectl run mypod --image=usm, I get ErrImagePull
How to run the pod with the local docker image?
master* $ kubectl run mypod --image=usm
pod/mypod created
master* $ kubectl get pods [15:07:49]
NAME READY STATUS RESTARTS AGE
mypod 0/1 ErrImagePull 0 6s
I'm trying to set the imagePullPolicy to never
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- image: usm
imagePullPolicy: Never
name: mypod
ports:
- containerPort: 80
name: http
protocol: TCP
master* $ kubectl apply -f kube-pod-usm.yaml [15:55:39]
pod/mypod created
master* $ kubectl get pods [15:55:54]
NAME READY STATUS RESTARTS AGE
mypod 0/1 ErrImageNeverPull 0 42s
You need that image to be available in someplace. Normally this is done through a registry, but for local development, you can just upload your local image to your minikukbe cluster with the following command:
minikube image load image:tag
You may also want to check minikube docker-env which allows pointing your terminal docker-cli to the docker inside minikube in a simple way.
When a Kubernetes cluster creates a new deployment or updates an existing deployment, it needs to pull an image. This is done through the kubelet process on each user node. In order for kubelets to pull this image successfully, they must be accessible from all nodes in the cluster that match the scheduling request.
Edit pod specification and provide the correct registry
If you set the Image pull Policy to Never :
the kubelet does not try fetching the image. If the image is somehow already present locally (in the local registry of Kubernetes), the kubelet attempts to start the container; otherwise, startup fails.
I am trying to create a Kubernetes deployment from local docker images. And using imagePullPolicy as Never such that Kubernetes would pick it up from local docker image imported via tar.
Environment
SingleNodeMaster # one node deployment
But Kubernetes always trying to fetch the private repository although local docker images are present.
Any pointers on how to debug and resolve the issue such that Kubernetes would pick the images from the local docker registry? Thank you.
Steps performed
docker load -i images.tar
docker images # displays images from myprivatehub.com/nginx/nginx-custom:v1.1.8
kubectl create -f local-test.yaml with imagepullPolicy as Never
Error
Pulling pod/nginx-custom-6499765dbc-2fts2 Pulling image "myprivatehub.com/nginx/nginx-custom:v1.1.8"
Failed pod/nginx-custom-6499765dbc-2fts2 Error: ErrImagePull
Failed pod/nginx-custom-6499765dbc-2fts2 Failed to pull image "myprivatehub.com/nginx/nginx-custom:v1.1.8": rpc error: code = Unknown desc = failed to pull and unpack image "myprivatehub.com/nginx/nginx-custom:v1.1.8": failed to resolve reference "myprivatehub.com/nginx/nginx-custom:v1.1.8": failed to do request: Head "https://myprivatehub.com/v2/nginx/nginx-custom/manifests/v1.1.8": dial tcp: lookup myprivatehub.com: no such host
docker pull <imagename>
Error response from daemon: Get https://myprivatehub.com/v2/: dial tcp: lookup myprivatehub.com on 172.31.0.2:53: no such host
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-custom
namespace: default
spec:
selector:
matchLabels:
run: nginx-custom
replicas: 5
template:
metadata:
labels:
run: nginx-custom
spec:
containers:
- image: myprivatehub.com/nginx/nginx-custom:v1.1.8
imagePullPolicy: Never
name: nginx-custom
ports:
- containerPort: 80
This happens due to container runtime being different than docker. I am using containerd , after switching container runtime to docker , it started working.
This is to update another approach that can be taken to achieve the similar result. In this case, one can use Docker Registry. Docker Registry Doc
We can create a Docker registry on the machine where Kubernetes is running and docker too is installed. One of the easiest way to achieve the same can be done as following:
Create a local private docker registry. If the registry:2 image is not present, then it would download it and run.
sudo docker run -d -p 5000:5000 --restart=always --name registry registry:2
Build the image or load the image from a tar as required. For my example, i am creating it to add it to the local repository.
sudo docker build -t coolapp:v1 .
Once the build is done, create a tag with this image such that it represents a host and a port.
sudo docker tag coolapp:v1 localhost:5000/coolapp:v1
Push the new tag to the local private registry
sudo docker push localhost:5000/coolapp:v1
Now in the Kubernetes YAML, we can specify the deployment as following:
apiVersion: apps/v1
kind: Deployment
metadata:
name: mycoolapp
spec:
replicas: 1
selector:
matchLabels:
app: mycoolapp
template:
metadata:
labels:
app: mycoolapp
spec:
containers:
- name: mycoolapp
image: localhost:5000/coolapp:v1
ports:
- containerPort: 3000
and we apply the YAML
sudo kubectl apply -f deployment.yaml
Once this is done, we will be able to see that Kubernetes has pulled the image from the local private repository and is running it.
I've used helm create helloworld-chart to create an application using a local docker image I created. i think the issue is that i have the ports all messed up.
DOCKER PIECES
--------------------------
Docker File
FROM busybox
ADD index.html /www/index.html
EXPOSE 8008
CMD httpd -p 8008 -h /www; tail -f /dev/null
(I also have an index.html file in the same directory as my Dockerfile)
Create Docker Image (and publish locally)
docker build -t hello-world .
I then ran this with docker run -p 8080:8008 hello-world and verified I am able to reach it from localhost:8080. (I then stopped that docker container)
I also verified this image was in docker locally with docker image ls and got the output:
REPOSITORY TAG IMAGE ID CREATED SIZE
hello-world latest 8640a285e98e 20 minutes ago 1.23MB
HELM PIECES
--------------------------
Created a helm chart via helm create helloworld-chart.
Edited the files:
values.yaml
# ...elided because left the same as default...
image:
repository: hello-world
tag: latest
pullPolicy: IfNotPresent
# ...elided because left the same as default...
service:
name: hello-world
type: NodePort # Chose this because MiniKube doesn't have LoadBalancer installed
externalPort: 30007
internalPort: 8008
port: 80
service.yaml
# ...elided because left the same as default...
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.port }}
targetPort: {{ .Values.service.internalPort }}
nodePort: {{ .Values.service.externalPort }}
deployment.yaml
# ...elided because left the same as default...
spec:
# ...elided because left the same as default...
containers:
ports:
- name: http
containerPort: {{ .Values.service.internalPort }}
protocol: TCP
I verified this "looked" correct with both helm lint helloworld-chart and helm template ./helloworld-chart
HELM AND MINIKUBE COMMANDS
--------------------------
# Packaging my helm
helm package helloworld-chart
# Installing into Kuberneters (Minikube)
helm install helloworld helloworld-chart-0.1.0.tgz
# Getting an external IP
minikube service helloworld-helloworld-chart
When I do that, it gives me an external ip like http://172.23.13.145:30007 and opens in a browser but just says the site cannot be reached. What do i have mismatched?
UPDATE/MORE INFO
---------------------------------------
When I check the pod, it's in a CrashLoopBackOff state. However, I see nothing in the logs:
kubectl logs -f helloworld-helloworld-chart-6c886d885b-grfbc
Logs:
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
I'm not sure why it's exiting.
The issue was that Minikube was actually looking in the public Docker image repo and finding something also called hello-world. It was not finding my docker image since "local" to minikube is not local to the host computer's docker. Minikube has its own docker running internally.
You have to add your image to minikube's local repo: minikube cache add hello-world:latest.
You need to change the pull policy: imagePullPolicy: Never
I have this that is working...
# Set docker env
eval $(minikube docker-env)
# Build image
docker build -t jrg/hw .
# Run in minikube
kubectl run hello-world --image=jrg/hw:latest --image-pull-policy=Never --port=8080
kubectl expose deployment hello-world --type=NodePort --name=hello-service
I can access the endpoint just as I expect. Now I am trying to use a .yml file to deploy like this...
apiVersion: v1
kind: Pod
metadata:
name: hello-world-dev
labels:
purpose: simple
spec:
containers:
- name: hello-world-dev-container
image: jrg/hw:latest
env:
- name: WORKING
value: "Yup Working"
But when I run kubectl apply -f k8s/ineject/dev.envvars.yml I get...
NAME READY STATUS RESTARTS AGE
hello-world-7d87b8ddd5-gqr8k 1/1 Running 1 2d22h
hello-world-dev 0/1 ErrImagePull 0 6s
So why can one see my local docker to get the image and 1 has an issue?
In the docs regarding Pre-pulling Images we can read:
By default, the kubelet will try to pull each image from the specified registry. However, if the imagePullPolicy property of the container is set to IfNotPresent or Never, then a local image is used (preferentially or exclusively, respectively).
Also please see other options with imagePullPolicy in Container Images docs.
The imagePullPolicy and the tag of the image affect when the kubelet attempts to pull the specified image.
imagePullPolicy: IfNotPresent: the image is pulled only if it is not already present locally.
imagePullPolicy: Always: the image is pulled every time the pod is started.
imagePullPolicy is omitted and either the image tag is :latest or it is omitted: Always is applied.
imagePullPolicy is omitted and the image tag is present but not :latest: IfNotPresent is applied.
imagePullPolicy: Never: the image is assumed to exist locally. No attempt is made to pull the image.
I have successfully built Docker images and ran them in a Docker swarm. When I attempt to build an image and run it with Docker Desktop's Kubernetes cluster:
docker build -t myimage -f myDockerFile .
(the above successfully creates an image in the docker local registry)
kubectl run myapp --image=myimage:latest
(as far as I understand, this is the same as using the kubectl create deployment command)
The above command successfully creates a deployment, but when it makes a pod, the pod status always shows:
NAME READY STATUS RESTARTS AGE
myapp-<a random alphanumeric string> 0/1 ImagePullBackoff 0 <age>
I am not sure why it is having trouble pulling the image - does it maybe not know where the docker local images are?
I just had the exact same problem. Boils down to the imagePullPolicy:
PC:~$ kubectl explain deployment.spec.template.spec.containers.imagePullPolicy
KIND: Deployment
VERSION: extensions/v1beta1
FIELD: imagePullPolicy <string>
DESCRIPTION:
Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always
if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated.
More info:
https://kubernetes.io/docs/concepts/containers/images#updating-images
Specifically, the part that says: Defaults to Always if :latest tag is specified.
That means, you created a local image, but, because you use the :latest it will try to find it in whatever remote repository you configured (by default docker hub) rather than using your local. Simply change your command to:
kubectl run myapp --image=myimage:latest --image-pull-policy Never
or
kubectl run myapp --image=myimage:latest --image-pull-policy IfNotPresent
I had this same ImagePullBack error while running a pod deployment with a YAML file, also on Docker Desktop.
For anyone else that finds this via Google (like I did), the imagePullPolicy that Lucas mentions above can also be set in the deployment yaml file. See the spec.templage.spec.containers.imagePullPolicy in the yaml snippet below (3 lines from the bottom).
I added that and my app deployed successfully into my local kube cluser, using the kubectl yaml deploy command: kubectl apply -f .\Deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app-deployment
labels:
app: web-app
spec:
replicas: 3
selector:
matchLabels:
app: web-app
template:
metadata:
labels:
app: web-app
spec:
containers:
- name: web-app
image: node-web-app:latest
imagePullPolicy: Never
ports:
- containerPort: 3000
You didn't specify where myimage:latest is hosted, but essentially ImagePullBackoff means that I cannot pull the image because either:
You don't have networking setup in your Docker VM that can get to your Docker registry (Docker Hub?)
myimage:latest doesn't exist in your registry or is misspelled.
myimage:latest requires credentials (you are pulling from a private registry). You can take a look at this to configure container credentials in a Pod.