I'm building docker images via jib and skaffold. When I provide the image key service-1 in skaffold build.artifacts.image, an image is created (Image named 'service-1:SHA256') and I can see that through docker image ls. Then in skaffold deploy, I provide kubectl commands, where its picking up the image created just fine. In my deployment.yaml manifest file, I specify just spec.template.spec.containers.image : service-1
The above method is working fine, but I want to do helm. I have created helm charts. But helm is not able to pickup any image from my local docker daemon repo. If I specify image:service-1:SHA256 that's present when I see docker image ls, I get ErrImagePull.
Why isn't helm able to pick the image from my local docker registry. I'm also not able to docker pull service-1:sha256.
What's the default docker registry that's being used, and how is skaffold with kubectl able to pull the image correctly, while helm and docker cannot.
I tried skaffold+helm also, but faced the same issue.
apiVersion: skaffold/v2alpha3
kind: Config
metadata:
name: service-1
build:
artifacts:
- image: service-1
jib:
project: com.sample.app:service-1
args:
- -Dmaven.test.skip
deploy:
helm:
releases:
- name: service-1
chartPath: service-1-chart
values:
image: service-1
valuesFiles:
- service-1-chart/values.yaml
Getting the following during pod describe
Normal Scheduled 75s default-scheduler Successfully assigned default/service1-df75c747b-g6qfm to docker-desktop
Normal Pulling 29s (x3 over 74s) kubelet, docker-desktop Pulling image "service-1:e7d693388f7f26a062257408e3cfb5904da4749342729802eb9ac65fab6d2f31"
Warning Failed 27s (x3 over 73s) kubelet, docker-desktop Failed to pull image "service-1:e7d693388f7f26a062257408e3cfb5904da4749342729802eb9ac65fab6d2f31": rpc error: code = Unknown desc = Error response from daemon: pull access denied for service-1, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
Warning Failed 27s (x3 over 73s) kubelet, docker-desktop Error: ErrImagePull
Normal BackOff 2s (x4 over 72s) kubelet, docker-desktop Back-off pulling image "service-1:e7d693388f7f26a062257408e3cfb5904da4749342729802eb9ac65fab6d2f31"
Warning Failed 2s (x4 over 72s) kubelet, docker-desktop Error: ImagePullBackOff
When I try to do docker pull:
Error response from daemon: pull access denied for service-1, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
If you don't want to use Docker registry, but to run the image directly from your Docker Daemon, then add imagePullPolicy: Never.
Related
I'm new with k8s and I've deployed a simple nginx deployment using eks and ecr. The problem only present when I'm using ecr image, it's not happens with nginx public image from docker hub.
Here is my deployment file.
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
imagePullSecrets:
- name: *-ecr-registry
containers:
- name: nginx
# image: nginx:latest
image: *.dkr.ecr.*.amazonaws.com/nginx:dev
Here is output of a pod.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 12s default-scheduler Successfully assigned *
Normal Pulled 10s (x2 over 12s) kubelet Container image * already present on machine
Normal Created 10s (x2 over 12s) kubelet Created container nginx
Normal Started 10s (x2 over 11s) kubelet Started container nginx
Warning BackOff 8s (x2 over 9s) kubelet Back-off restarting failed container
Here is result log of the pod
exec /docker-entrypoint.sh: exec format error
I also ran the ecr image with docker in my local machine without any error.
Can someone give me some hints, how can I overcome this issue, thanks in advance.
The requested image's platform (linux/amd64) does not match the
detected host platform (linux/arm64/v8) and no specific platform was
requested
it's due to you have built the Docker image on an ARM M1 chip either on Mac and try to run it on AMD or the opposite due to that it's throwing an error.
You can build the docker image again by passing ARG --platform
--platform linux/amd64
Or you can also set the Environment variable export DOCKER_DEFAULT_PLATFORM=linux/amd64
You can pass ARG with FROM in Dockerfile or while running command docker build --platform
Dockerfile
FROM --platform=linux/amd64 node
ARG with build command
docker build -t <image-name> --platform linux/amd64
ref doc: https://docs.docker.com/engine/reference/builder/#from
You can also run the image with docker like
docker run --platform linux/amd64 node
After hours of trying many ways, I realize that I'm using base-image keymetrics/pm2:18-alpine only supports amd64 kernel. Even I tried to rebuild my image in arm64, that doesn't not work in my work nodes with arm64 kernel type. So I see 2 ways to overcome this issue is:
recreate my eks cluster using amd64 worker nodes.
Or switch to other base-image like node:18.14-alpine that supports both arm64 and amd64 kernel.
I'm trying to run some docker images on Kubernetes.
docker images
master* $ docker images [15:16:49]
REPOSITORY TAG IMAGE ID CREATED SIZE
usm latest 4dd5245393bf About an hour ago 158MB
kuard latest 497961f486c7 4 days ago 22.9MB
docker container
master* $ docker ps [15:21:40]
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a46850d28303 usm "/docker-entrypoint.…" About an hour ago Up About an hour 0.0.0.0:6061->80/tcp, :::6061->80/tcp usm
88471e086486 gcr.io/k8s-minikube/kicbase:v0.0.32 "/usr/local/bin/entr…" 2 days ago Up 2 hours 127.0.0.1:49157->22/tcp, 127.0.0.1:49156->2376/tcp, 127.0.0.1:49155->5000/tcp, 127.0.0.1:49154->8443/tcp, 127.0.0.1:49153->32443/tcp minikube
Dockerfile
FROM nginx
COPY ./dist /usr/share/nginx/html
EXPOSE 80
kube version
master* $ minikube version [15:37:13]
minikube version: v1.26.0
commit: f4b412861bb746be73053c9f6d2895f12cf78565
When I run kubectl run mypod --image=usm, I get ErrImagePull
How to run the pod with the local docker image?
master* $ kubectl run mypod --image=usm
pod/mypod created
master* $ kubectl get pods [15:07:49]
NAME READY STATUS RESTARTS AGE
mypod 0/1 ErrImagePull 0 6s
I'm trying to set the imagePullPolicy to never
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- image: usm
imagePullPolicy: Never
name: mypod
ports:
- containerPort: 80
name: http
protocol: TCP
master* $ kubectl apply -f kube-pod-usm.yaml [15:55:39]
pod/mypod created
master* $ kubectl get pods [15:55:54]
NAME READY STATUS RESTARTS AGE
mypod 0/1 ErrImageNeverPull 0 42s
You need that image to be available in someplace. Normally this is done through a registry, but for local development, you can just upload your local image to your minikukbe cluster with the following command:
minikube image load image:tag
You may also want to check minikube docker-env which allows pointing your terminal docker-cli to the docker inside minikube in a simple way.
When a Kubernetes cluster creates a new deployment or updates an existing deployment, it needs to pull an image. This is done through the kubelet process on each user node. In order for kubelets to pull this image successfully, they must be accessible from all nodes in the cluster that match the scheduling request.
Edit pod specification and provide the correct registry
If you set the Image pull Policy to Never :
the kubelet does not try fetching the image. If the image is somehow already present locally (in the local registry of Kubernetes), the kubelet attempts to start the container; otherwise, startup fails.
I have a problem to deploy docker image via kubernetes.
One issue is that, we cannot use any docker image registry service e.g. docker hub or any cloud services. But, yes I have docker images as .tar file.
However, it always fails with following message
Warning Failed 1s kubelet, dell20
Failed to pull image "test:latest": rpc
error: code = Unknown
desc = failed to resolve image "docker.io/library/test:latest":
failed to do request: Head https://registry-1.docker.io/v2/library/test/manifests/latest: dial tcp i/o timeout
I also change deployment description with IfNotPresent or Never. In this case it will fail anyway with ErrImageNeverPull.
My guess is: kubernetes tries to use Docker Hub anyway, since it https://registry-1.docker.io in order to pull the image. I just want to use tar docker image in local disk, rather than pulling from some services.
And yes the image is in docker:
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
test latest 9f4916a0780c 6 days ago 1.72GB
Can anyone give me any advices on this problem?
I was successful with using local image with Kubernetes cluster. I provided the explanation with example below:
The only prerequisite is that you need to make sure you have access to upload this image directly to nodes.
Create the image
Pull the default nginx image from docker registry with below command:
$ docker pull nginx:1.17.5
Nginx image is used only for demonstration purposes.
Tag this image with new name as nginx-local with command:
$ docker tag nginx:1.17.5 nginx-local:1.17.5
Save this image as nginx-local.tar executing command:
$ docker save nginx-local:1.17.5 > nginx-local.tar
Link to documentation: docker save
File nginx-local.tar is used as your image.
Copy the image to all of the nodes
The problem with this technique is that you need to ensure all of the nodes have this image.
Lack of image will result in failed pod creation.
To copy it you can use scp. It's secure way to transer files between machines.
Example command for scp:
$ scp /path/to/your/file/nginx-local.tar user#ip_adddress:/where/you/want/it/nginx-local.tar
If image is already on the node, you will need to load it into local docker image repository with command:
$ docker load -i nginx-local.tar
To ensure that image is loaded invoke command
$ docker images | grep nginx-local
Link to documentation: docker load:
It should show something like that:
docker images | grep nginx
nginx-local 1.17.5 540a289bab6c 3 weeks ago 126MB
Creating deployment with local image
The last part is to create deployment with use of nginx-local image.
Please note that:
The image version is explicitly typed inside yaml file.
ImagePullPolicy is set to Never. ImagePullPolicy
Without this options the pod creation will fail.
Below is example deployment which uses exactly that image:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-local
namespace: default
spec:
selector:
matchLabels:
run: nginx-local
replicas: 5
template:
metadata:
labels:
run: nginx-local
spec:
containers:
- image: nginx-local:1.17.5
imagePullPolicy: Never
name: nginx-local
ports:
- containerPort: 80
Create this deployment with command:
$ kubectl create -f local-test.yaml
The result was that pods were created successfully as shown below:
NAME READY STATUS RESTARTS AGE
nginx-local-84ddb99b55-7vpvd 1/1 Running 0 2m15s
nginx-local-84ddb99b55-fgb2n 1/1 Running 0 2m15s
nginx-local-84ddb99b55-jlpz8 1/1 Running 0 2m15s
nginx-local-84ddb99b55-kzgw5 1/1 Running 0 2m15s
nginx-local-84ddb99b55-mc7rw 1/1 Running 0 2m15s
This operation was successful but I would recommend you to use local docker repository. It will easier management process with images and will be inside your infrastructure.
Link to documentation about it: Local Docker Registry
I have this that is working...
# Set docker env
eval $(minikube docker-env)
# Build image
docker build -t jrg/hw .
# Run in minikube
kubectl run hello-world --image=jrg/hw:latest --image-pull-policy=Never --port=8080
kubectl expose deployment hello-world --type=NodePort --name=hello-service
I can access the endpoint just as I expect. Now I am trying to use a .yml file to deploy like this...
apiVersion: v1
kind: Pod
metadata:
name: hello-world-dev
labels:
purpose: simple
spec:
containers:
- name: hello-world-dev-container
image: jrg/hw:latest
env:
- name: WORKING
value: "Yup Working"
But when I run kubectl apply -f k8s/ineject/dev.envvars.yml I get...
NAME READY STATUS RESTARTS AGE
hello-world-7d87b8ddd5-gqr8k 1/1 Running 1 2d22h
hello-world-dev 0/1 ErrImagePull 0 6s
So why can one see my local docker to get the image and 1 has an issue?
In the docs regarding Pre-pulling Images we can read:
By default, the kubelet will try to pull each image from the specified registry. However, if the imagePullPolicy property of the container is set to IfNotPresent or Never, then a local image is used (preferentially or exclusively, respectively).
Also please see other options with imagePullPolicy in Container Images docs.
The imagePullPolicy and the tag of the image affect when the kubelet attempts to pull the specified image.
imagePullPolicy: IfNotPresent: the image is pulled only if it is not already present locally.
imagePullPolicy: Always: the image is pulled every time the pod is started.
imagePullPolicy is omitted and either the image tag is :latest or it is omitted: Always is applied.
imagePullPolicy is omitted and the image tag is present but not :latest: IfNotPresent is applied.
imagePullPolicy: Never: the image is assumed to exist locally. No attempt is made to pull the image.
Im trying to create a pod using my local docker image as follow.
1.First I run this command in terminal
eval $(minikube docker-env)
2.I created a docker image as follow
sudo docker image build -t my-first-image:3.0.0 .
3.I created the pod.yml as shown below and I run this command
kubectl -f create pod.yml.
4.then i tried to run this command
kubectl get pods
but it shows following error
NAME READY STATUS RESTARTS AGE
multiplication-b47499db9-phpb7 0/1 ImagePullBackOff 0 23h
my-first-pod 0/1 ErrImagePull 0 7s
5.i get the pods logs
kubectl describe pod my-first-pod
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 99s default-scheduler Successfully assigned default/my-first-pod to minikube
Warning Failed 41s (x3 over 94s) kubelet, minikube Failed to pull image "my-first-image:3.0.0": rpc error: code = Unknown desc = Error response from daemon: pull access denied for my-first-image, repository does not exist or may require 'docker login'
Warning Failed 41s (x3 over 94s) kubelet, minikube Error: ErrImagePull
Normal BackOff 12s (x4 over 93s) kubelet, minikube Back-off pulling image "my-first-image:3.0.0"
Warning Failed 12s (x4 over 93s) kubelet, minikube Error: ImagePullBackOff
Normal Pulling 0s (x4 over 98s) kubelet, minikube pulling image "my-first-image:3.0.0"
Dockerfile
FROM node:carbon
WORKDIR /app
COPY . .
CMD [ "node", "index.js" ]
pods.yml
kind: Pod
apiVersion: v1
metadata:
name: my-first-pod
spec:
containers:
- name: my-first-container
image: my-first-image:3.0.0
index.js
var http = require('http');
var server = http.createServer(function(request, response) {
response.statusCode = 200;
response.setHeader('Content-Type', 'text/plain');
response.end('Welcome to the Golden Guide to Kubernetes
Application Development!');
});
server.listen(3000, function() {
console.log('Server running on port 3000');
});
Reason
This is because it can't download the docker image defined in your pod definition file. By default it downloads required images from DockerHub.
Way 1
So after creating your my-first-image:3.0.0 image you have to publish it at DockerHub. For that create an account at DockerHub and login from terminal using login command
sudo docker login
After successful login, rebuild your docker image with your DockerHub username in tag and push it to DockerHub (more details)
sudo docker image build -t YOUR_DOCKERHUB_USERNAME/my-first-image:3.0.0 .
sudo docker push YOUR_DOCKERHUB_USERNAME/my-first-image:3.0.0
Update your image in pod.yml as YOUR_DOCKERHUB_USERNAME/my-first-image:3.0.0 and create your pods as before.
Way 2
You can instruct to find required docker image from your local machine instead of downloading from DockerHub. To do so you have to add imagePullPolicy: Never in your pod.yml file under specific container description. Below is an example of your pod.yml file to show where to define image pull policy
kind: Pod
apiVersion: v1
metadata:
name: my-first-pod
spec:
containers:
- name: my-first-container
image: YOUR_DOCKERHUB_USERNAME/my-first-image:3.0.0
imagePullPolicy: Never
Since you are using an image without uploading it .You will have to set the imagePullPolicy to Never, otherwise Kubernetes will try to download the image.
Start minikube
minikube start
Set docker env
eval $(minikube docker-env)
Build image
docker build -t my-first-image:3.0.0 .
Run in minikube
kubectl run my-first-container --image=my-first-image:3.0.0 --image-pull-policy=Never
Check that it's running
kubectl get pods
Your pod spec should be like below
pods.yml
kind: Pod
apiVersion: v1
metadata:
name: my-first-pod
spec:
containers:
- name: my-first-container
image: my-first-image:3.0.0
imagePullPolicy: Never
You have build the image but you need to push to docker repository.
In case you are building docker image on kubernetes node, it can work but better to push to repository so docker will pull from central repo, as the number of nodes increases you cant control where pod will invoke and image will be unavailable.
docker push -t my-first-image:3.0.0
In case you have private docker repositiory, push to private repository and use full qualified image name.
repository name is missing.
create an account in dockerHub.
then build and push the image to your repo in dockerhub
sudo docker image build -t niranga/my-first-image:3.0.0 .
sudo docker login
sudo docker push niranga/my-first-image:3.0.0
update the image name in pod.yaml file as niranga/my-first-image:3.0.0
it should work
When you build image to local Minikube registry you might want to set imagePullPolicy: Never so that it will not try pulling the image from remote registry.
The pod spec would look something like that:
kind: Pod
apiVersion: v1
metadata:
name: my-first-pod
spec:
containers:
- name: my-first-container
image: my-first-image:3.0.0
imagePullPolicy: Never