Kubernetes does not pull docker image from private repository without https - docker

I configured docker on the same host as my kubernetes-master for the private docker registry.Docker pushing to the private docker registry without https was successful. I also can pull the image just using docker.
When I run kubernetes for this image, I get with 'kubectl describe pods' following log :
kubectl describe pods
Name: fgpra-250514157-yh6vb
Namespace: default
Node: 5.179.232.64/5.179.232.64
Start Time: Tue, 11 Oct 2016 18:06:59 +0200
Labels: pod-template-hash=250514157,run=fgpra
Status: Pending
IP: <removed myself>
Controllers: ReplicaSet/fgpra-250514157
Containers:
fgpra:
Container ID:
Image: 5.179.232.65:5000/some_api_image
Image ID:
Port: 3000/TCP
QoS Tier:
cpu: BestEffort
memory: BestEffort
State: Waiting
Reason: ErrImagePull
Ready: False
Restart Count: 0
Environment Variables:
Conditions:
Type Status
Ready False
Volumes:
default-token-q7u3x:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-q7u3x
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
4s 4s 1 {default-scheduler } Normal Scheduled Successfully assigned fgpra-250514157-yh6vb to 5.179.232.64
4s 4s 1 {kubelet 5.179.232.64} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
4s 4s 1 {kubelet 5.179.232.64} spec.containers{fgpra} Normal Pulling pulling image "5.179.232.65:5000/some_api_image"
4s 4s 1 {kubelet 5.179.232.64} spec.containers{fgpra} Warning Failed Failed to pull image "5.179.232.65:5000/some_api_image": unable to ping registry endpoint https://5.179.232.65:5000/v0/
v2 ping attempt failed with error: Get https://5.179.232.65:5000/v2/: http: server gave HTTP response to HTTPS client
v1 ping attempt failed with error: Get https://5.179.232.65:5000/v1/_ping: http: server gave HTTP response to HTTPS client
4s 4s 1 {kubelet 5.179.232.64} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "fgpra" with ErrImagePull: "unable to ping registry endpoint https://5.179.232.65:5000/v0/\nv2 ping attempt failed with error: Get https://5.179.232.65:5000/v2/: http: server gave HTTP response to HTTPS client\n v1 ping attempt failed with error: Get https://5.179.232.65:5000/v1/_ping: http: server gave HTTP response to HTTPS client"
3s 3s 1 {kubelet 5.179.232.64} spec.containers{fgpra} Normal BackOff Back-off pulling image "5.179.232.65:5000/some_api_image"
3s 3s 1 {kubelet 5.179.232.64} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "fgpra" with ImagePullBackOff: "Back-off pulling image \"5.179.232.65:5000/some_api_image\""
I already configured my /etc/init.d/sysconfig/docker to use my insecure private registry.
This is the command to start the kubernetes deployment :
kubectl run fgpra --image=5.179.232.65:5000/some_api_image --port=3000
How can I set kubernetes to pull from my private docker registry without using ssl?

This rather a docker issue than a kubernetes one. You need to add your http registry as a insecure-registry to your docker daemon on each kubernetes node.
docker daemon --insecure-registry=5.179.232.65:5000
In most environment there is a file like /etc/default/docker where you can add this parameter.

Related

kubernetes unable to pull image docker private registry

I tried to deploy 'deployments' in kubernetes which is pull docker image from private registry (I don't know who did this setup) but during "docker pull images" through kubernetes i'm getting following error
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 85s default-scheduler Successfully assigned default/trusted-enc-assettag1-deployment-8467b74958-6fbp7 to k8s-node
Normal BackOff 24s (x2 over 61s) kubelet, k8s-node Back-off pulling image "10.105.168.81:5000/simplehttpserverenc:enc_v1"
Warning Failed 24s (x2 over 61s) kubelet, k8s-node Error: ImagePullBackOff
Normal Pulling 12s (x3 over 82s) kubelet, k8s-node Pulling image "10.105.168.81:5000/simplehttpserverenc:enc_v1"
Warning Failed 0s (x3 over 62s) kubelet, k8s-node Failed to pull image "10.105.168.81:5000/simplehttpserverenc:enc_v1": rpc error: code = Unknown desc = Error response from daemon: Get https://10.105.168.81:5000/v2/: net/http: TLS handshake timeout
Warning Failed 0s (x3 over 62s) kubelet, k8s-node Error: ErrImagePull
[root#k8s-master ~]# docker pull 10.105.168.81:5000/simplehttpserverenc:enc_v1
ImagePullBackOff and net/http: TLS handshake timeout error.
Initially this "net/http: TLS handshake timeout" error is observed in docker pull as well. I referred some answers and
configured certificate(/etc/docker/certs.d//ca.crt ) and
proxy (/etc/systemd/system/docker.service.d/proxy.conf)
after that able to perform docker pull from private image.
[root#k8s-master ~]# docker pull 10.105.168.81:5000/simplehttpserverenc:enc_v1
enc_v1: Pulling from simplehttpserverenc
54fec2fa59d0: Pull complete
cd3f35d84cab: Pull complete
a0afc8e92ef0: Pull complete
9691f23efdb7: Pull complete
6512e60b314b: Pull complete
a8ac6632d329: Pull complete
68f4c4e0aa8c: Pull complete
Digest: sha256:0358708cd11e96f6cf6f22b29d46a8eec50d7107597b866e1616a73a198fe797
Status: Downloaded newer image for 10.105.168.81:5000/simplehttpserverenc:enc_v1
10.105.168.81:5000/simplehttpserverenc:enc_v1
[root#k8s-master ~]#
But still unable to perform this docker pull through kubernetes. How to solve this ?
If you use docker as container engine in your k8s, AFAIK it's the same with Understand the configuration. Because the image pulling is conducted by the container engine and it depends the proprietary configuration of each one on the certificates. How about pulling the same image on the worker node in your k8s ? Is it possible to pull the one without errors ?
As your dockerconfigjson is not working properly. Try this method :
kubectl create secret docker-registry regcred --docker-server=10.105.168.81:5000 --docker-username=<your-name> --docker-password=<your-pword>
And in Kubernetes manifest :
apiVersion: v1
kind: Pod
metadata:
name: private-reg
spec:
containers:
- name: private-reg-container
image: 10.105.168.81:5000/simplehttpserverenc:enc_v1
imagePullSecrets:
- name: regcred
I had encounted this many times, when I forgot to configure these secrets. Also if you have any othernamespace, you will have to generate secrets for each of these namespaces separately passing -n <your-ns> to above kubectl create secret
Edit : As you can not pull the image from worker node.
Make sure you copied docker-registry ca.crt to /etc/docker/certs.d/ca.crt
and then try docker pull .

Kubectl create deployment fails to pull image from local docker repo connection refused

I am encountering a very basic error:
I have docker-desktop and minikube setup on my windows 10 machine.
Further I setup a local docker registry using the steps here.
Here is what I have when I run docker-ps :
c:\>docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6675d2c57a74 registry:2 "/entrypoint.sh /etc…" 7 hours ago Up 2 hours 0.0.0.0:5000->5000/tcp registry
d05edc8f05b0 gcr.io/k8s-minikube/kicbase:v0.0.10 "/usr/local/bin/entr…" 8 hours ago Up 8 hours 127.0.0.1:32771->22/tcp, 127.0.0.1:32770->2376/tcp, 127.0.0.1:32769->5000/tcp, 127.0.0.1:32768->8443/tcp minikube
I pushed my local image to my local registry using docker push, and I deleted the local image using docker image remove command to avoid any confusion.
I now tried pulling the local registry image to see if it works, and it does
docker pull localhost:5000/dev/my-web:v1
v1: Pulling from dev/my-web
Digest: sha256:b3a0cf5c66ade8d39709c0cbbd0e08c9cc5f5e1c97f039a2bd1afed657dc8b74
Status: Downloaded newer image for localhost:5000/dev/my-web:v1
localhost:5000/dev/my-web:v1
Now I run my kubectl create commands and they fail with the error connection refused.
C:\>kubectl create deployment myweb --image=localhost:5000/dev/my-web:v1
deployment.apps/myweb created
C:\>kubectl get pods
NAME READY STATUS RESTARTS AGE
myweb-7d467f4bc4-r9xhc 0/1 ErrImagePull 0 12s
C:\>kubectl describe pod/myweb-7d467f4bc4-r9xhc
Name: myweb-7d467f4bc4-r9xhc
Namespace: default
Priority: 0
Node: minikube/172.17.0.3
Start Time: Sun, 09 Aug 2020 23:30:07 -0400
Labels: app=myweb
pod-template-hash=7d467f4bc4
Annotations: <none>
Status: Pending
IP: 172.18.0.3
IPs:
IP: 172.18.0.3
Controlled By: ReplicaSet/myweb-7d467f4bc4
Containers:
my-web:
Container ID:
Image: localhost:5000/dev/my-web:v1
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-nr7vj (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-nr7vj:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-nr7vj
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 93s default-scheduler Successfully assigned default/myweb-7d467f4bc4-r9xhc to minikube
Normal Pulling 53s (x3 over 92s) kubelet, minikube Pulling image "localhost:5000/dev/my-web:v1"
Warning Failed 53s (x3 over 92s) kubelet, minikube Failed to pull image "localhost:5000/dev/my-web:v1": rpc error: code = Unknown desc = Error response from daemon: Get http://localhost:5000/v2/: dial tcp 127.0.0.1:5000: connect: connection refused
Warning Failed 53s (x3 over 92s) kubelet, minikube Error: ErrImagePull
Normal BackOff 13s (x5 over 91s) kubelet, minikube Back-off pulling image "localhost:5000/dev/my-web:v1"
Warning Failed 13s (x5 over 91s) kubelet, minikube Error: ImagePullBackOff
I am not able to figure out why am I getting connection refused error when I am trying through kubectl command as against docker pull command.
Please help.
Additional notes: (1) I am using in-built windows hypervisor (2) Using default networking
Well, Kubernetes cannot find your registry. It depends on the setup you have, but Docker for Windows runs in a VM and you didn't specify how you are running minikube, but most likely in another VM. So potentially here you have two VMs that can or cannot talk to each depending on how you set them up.
And localhost is almost never going to work with Kubernetes because that always resolves to the local IP of your Kubernetes node when it comes to pulling the image. That means that you would have to have your registry and the kubelet pulling the image on the exact same VM.
I would just focus on making it work with the VM IP address where your registry is running and make sure that your Kubernetes VM can reach the registry VM IP address. Also, remember that when you run your registry you also have to expose you container port in this case 5000
P.S. You didn't specify what Hypervisor you are running? VirtualBox? VMware? are you using bridged networking? host only❓
✌️

kubectl can't connect to docker registry to download image

I'm stepping through Kubernetes in Action to get more than just familiarity with Kubernetes.
I already had a Docker Hub account that I've been using for Docker-specific experiments.
As described in chapter 2 of the book, I built the toy "kubia" image, and I was able to push it to Docker Hub. I verified this again by logging into Docker Hub and seeing the image.
I'm doing this on Centos7.
I then run the following to create the replication controller and pod running my image:
kubectl run kubia --image=davidmichaelkarr/kubia --port=8080 --generator=run/v1
I waited a while for statuses to change, but it never finishes downloading the image, when I describe the pod, I see something like this:
Normal Scheduled 24m default-scheduler Successfully assigned kubia-25th5 to minikube
Normal SuccessfulMountVolume 24m kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-x5nl4"
Normal Pulling 22m (x4 over 24m) kubelet, minikube pulling image "davidmichaelkarr/kubia"
Warning Failed 22m (x4 over 24m) kubelet, minikube Failed to pull image "davidmichaelkarr/kubia": rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
So I then constructed the following command:
curl -v -u 'davidmichaelkarr:**' 'https://registry-1.docker.io/v2/'
Which uses the same password I use for Docker Hub (they should be the same, right?).
This gives me the following:
* About to connect() to proxy *** port 8080 (#0)
* Trying **.**.**.**...
* Connected to *** (**.**.**.**) port 8080 (#0)
* Establish HTTP proxy tunnel to registry-1.docker.io:443
* Server auth using Basic with user 'davidmichaelkarr'
> CONNECT registry-1.docker.io:443 HTTP/1.1
> Host: registry-1.docker.io:443
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject: CN=*.docker.io
* start date: Aug 02 00:00:00 2017 GMT
* expire date: Sep 02 12:00:00 2018 GMT
* common name: *.docker.io
* issuer: CN=Amazon,OU=Server CA 1B,O=Amazon,C=US
* Server auth using Basic with user 'davidmichaelkarr'
> GET /v2/ HTTP/1.1
> Authorization: Basic ***
> User-Agent: curl/7.29.0
> Host: registry-1.docker.io
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: application/json; charset=utf-8
< Docker-Distribution-Api-Version: registry/2.0
< Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io"
< Date: Wed, 24 Jan 2018 18:34:39 GMT
< Content-Length: 87
< Strict-Transport-Security: max-age=31536000
<
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}
* Connection #0 to host *** left intact
I don't understand why this is failing auth.
Update:
Based on the first answer and the info I got from this other question, I edited the description of the service account, adding the "imagePullSecrets" key, then I deleted the replicationcontroller again and recreated it. The result appeared to be identical.
This is the command I ran to create the secret:
kubectl create secret docker-registry regsecret --docker-server=registry-1.docker.io --docker-username=davidmichaelkarr --docker-password=** --docker-email=**
Then I obtained the yaml for the serviceaccount, added the key reference for the secret, then set that yaml as the settings for the serviceaccount.
This are the current settings for the service account:
$ kubectl get serviceaccount default -o yaml
apiVersion: v1
imagePullSecrets:
- name: regsecret
kind: ServiceAccount
metadata:
creationTimestamp: 2018-01-24T00:05:01Z
name: default
namespace: default
resourceVersion: "81492"
selfLink: /api/v1/namespaces/default/serviceaccounts/default
uid: 38e2882c-009a-11e8-bf43-080027ae527b
secrets:
- name: default-token-x5nl4
Here's the updated events list from the describe of the pod after doing this:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 7m default-scheduler Successfully assigned kubia-f56th to minikube
Normal SuccessfulMountVolume 7m kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-x5nl4"
Normal Pulling 5m (x4 over 7m) kubelet, minikube pulling image "davidmichaelkarr/kubia"
Warning Failed 5m (x4 over 7m) kubelet, minikube Failed to pull image "davidmichaelkarr/kubia": rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Normal BackOff 4m (x6 over 7m) kubelet, minikube Back-off pulling image "davidmichaelkarr/kubia"
Warning FailedSync 2m (x18 over 7m) kubelet, minikube Error syncing pod
What else might I be doing wrong?
Update:
I think it's likely that all these issues with authentication are unrelated to the real issue. The key point is what I see in the pod description (breaking into multiple lines to make it easier to see):
Warning Failed 22m (x4 over 24m) kubelet,
minikube Failed to pull image "davidmichaelkarr/kubia": rpc error: code =
Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/:
net/http: request canceled while waiting for connection
(Client.Timeout exceeded while awaiting headers)
The last line seems like the most important piece of information at this point. It's not failing authentication, it's timing out the connection. In my experience, something like this is usually caused by issues getting through a firewall/proxy. We do have an internal proxy, and I have those environment variables set in my environment, but what about the "serviceaccount" that kubectl is using to make this connection? Do I have to somehow set a proxy configuration in the serviceaccount description?
You need to make sure the Docker daemon running in the Minikube VM uses your corporate proxy by starting minikube along these lines:
minikube start --docker-env http_proxy=http://proxy.corp.com:port --docker-env https_proxy=http://proxy.corp.com:port --docker-env no_proxy=192.168.99.0/24
I faced same issue couple of time.
Updating here, might be useful for someone.
First describe the POD(kubectl describe pod <pod_name>),
1. If you see access denied/repository does not exist errors like
Error response from daemon: pull access denied for test/nginx,
repository does not exist or may require 'docker login': denied:
requested access to the resource is denied
Solution:
If local K8s, you need to login into docker registry first OR
if Kubernetes Cluster on Cloud, create secret for Registry and add imagepullsecret
along with secret name
2. If you get timeout error,
Error: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while
awaiting headers)
Solution:
check the node is able to connect network OR able to reach private/public Registry.
If AWS EKS Cluster, you need to enable auto-assign ip to Subnet where EC2 is running.
To fetch images stored on registries that require credentials, you need to create a special type of secret called imagePullSecrets.
kubectl create secret docker-registry regsecret --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>
Then create the Pod specifying the imagePullSecrets field
apiVersion: v1
kind: Pod
metadata:
name: private-reg
spec:
containers:
- name: private-reg-container
image: <your-private-image>
imagePullSecrets:
- name: regsecret
As mentioned in my comment to the original post, I had the same issue. The only thing of note is the minikube was up as at creation. I restarted the underlying VM and image pulls started working.
This seems to be quite old issue, but I have similar issue and solved by logged in to your docker account.
You can try it by deleting the existing failed pods, firing "docker login" command (login to your acc), then retry for the pod creation.

Kubernetes cannot pull from insecure registry ans cannot run container from local image on offline cluster

I am working on a offline cluster (machines have no internet access), deploying docker images using ansible and docker compose scripts.
My servers are Centos7.
I have set up an insecure docker registry on the machines. We are going to change environnement, and I am installing kubernetes in order to manage my pull of container.
I follow this guide to install kubernetes:
https://severalnines.com/blog/installing-kubernetes-cluster-minions-centos7-manage-pods-services
After the installation, I tried to launch a testing pod. here is the yml for the pod, launching with
kubectl -f create nginx.yml
here the yml:
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: [my_registry_addr]:[my_registry_port]/nginx:v1
ports:
- containerPort: 80
I used kubectl describe to get more information on what was wrong:
Name: nginx
Namespace: default
Node: [my node]
Start Time: Fri, 15 Sep 2017 11:29:05 +0200
Labels: <none>
Status: Pending
IP:
Controllers: <none>
Containers:
nginx:
Container ID:
Image: [my_registry_addr]:[my_registry_port]/nginx:v1
Image ID:
Port: 80/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Volume Mounts: <none>
Environment Variables: <none>
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
No volumes.
QoS Class: BestEffort
Tolerations: <none>
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
2m 2m 1 {default-scheduler } Normal Scheduled Successfully assigned nginx to [my kubernet node]
1m 1m 2 {kubelet [my kubernet node]} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with ErrImagePull: "Error while pulling image: Get https://index.docker.io/v1/repositories/library/[my_registry_addr]/images: dial tcp: lookup index.docker.io on [kubernet_master_ip]:53: server misbehaving"
54s 54s 1 {kubelet [my kubernet node]} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with ImagePullBackOff: "Back-off pulling image \"[my_registry_addr]:[my_registry_port]\""
8s 8s 1 {kubelet [my kubernet node]} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with ErrImagePull: "Network timed out while trying to connect to https://index.docker.io/v1/repositories/library/[my_registry_addr]/images. You may want to check your internet connection or if you are behind a proxy."
then, I go to my node and use journalctl -xe
sept. 15 11:22:02 [my_node_ip] dockerd-current[9861]: time="2017-09-15T11:22:02.350930396+02:00" level=info msg="{Action=create, LoginUID=4294967295, PID=11555}"
sept. 15 11:22:17 [my_node_ip] dockerd-current[9861]: time="2017-09-15T11:22:17.351536727+02:00" level=warning msg="Error getting v2 registry: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
sept. 15 11:22:17 [my_node_ip] dockerd-current[9861]: time="2017-09-15T11:22:17.351606330+02:00" level=error msg="Attempting next endpoint for pull after error: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
sept. 15 11:22:32 [my_node_ip] dockerd-current[9861]: time="2017-09-15T11:22:32.353946452+02:00" level=error msg="Not continuing with pull after error: Error while pulling image: Get https://index.docker.io/v1/repositories/library/[my_registry_ip]/images: dial tcp: lookup index.docker.io on [kubernet_master_ip]:53: server misbehaving"
sept. 15 11:22:32 [my_node_ip] kubelet[11555]: E0915 11:22:32.354309 11555 docker_manager.go:2161] Failed to create pod infra container: ErrImagePull; Skipping pod "nginx_default(8b5c40e5-99f4-11e7-98db-f8bc12456ee4)": Error while pulling image: Get https://index.docker.io/v1/repositories/library/[my_registry_ip]/images: dial tcp: lookup index.docker.io on [kubernet_master_ip]:53: server misbehaving
sept. 15 11:22:32 [my_node_ip] kubelet[11555]: E0915 11:22:32.354390 11555 pod_workers.go:184] Error syncing pod 8b5c40e5-99f4-11e7-98db-f8bc12456ee4, skipping: failed to "StartContainer" for "POD" with ErrImagePull: "Error while pulling image: Get https://index.docker.io/v1/repositories/library/[my_registry_ip]/images: dial tcp: lookup index.docker.io on [kubernet_master_ip]:53: server misbehaving"
sept. 15 11:22:44 [my_node_ip] dockerd-current[9861]: time="2017-09-15T11:22:44.350708175+02:00" level=error msg="Handler for GET /v1.24/images/[my_registry_ip]:[my_registry_port]/json returned error: No such image: [my_registry_ip]:[my_registry_port]"
I sure thant my docker configuration is good, cause I am using it every day with ansible or mesos.
docker version is 1.12.6, kubernetes version is 1.5.2
What can I do now? I didn't find any configuration key for this usage.
When I saw that pulling was failing, I manually pull the image on all the nodes. I put a tag to ensure that kubernetes will to try to pull as default, and set " imagePullPolicy: IfNotPresent "
The syntax for specifying the docker image is :
[docker_registry]/[image_name]:[image_tag]
In your manifest file, you have used ":" to separate docker repository host and the port the repository is listening on. The default port for docker private registry I guess is 5000.
So change your image declaration from
Image: [my_registry_addr]:[my_registry_port]/nginx:v1
to
Image: [my_registry_addr]/nginx:v1
Also, check the network connectivity from the worker node to your docker registry by doing a ping.
ping [my_registry_addr]
If you still want to check if the port 443 is opened on the registry you can do a tcp check on that port on the host running docker registry
curl telnet://[my_registry_addr]:443
Hope that helps.
I finally find what was the problem.
To work, Kubernetes need a pause container. Kubernetes was trying to find the pause container on the internet.
I deployed a custom pause container on my registry, I set up kubernetes pause container to this image.
After that, kubernetes is working like a charm.

openshift online: Error on StartContainer - ErrImagePull: "unauthorized: authentication required"

I'm new to openshift and I have a showstopper:
On my Computer I created a Dockerimage called restservice and I successfully tested it:
docker run -d -p 8080:8080 restservice
Then I created an app in Openshift-Online with the image:
oc new-app restservice
I can see the deployment-pod is starting and after the creating of the running pod is failing.
with
oc describe pod restservice-2-50n0h
I get the following Error:
...
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
1m 1m 1 {default-scheduler } Normal Scheduled Successfully assigned restservice-2-50n0h to ip-172-31-54-238.us-west-2.compute.internal
41s 41s 1 {kubelet ip-172-31-54-238.us-west-2.compute.internal} spec.containers{restservice} Normal Pulling pulling image "restservice:latest"
39s 39s 1 {kubelet ip-172-31-54-238.us-west-2.compute.internal} spec.containers{restservice} Warning Failed Failed to pull image "restservice:latest": unauthorized: authentication required
39s 39s 1 {kubelet ip-172-31-54-238.us-west-2.compute.internal} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "restservice" with ErrImagePull: "unauthorized: authentication required"
55s 9s 2 {kubelet ip-172-31-54-238.us-west-2.compute.internal} Warning FailedSync Error syncing pod, skipping: failed to "SetupNetwork" for "restservice-2-50n0h_wgbeckmann" with SetupNetworkError: "Failed to setup network for pod \"restservice-2-50n0h_wgbeckmann(06f892b4-7568-11e7-914e-0a69cdf75e6f)\" using network plugins \"cni\": CNI request failed with status 400: 'Failed to execute iptables-restore: exit status 1 (iptables-restore: line 3 failed\n)\n'; Skipping pod"
I have no Idea what Authentication is needet.
The missing step is to pusch the Image to openshift online.
So these are the Steps:
Build a Image on the local Computer
docker build -t restservice .
Tag it with registry/username/image-name
docker tag restservice registry.starter-us-west2.openshift.com/myusername/myrestservice
Get your Secret for the Login into the Openshift Registry
oc whoami -t
sr3grwkegr3kjrk42k2jrg34kb5k43g5k4jg53 (sr3... is the output)
Log into Openshift Registry
docker login -u name#mail.com -p sr3grwkegr3kjrk42k2jrg34kb5k43g5k4jg53 https://registry.starter-us-west-2.openshift.com
Push the Image to the Registry
docker push registry.starter-us-west-2.openshift.com/myusername/myrestservice
Create the new App with the Image
oc new-app myrestservice
Thats all ....

Resources