Failed to pull container images via short reference format - docker

I have a problem downloading container images from a private docker registry.
The registry need no authentication credentials and is only available over the
internal network.
I have configured the registries in /etc/docker/daemon.json like the example
below. I have replaced the FQDN.
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors": [
"https://customer-registry.example.com",
"https://location-a.dev.example.com:17519",
"https://location-b.dev.example.com:17519"
]
}
When I use the short reference format to download the container image
apvxw/build-env-go:4.2.1 I get the following error:
$ docker pull apvxw/build-env-go:4.2.1
Unable to find image 'apvxw/build-env-go:4.2.1' locally
/usr/bin/docker: Error response from daemon: pull access denied for
orbis-u/build-env-go, repository does not exist or may require 'docker login':
denied: requested access to the resource is denied.
When I use the long reference format docker can pull the image
$ docker pull customer-registry.example.com/apvxw/build-env-go:4.2.1
4.2.1: Pulling from apvxw/build-env-go:4.2.1
ac9208207ada: Already exists
5cf798ece9e5: Already exists
510bf5361e28: Already exists
b2f42d2b54d9: Pull complete
4b8be0bf5345: Pull complete
311322fb5cb6: Pull complete
Digest: sha256:f522ce0974ee41dfc7f16fc44682fee77e57bc056e37b27a4bf4885af3f5c375
Status: Downloaded newer image for customer-registry.example.com/apvxw/build-env-go:4.2.1
customer-registry.example.com/apvxw/build-env-go:4.2.1
The same behavior applies to the other configured docker registries.
I have also tested downloading container images via the registries with podman.
There the download via the short reference works fine.
Does anyone have an idea what the problem is because I can't download container
images under Arch using the short notation?
System and docker informations:
$ uname -a
Linux markus-pc 5.5.9-arch1-2 #1 SMP PREEMPT Thu, 12 Mar 2020 23:01:33 +0000 x86_64 GNU/Lin
$
$ docker info
Client:
Debug Mode: false
Server:
Containers: 10
Running: 0
Paused: 0
Stopped: 10
Images: 71
Server Version: 19.03.7-ce
Storage Driver: overlay2
Backing Filesystem: <unknown>
Supports d_type: true
Native Overlay Diff: false
Logging Driver: json-file
Cgroup Driver: systemd
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: d76c121f76a5fc8a462dc64594aea72fe18e1178.m
runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 5.5.9-arch1-2
Operating System: Arch Linux
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.54GiB
Name: markus-pc
ID: DKFK:PHVZ:LDGJ:54OG:5VJ5:5XYK:YDZR:DJFR:HZ7B:4HDT:LBEK:7JQ4
Docker Root Dir: /var/lib/docker
Debug Mode: false
Username: volkerraschek
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
https://customer-registry.example.com/
https://location-a.dev.example.com:17519/
https://location-b.dev.example.com:17519/
Live Restore Enabled: false

In Docker
The default registry is configured to search images on docker hub which is docker.io. When you pull any image without domain name it will try to pull from docker.io, not from any other registry.
If you want to pull an image from a private registry, then you need to use long reference which is
docker pull <YOUR-DOMAIN>/apvxw/build-env-go:4.2.1
In Podman
You can configure multiple registries and podman searches in all registries whichever configured in the config file.
Location of podman configuration file is /etc/containers/registries.conf. and you can add the following lines in that which you might have already done.
[registries.search]
registries = ['docker.io', 'customer-registry.example.com', 'location-a.dev.example.com:17519', 'location-b.dev.example.com:17519']

Related

docker 20: change docker image location

i want to change the pull location for the docker image, i already follow the instruction from this link https://quick-adviser.com/how-do-i-change-docker-location/.
i already try adding file daemon.json to C:\ProgramData\Docker\config and fill it with following code:
{
"data-root": "D:\\docker-image"
}
then restarting docker desktop. After that, i trying pull a small size docker image like adminer docker pull adminer and check to D:\docker-image. After the execution is completed, the folder still empty
here's my docker info output:
$ docker info
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., v0.7.1)
compose: Docker Compose (Docker Inc., v2.2.3)
scan: Docker Scan (Docker Inc., v0.16.0)
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 20.10.12
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
runc version: v1.0.2-0-g52b36a2
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.16.3-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 6.04GiB
Name: docker-desktop
ID: V6FY:3JES:DVIP:5ZLG:6J26:IXE7:RKCB:T3MK:RR4B:X2XC:JR7B:LEIH
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
Basically you're running Docker Desktop with WSL2. So files are no longer located in Windows directly. Docker creates an Linux-Virtual-Machine (WSL2) where all files are located inside in a Linux folder structure.
# Image-Layers are somewhere here:
\\wsl$\docker-desktop-data\version-pack-data\community\docker\image\overlay2
\\wsl$\docker-desktop-data\version-pack-data\community\docker\overlay2
If you want to change the folder on Windows side (e.g. to use disk space on another drive) then you have to move the image file. This is described in the question I posted. How can I change the location of docker images when using Docker Desktop on WSL2 with Windows 10 Home?
Two different possibilities given: Moving the WSL2 in total or Move the file and generate a Symlink (mklink) to keep WSL2 like it is and just redirect to the file in another location.
By default docker locates the WSL-Image-File for the Data here: %homepath%\AppData\Local\Docker\wsl\data as ext4.vhdx
With this Docker Desktop and WSL2 setup you can't just change the path where the images are located by just using a windows path because under the hood docker works in Linux. So images are stored in Linux.
If you just want to get the image data to move them to another system check docker save and docker load https://docs.docker.com/engine/reference/commandline/save/
Does this fit your use case?

Error response "proxyconnect tcp: EOF" from daemon when running docker run hello-world in docker quickstart terminal

I'm running docker toolbox on windows version 10.0.15063 behind corporate proxy, and I'm getting the following error when i run the command docker run hello-world
Unable to find image 'hello-world:latest' locally
C:\Program Files\Docker Toolbox\docker.exe: Error response from daemon: Get https://registry-1.docker.io/v2/: proxyconnect tcp: EOF.
See 'C:\Program Files\Docker Toolbox\docker.exe run --help'
I'm getting the same error on attempting to login to docker using docker login.
I've already tried exporting HTTPS_PROXY and HTTP_PROXY in /Docker Toolbox/start.sh and adding the same to my environment variables but none of these approaches seem to be working.
Here's my docker info
$ docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 18.09.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.14.116-boot2docker
Operating System: Boot2Docker 18.09.6 (TCL 8.2.1)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 989.4MiB
Name: default
ID: PNCR:EWZF:D4O3:7KDP:UW7W:4NEO:YGTT:CHWV:VF5U:YTE6:GD66:WKHS
Docker Root Dir: /mnt/sda1/var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
HTTP Proxy: http://xxxx:xxxx
HTTPS Proxy: https://xxxxx:xxxx
No Proxy: 192.168.99.100
Registry: https://index.docker.io/v1/
Labels:
provider=virtualbox
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Here's what i get on ssh-ing into the default machine and running sudo vi /var/lib/boot2docker/profile
CACERT=/var/lib/boot2docker/ca.pem
DOCKER_HOST='-H tcp://0.0.0.0:2376'
DOCKER_STORAGE=aufs
DOCKER_TLS=auto
SERVERKEY=/var/lib/boot2docker/server-key.pem
SERVERCERT=/var/lib/boot2docker/server.pem
export "HTTP_PROXY=http://proxy2:8080"
export "HTTPS_PROXY=https://proxy2:8080"
export "NO_PROXY=192.168.99.100"
I'm I missing something? I've been working on this for the last hour without any luck.
I actually figured it out.
I was initially using corporate network while working on this, and the networking team has blacklisted every ip and then white-listed the trusted ones. That was what was causing the whole problem while trying to connect to an external registry.
These are the steps I followed to fix the issue:
I switched to my personal network then disabled the proxy ie. Under Internet Properties > Switch to automatic Configuration > Automatically detect settings.
Removed the HTTP_PROXY and HTTPS_PROXY exports from start.sh in C:\Program Files\Docker Toolbox
Opened Windows Firewall > Advanced Settings > Inbound Rules > Add New Rule; Then
Created a Custom rule with a range ip range of 1-100, then named it Docker.
Deleted the .Virtualbox folder from C:\Users\myusername*
Restarted Docker Quickstart Terminal. It created a new default docker-machine and updated the configuration settings, and now pulling images from external registries is working fine.

'docker: invalid reference format' when docker repository is not explicitly specified

I'm running a Docker in Docker container in kubernetes through the Gitlab CI (there is a main image and dind is as a service). In the pod I can build docker images and push them, however any further manipulation or a simple docker run triggers an error of invalid reference format.
There are two additional insecure private repositories specified in the dind service specification. If I try to run docker images without a repo prefix specified, it fails. (note that the images are there, it's just their tag which doesn't have the repo specified)
bash-4.2# docker run hello-world
docker: invalid reference format.
See 'docker run --help'.
bash-4.2# docker run docker.io/hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
bash-4.2#
I would expect the basic docker pull/run commands to pass and to use the images cached in the daemon even when their current tag doesn't contain a repository specified.
Added docker info
Containers: 3
Running: 1
Paused: 0
Stopped: 2
Images: 21
Server Version: 18.09.5
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.14.35-1844.3.2.el7uek.x86_64
Operating System: Alpine Linux v3.9 (containerized)
OSType: linux
Architecture: x86_64
CPUs: 20
Total Memory: 125.5GiB
Name: runner-vppr4q7s-project-14503-concurrent-08qxbj
ID: HQ3U:HATA:TZUT:OLVS:MHRZ:F4DY:ENBD:Z5WH:FQBC:SGI3:BUUY:QIME
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
HTTP Proxy: xxxxxxxx
HTTPS Proxy: xxxxxxx
No Proxy: localhost,127.0.0.1,.xxxxxxxxxx
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
xxxxxxxxxxxx:5000
yyyyyyyyyyyy:5000
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
Access to the remote API is equivalent to root access on the host. Refer
to the 'Docker daemon attack surface' section in the documentation for
more information: https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Registries:
Additionally:
bash-4.2# docker tag docker.io/hello-world:latest hello-world:v1
Error parsing reference: "hello-world:v1" is not a valid repository/tag: invalid reference format
bash-4.2# docker tag docker.io/hello-world:latest some:2000/hello-world:v1
bash-4.2# docker tag docker.io/hello-world:latest docker.io/hello-world:v1
(ok)
And docker images doesn't show the docker.io/hello-world:latest and docker.io/hello-world:v1 but shows some:2000/hello-world:v1

Docker Hub login behind a proxy

I recently installed Docker 18.04.0-ce on my Ubuntu 17.10 VM. While working behind a proxy I got stuck trying to log in into the Docker Hub registry. So when I run:
docker login -u <username> -p <cool password>
And I get this message:
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http:
request canceled while waiting for connection (Client.Timeout exceeded while
awaiting headers)
As far as my understanding goes from the research I did is that when you are behind a proxy and you want to use Docker you need to configure it accordingly even though having set your environment variable like so:
http_proxy=<my_proxy>
https_proxy=<my_proxy>
HTTP_PROXY=<my_proxy>
HTTPS_PROXY=<my_proxy>
Second, it looks like that Docker has multiple levels of proxy configurations meaning different configuration for build, containers, and the daemon.
I went and configured all of those to match my proxy, but I still get that message. YES, I DID RESTART the service and the daemon.
UPDATE:
$ docker info
Containers: 8
Running: 0
Paused: 0
Stopped: 8
Images: 12
Server Version: 18.04.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.13.0-39-generic
Operating System: Ubuntu 17.10
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 7.79GiB
Name: docker-vb
ID: 447I:6DFI:JZ7V:F6SZ:BUCB:IFB2:4HGT:MXK2:Y5H5:EECC:FIQN:SZOH
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
HTTP Proxy: http://<proxy_ip>:<proxy_port>
HTTPS Proxy: http://<proxy_ip>:<proxy_port>
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
The docker login typically needs to be run with sudo priviledges, so ensure that the HTTPS_PROXY environment is actually preserved. Simply done by
export HTTPS_PROXY=<your proxy>
Or simply pass it on the command line:
sudo HTTPS_PROXY=<your proxy> docker login -u <user> <registry>
The docker daemon doesn't need to be restarted.
The above works fine (we use extensivelye proxies at work).
There may be some issues with your network reachability, proxy, and possibly the registry. The docker log can provide some clues:
Ubuntu: sudo journalctl -fu docker.service
RHEL: /var/log/messages | grep docker

How to find whether a image is present in the docker repository

Trying to pull a docker image in my local repo.
Command used:
docker pull buzybox:1.24
Error response from daemon: repository buzybox not found: does not exist or no pull access
My questions:
1:How to find the repository name which am connected with?
2.Whats the access issue in the error its talking about? Do we also have any kind of access issues in the default repo?? How to find that out?
3.How to change the repo if in case i dont find a image in my currect repo.
4.Do we have any web portal from where we can search if the image is present in repository.
Output id docker info :
docker info
Containers: 5
Running: 0
Paused: 0
Stopped: 5
Images: 2
Server Version: 17.06.2-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk
syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 6e23458c129b551d5c9871e5174f6b1b7f6d1170
runc version: 810190ceaa507aa2727d7ae6f4790c76ec150bd2
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.41-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.941GiB
Name: moby
ID: RJWZ:HXFZ:5FQX:V2AB:YAXP:BKSU:RLQH:H24C:LNUD:USDA:VIF7:W2M3
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 15
Goroutines: 26
System Time: 2018-01-28T06:34:45.7011515Z
EventsListeners: 0
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Thanks in advance Guys!!
Nilotpal
You do not connect to a repository. You connect to a registry. You are probably connected to https://index.docker.io/v1/ but to be sure, you can run sudo docker info and look for the registry line. The repository in your case is buzybox and you are trying to download the image with tag 1.24 from that repository.
The access issue is probably that you mispelled busybox (notice the 's' instead of a 'z').
To log into a different registry, then run
docker login https://<YOUR-DOMAIN>:8080
and then make sure to specify the new registry when you download an image like so:
docker pull <YOUR-DOMAIN>:8080/busybox:1.24

Resources