I am trying to pull docker image from Nexus repo without using the registry mirror in the command line and it is throwing an error. If I use the registry mirror in the pull it is succeeding but the image name is not I would like.
My docker version is:
Docker version 20.10.8, build 3967b7d
My nexus version is
Sonatype Nexus Repository ManagerOSS 3.31.1-01
docker system info:
Insecure Registries:
xxx.xxx.x.xxx:8083
127.0.0.0/8
Registry Mirrors:
http://xxx.xxx.x.xxx:8083/
When I run: sudo docker pull xxx.xxx.x.xxx:8083/mongo:4.2.3, it succeeds and the debug info is:
DEBU[2021-08-17T10:37:19.364681226-04:00] Calling HEAD /_ping
DEBU[2021-08-17T10:37:19.365301100-04:00] Calling POST /v1.41/images/create?fromImage=192.168.9.175%3A8083%2Fmongo&tag=4.2.3
DEBU[2021-08-17T10:37:19.367151579-04:00] Trying to pull xxx.xxx.x.xxx:8083/mongo from https://xxx.xxx.x.xxx:8083 v2
WARN[2021-08-17T10:37:19.374915464-04:00] Error getting v2 registry: Get https://xxx.xxx.x.xxx:8083/v2/: http: server gave HTTP response to HTTPS client
INFO[2021-08-17T10:37:19.374944418-04:00] Attempting next endpoint for pull after error: Get https://xxx.xxx.x.xxx:8083/v2/: http: server gave HTTP response to HTTPS client
DEBU[2021-08-17T10:37:19.374964188-04:00] Trying to pull xxx.xxx.x.xxx:8083/mongo from http://xxx.xxx.x.xxx:8083 v2
DEBU[2021-08-17T10:37:19.398630498-04:00] Fetching manifest from remote digest="sha256:92814bb60dc673bb68b6aca0b24bcb8738d7b2c267b97ce62fa92adc3746a0ea" error="<nil>" remote="192.168.9.175:8083/mongo:4.2.3"
DEBU[2021-08-17T10:37:19.429454057-04:00] Pulling ref from V2 registry: xxx.xxx.x.xxx:8083/mongo:4.2.3
When I run: sudo docker pull mongo:4.2.3 it fails to pull the image from Nexus with an error and pulls from docker.io on the next try. Debug info as below:
DEBU[2021-08-17T10:26:25.078886904-04:00] Calling HEAD /_ping
DEBU[2021-08-17T10:26:25.079306196-04:00] Calling GET /v1.41/info
DEBU[2021-08-17T10:26:25.097994642-04:00] Calling POST /v1.41/images/create?fromImage=mongo&tag=4.2.3
DEBU[2021-08-17T10:26:25.099642151-04:00] Trying to pull mongo from http://xxx.xxx.x.xxx:8083/ v2
INFO[2021-08-17T10:26:25.116000813-04:00] **Attempting next endpoint for pull after error: manifest unknown: manifest unknown**
DEBU[2021-08-17T10:26:25.116039299-04:00] Trying to pull mongo from https://registry-1.docker.io v2
DEBU[2021-08-17T10:26:25.305043063-04:00] Fetching manifest from remote digest="sha256:58b25d51baa11a85b6aedf7c4e05710d12a27ddc2883e2692e7d58527d98bd73" error="<nil>" remote="docker.io/library/mongo:4.2.3"
DEBU[2021-08-17T10:26:25.360955030-04:00] Pulling ref from V2 registry: mongo:4.2.3
DEBU[2021-08-17T10:26:25.361036645-04:00] docker.io/library/mongo:4.2.3 resolved to a manifestList object with 5 entries; looking for a unknown/amd64 match
Issue with Image name:
REPOSITORY TAG IMAGE ID CREATED SIZE
xxx.xxx.x.xxx:8083/mongo 4.2.3 97a9a3e85158 17 months ago 386MB
Any guidance on this would help.
Nexus Docker ( xxx.xxx.x.xxx:8083) is pointed to hosted Type on port 8083 and the mongo:4.2.3 is uploaded into this docker type. We ultimately want to use this in a air gapped system where there is no internet connection.
There are three things going on here:
I am trying to pull docker image from Nexus repo without using the registry mirror in the command line and it is throwing an error. If I use the registry mirror in the pull it is succeeding but the image name is not I would like.
I'm going to recommend changing your likes. :)
If you want to pull from a specific registry, then use that registry in the image name. Trying to refer to your local registry with short names is merging two different image registry namespaces, which means it's trivial to run an image from the wrong namespace and result in a security breach. This was a large issue for other package repositories (see "dependency confusion" attacks) that docker was not susceptible to because they require the registry name as part of the image name (the only exception being Docker Hub). Even RedHat who tried to get options like add-registry and block-registry into the upstream docker engine (and failed, these options only ever appeared in a RedHat specific fork) is now telling users that it was a very bad idea and now their users are exposed to security vulnerabilities they can't easily fix because removing the feature will break lots of user environments.
Next, why doesn't the pull go to your registry? Because your image name doesn't match that of Docker Hub. Official images without a username are actually under the library repository. This is typically hidden from view, but you can do things like docker pull library/alpine or even docker pull docker.io/library/alpine instead of docker pull alpine, and all 3 will be pulling from the same place.
The fix is to run
docker pull xxx.xxx.x.xxx:8083/mongo:4.2.3
docker tag xxx.xxx.x.xxx:8083/mongo:4.2.3 xxx.xxx.x.xxx:8083/library/mongo:4.2.3
docker push xxx.xxx.x.xxx:8083/library/mongo:4.2.3
The last issue I actually can't help you with, it comes from the error message you're seeing when pulling from Hub, which should work:
docker.io/library/mongo:4.2.3 resolved to a manifestList object with 5 entries; looking for a unknown/amd64 match
The unknown/amd64 is unexpected to me, typically that would be linux/amd64 so there is something unexpected with the platform you're running your commands on. If you want to get into debugging that, update your question with docker info. You can try working around that with:
docker pull --platform linux/amd64 mongo:4.2.3
to force the platform, but that still doesn't explain why it doesn't know your current platform.
I guess you are trying to set your nexus docker repository to be the default one for the machine in the sealed network.
that needs changing because of the following from docker documentation:
Tag an image for a private repository
To push an image to a private registry and not the central Docker registry you must tag it with the registry hostname and port (if needed).
$ docker tag 0e5574283393 myregistryhost:5000/fedora/httpd:version1.0
with more upfront configuration and upkeep but no changes requiered for the client machines
Is if you have a DNS server in your network you could point docker.io to your nexus host ip address and put a proxy to intercept the communication and redirect and adapt the requests as they were to the nexus docker registry
Hopes this solves your pickle :)
Update 1:
It could be that you need to also change /etc/containers/registries.conf like specified here to only or also specify your nexus docker registry.
Update 2:
Before letting Gopi give up entirely, I would suggest using Podman as an alternative to Docker. Podman is a daemon-less container engine that works by forking processes to handle each running container. It seamlessly works with docker images thanks to the OCI standard, and on top of that, the only change when using it is replacing the docker command prefix with podman since all the commands are exactly the same. Podman was created by RedHat so by default it searches RedHat repos and you can add your own too as shown in this article that I mentioned before.
Related
Try to login to login to my private compagny docker (jfrog) registry, it was working ...
[rabxxxx.xxx#vmxxx]$ docker login registry.x.xx-xx-xxxxx.xx.xx.xxxx.net -u xxx
Password:
WARNING! Your password will be stored unencrypted in /home/rabxxxx.xxx/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[rabxxxx.xxx#vmxxx]$ docker run -p 8080:8080 -p 50000:50000 jenkins
Unable to find image 'jenkins:latest' locally
docker: Error response from daemon: Get "https://registry-1.docker.io/v2/": context deadline exceeded.
See 'docker run --help'.
cat /etc/docker/daemon.json
{
"insecure-registries": ["registry.x.xx-xx-xxxxx.xx.xx.xxxx.net"],
"registry-mirrors": ["registry.x.xx-xx-xxxxx.xx.xx.xxxx.net"]
}
Please can you tell what I must check and why it's return me this
Error response from daemon: Get "https://registry-1.docker.io/v2/"
Thx
docker pull registry.x.xx-xx-xxxxx.xx.xx.xxxx.net/jenkins`
Using default tag: latest Error response from daemon: manifest for registry.repo.proxy-dev-forge.asip.hst.fluxus.net/jenkins:latest not found: manifest unknown: The named manifest is not known to the registry.
( same with /library/jenkins ]
When you docker run jenkins, docker will expand that to docker.io/library/jenkins, where docker.io is Docker Hub and the actual underlying registry server is registry-1.docker.io. When you define a mirror for that, you need to keep the repository names the same, so not registry.example.org/jenkins, but registry.example.org/library/jenkins.
In your case, it appears your mirror has neither of those, making it an incomplete mirror, assuming it's a mirror at all. But you don't really want to run the library/jenkins image anyway, since they've indicated it's deprecated with a recommendation to use the jenkins/jenkins repo instead (likely maintained by Cloudbees).
Please can you tell what I must check and why it's return me this
Error response from daemon: Get "https://registry-1.docker.io/v2/"
Mirroring in docker is a best effort. If the mirror fails for any reason, or you try to do a push instead of a pull, docker will fall back to going direct to Docker Hub.
If you only want to talk to your local registry, then you should put that registry in all your image names. This is particularly important if your mirror doesn't match Docker Hub and you make assumptions based on your local registry. In those cases, the same definition for an image on one machine may run very different code if the network has an issue or you deploy on a different machine without the mirroring specification. RedHat documented many of these risks after pushing hard to add the ability to overload the top level namespace, and we had a wave of dependency confusion attacks impact other software repositories.
Your company registry did not cache image from official registry. Try to make it auto cache or pull the jenkins image from docker official registry and push to your company registry then you can use it. I will suggest you to make the company registry auto cache, when the registry exists in your private registry, just use it, if did not exists, cache from the official registry.
I'm trying to set a docker mirror to be the default mirror to pull/push images.
As per documentation I already set the file /etc/docker/daemon.json with the following:
{
"registry-mirrors": ["https://localregistry"]
}
Then I try the following:
docker login localregistry
docker pull localregistry/image:tag > it works
docker pull image:tag > doesn't work
I'm always getting "no basic auth credentials error" from the docker daemon, but from the registry log I get err.code="manifest unknown" err.detail="unknown tag"
Any idea?
I'm using docker version 19.03.08
docker login localregistry
First, I hope this is changing the name for the question, because the registry name localregistry will not work...
docker pull localregistry/image:tag > it works
The fact that this works indicates that you likely have a registry name with a . or : in the hostname. Otherwise docker would try to pull localregistry/image:tag from the localregistry user on Docker Hub.
docker pull image:tag > doesn't work
This should always work, failures should be transparent to the user if it's really a mirror of Docker Hub. What happens is it resolves that name to docker.io/library/image:tag, first tries to pull from localregistry/library/image:tag, and any error falls back to a pull from Docker Hub, and any error there finally shows to the user.
Most likely the issue is that you didn't include library as the repo name for your image in the local registry.
If you are using this to include images that don't exist on Docker Hub, then I would skip the mirror and simply refer to the mirror explicitly. Doing otherwise creates many opportunities for nonintuitive failures that aren't easy to see. E.g. a stale image can be pushed to your mirror in place of an upstream image, and Docker will stop pulling updates from upstream. And because any mirror errors fall back to Hub, if you use an image name that you have no control over upstream, someone else could take that name on Hub and begin injecting unknown or even malicious images into your server.
If this doesn't answer your question, then I'd recommend using your question with actual image names and error messages from the logs showing what specifically failed (you can mask out part of the registry name of necessary).
In our current setup we are do proxy to entire docker hub to pull the images, but what we want is limit this proxy to only official indexes like ubuntu, apache etc.
During configuration of repository there was this option to set custom index for the repository, so my two questions are :
With this custom index option will I be able to achieve what I explained above?
If yes, then from where I should get the URL for custom index. so far I tried using https://hub.docker.com/_/centos/ for centos but it didn't worked.
Please suggest how to go about this.
Yes, you can use a custom index to limit what's available in your repo. To achieve that you have to deploy and maintain your own docker registry index service and use that when configuring your docker proxy repository.
However, an easier solution would be to use a content selector to limit what's allowed to be access via your docker proxy repository while still using Docker Hub with it's original index.
Using the simplified content selector solution is a bit hacky, but might work for you. Essentially you have to provide a whitelist (or a blacklist) in a form of a regex. The implication of this solution is that users of your registry will still be able to search for any available image, because you can keep using the Docker Hub's index for simplicity.
Create a docker proxy repository pointing to the Docker Hub.
Create a content selector with a white/black list of images, eg. path =~ '^/v2/library/(alpine|ubuntu).*$' will only allow alpine and ubuntu to be downloaded.
Create a new privilege of type Repository Content Selector.
Create a role containing the above privilege.
Create a user with an above role.
Now, the user you have created shall log into your registry, eg. $ docker login nexus.local:8085. That user will be authorised to pull both alpine and ubuntu (based on the content selector from step 2), but will fail to download anything else.
A sample succesful pull:
$ docker pull nexus.local:8085/alpine
Using default tag: latest
latest: Pulling from alpine
Digest: sha256:0873c923e00e0fd2ba78041bfb64a105e1ecb7678916d1f7776311e45bf5634b
Status: Image is up to date for nexus.local:8085/alpine:latest
A sample failed pull:
$ docker pull nexus.local:8085/postgres
Using default tag: latest
Error response from daemon: unauthorized: access to the requested resource is not authorized
we are do proxy to entire docker hub to pull the images, but what we want is limit this proxy to only official indexes like ubuntu, apache
What I did was:
Create a routing rule
Mode: ALLOW
I wanted only alpine and centos so I created two matches with:
^/v2/library/centos.*$
^/v2/library/alpine.*$
Go to configuration of the docker proxy repository
Go to Routing Rule
And pick the routing rule you created.
That way all except centos and alpine are available from that docker proxy repo. Pulling anything else results in "manigest unknown" error from docker.
I'm unable to pull docker images in my environment. I think it's blocked by company firewall, but I'm not sure why It gets layer info and later It prints that repository is not found.
sudo docker pull hello-world
latest: Pulling from hello-world
50a54e1f9180: Pulling fs layer
7a5a2d73abce: Pulling fs layer
Pulling repository hello-world
Repository not found
Docker version: (I cannot upgrade to newest docker on RHEL 6.9)
Docker version 1.7.1, build 786b29d/1.7.1
Could somebody explain me which protocols (https only?) are used during docker image pulling phase and what addresses are contacted ("https://registry-1.docker.io/v2" only?) ?
Docker images can consist of multiple layers. By default, the Docker daemon will pull three layers of an image at a time but will pull less in case an image has lesser layers. Also, if no tag is provided, Docker Engine uses the :latest tag as a default. Above is a basic log of your pull request indicating docker trying to pull layers of the image but failing, may be due to firewall restrictions or older docker version.
Docker uses the https:// protocol to communicate with a registry,
unless the registry is allowed to be accessed over an insecure
connection.
Not sure what all addresses it tries to connect to pull an image.
Problem was that firewall was blocking connections during pulling images.
Docker registry uses CDN so more URLs need to be allowed and not only registry URL.
I requested to allow the following URLs on company firewall and it is working now.
dseasb33srnrn.cloudfront.net
auth.docker.io
elb-registry.us-east-1.aws.dckr.io.
us-east-1-elbregis-10fucsvj1tcgy-133821800.us-east-1.elb.amazonaws.com
registry-1.docker.io
registry-origin.docker.io
index.docker.io
elb-io.us-east-1.aws.dckr.io
us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Docker log file (/var/log/docker) help me to identify root problem.
There were the following errors:
level=error msg="Error from V2 registry: Get https://dseasb33srnrn.cloudfront.net/registry-v2/docker/registry/v2/blobs/sha256/78/78445dd45222097f5f8d5a16e48dc19c4ca162dcdb80010ab6f1ccfc7e2c0fa3/data?Expires=1493033299&Signature=DiEmffSxF1F9z-SRoGyX3NwzfeQY3BhE2Du3aPb1qy9VglXyn1mus7Xy9Y~DQnwaQ9IIN71FboK5lOAiN1Qj-x662qhioi72CJ-v02fiMHqC03FDb0l4LyULquU8GaalW3uZG4hdfuSqOBQ1qo9HEcxhMyQGqOqpfPUKjUlHqm8_&Key-Pair-Id=APKAJECH5M7VWIS5YZ6Q: read tcp 52.85.173.110:443: connection reset by peer"
The list of URLs which needs to be allowed I found here:
https://forums.docker.com/t/list-of-docker-hub-mirror-sites-to-configure-proxy-whitelist/20845/2
Am new to openshift and am trying a basic setup. I have docker and openshift installed on my Ubuntu VM inside my windows laptop. I dont have any problem in using docker alone and pushing or pulling from my local registry. I have problem in using the oc new-app command in the openshift origin.
I have a local registry running in my VM
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
26d3a527398d registry:2 "/bin/registry serve " 16 minutes ago Up 16 minutes 0.0.0.0:443->443/tcp, 0.0.0.0:5000->5000/tcp registry
and I have pushed a image to this registry
docker images myregistry.com:5000/ubuntu
REPOSITORY TAG IMAGE ID CREATED SIZE
myregistry.com:5000/ubuntu latest cf62323fa025 7 days ago 125 MB
When i try to use this image in the openshift, am getting below error
./oc new-app --docker-image=myregistry.com:5000/ubuntu
error: can't look up Docker image "myregistry.com:5000/ubuntu": Internal error occurred: Get https://myregistry.com:5000/v2/: http: server gave HTTP response to HTTPS client
error: no match for "myregistry.com:5000/ubuntu"
The 'oc new-app' command will match arguments to the following types:
1. Images tagged into image streams in the current project or the 'openshift' project
- if you don't specify a tag, we'll add ':latest'
2. Images in the Docker Hub, on remote registries, or on the local Docker engine
3. Templates in the current project or the 'openshift' project
4. Git repository URLs or local paths that point to Git repositories
--allow-missing-images can be used to point to an image that does not exist yet.
See 'oc new-app -h' for examples.
Is this something to do with my DNS resolution ? How can I use my local registry inside my openshift origin
The code in openshift/origin/pkg/diagnostics/pod/auth.go#authenticateToRegistry() mentions:
case strings.Contains(secError.Error(), "tls: oversized record received"),
strings.Contains(secError.Error(), "server gave HTTP response to HTTPS"):
r.Debug("DP1015", "docker-registry not secured; falling back to cleartext connection")
Issue 6516 mentions:
if your registry requires authentication, then that is the problem. new-app cannot pull images from registries that require authentication (there is an existing issue for this), so as far as it's concerned the image does not exist in the registry.
Issue 6540 (OC 1.1.1) is supposed to resolve this:
new-app can use an image that appears to only exist locally (because new-app can't access the registry). It will proceed to define the expected openshift objects (deploymentconfig, etc) with a reference to that image.
If the nodes are also not able to pull the image, the deployment will ultimately still fail. (however unlike new-app, nodes are able to pull images from secure registries)
The OP Raghavan reports in the comments having setup the DNS with "How to Setup a DNS Server for a Home Lab on Ubuntu 14.04"
I added the DNS and --insecure-registry option in /etc/default/docker file as
DOCKER_OPTS=--dns 10.0.3.1 --insecure-registry ns1.myregistry.com:5000
Here 10.0.3.1 was my DNS server and also the server where docker registry is running with hostname as ns1.myregistry.com