Error response from daemon: Unexpected status code 404 - docker

I am configuring docker registry on nexus 3 configuration. I Am running nexus behind apache and has https enabled.
On command line, when I do a docker search, I get the below error:
docker search my.nexus.net/ubantu
Error response from daemon: Unexpected status code 404
Here is the daemon log on debug mode:
DEBU[7519] Calling GET /images/search
INFO[7519] GET /v1.19/images/search?term=my.nexus.net%2Fubantu
DEBU[7519] pinging registry endpoint https://my.nexus.net/v0/
DEBU[7519] attempting v2 ping for registry endpoint https://my.nexus.net/v2/
DEBU[7519] hostDir: /etc/docker/certs.d/my.nexus.net
DEBU[7519] attempting v1 ping for registry endpoint https://my.nexus.net/v1/
DEBU[7519] hostDir: /etc/docker/certs.d/my.nexus.net
DEBU[7519] Error unmarshalling the _ping RegistryInfo: invalid character '<' looking for beginning of value
DEBU[7519] RegistryInfo.Version: ""
DEBU[7519] Registry standalone header: ''
DEBU[7519] RegistryInfo.Standalone: true
DEBU[7519] attempting v1 ping for registry endpoint https://my.nexus.net/v1/
DEBU[7519] hostDir: /etc/docker/certs.d/my.nexus.net
DEBU[7519] Error unmarshalling the _ping RegistryInfo: invalid character '<' looking for beginning of value
DEBU[7519] RegistryInfo.Version: ""
DEBU[7519] Registry standalone header: ''
DEBU[7519] RegistryInfo.Standalone: true
DEBU[7519] Endpoint https://my.nexus.net/v1/ is eligible for private registry. Enabling decorator.
DEBU[7519] Index server: https://my.nexus.net/v1/
DEBU[7519] hostDir: /etc/docker/certs.d/my.nexus.net
ERRO[7519] Handler for GET /images/search returned error: Unexpected status code 404
ERRO[7519] HTTP Error err=Unexpected status code 404 statusCode=500
If any one has any idea on it, please let me know.

From the logs it seems you try pining different versions of the registry endpoint. Did you use v1Enabled:falseoption on the repository configuration? It seems to get an error during v1 ping but still uses that endpoint. It's rather strange & unexpected behaviour.

I had the same problem.
After googling, it looks like 'docker search' uses the V1 API: see Issue https://github.com/docker/distribution/issues/206
So after I have enabled the V1 API on all docker registries of the group corresponding with the port, it works perfectly.

The docker search command use a v1 API. You have 2 alternatives :
Enable v1 API before using docker search
Use a curl like that: curl -X GET localhost:5000/v2/_catalog

To do pretty much anything in docker using NXRM3, you need to specify the port you are searching so the repository manager knows what repository you are looking for. If you just specify the root port (or in your case, looks like no port), NXRM3 has no idea which you are looking at.
So if your group is setup to use HTTPS connector 18075 try "docker search my.nexus.net:18075/ubantu"
Reference: http://books.sonatype.com/nexus-book/reference3/docker.html#docker-search

I used a VPN and it worked. Not psiphon.

Related

docker login fails with bad gateway. Potential V1 vs V2 issue?

I received a docker registry location to login to and pull some images from at my organization. On attempts to login, I receive the following error:
docker login -u test-user internal.server.com:8080/test
Password:
Error response from daemon: Get https://internal.server.com:8080/v2/: Bad Gateway
I turned debug on for the docker daemon and tried again and receive the following errors:
level=debug msg="attempting v2 login to registry endpoint https://internal.server.com:8080/v2/"
level=info msg="Error logging in to v2 endpoint, trying next endpoint: Get https://internal.server.com:8080/v2/: Bad Gateway"
level=debug msg="FIXME: Got an API for which error does not match any expected type!!!: Get https://internal.server.com:8080/v2/: Bad Gateway" error...l.Error" module=api
If I am interpreting the error correctly, this means the my docker client is attempting to access the registry at v2 yet the registry doesn't seem to support v2? Or do I have this backwards (or some other issue entirely)? Thanks!
/v2 is part of the registry API. It will exist on all requests, so receiving a Bad Gateway from this indicates an error with the registry server.

Artifactory docker login /v2/ failed with status: 404 Not Found

We are running Artifactory 7.35.2 on AWS EKS v1.21, using helm chart to deploy. We are also using nginx-ingress for all our domains in the cluster with NLB.
Issue we are having now is i am getting error when trying to do docker login:
❯ docker login artifactory.example.com
Error response from daemon: login attempt to https://artifactory.example.com/v2/ failed with status: 404 Not Found
I went to Artifactory > HTTP Settings and used both subdomain and repository path with same result, using Tomcat as Server Provider. When using Nginx as provider, i get:
Login did not succeed, error: Error response from daemon: Get "https://artifactory.example.com/v2/": received unexpected HTTP status: 503 Service Unavailable
The ingress code in helm chart is from:
https://www.jfrog.com/confluence/display/JFROG/Helm+Charts+for+Advanced+Users#HelmChartsforAdvancedUsers-ArtifactoryAdvancedOptions
ingress:
enabled: true
defaultBackend:
enabled: false
hosts:
- artifactory.example.com
annotations:
kubernetes.io/ingress.class: nginx
kubernetes.io/ingress.provider: ingress-nginx
ingress.kubernetes.io/force-ssl-redirect: "true"
ingress.kubernetes.io/proxy-body-size: "0"
ingress.kubernetes.io/proxy-read-timeout: "600"
ingress.kubernetes.io/proxy-send-timeout: "600"
nginx.ingress.kubernetes.io/proxy-body-size: "0"
nginx.ingress.kubernetes.io/configuration-snippet: |
rewrite ^/(v2)/token /artifactory/api/docker/null/v2/token;
rewrite ^/(v2)/([^\/]*)/(.*) /artifactory/api/docker/$2/$1/$3;
Although, rewrite ^/(v2)/([^\/]*)/(.*) /artifactory/api/docker/$2/$1/$3; is currently giving me error in helm:
Error: UPGRADE FAILED: cannot patch "artifactory" with kind Ingress: admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: nginx.ingress.kubernetes.io/configuration-snippet annotation contains invalid word \
I changed it to rewrite ^/(v2)/(.*)/(.*) /artifactory/api/docker/$2/$1/$3; but still getting same 404 error.
Has anyone else ran into similar issue that can point me in right direction?

Installing Zookeeper on offline openshift

I've an Openshift Origin cluster running offline on 3 Centos 7 vm. It's working fine, I've a registry where I push my images like this :
docker login -u <username> -e <any_email_address> -p <token_value> <registry_ip>:<port>
Login is successful, then :
oc tag <image-id> <docker-registry-IP>:<port>/<project-name>/<image>
So, for nginx for example :
oc tag 49011ce3b713 172.30.222.111:5000/test/nginx
Then I push it to the internal registry :
docker push 172.30.222.111:5000/test/nginx
And finaly :
oc new-app nginx --name="nginx"
With nginx, everything is working fine, now my problem :
I'm actually wanting to put Zookeeper on it, so : I do the same steps than above, I also install "jboss/base-jdk:7" which is a dependancy of Zookeeper, problem is :
docker push 172.30.222.111:5000/test/jboss/base-jdk:7
Giving :
[root#master 994089]# docker push 172.30.222.111:5000/test/jboss/base-jdk:7
The push refers to a repository [172.30.222.111:5000/test/jboss/base-jdk]
c4c6a9114a05: Layer already exists
3bf2c105669b: Layer already exists
85c6e373d858: Layer already exists
dc1e2dcdc7b6: Layer already exists
Received unexpected HTTP status: 500 Internal Server Error
The problem seems to be the "/" here jboss**/**base-jdk:7
I also tried to push just like this :
docker push 172.30.222.111:5000/test/base-jdk:7
This is working , but Zookeeper is looking for exactly "jboss/base-jdk:7", and not just "base-jdk:7"
Finally, I'm blocked here, when trying this command : oc new-app zookeeper --name="zookeeper" --loglevel=8 --insecure-registry --allow-missing-images
I0628 14:31:54.009713 53407 dockerimagelookup.go:92] checking local Docker daemon for "jboss/base-jdk:7"
I0628 14:31:54.030546 53407 dockerimagelookup.go:380] partial match on "172.30.222.111:5000/test/base-jdk:7" with 0.375000
I0628 14:31:54.030571 53407 dockerimagelookup.go:346] exact match on "jboss/base-jdk:7"
I0628 14:31:54.030578 53407 dockerimagelookup.go:107] Found local docker image match "172.30.222.111:5000/test/base-jdk:7" with score 0.375000
I0628 14:31:54.030589 53407 dockerimagelookup.go:107] Found local docker image match "jboss/base-jdk:7" with score 0.000000
I0628 14:31:54.032799 53407 componentresolvers.go:59] Error from resolver: [can't look up Docker image "jboss/base-jdk:7": Internal error occurred: Get http://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 10.253.158.90:53: no such host]
I0628 14:31:54.032831 53407 dockerimagelookup.go:169] Added missing image match for jboss/base-jdk:7
F0628 14:31:54.032882 53407 helpers.go:110] error: can't look up Docker image "jboss/base-jdk:7": Internal error occurred: Get http://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 10.253.158.90:53: no such host
We can see that 172.30.222.111:5000/test/base-jdk:7 is found but it's not exactly what the command is looking for so it doesn't use it...
So, if you have any idea how to solve this ! :)
Resolved by upgrading to Openshift 1.5.1, previous was 1.3.1.

Push docker image to Google Container Registry failure on Mac

I was trying to upload my image to Google Container Registry, but it return some error and I don't know how to troubleshooting.
$> gcloud docker -- push asia.gcr.io/dtapi-1314/web
The push refers to a repository [asia.gcr.io/dtapi-1314/web]
53ccd4e59f47: Retrying in 1 second
32ca8635750d: Retrying in 1 second
e5363ba7dd4d: Retrying in 1 second
d575d439624a: Retrying in 1 second
5c1cba20b78d: Retrying in 1 second
7198e99c156d: Waiting
6ca37046de16: Waiting
b8f2f07b3eab: Waiting
16681562a534: Waiting
92ea1d98cb79: Waiting
97ca462ad9ee: Waiting
unable to decode token response: read tcp 10.0.2.10:54718->74.125.23.82:443: read: connection reset by peer
I checked permission on my Mac.
$> gsutil acl get gs://asia.artifacts.dtapi-1314.appspot.com
It returned a list of correct permission.
I'd tested push on the cloud console, it works.
Does anyone have clue?
Thanks a lot if anyone could help. :)
Other troubleshooting
gcloud auth login
gcloud docker -- login -p $(gcloud auth print-access-token) -u _token https://asia.gcr.io
gsutil acl get gs://asia.artifacts.{%PROJECT_ID}.appspot.com
Add insecure-registry to dockerd startup command.
--insecure-registry asia.gcr.io
Might be the same cause
gcloud docker -- pull google/python
The error was
Error response from daemon: Get https://registry-1.docker.io/v2/google/python/manifests/latest: read tcp 10.0.2.15:37762->52.45.33.149:443: read: connection reset by peer
docker server log
DEBU[0499] Increasing token expiration to: 60 seconds
ERRO[0500] Error trying v2 registry: Get https://registry-1.docker.io/....../python/manifests/latest: read tcp 10.0.2.15:37762->52.45.33.149:443: read: connection reset by peer
ERRO[0500] Attempting next endpoint for pull after error: Get https://registry-1.docker.io/....../python/manifests/latest: read tcp 10.0.2.15:37762->52.45.33.149:443: read: connection reset by peer
DEBU[0500] Skipping v1 endpoint https://index.docker.io because v2 registry was detected
ERRO[0500] Handler for POST /v1.24/images/create returned error: Get https://registry-1.docker.io/....../python/manifests/latest: read tcp 10.0.2.15:37762->52.45.33.149:443: read: connection reset by peer
Environment
MacOS: 10.11.6
Docker Toolbox (on MAC)
Docker 1.12.3 (Git commit: 6b644ec, Built: Wed Oct 26 23:26:11 2016)
The root cause was stupid, but I'd like to update this for anyone who see this question. I found when I attached my computer to company's WIFI. Then It would work (Still some reset). The cable network of my company is mysterious broken to Google Container Registry. The cable network works for all other services (google/youtube/mobile services) we used but broken to Google Container Registry.
Seems like a permission issue. Try running
gcloud auth login
I remember running into a similar issue and this helped.

Could not authenticate with IBM Containers registry at registry.ng.bluemix.net

When I login to the IBM containers on Bluemix, I get the following error:
cf ic login
** Retrieving client certificates from IBM Containers
** Storing client certificates in /Users/triplez/.ice/certs Successfully retrieved client certificates
** Checking local docker configuration OK
** Authenticating with registry at registry.ng.bluemix.net
Could not authenticate with IBM Containers registry at registry.ng.bluemix.net
**** exit status 1
****Error response from daemon: invalid registry endpoint registry.ng.bluemix.net/v0/: unable to ping registry endpoint
registry.ng.bluemix.net/v0/
v2 ping attempt failed with error: Get registry.ng.bluemix.net/v2/: dial tcp: lookup
registry.ng.bluemix.net on 103.11.48.126:53: read udp
103.11.48.126:53: i/o timeout
v1 ping attempt failed with error: Get registry.ng.bluemix.net/v1/_ping: dial tcp: lookup registry.ng.bluemix.net on 103.11.48.126:53: read udp
103.11.48.126:53: i/o timeout. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add
--insecure-registry registry.ng.bluemix.net to the daemon's
arguments. In the case of HTTPS, if you have access to the registry's
CA certificate, no need for the flag; simply place the CA certificate
at /etc/docker/certs.d/registry.ng.bluemix.net/ca.crt
I've already uninstalled docker and reinstalled, giving me the same error. I've also deleted ~/.cf/config.json and ~/.ice/ice-cfg.ini and ~/.ice/certs/ and tried logging in again. I still receive the same error.
I've also tested this with ice cli with the same error.
EDITED
Tried this command:
ice login -a 'https://api.ng.bluemix.net' --registry 'registry.ng.bluemix.net' --host 'https://containers-api.ng.bluemix.net/v3/containers'
Got the same error:
Authentication with container cloud service at
containers-api.ng.bluemix.net/v3/containers completed
successfully
You can issue commands now to the container service
Proceeding to authenticate with the container cloud registry at
registry.ng.bluemix.net/v3
Error response from daemon: invalid registry endpoint
registry.ng.bluemix.net/v3/v0/:
unable to ping registry endpoint registry.ng.bluemix.net/v3/v0/
v2 ping attempt failed with error: Get registry.ng.bluemix.net/v3/v2/:
dial tcp: lookup registry.ng.bluemix.net on 192.168.0.1:53: read ump
192.168.0.1:53: i/o timeout
v1 ping attempt failed with error: Get registry.ng.bluemix.net/v3/v1/_ping: dial tcp: lookup registry.ng.bluemix.net on 192.168.0.1:53: read udp 192.168.0.1:53: i/o timeout. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add --insecure-registry registry.ng.bluemix.net to the daemon's arguments. In the case of
HTTPS, if you have access to the registry's CA certificate, no need
for the flag; simply place the CA certificate at
/etc/docker/certs.d/registry.ng.bluemix.net/ca.crt
docker is not available on your system or is not properly configured
Could not authenticate with cloud registry at registry.ng.bluemix.net/v3
You can still use IBM Containers but will not be able to run local docker containers, push, or pull images
It looks like the container cloud service host name is not correct.
Please try to run the ice login command as below:
ice login -a 'https://api.ng.bluemix.net' --registry 'registry.ng.bluemix.net' --host 'https://containers-api.ng.bluemix.net/v3/containers'
Based on your comment "boot2docker has been deprecated" I assume you are using Docker 1.8.
This version of Docker is not supported by IBM Containers on Bluemix yet.
Please see documentation in link below saying Docker 1.6 or 1.7 is required:
https://www.ng.bluemix.net/docs/containers/container_cli_ov.html
I know IBM Containers team is working to add support for Docker 1.8 as well. Please keep an eye on documentation for updates in the future.

Resources