Docker remote api v1.6.:
I am using it to POST and build dockerfile. I can see my newly built images in the json image list output by same remote api, http://docs.docker.io/en/latest/api/docker_remote_api_v1.6/#id20 . During build i'm also tagging it with repository name, http://docs.docker.io/en/latest/api/docker_remote_api_v1.6/#id30.
To push this so generated image on the registry am using docker cli. But the reply from docker cli says
"No such id...".
Since am using docker cli to pull images as well i can't pull it as well.
Please help solving this sync issue in reflecting changes make through docker remote api's into docker cli result?
Related
I have a docker container with Trivy installed.
I have a remote registry with docker images.
and
I would like to download the docker images to the container for scanning
Challenges
It is hard to run docker within a docker container for pulling the images.
Trivy requires that you have the images locally before it can scan the images, either in a local registry or as a file.
I found two solutions:
Download the images with Skopeo
Download the images with the HTTP API V2
For the API I had a hard time making the authentication work, as it is repository specific, and Scaleways' authentication had unexpected behaviour.
I am trying to pull docker image from Nexus repo without using the registry mirror in the command line and it is throwing an error. If I use the registry mirror in the pull it is succeeding but the image name is not I would like.
My docker version is:
Docker version 20.10.8, build 3967b7d
My nexus version is
Sonatype Nexus Repository ManagerOSS 3.31.1-01
docker system info:
Insecure Registries:
xxx.xxx.x.xxx:8083
127.0.0.0/8
Registry Mirrors:
http://xxx.xxx.x.xxx:8083/
When I run: sudo docker pull xxx.xxx.x.xxx:8083/mongo:4.2.3, it succeeds and the debug info is:
DEBU[2021-08-17T10:37:19.364681226-04:00] Calling HEAD /_ping
DEBU[2021-08-17T10:37:19.365301100-04:00] Calling POST /v1.41/images/create?fromImage=192.168.9.175%3A8083%2Fmongo&tag=4.2.3
DEBU[2021-08-17T10:37:19.367151579-04:00] Trying to pull xxx.xxx.x.xxx:8083/mongo from https://xxx.xxx.x.xxx:8083 v2
WARN[2021-08-17T10:37:19.374915464-04:00] Error getting v2 registry: Get https://xxx.xxx.x.xxx:8083/v2/: http: server gave HTTP response to HTTPS client
INFO[2021-08-17T10:37:19.374944418-04:00] Attempting next endpoint for pull after error: Get https://xxx.xxx.x.xxx:8083/v2/: http: server gave HTTP response to HTTPS client
DEBU[2021-08-17T10:37:19.374964188-04:00] Trying to pull xxx.xxx.x.xxx:8083/mongo from http://xxx.xxx.x.xxx:8083 v2
DEBU[2021-08-17T10:37:19.398630498-04:00] Fetching manifest from remote digest="sha256:92814bb60dc673bb68b6aca0b24bcb8738d7b2c267b97ce62fa92adc3746a0ea" error="<nil>" remote="192.168.9.175:8083/mongo:4.2.3"
DEBU[2021-08-17T10:37:19.429454057-04:00] Pulling ref from V2 registry: xxx.xxx.x.xxx:8083/mongo:4.2.3
When I run: sudo docker pull mongo:4.2.3 it fails to pull the image from Nexus with an error and pulls from docker.io on the next try. Debug info as below:
DEBU[2021-08-17T10:26:25.078886904-04:00] Calling HEAD /_ping
DEBU[2021-08-17T10:26:25.079306196-04:00] Calling GET /v1.41/info
DEBU[2021-08-17T10:26:25.097994642-04:00] Calling POST /v1.41/images/create?fromImage=mongo&tag=4.2.3
DEBU[2021-08-17T10:26:25.099642151-04:00] Trying to pull mongo from http://xxx.xxx.x.xxx:8083/ v2
INFO[2021-08-17T10:26:25.116000813-04:00] **Attempting next endpoint for pull after error: manifest unknown: manifest unknown**
DEBU[2021-08-17T10:26:25.116039299-04:00] Trying to pull mongo from https://registry-1.docker.io v2
DEBU[2021-08-17T10:26:25.305043063-04:00] Fetching manifest from remote digest="sha256:58b25d51baa11a85b6aedf7c4e05710d12a27ddc2883e2692e7d58527d98bd73" error="<nil>" remote="docker.io/library/mongo:4.2.3"
DEBU[2021-08-17T10:26:25.360955030-04:00] Pulling ref from V2 registry: mongo:4.2.3
DEBU[2021-08-17T10:26:25.361036645-04:00] docker.io/library/mongo:4.2.3 resolved to a manifestList object with 5 entries; looking for a unknown/amd64 match
Issue with Image name:
REPOSITORY TAG IMAGE ID CREATED SIZE
xxx.xxx.x.xxx:8083/mongo 4.2.3 97a9a3e85158 17 months ago 386MB
Any guidance on this would help.
Nexus Docker ( xxx.xxx.x.xxx:8083) is pointed to hosted Type on port 8083 and the mongo:4.2.3 is uploaded into this docker type. We ultimately want to use this in a air gapped system where there is no internet connection.
There are three things going on here:
I am trying to pull docker image from Nexus repo without using the registry mirror in the command line and it is throwing an error. If I use the registry mirror in the pull it is succeeding but the image name is not I would like.
I'm going to recommend changing your likes. :)
If you want to pull from a specific registry, then use that registry in the image name. Trying to refer to your local registry with short names is merging two different image registry namespaces, which means it's trivial to run an image from the wrong namespace and result in a security breach. This was a large issue for other package repositories (see "dependency confusion" attacks) that docker was not susceptible to because they require the registry name as part of the image name (the only exception being Docker Hub). Even RedHat who tried to get options like add-registry and block-registry into the upstream docker engine (and failed, these options only ever appeared in a RedHat specific fork) is now telling users that it was a very bad idea and now their users are exposed to security vulnerabilities they can't easily fix because removing the feature will break lots of user environments.
Next, why doesn't the pull go to your registry? Because your image name doesn't match that of Docker Hub. Official images without a username are actually under the library repository. This is typically hidden from view, but you can do things like docker pull library/alpine or even docker pull docker.io/library/alpine instead of docker pull alpine, and all 3 will be pulling from the same place.
The fix is to run
docker pull xxx.xxx.x.xxx:8083/mongo:4.2.3
docker tag xxx.xxx.x.xxx:8083/mongo:4.2.3 xxx.xxx.x.xxx:8083/library/mongo:4.2.3
docker push xxx.xxx.x.xxx:8083/library/mongo:4.2.3
The last issue I actually can't help you with, it comes from the error message you're seeing when pulling from Hub, which should work:
docker.io/library/mongo:4.2.3 resolved to a manifestList object with 5 entries; looking for a unknown/amd64 match
The unknown/amd64 is unexpected to me, typically that would be linux/amd64 so there is something unexpected with the platform you're running your commands on. If you want to get into debugging that, update your question with docker info. You can try working around that with:
docker pull --platform linux/amd64 mongo:4.2.3
to force the platform, but that still doesn't explain why it doesn't know your current platform.
I guess you are trying to set your nexus docker repository to be the default one for the machine in the sealed network.
that needs changing because of the following from docker documentation:
Tag an image for a private repository
To push an image to a private registry and not the central Docker registry you must tag it with the registry hostname and port (if needed).
$ docker tag 0e5574283393 myregistryhost:5000/fedora/httpd:version1.0
with more upfront configuration and upkeep but no changes requiered for the client machines
Is if you have a DNS server in your network you could point docker.io to your nexus host ip address and put a proxy to intercept the communication and redirect and adapt the requests as they were to the nexus docker registry
Hopes this solves your pickle :)
Update 1:
It could be that you need to also change /etc/containers/registries.conf like specified here to only or also specify your nexus docker registry.
Update 2:
Before letting Gopi give up entirely, I would suggest using Podman as an alternative to Docker. Podman is a daemon-less container engine that works by forking processes to handle each running container. It seamlessly works with docker images thanks to the OCI standard, and on top of that, the only change when using it is replacing the docker command prefix with podman since all the commands are exactly the same. Podman was created by RedHat so by default it searches RedHat repos and you can add your own too as shown in this article that I mentioned before.
I have created docker hub repo and also created and pushed a docker image of python application to the repo.
However, I cannot find the correct Url of the image that I have to provide to the other services which will use this image. for eg azure service fabric or Kubernetes.
How can I find the exact URL? Through PowerShell or through the browser...
You don't usually download images by url. Instead, you use the docker CLI with the repository and image name.
If it's a private repo, login first, by using docker login
more about login
Use docker pull {reponame/imagename:tag} to download an image to your machine.
more about pull
Replace {reponame} with the repository name.
Replace {imagename} with the name you used with docker push.
Replace {tag} with the tag you put on the image (or latest).
For example, I use this line to get my docker hub image:
docker pull loekd/nanoserver:2.0
I have successfully built some Docker images:
Now I would like to start my microservices by docker-compose, unfortunatelly I am unable to pull those images i.e. repository callista/discovery-server not found: does not exist or no pull access I solved this error by logging into my DockerHub account and pushining those images to remote server. But it seems to me like a little overkill to send such larges images (which are likely to change pretty soon) over the Internet over and over again twice (push&pull).
Is it possible to configure Docker to install those images locally and not to pull from remote server?
I use Docker 1.8 and work on Windows 10.
Do you need to run this images in a server different from the one you build then?
If you need you have some alternatives:
As #engineer-dollery said, you can run a registry into your network, than you would not need to send it over the internet, only in your network. Docs: https://docs.docker.com/registry/deploying/
You could use the docker save and docker import to move then around too. Docs: https://docs.docker.com/engine/reference/commandline/save/
But if the server you run the images is the same you build then...
...than you could just add the tag image to your docker-compose services, and do a docker-compose build, as #lauri said, but with the image docker-compose will create a image with that name after the build, and then you could do docker run using than. Or do a docker-compose up --build so it will always build than again if something changes into the Dockerfile
If you define build option in docker-compose.yml, you should be able to build images locally with Docker Compose and then it uses those images without pulling. By default Docker Compose builds images if they are not found locally. If you want to rebuild images just add --build option docker-compose up command docker-compose up --build
Docker Compose build reference:
https://docs.docker.com/compose/compose-file/#build
Im building a deployment script in nodejs, with 1 part being calling the gcloud cli through require('child_process').spawn(...); to push the already build docker images. i execute the following command:
gcloud docker -- push myImage
This all works great, the images gets uploaded. But the problem is that gcloud docker opens a new process to push my image and the process i spawned, closes before the pushing of the image is done.
Problem is, I want to delete the builded images locally, directly afterwards.
I've been looking in the gcloud docker documentation but i don't see any argument for this.
Is there a way to know that the process of uploading the images was completed?
edit:
i did find a way to do it only through docker but i'd like a universal solution (both working on windows and linux environments)
After some more research on the google documentation, i found this authentication page
They tell you to create a service account and use the json private key you get as token to use into docker login. This way you don't need an oauth token for your automated services, but you can use this json key instead.
You can check all the images by running this command:
[sudo docker images]
Take a note of the "IMAGE ID" it will used when Tagging and deleting the image.
When you build a docker images, tag it before By running this command:
[docker tag "IMAGE ID" gcr.io/{the Google Container Registry path}:{version} ]
You can push any built image by running this command:
[gcloud docker -- push gcr.io/{the google container registry path}:{version}].
When pushing you will notice that list of container are pushed to your Google Container registry see the example below:
$ sudo gcloud docker -- push gcr.io/{the google container registry path}:{version}
The push refers to repository [gcr.io/{the google container registry path}]
43d35f91f441: =================> Pushed
3b93beb428bf: Layer already exists
629fa6a1373d: =================> Pushed
0f82335d5733: Layer already exists
c216b39a9ab6: Layer already exists
ccbd0c2af699: Layer already exists
38788b6810d3: Layer already exists
cd7100a72410: Layer already exists
v1: digest: sha256:**************************************************************** size: 1992
You can check all the images by running this command:
[sudo docker images]
Take a note of the "IMAGE ID" of the image you need to delete.
Run the command :
[sudo docker rmi "IMAGE ID"].
If the image doesn't allow to be deleted, you have to stop the container that is still running and prune the docker
[sudo docker container stop "the container ID"]
[sudo docker container prune]
Then you can delete the image.