I have a docker container with Trivy installed.
I have a remote registry with docker images.
and
I would like to download the docker images to the container for scanning
Challenges
It is hard to run docker within a docker container for pulling the images.
Trivy requires that you have the images locally before it can scan the images, either in a local registry or as a file.
I found two solutions:
Download the images with Skopeo
Download the images with the HTTP API V2
For the API I had a hard time making the authentication work, as it is repository specific, and Scaleways' authentication had unexpected behaviour.
Expand RED HAT CONTAINER REGISTRY with a local Docker image repository.
As a Docker user, I can work with the registry (pull, push) everything is OK!
But also, as a user, I can upload an image with the docker hub with the pull command.
For example:
$ docker pull debian
Question:
How to prevent the user to "pull" the image with docker hub and allow using only the openshift internal registry?
Those. Is it possible to configure any config so that the user accesses the internal repository of images?
I'm confused as to the difference between docker registries and repositories. It seems like the Docker documentation uses the two words interchangeably. Also, repositories are sometimes referred to as images, such as this from their docs:
In order to push a repository to its registry, you need to have named
an image or committed your container to a named image as we saw here.
Now you can push this repository to the registry designated by its
name or tag.
How can you push a repository to a registry? Aren't you pushing the image to the repository?
Docker registry is a service that is storing your docker images.
Docker registry could be hosted by a third party, as public or private registry, like one of the following registries:
Docker Hub,
Quay,
Google Container Registry,
AWS Container Registry
or you can host the docker registry by yourself
(see https://docs.docker.com/ee/dtr/ for more details).
Docker repository is a collection of different docker images with same name, that have different tags. Tag is alphanumeric identifier of the image within a repository.
For example see https://hub.docker.com/r/library/python/tags/. There are many different tags for the official python image, these tags are all members of the official python repository on the Docker Hub. Docker Hub is a Docker Registry hosted by Docker.
To find out more read:
https://docs.docker.com/registry/
https://github.com/docker/distribution
From the book Using Docker, Developing and deploying Software with Containers
Registries, Repositories, Images, and Tags
There is a hierarchical system for storing images.
The following terminology is used:
Registry
A service responsible for hosting and distributing images. The default registry is the Docker Hub.
Repository
A collection of related images (usually providing different versions of the same application or service).
Tag
An alphanumeric identifier attached to images within a repository (e.g., 14.04 or stable ).
So the command docker pull amouat/revealjs:latest will download the image tagged latest within the amouat/revealjs repository from the Docker Hub registry.
Complementing the information:
You usually push a repository to a registry (and all images that are part of it). But you can push a single image to a registry. In all cases, you use docker push.
An image has a 12-hex-digit Image ID, but is also identified by: namespace/repo-name:tag
The image full name can be optionally prefixed by the registry host name and port: myregistryhost:5000/namespace/repo-name:tag
A common naming convention is to use your registry user-name as what I called "namespace".
A docker repository is a cute combination of registry and image.
docker tag foo <registry>/<image>:<tag>
is the same as
docker tag foo <repository>:<tag>
Docker Registry is a service, which you can either host yourself (Trusted and Private) or you can let docker hub be the host for this service. Usually, if your software is commercial, you will have hosted this as a "Private and Trusted" registry. For Java Developers, this is somewhat analogous to Maven Artifactory setup.
Docker Repository is a set of "Tagged" images. An example is that you might have tagged 5 of ubuntu:latest images:
a) Nano editor (image1_tag:v1)
b) A specific software 1 (image1_tag:v2)
c) Sudo (image1_tag:v3)
d) apache http daemon (image1_tag:v4)
e) tomcat (image1_tag:v5)
You can use docker push command to push each of the above images to your repository. As long as the repository names match, they will be pushed successfully, and appear under your chosen repository and correctly tagged.
Now, your question is, "So where is this repository hosted/who is managing the service"? That is where Docker Registry comes into picture. By default you will get a docker hub registry (Open Source) which you can use to keep your private/public repository. So without any modification, your images will be pushed to your private repository in docker hub. An example output when you pushing your image tags are the following:
docker#my-docker-vm:/$ docker push mydockerhub/my-helloworld-repo:my_tag
The push refers to repository [docker.io/mydockerhub/my-helloworld-repo]
bf41e934d39d: Pushed
70d93396f87f: Pushed
6ec525dfd060: Pushed
705419d10b13: Pushed
a4aaef726d02: Pushed
04964fddc946: Pushed
latest: digest: sha256:eb93c92351bce785aa3ec0de489cfeeaafd55b7d90adf95ecea02629b376e577 size: 1571
docker#my-docker-vm:/$
And if you type immediately docker images --digests -a you can confirm that your pushed image tags are now showing new signature against the private repository managed by docker hub registry.
A Docker image registry is the place to store all your Docker images. The image registry allows you to push and pull the container images as needed.
Registries can be private or public. When the registry is public, the images are shared with the whole world whereas in the private registry the images are shared only amongst the members of an enterprise or a team.
A registry makes it possible for the Docker daemon to easily pull and run your Docker images.
Docker Hub and other third party repository hosting services are called “registries”. A registry stores a collection of repositories.
As a registry can have many repositories and a repository can have many different versions of the same image which are individually versioned with tags.
The confusion starts with this definition of a tag: "An alphanumeric identifier attached to images in a repository"
I'd rather call that alphanumeric identifier that you append with a ':' a tag-suffix for now. When somebody says "'latest' is the default tag", then this kind of tag-suffix is meant.
In reality, the :latest' suffix is technically part of the tag. The entire name is a tag. All these are tags (possibly referring to the same image):
myimagename
myimagename:latest
username/theirimagename:1.0
myrepo:5000/username/imagename:1.0
(I say imagename here, just to illustrate the other main source of confusion. That's the repositoryname, of course. Sorry.)
Examples:
a) When you want to name your image while building, you use docker build -t thisname ... -- that is -t for tag, (not -n for name).
b) When you want to push that image to a registry, you need to have the full URL (starting with registryname and ending with a tag-suffix) as a tag:
docker tag thisname mylocalregistry:5000/username/repoimagething:1.0
Now you push the image known as thisname by saying:
docker push mylocalregistry:5000/username/repoimagething:1.0
Naming things is hard.
Alas! A repository is not a "container" (aaargh...) where you put things in, that is what muggles think...
I'm new to docker, then the problem may describe not clearly in title. I want to do it that when I use pull image, it will check local registry at first. Then, if there is no this image in local registry, it will download from docker hub and auto push it to local registry. How to do it?
docker hub <-> registry <-> client pull image
It can use Mirror to do this work:
it pulls the image from the public Docker registry and stores it
locally before handing it back to you. On subsequent requests, the
local registry mirror is able to serve the image from its own storage.
I'm wondering where Docker's images are exactly stored to in my local host machine.
Can I share my Docker-Image without using the Docker-Hub or a Dockerfile but the 'real' Docker-Image? And what is exactly happening when I 'push' my Docker-Image to Docker-Hub?
Docker images are stored as filesystem layers. Every command in the Dockerfile creates a layer. You can also create layers by using docker commit from the command line after making some changes (via docker run probably).
These layers are stored by default under /var/lib/docker. While you could (theoretically) cherry pick files from there and install it in a different docker server, is probably a bad idea to play with the internal representation used by Docker.
When you push your image, these layers are sent to the registry (the docker hub registry, by default… unless you tag your image with another registry prefix) and stored there. When pulling, the layer id is used to check if you already have the layer locally or it needs to be downloaded. You can use docker history to peek at which layers (other images) are used (and, to some extent, which command created the layer).
As for options to share an image without pushing to the docker hub registry, your best options are:
docker save an image or docker export a container. This will output a tar file to standard output, so you will like to do something like docker save 'dockerizeit/agent' > dk.agent.latest.tar. Then you can use docker load or docker import in a different host.
Host your own private registry. - Outdated, see comments See the docker registry image. We have built an s3 backed registry which you can start and stop as needed (all state is kept on the s3 bucket of your choice) which is trivial to setup. This is also an interesting way of watching what happens when pushing to a registry
Use another registry like quay.io (I haven't personally tried it), although whatever concerns you have with the docker hub will probably apply here too.
Based on this blog, one could share a docker image without a docker registry by executing:
docker save --output latestversion-1.0.0.tar dockerregistry/latestversion:1.0.0
Once this command has been completed, one could copy the image to a server and import it as follows:
docker load --input latestversion-1.0.0.tar
Sending a docker image to a remote server can be done in 3 simple steps:
Locally, save docker image as a .tar:
docker save -o <path for created tar file> <image name>
Locally, use scp to transfer .tar to remote
On remote server, load image into docker:
docker load -i <path to docker image tar file>
[Update]
More recently, there is Amazon AWS ECR (Elastic Container Registry), which provides a Docker image registry to which you can control access by means of the AWS IAM access management service. ECR can also run a CVE (vulnerabilities) check on your image when you push it.
Once you create your ECR, and obtain the "URL" you can push and pull as required, subject to the permissions you create: hence making it private or public as you wish.
Pricing is by amount of data stored, and data transfer costs.
https://aws.amazon.com/ecr/
[Original answer]
If you do not want to use the Docker Hub itself, you can host your own Docker repository under Artifactory by JFrog:
https://www.jfrog.com/confluence/display/RTF/Docker+Repositories
which will then run on your own server(s).
Other hosting suppliers are available, eg CoreOS:
http://www.theregister.co.uk/2014/10/30/coreos_enterprise_registry/
which bought quay.io