I have access to a GitLab Container Registry and can push images as follows:
docker login --username $my_username -p $my_token $my_server/$my_project
docker tag $my_image:latest $my_server/$my_project/$my_image:latest
docker push $my_server/$my_project/$my_image:latest
I'd now like to use the Container Registry API as well and have tried this for a start (I'd like to list all tags subsequenty):
curl -H "PRIVATE-TOKEN: $my_token" https://$my_server/api/v4/projects/$my_project/registry/repositories
However, this results in "404 page not found". What am I missing, shouldn't the URL be valid according to the documentation?
I have by now concluded that the most likely cause is that my access token $my_token has been granted only limited privileges by the GitLab admin.
This perhaps includes the scopes read_registry and write_registry, but not api. Apparently there is no scope that specifically covers the Container Registry API without the rest of the GitLab API.
Related
I am trying to pull and push images between Docker Desktop and Azure and Visual Studio 2019.
currently I can push from VS2019 by Publish option and I can push to Docker and Azure Container Registry.
How do I pull from Azure to Docker? I believe there is an issue with security accounts between the 2 systems. After all, my Docker account is not my Azure account. I came across this article
https://learn.microsoft.com/en-us/azure/container-registry/container-registry-auth-service-principal
which contains a script. Is this the right article to solve my problem? I made a copy of the script but I am struggling to run it. If I save it to assignpermissions.sh file and run wsl ./assignpermissions.sh it complains that az does not exist.
So
Is that the right article to help me (eventually) pull and push between Azure and Docker?
How do I run the script when calling az is causing an error?
Any other things I need to watch out for in the next step?
Log in to a registry
There are several ways to authenticate to your private container registry.
Azure CLI
The recommended method when working in a command line is with the Azure CLI command az acr login. For example, to log in to a registry named myregistry, log into the Azure CLI and then authenticate to your registry:
az login
az acr login --name myregistry
Azure PowerShell
The recommended method when working in PowerShell is with the Azure PowerShell cmdlet Connect-AzContainerRegistry. For example, to log in to a registry named myregistry, log into Azure and then authenticate to your registry:
Connect-AzAccount
Connect-AzContainerRegistry -Name myregistry
You can also log in with docker login. For example, you might have assigned a service principal to your registry for an automation scenario. When you run the following command, interactively provide the service principal appID (username) and password when prompted. For best practices to manage login credentials, see the docker login command reference:
docker login myregistry.azurecr.io
Both commands return Login Succeeded once completed.
Note: You might want to use Visual Studio Code with Docker extension for a faster and more convenient login.
Tip: Always specify the fully qualified registry name (all lowercase) when you use docker login and when you tag images for pushing to your registry. In the examples in this article, the fully qualified name is myregistry.azurecr.io.
Push the image to your registry
Now that you've tagged the image with the fully qualified path to your private registry, you can push it to the registry with docker push:
docker push myregistry.azurecr.io/samples/nginx
Pull the image from your registry
Use the docker pull command to pull the image from your registry:
docker pull myregistry.azurecr.io/samples/nginx
In my server, I have deployed a custom docker registry with Docker Registry.
So commands below works (assuming the host is registry.example.com):
docker pull alpine
docker tag alpine registry.example.com/alpine
docker push registry.example.com/alpine
docker system prune -af # just clean up
docker pull registry.example.com/alpine # It works!
But then, when I navigate to registry.example.com on a browser, it shows a completely blank page (HTTP returns empty).
I would like to do something similar to hub.docker.com.
How can I add a custom HTML page, so that when navigating to registry.example.com it will return some HTML instead of an empty body? (I don't need a backend server for registration or database. Just a simple HTML document will suffice.)
Do I need to modify some files or config in /var/lib/registry?
Thanks a lot
on the Docker documentation, https://docs.docker.com/registry/configuration/
i was trying to find a setting to have a custom webpage, i could not find a setting for this. The docker hub probably has a page and retrieves the data from the Docker API
Docker HUB != Docker Registry
because
Docker HUB > Docker Registry
You need to create own RestAPI or WEB service that will call docker via CLI/docker internal API and response JSON data/HTML pages.
If you want to use ready solution, I recommend portainer as docker WebUI
I have a private GitLab project with a pipeline for building and pushing a Docker image. Therefore I have to authenticate to GitLab's Docker registry first.
Research
I read Authenticating to the Container Registry with GitLab CI/CD:
There are three ways to authenticate to the Container Registry via GitLab CI/CD which depend on the visibility of your project.
Available for all projects, though more suitable for public ones:
Using the special CI_REGISTRY_USER variable: The user specified by this variable is created for you in order to push to the Registry connected to your project. Its password is automatically set with the CI_REGISTRY_PASSWORD variable. This allows you to automate building and deploying your Docker images and has read/write access to the Registry. This is ephemeral, so it’s only valid for one job. You can use the following example as-is:
docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
For private and internal projects:
Using a personal access token: You can create and use a personal access token in case your project is private:
For read (pull) access, the scope should be read_registry.
For read/write (pull/push) access, use api.
Replace the <username> and <access_token> in the following example:
docker login -u <username> -p <access_token> $CI_REGISTRY
Using the GitLab Deploy Token: You can create and use a special deploy token with your private projects. It provides read-only (pull) access to the Registry. Once created, you can use the special environment variables, and GitLab CI/CD will fill them in for you. You can use the following example as-is:
docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY
and Container Registry:
With the update permission model we also extended the support for accessing Container Registries for private projects.
Version history
Your jobs can access all container images that you would normally have access to. The only implication is that you can push to the Container Registry of the project for which the job is triggered.
This is how an example usage can look like:
test:
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker pull $CI_REGISTRY/group/other-project:latest
- docker run $CI_REGISTRY/group/other-project:latest
I tried the first and the fourth way and I could authenticate.
Question
What are the pros and cons? I guess the third way is for deployment only, not for building and pushing. Same could be for the second way. Is that right?
And why is the fourth way not listed in the other documentation? Is that way deprecated?
I prefer the fourth option. A note: "If a user creates one named gitlab-deploy-token, the username and token of the deploy token is automatically exposed to the CI/CD jobs as CI/CD variables: CI_DEPLOY_USER and CI_DEPLOY_PASSWORD respectively.
When creating deploy token, you can grant permission read/write to registry/package registry.
The CI_REGISTRY_PASSWORD is ephemeral so avoid using it if you have multiple deploy jobs (which need to pull private image) run parallel.
I believe the differences are just about user skill and permissions.
The first way anyone can do since the variables are automatically present in a running job.
Second, anyone, with any permissions, can create a personal access token (but has an extra step compared to 1 to create the access token).
Third, someone with the correct permissions could create a deploy key. Deploy keys don't give access to the API like personal access tokens can, and only have permission to pull/read the data in the repository, they cannot write/push.
Fourth option, it allows you to both read/pull container images from the registry, but it also allows you to push to the registry. This is helpful if you have a CI step that builds an app in an image, or anything else where you're generating a container image and want to push it into the registry (so another step in the pipeline can pull it down and use it). My guess is that this option isn't listed with the others since it's meant for the building of container images. You probably could use it like any of the others though.
I am new to Docker. I know the default registry is 'docker hub'. And there are tutorials on navigating 'Docker Hub', e.g. search image etc. But that kind of operations are performed in Docker Hub UI via web.
I was granted a private Docker registry. After I login using the command like docker login someremotehost:8080, I do not know what command to use to navigate around inside the registry. I do not know what images are available and what their tags are.
Could anyone share some info/link on what command to use to explore private remote registry after user login?
Also, to use images from the private registry, the name I need to use becomes something like 'my.registry.address:port/repositoryname.
Is there a way to change the configuration of my docker application, so that it will make my.registry the default registry, and I can just use repositoryname, without specifying registry name in every docker command?
There are no standard CLI commands to interact with remote registries beyond docker pull and docker push. The registry itself might provide some sort of UI (for example, Amazon ECR can list images through the standard AWS console), or your local development team might have a wiki that lists out what's generally available.
You can't change the default Docker registry. You have a pretty strong expectation that e.g. ubuntu is really docker.io/library/ubuntu and not something else.
For the Docker there are only two commands for communication of registry:
Docker Pull and Docker Push
And about the docker private registry there is no any default setting in docker to get the pull from only from the specific registry. The reason for this is name of docker image.For official docker image there is direct name like Centos . But in the docker registry there is also some images which is created by non-official organisation or person. In that kind of docker images there is always name of user or organisation like pivotaldata/centos. So this naming convention is help to docker find the image in docker registry in public(via login) or public registry.
In the case you want to interact more with private repo you can write your own batch script or bash script. Like I have created a batch script which pull all the tag from the private repo if user give the wrong tag.
#echo off
docker login --username=xxxx --password=xxxx
docker pull %1:%2
IF NOT %ERRORLEVEL%==0 (
echo "Specified Version is Not Found "
echo "Available Version for this image is :"
for /f %%i in (' curl -s -H "Content-Type:application/json" -X POST -d "{\"username\":\"user\",\"password\":\"password\"}" https://hub.docker.com/v2/users/login ^|jq -r .token ') do set TOKEN=%%i
curl -sH "Authorization: JWT %TOKEN%" "https://hub.docker.com/v2/repositories/%1/tags/" | jq .results[].name
)
I'm using the official Docker registry image, and have configured it as a pull though cache.
My clients can log in and push/pull local images, such as this:
docker login -u username -p secret docker.example.local:5000
docker pull docker.example.local:5000/myImage
I've configured my clients to use the Docker registry server as a proxy:
root#server:/# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://docker.example.local:5000"]
}
But when my clients tries to pull images not already present on the registry server, I get an error. Example pull command:
docker pull alpine
The registry server then responds with this message in its log file:
error authorizing context: basic authentication challenge for realm \"Registry Realm\": invalid authorization credential
I came across this SO post suggesting putting a Nginx proxy server in front, but this seems like a hack and I'd prefer some cleaner way of doing this if possible.
How have others set up their registry server in a pull through cache mode - did you find a better solution than setting up an Nginx proxy in front of the registry server?
You are using wrong name of registry-server-name.
Do not use https:// prefix
#>docker login -u username -p secret docker.example.local:5000
You should ensure that you either provide environment variable REGISTRY_HTTP_HOST=https://docker.example.local:5000 or specify it in /etc/docker/registry/config.yml file of registry image
http:
addr: localhost:5000
prefix: /my/nested/registry/
host: https://docker.example.local:5000
# see https://docs.docker.com/registry/configuration/
Reason is that address used in docker login should match host configuration of docker registry.
It's been a bit since I dug through that code, but I believe docker will attempt to login to your pull through cache using your Hub credentials. It only uses that registries individual credentials when you pull from it directly. So you need to run docker login without a hostname to configure the Hub login. This is only between the docker engine and the mirror.
From the pull through cache to Hub, you configure the user/password in the pull through cache and anyone that can reach the cache will use those credentials when pulling from Hub. This means you need to ensure the cache is configured with a minimal access user or is only accessible by devices on the network that you trust.