Can you have a Docker registry with credentials? - docker

Assume I am running a docker registry at example.com, i would pull down images by:
docker pull example.com:5000/image:1
How can I password protect this registry? It might contain confidential code/data.

This is mentioned in the official documentation. There are several ways to secure a Docker registry. Note that all of them will only work with a TLS-secured registry (otherwise you'd be sending password or other credentials in plain-text, which would not add any security).
Authentication within the registry itself
Start by creating a credentials file in htpasswd format:
$ htpasswd -Bbn testuser testpassword > auth/htpasswd
Then mount this file into your registry container and pass the REGISTRY_AUTH_HTPASSWD_* environment variables:
$ docker run -d -p 5000:5000 --restart=always --name registry \
-v `pwd`/auth:/auth \
-e "REGISTRY_AUTH=htpasswd" \
-e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
-e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
-v `pwd`/certs:/certs \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
registry:2
Proxy authentication
Use a reverse proxy (like Nginx) running on the host or another container to handle authentication (documented in-depth here):
upstream docker-registry {
server registry:5000;
}
server {
listen 443 ssl;
server_name myregistrydomain.com;
ssl_certificate /etc/nginx/conf.d/domain.crt;
ssl_certificate_key /etc/nginx/conf.d/domain.key;
client_max_body_size 0;
location /v2/ {
auth_basic "Registry realm";
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
proxy_pass http://docker-registry;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 900;
}
}
External authentication with token server
As you've mentioned username/password protections, one of the solutions above will probably be sufficient for you. For completeness, you can also use authentication with an external authentication provider. The interface that a token server needs to implement is specified here, with the necessary configuration options being described here.

Related

I have to enter my login twice and ssh-key doesn't work on a dockerized gitlab

I launch gitlab with this command:
sudo docker run --detach --hostname example.com --publish 4433:443 --publish 8080:80 --publish 2222:22 --name gitlab --restart always --volume /data/gitlab/config:/etc/gitlab --volume /data/gitlab/logs:/var/log/gitlab --volume /data/gitlab/data:/var/opt/gitlab gitlab/gitlab-ce:latest
example.com being another URL, as you may have guessed.
I have an nginx server with this config:
server {
server_name example.com;
client_max_body_size 50m;
location / {
proxy_pass http://127.0.0.1:8080/;
proxy_set_header Host $http_host;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote-addr;
}
listen 443 ssl;
ssl_certificate [MY PATH TO THE .pem FILE];
ssl_certificate_key [OTHER PATH];
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}
server {
if ($host = example.com) {
return 301 https://$host$request_uri;
}
}
When I use HTTPS to clone and push repos, I have to enter my login/password twice, and when I use ssh (for example git clone git#example.com:myuser/myproject.git), it asks me for a password.
I triple checked, my ssh key configuration is correct.
I left the gitlab.rb config by default, except for this line:
external_url 'https://example.com'
What happens here?
For this particular key, I don't use a passphrase
That means SSH fails to connect to example.com as git, and falls back to the Identity authentication: git's password (which you are not supposed to have).
Using a port syntax HOST_PORT:CONTAINER_PORT, you are supposed to launched your GitLab Docker container with a host port (for instance 2222) mapped to GitLab internal SSH daemon (port 22)
sudo docker run [...] -port 2222:22
Then check it is working with:
ssh -T git#example.com -p 2222
Welcome to GitLab, #you!
With a ~/.ssh/config file, it is easier:
Host gl
hostname example.com
port 2222
User git
IdentityFile ~/.ssh/yourGitLabkey
Then:
ssh -T gh
Welcome to GitLab, #you!
See as examples this thread, or this thread, based on the official documentation "Install GitLab using Docker Compose", mentioned by issue 1767.

Gitlab docker instance doesn't take my external URL

I launched a gitlab container like this:
sudo docker run --detach --hostname MY_URL.com --publish 4433:443 --publish 8080:80 --publish 2222:22 --name gitlab --og/gitlab --volume /data/gitlab/data:/var/opt/gitlab gitlab/gitlab-ce:latest
And I have a NGINX configuration like this:
server {
server_name MY_URL.com;
location / {
proxy_pass http://127.0.0.1:8080/;
proxy_set_header Host $http_host;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwardedd_for;
proxy_set_header X-Forwarded_Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/MY_URL.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/MY_URL.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = MY_URL.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name MY_URL.com;
return 404; # managed by Certbot
}
With this configuration, everything works fine, I can type https://MY_URL.com in the address bar of my browser and I can get access to my Gitlab.
The problem is that the link to clone in the repositories is "HTTP" and not "HTTPS". Moreover, it seems that there is a configuration somewhere telling my CI jobs to use "HTTP://MY_URL.com" (and it doesn't work because I get an HTTP basic auth error, which I wouldn't get if I used https I think). I read the documentation and I thought I just had to modify the external_url parameter:
sudo vi /data/gitlab/config/gitlab.rb
Adding external_url 'https://MY_URL.com'
sudo docker exec -it gitlab gitlab-ctl reconfigure
But after doing that I always have a "bad redirection" if I write "http://MY_URL.com" or "https://MY_URL.com". In the nginx logs, I don't have any error but only 301 in the access.log.
What am I doing wrong here?
Thanks a lot in advance...
Because you are providing an external NGINX configuration that also terminates SSL, you have to apply a configuration to your GitLab instance for external proxy/load-balancer SSL termination.
Normally, when you don't provide external_url, the system host name is used and HTTPS is disabled. If you provide an external_url with an https:// scheme, this will activate HTTPS, which is not what you want since you are using an external server (NGINX) for SSL/TLS termination.
external_url "https://myhost.com"
nginx['listen_port'] = 80
nginx['listen_https'] = false
This should be all you need to get GitLab to display the correct hostname in the UI without any other behavior changes.
You'll probably also want to change the proxy headers since you already have a proxy server in front of GitLab. You'll want to configure trusted proxies as well as the real-ip header to make sure GitLab correctly logs the IP address of your users (instead of the IP of your proxy).

Docker/GitLab authentication for docker registry returns 401 error

I'm trying to configure my docker registry using auth of gitlab (docker).
Doing docker login registry.website.com gives me a 401 Unauthorized error:
Error response from daemon: login attempt to https://registry.website.com/v2/ failed with status: 401 Unauthorized
In the docker logs I find
{ "level":"info","msg":"token signed by untrusted key with ID: \"IWNY:KT2H:YUN5:STQP:22LM:YNIU:RT4T:AZO7:TBVL:ZQ3I:Z4JZ:LA3T\"","time":"2018-12-17T23:36:03.538232467Z" }
{ [...] "level":"warning","msg":"error authorizing context: invalid token","service":"registry","source":"registry","time":"2018-12-17T23:36:03.53860308Z","version":"v2.6.2" }
My keys are generated by doing
$ sudo openssl req -new -newkey rsa:4096 -subj "/CN=gitlab-issuer" -nodes -x509 -keyout registry-auth.key -out registry-auth.crt
$ sudo chmod 400 registry-auth.key
In my debugging attempt I do get different sha256 digest:
459b854f47c51bd94e0fd696cc35148cf93065df986abcc368cf13958373d298
459b854f47c51bd94e0fd696cc35148cf93065df986abcc368cf13958373
As #VDR has shown this is ok, as the first 30 characters are used. So with that there should not be a problem with the keys. But why do I get the 401 error?
This is how I configured gitlab and the registry:
The configuration of docker gitlab (gitlab.rb) uses the key by
gitlab_rails['registry_key_path'] = "/certs/registry-auth.key"
Config of registry has
auth.token.rootcertbundle: /root/certs/registry-auth.crt
nginx-proxy/vhost.d/docker-registry.conf
proxy_pass http://registry.website.com;
proxy_set_header Host $http_host; # required for docker client's sake
proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 900;
I don't see what I'm missing here...
If your Gitlab is behind a proxy, you will need to configure the proxy in docker.
To configure docker to use the proxy, put the following in your ~/.docker/config.json file where your docker is running.
{
"auths" : {
},
"proxies":
{
"default":
{
"httpProxy": "http://myproxy/",
"httpsProxy": "http://myproxy/"
}
}
}
If there is anything already in the "auths": {} section, you should leave it as is.
Save this file then restart your docker daemon. Once docker is back up, you should be able to run docker login ... without issues.

How to login to a Docker Registry that has an HTTP prefix set?

I have setup my own Docker Registry, but I did not want it on the root URL so when I created the service I used the REGISTRY_HTTP_PREFIX environment variable and set it to /registry/, thus the URL to the registry is https://tools.example.com/registry. This is being proxied by Nginx which has Basic Auth setup on it.
I tested access to the registry using a Browser and I was able to get it to show that there are no repositories by going to http://tools.example.com/registry/v2/_catalog:
This led me to think that it was workoing. However when I try to login to the registry using the Docker command line, I get the Basic Auth challenge but then it fails to login because the URL is incorrect, e.g.
docker login -u russells -p xxxxxxxx https://tools.example.com/registry/
Error response from daemon: login attempt to https://tools.example.com/v2/ failed with status: 404 Not Found
As can be seen from the error, the prefix is not being added properly. SO how can I login to the registry so I can push images. Is there an environment variable or something that I am missing to make the docker login work properly?
Update - 2017-08-12 2253 BST
I Have been playing around with the configuration a bit, but I am still not getting very far.
As requested here are my configuration files.
nginx.conf
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
keepalive_timeout 65;
upstream docker-registry {
server registry:5000;
}
map $upstream_http_docker_distribution_api_version $docker_distribution_api_version {
'' 'registry/2.0';
}
server {
listen 15000;
server_name tools.example.com;
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
# required to avoid HTTP 411
chunked_transfer_encoding on;
location /registry/ {
# Do not allow connections from docker 1.5. and earlier
# docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$") {
return 404;
}
auth_basic "Docker Registry";
auth_basic_user_file /etc/nginx/.htpasswd;
add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;
proxy_pass http://docker-registry/registry/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 900;
}
}
}
My Docker Registry service is deployed as registry and is running on the default port of 5000. Looking at this now I think I have got things confused. I do not need the registry to answer on the prefix itself, just Nginx.
For example if I leave the location set to / then I can login, but if I change this to /registry/ then I am not able to. I am beginning to think that the two are conflicting each other.
Registry
I have not set a configuration for the Registry other than the one environment variable - REGISTRY_HTTP_PREFIX, which maybe surplus to requirements in this setup.
Update - 2017-08-15 1100 BST
In order to test the prefix for the registry I created a registry container with the following configuration file:
version: 0.1
auth:
htpasswd:
realm: Docker Registry
path: /auth/etc/htpasswd
storage:
filesystem:
rootdirectory: /var/lib/registry
maxthreads: 100
http:
addr: 0.0.0.0:5000
prefix: /registry/
tls:
certificate: /auth/ssl/certs/registry.cert
key: /auth/ssl/private/registry.key
As this is using self signed certificates I updated my Docker engine by placing the certificate in /etc/docker/certs.d/host-lin-01:5000.
I then created the container with the following command:
docker run -it --rm -p 5000:5000 --name registry_test -v ~/workspaces/docker/registry/etc/registry.yml:/etc/docker/registry/config.yml -v ~/workspaces/docker/registry:/auth registry:2
If I try and login to the registry with the command:
docker login -u russells -p xxxxxx https://host-lin-01:5000/registry
I get the following error:
Error response from daemon: login attempt to https://host-lin-01:5000/v2/ failed with status: 404 Not Found
Now if I remove the perfix: /registry/ line from the registry yaml file and restart the container and then login all is well:
docker login -u russells -p xxxxxx https://turtle-host-03:5000/
Login Succeeded
What is strange, however, is that the login works for any prefix I put on the end of the login URL, e.g.
docker login -u russells -p xxxxxx https://turtle-host-03:5000/registry/fred/34
Login Succeeded
I do not understand this. I must be misunderstanding what the prefix setting does.
You issue is the application of your Basic Auth. So you have Nginx with Basic Auth which is backed by a plain registry.
You are able to authenticate urls and see the _catalog blank json, and that make you feel it is working. But technically what is happening is that your Nginx is asking for username/password, which gets it and then passes is on to you docker registry. Which in turn has no authentication.
Now when you use docker login you are expecting a authenticated registry but you have a authenticated nginx and non-authenticated registry. So you need to ditch the below lines of code from your nginx config
auth_basic "Docker Registry";
auth_basic_user_file /etc/nginx/.htpasswd;
Also when launching your registry you need to define the below environment variables
REGISTRY_AUTH: htpasswd
REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
Make sure you map /auth/htpasswd from your host to the registry container. Do this and the setup should work. Also make sure to setup the server certificates in your docker client system
Optional Changes
Next part of this answer is optional as such. Since you are using Nginx and Registry both. I would suggest you ditch the REGISTRY_HTTP_PREFIX from your registry and change the proxy_pass to
proxy_pass http://docker-registry/;

Gitlab links to "https://gitlab/"

I installed gitlab in a docker container from the official image gitlab/gitlab-ce:latest. This image has all config in the file gitlab.rb. Https is done by a nginx reverse proxy.
My problem is, that when gitlab has an absolute link to itself, it links always to https://gitlab/. This host also can be seen in the "New group" dialog:
Docker call:
docker run \
--name git \
--net mydockernet \
--ip 172.18.0.2 \
--hostname git.mydomain.com \
--restart=always \
-p 766:22 \
-v /docker/gitlab/config:/etc/gitlab \
-v /docker/gitlab/logs:/var/log/gitlab \
-v /docker/gitlab/data:/var/opt/gitlab \
-d \
gitlab/gitlab-ce:latest
gitlab.rb:
external_url 'https://git.mydomain.com'
ci_nginx['enable'] = false
nginx['listen_port'] = 80
nginx['listen_https'] = false
gitlab_rails['gitlab_shell_ssh_port'] = 766
Nginx config:
upstream gitlab {
server 172.18.0.2;
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name git.mydomain.com;
ssl on;
ssl_certificate /etc/ssl/certs/mydomain.com.chained.pem;
ssl_certificate_key /etc/ssl/private/my.key;
location / {
proxy_pass http://gitlab/;
proxy_read_timeout 10;
}
}
I tried to replace the wrong url wit nginx. This worked for the appearance like in the screen shot, but not for the links:
sub_filter 'https://gitlab/' 'https://$host/';
sub_filter_once off;
You've set your url correctly in your mapped volume /etc/gitlab/gitlab.rb
external_url 'https://git.mydomain.com'
Run sudo gitlab-ctl reconfigure for the change to take effect.
Problem solved. It was in the nginx reverse proxy config. I named the upstream "gitlab" and somehow this name found it's way into the web pages:
upstream gitlab {
server 172.18.0.2;
}
...
proxy_pass http://gitlab/;
I didn't expect that. I even omitted the upstream part in my original question because I thought the upstream name is just an internal name and will be replaced by the defined ip.
So the fix for me is, to write the full domain into the upstream:
upstream git.mydomain.com{
server 172.18.0.2;
}
...
proxy_pass http://git.mydomain.com/;

Resources