I'm trying to set up a workflow where I can git pull a docker container from a git repository on a local machine, then push it to a private docker registry where many people can access it. The issue is, I want it so anyone from any machine anywhere will be able to pull from this registry GRANTED they have some sort of authentication. Sort of like a private web hosted docker cloud. Is that possible?
If you aren't squeamish, I would thoroughly recommend Portus
[https://github.com/SUSE/Portus][1]
as a means to secure and manage your registry.
The registry itself can be set up in one command;
docker run -d -e SEARCH_BACKEND=sqlalchemy --restart always -v /var/docker/registry/conf:/etc/docker -v /var/docker/registry/data:/var/lib/registry -p 5000:5000 --name registry registry:2
See https://docs.docker.com/registry/deploying/ for a detailed reference.
I generally prefer to run the registry without SSL, offloading all SSL to a shared haproxy (also a docker container).
Related
I'm new to Docker. Is it possible to have a Docker base image repository outside Docker Hub? Let's say store them in your cloud rather than having DH account? Thanks.
You can host your own registry as you would like. Full details of hosting one's own registry server can be found at Deploy a registry server.
The the highest level, the following will suffice:
docker run -d -p 5000:5000 --restart=always --name registry registry:2
Some cloud providers give you your own registries ready to use. For example, Google Container Registry.
Google has its own registry for containers which I assume may be relevant based on the GCP tag. Check out the Google Container Registry at gcr.io. You can upload and pull containers from there just as you would with the Docker Hub.
Not sure if SO is the correct forum to ask the following question. Please move them to the correct one if it's not.
I'd like to set up a Docker Private Registry, but after reading Docker's documentation (and related SO questions) - am not sure if:
Docker Trusted Registry (DTR) is needed?
Can I set up a single DTR under the Docker free plan?
Can anyone answer the above?
Docker Trusted Registry is a commercial offering from Docker Inc. It includes the on-premises registry server, optional integration with their UCP product, RBAC, integration with notary (rebranded as Content Trust) for image signing, and vulnerability scanning. There is no free or open source version of DTR itself.
Docker does have an open source registry product that you can download and run as a container in your own environment. It's available on the docker hub. Running this is as easy as:
$ docker run -d -p 5000:5000 --restart=unless-stopped --name registry \
-v registry-data:/var/lib/registry \
-e "REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/var/lib/registry" \
registry:2
And then you can push/pull to your local registry with:
$ docker tag your_image 127.0.0.1:5000/your_image:latest
$ docker push 127.0.0.1:5000/your_image:latest
$ docker pull 127.0.0.1:5000/your_image:latest
Note that this is configured as an insecure registry, there are more steps to make it secure with TLS and add authorization. To connect to it from other hosts, you'd need to either add TLS to the registry, or update the other docker hosts with your IP as an insecure registry for dockerd.
You can also use Docker Hub itself for your registry if you are posting public images in the cloud. More than a single private repo would required a paid plan.
There are also third party implementations of the registry api. Two that I'm aware of are Nexus and Artifactory.
I have two docker containers - one running jenkins and one running docker registry. I want to build/push images from jenkins to docker registry. How do I achieve this in an easy and secure way (meaning no hacks)?
The easiest would be to make sure the jenkins container and registry container are on the same host. Then you can mount the docker socket onto the jenkins container and use the dockerd from the host machine to push the image to the registry. /var/run/docker.sock is the unix socket the dockerd is listening to.
By mounting the docker socket any docker command you run from that container executes as if it was the host.
$ docker run -dti --name jenkins -v /var/run/docker.sock:/var/run/docker.sock jenkins:latest
If you use pipelines, you can install this Docker Plugin https://plugins.jenkins.io/docker-workflow,
create a credentials resource on Jenkins,to access the Docker registry, and do this in your pipeline:
stage("Build Docker image") {
steps {
script {
docker_image = docker.build("myregistry/mynode:latest")
}
}
}
stage("Push images") {
steps {
script {
withDockerRegistry(credentialsId: 'registrycredentials', url: "https://myregistry") {
docker_image.push("latest")
}
}
}
}
Full example at: https://pillsfromtheweb.blogspot.com/2020/06/build-and-push-docker-images-with.html
I use this type of workflow in a Jenkins docker container, and the good news is that it doesn't require any hackery to accomplish. Some people use "docker in docker" to accomplish this, but I can't help you if that is the route you want to go as I don't have experience doing that. What I will outline here is how to use the existing docker service (the one that is running the jenkins container) to do the builds.
I will make some assumptions since you didn't specify what your setup looks like:
you are running both containers on the same host
you are not using docker-compose
you are not running docker swarm (or swarm mode)
you are using docker on Linux
This can easily be modified if any of the above conditions are not true, but I needed a baseline to start with.
You will need the following:
access from the Jenkins container to docker running on the host
access from the Jenkins container to the registry container
Prerequisites/Setup
Setting that up is pretty straight forward. In the case of getting Jenkins access to the running docker service on the host, you can do it one of two ways. 1) over TCP and 2) via the docker unix socket. If you already have docker listening on TCP you would simply take note of the host's IP address and the default docker TCP port number (2375 or 2376 depending on whether or not you use TLS) along with and TLS configuration you may have.
If you prefer not to enable the docker TCP service it's slightly more involved, but you can use the UNIX socket at /var/run/docker.sock. This requires you to bind mount the socket to the Jenkins container. You do this by adding the following to your run command when you run jenkins:
-v /var/run/docker.sock:/var/run/docker.sock
You will also need to create a jenkins user on the host system with the same UID as the jenkins user in the container and then add that user to the docker group.
Jenkins
You'll now need a Docker build/publish plugin like the CloudBees Docker Build and Publish plugin or some other plugin depending on your needs. You'll want to note the following configuration items:
Docker URI/URL will be something like tcp://<HOST_IP>:2375 or unix:///var/run/docker.sock depending on how we did the above setup. If you use TCP and TLS for the docker service you will need to upload the TLS client certificates for your Jenkins instance as "Docker Host Certificate Authentication" to your usual credentials section in Jenkins.
Docker Registry URL will be the URL to the registry container, NOT localhost. It might be something like http://<HOST_IP>:32768 or similar depending on your configuration. You could also link the containers, but that doesn't easily scale if you move the containers to separate hosts later. You'll also want to add the credentials for logging in to your registry as a username/password pair in the appropriate credentials section.
I've done this exact setup so I'll give you a "tl;dr" version of it as getting into depth here is way outside of the scope of something for StackOVerflow:
Install PID1 handler files in container (i.e. tini). You need this to handle signaling and process reaping. This will be your entrypoint.
Install some process control service (i.e. supervisord) packages. Generally running multiple services in containers is not recommended but in this particular case, your options are very limited.
Install Java/Jenkins package or base your image from their DockerHub image.
Add a dind (Docker-in-Docker) wrapper script. This is the one I based my config on.
Create the configuration for the process control service to start Jenkins (as jenkins user) and the dind wrapper (as root).
Add jenkins user to docker group in Dockerfile
Run docker container with --privileged flag (DinD requires it).
You're done!
Thanks for your input! I came up with this after some experimentation.
docker run -d \
-p 8080:8080 \
-p 50000:50000 \
--name jenkins \
-v pwd/data/jenkins:/var/jenkins_home \
-v /Users/.../.docker/machine/machines/docker:/Users/.../.docker/machine/machines/docker \
-e DOCKER_TLS_VERIFY="1" \
-e DOCKER_HOST="tcp://192.168.99.100:2376" \
-e DOCKER_CERT_PATH="/Users/.../.docker/machine/machines/docker" \
-e DOCKER_MACHINE_NAME="docker" \
johannesw/jenkins-docker-cli
I am trying to set up a private Docker registry. I [found this tutorial](https://github.com/docker/distribution/blob/master/docs/deploying.md
) which states I need to run:
docker run -d -p 5000:5000 --restart=always --name registry registry:2
But this implies that Docker is already installed & running on the server. So I created a new Ubuntu 14.04 (upstart-based) VM and installed Docker [using the recommended procedure](https://docs.docker.com/installation/ubuntulinux/#installation
) and verified that Docker is running by using docker -v.
A few concerns/issues here:
If Docker is already installed as an upstart service/daemon, how do I configure it to run using the “Registry Mode” command?
I need Docker to run using the registry command shown above, but it’s already running on the VM. How do I get modify the service/daemon to run the registry command, do I need to configure upstart?
Where can I find docs on the arguments passed into this registry command?
I can’t find any docs on what these various command-line args are. 5000:5000…what does that do?!? --restart=always? Any links/ideas?
Docker Registry is just another container that runs on your Docker Host.
The --restart=always will set the container to restart if it goes down. (Like after a system reboot)
The 5000:5000 is the published port mapping for the container, Docker Registry will listen on port 5000.
Good documentation can be found here
I'm trying to run an image from a private registry with docker swarm.
I have an image I've tagged and pushed to a private registry. If I run this locally:
docker run -p 8000:8000 -d registry.mydomain.com:8080/myimage
it runs fine.
If I activate my swarm and try and run from there:
$(docker-machine env --swarm swarm-master)
docker login registry.mydomain.com:8080
docker run -p 8000:8000 -d registry.mydomain.com:8080/myimage
I get "Authentication is required".
I'm actually trying to do this via the docker remote API, but first I figure I should get it running on the command line.
Is this possible?
Thanks!
Just curious, you are using authentication, but no SSL? I think docker only supports basic authentication over SSL. You could try to start docker with the insecure flag to at least try out the capabilities of swarm.
docker -d --insecure-registry registry.mydomain.com:8080
The error you are getting is probably docker swarm host trying to pull down the image from your registry first since run can be short hand for pull me this image and run it.