Background
There is a private docker registry I have no control of. This registry is not accessible from my computer but is accessible from a remote server I have access to.
This is my current (quite inefficient) workflow:
# On my machine
$ docker save IMAGE > FILE
$ scp FILE SERVER
$ ssh SERVER
# On the server
$ docker load < FILE
$ docker tag -f IMAGE REGISTRY:5000/IMAGE
$ docker push REGISTRY:5000/IMAGE
Problem
It takes forever to push the image as I need to save, upload and load the whole tarball even if there are no changes in most of the docker layers.
I tried to use ssh to forward the docker registry port (5000) to a port on my machine:
$ ssh -L 5042:REGISTRY:5000 SERVER
Now I can communicate with the registry from my machine:
$ curl localhost:5042/v2/
{}
But the docker wont push images to it:
$ docker tag IMAGE localhost:5042/IMAGE
$ docker push localhost:5042/IMAGE
The push refers to a repository [localhost:5042/IMAGE] (len: 1)
Sending image list
FATA[0000] Put http://localhost:5042/v1/repositories/IMAGE/: dial tcp 127.0.0.1:5042: connection refused
I have a feeling that the problem is in different name/tag of the image. On the server I need to tag it as REGISTRY:5000/IMAGE but on localhost it would make no sense as the REGISTRY url is not accessible from my computer.
Or the problem may be caused by the fact that I am running docker through docker-machine.
Question
Can I somehow push to a private docker registry that is port-forwarded to a local port?
There need to be two ssh tunnels. One from the local machine (this one is used by the docker client) and one from the docker-machine (this one is used by docker daemon).
docker-machine ssh
ssh -L 5042:REGISTRY:5000 SERVER
Related
So far, I have always copied files from the docker container to my VM first (web host), and later, run scp command line from my local machine to download it from the VM. Similar scenario happening for uploading files/folders. Is there a direct way to do that using scp?
In order to directly copy from your container you need sshd installed on the container and expose an port for ssh to public when you run the container.
Take in count that if you do you have to make sure that ssh is properly configured and secured.
Example:
*We take in count that you already have ssh configured on the container
docker run -d -p 8000:22 --name docker image
scp -P 8000 username#myserver.com:/root/file.txt ~/file.txt
I have an image that I'm using to run my CI/CD builds (using GitLab CE). I'd like to deploy my app doing something like this from within the container:
eval "$(docker-machine env manager)"
sudo docker stack deploy --compose-file docker-stack.yml web
However, I'd like the docker-machine to access machines defined on the host system since the container will be destroyed and I don't want to include access details in the image.
I've tried a few things
Accessing the Remote Host via docker-machine
Create the docker-machine on the host and mount the MACHINE_STORAGE_PATH so that it is available to the container
Connect to the remote docker-machine manually from within the container and setting the MACHINE_STORAGE_PATH equal to a mounted volume
Mounting the docker socket
In both cases, I can see the machine storage is persisted, but whenever I create a new container and run docker-machine ls none of the machines are listed.
Accessing the Remote Host via DOCKER_HOST
Forward the remote machine docker port to the host docker port docker-machine ssh manager-1 -N -L 2376:localhost:2376
export DOCKER_HOST=:2376
Tell docker to use the same certs that are used by docker-machine: export DOCKER_TLS_VERIFY=1 and export DOCKER_CERT_PATH=/Users/me/.docker/machine/machines/manager-1
Test with docker info
This gives me error during connect: Get https://localhost:2376/v1.26/info: x509: certificate signed by unknown authority
Any ideas on how I can perform a remote deployment from within a container?
Thanks
EDIT
Here is a diagram to try and help better communicate the scenario.
Don't use docker-machine for this.
Docker-machine stores files in $HOME/.docker/machine, so when you restart with a fresh copy of this folder, all previously defined machines will be removed. You could store this folder as a volume, but there's a much easier way for your purposes.
The solution is to mount the docker socket, and either as root or from a user with the same gid as the docker socket (note that group names themselves inside and outside the container may not match, so gid is important), run your docker ... commands as normal. You can skip the docker-machine eval completely since you are running the commands against the local docker socket.
If you need to run commands remotely, I find it easier to define the DOCKER_HOST and DOCKER_TLS_VERIFY variables manually rather than using docker-machine.
In case you want to communicate from your CI container to the Docker host you can simply mount the Docker socket when starting the CI container:
docker run -v /var/run/docker.sock:/var/run/docker.sock <gitlab-image>
Now you can run docker commands on the host from within the CI container.
Cannot pull image from local docker insecured registry repository inside Minikube.
I'm running Docker-toolbox v1.12.2 using Linux VM (Upstart) installed on Oracle VirtualBox 5.1.6 under Windows 7.
I've created a docker image and push (tag and then push) it into a local insecured docker-registry v2 that running on 192.168.99.100:5000/image/name.
docker run -d -p 5000:5000 --restart=always --name registry registry:2
and inside the VM, on /var/lib/boot2docker/profile I've add to the EXTRA_ARGS the flag --insecure-registry 192.168.99.100:5000 .
docker push & docker pull from localhost:5000/image/name are working fine within Docker(VM).
_catalog is reachable from Postman :GET http:192.168.99.100:5000/v2/_catalog and I'm able to get the images inside the registry.
I'm starting my Minikube v0.15.0 VM with the command:
minikube start --insecure-registry=192.168.99.100:5000
I'm under company PROXY so I've added the proxy in the command line (CMD):
set HTTP/HTTPS_PROXY=my.company.proxy:8080 and set NO_PROXY={minikube ip}.
Then Kubernetes dashboard started to work for me.
Now for the real problem, when running the command:
kubectl run image-name --image=192.168.99.100:5000/image/name --port=9999
to pull image from my local docker registry into Kubernetes its saying
deployment "image-name" created
But inside Kubernetes > Deployments I'm getting the following error:
Failed to pull image "192.168.99.109:5000/image/name": image pull failed for 192.168.99.100:5000/image/name:latest, this may be because there are no credentials on this request. details: (Error response from daemon: Get https://192.168.99.100:5000/v1/_ping: Tunnel or SSL Forbidden)
Can anyone help here with that Tunnel or SSL Forbidden error, it's driving me crazy, and I've tried so many solutions to configure --insecrue-registery inside docker, inside Kubernetes or when running the dokcer-registry.
BTW why it's refering to v1/_ping? i'm using the docker registry v2.
Seems like minikube cannot see the same network that your registry is running. Can you try running minikube ssh then run your curl for the catalog?
Also, as an alternative, you could run eval(minikube docker-env) which then will set your local docker client to use the docker server inside minikube.
So for example if you built an image tagged with myimage/foo it would build and put that image on the minikube docker host, so when you deployed the image, it wouldn't need to be pulled.
I want to set up my own private docker hub from where I can pull docker images on docker clients.
Taking this link as reference, I executed following commands on one machine:
docker pull registry
docker run -d -p 5000:5000 --name localregistry registry
docker ps
docker pull alpine
docker tag alpine:latest localhost:5000/alpine:latest
docker push localhost:5000/alpine:latest
I want to pull this image on some other machine which is reachable to/from this machine.
$ docker pull <ip_of_machine>:5000/alpine
Using default tag: latest
Error response from daemon: Get https://<ip_of_machine>:5000/v1/_ping: http: server gave HTTP response to HTTPS client
Is it possible to pull docker image from one machine which acts as a docker hub to another machine which is reachable?
Adding below line in docker client machine's /etc/sysconfig/docker file resolved the issue:
INSECURE_REGISTRY='--insecure-registry <ip>:5000'
Assuming by the tags you are using boot2docker or DockerToolbox:
You must open VirtualBox Manager
Select the default machine
Network
NAT
Port forwarding
Add an entry for the 5000 port
Regards
I have a Docker image I want to push to my registry (hosted on localhost). I do:
docker push localhost:5000/my_image
and works properly. However, if I tag the image and push it by:
docker push 172.20.20.20:5000/my_image
I get an error.
The push refers to a repository [172.20.20.20:5000/my_tomcat] (len: 1)
unable to ping registry endpoint https://172.20.20.20:5000/v0/ v2
ping attempt failed with error:
Get https://172.20.20.20:5000/v2/: Gateway Time-out
Can't I refer to registry by IP? If so, how could I push an image from another host that it is not localhost?
EDIT
I'm running the registry this way:
docker run -d -p 5000:5000 --restart=always --name registry registry:2
As mentioned in "IPs for all the Things" (by Jess Frazelle), you should be able, with docker 1.10, to run your registry with a fixed IP address.
It uses the --net= --ip= options of docker run.
# create a new bridge network with your subnet and gateway for your ip block
$ docker network create --subnet 203.0.113.0/24 --gateway 203.0.113.254 iptastic
# run a nginx container with a specific ip in that block
$ docker run --rm -it --net iptastic --ip 203.0.113.2 nginx
# curl the ip from any other place (assuming this is a public ip block duh)
$ curl 203.0.113.2
You can adapt this example to your registry docker run parameters.
First of all please check whether you are able to connect the registry on the port 5000. In Linux/windows you can do this using telnet. below is the command.
$ telnet 172.20.20.20 5000
If the connectivity check is failed then please check your firewall settings.
I am not sure whether you are running your running registry with login functionality. But from the question i can assume that you are not using it. In this case please add your registry as an insecure registry in the docker daemon from which you are trying to access the registry. The process is described here - https://docs.docker.com/registry/insecure/
Please let me know if are successful.