I installed Docker on Windows 10, and I pulled jenkins docker from Docker Hub. Next, I started my jenkins docker,
docker run --rm -u root -p 8080:8080 -v my_host_path:/var/jenkins_home jenkins
Next, I used Manage Jenkins and Manage Plugins to install Docker plugin, then went to the Configure page and tried to add Docker Cloud.
After I entered Docker Host URI : tcp://127.0.0.1:2375, I wanted to "Test Connection", but unfortunately got failed.
I tried to follow the instruction as below link:
How to find "Docker Host URI" to be used in Jenkins "Docker Plugin"?
But I can't not find any docker setting file under /etc/default/* in my jenkins container, so I can't set the DOCKER_OPTS argument.
Could someone give me any advise? Thank you !
Problem context: end of Chapter 3 exercise from the book "Continuous Delivery with Docker and Jenkins" by Rafal Leszko
from Configure and troubleshoot the Docker daemon page
Important: Setting hosts in the daemon.json is not supported on Docker Desktop for Windows or Docker Desktop for Mac.
Setting the docker host uri does NOT work on Windows. So either of these won't work in the Settings > Daemon tab:
"hosts" : "-H tcp://0.0.0.0:2375"
"DOCKER_OPTS" : "-H tcp://0.0.0.0:2375"
Exposing the daemon without TLS (checkbox on General tab) as recommended in some places did not work for me either.
The solution to connecting the Docker plugin in Jenkins with the docker host, is:
use the special DNS name host.docker.internal
From the docs:
How do I connect from a container to a service on the host?
Windows has a changing IP address (or none if you have no network access). We recommend that you connect to the special DNS name host.docker.internal, which resolves to the internal IP address used by the host. This is for development purposes and will not work in a production environment outside of Docker Desktop for Windows.
The gateway is also reachable as gateway.docker.internal.
For more information about the networking features in Docker Desktop for Windows, see Networking.
While the 'will not work in a production environment outside of Docker Desktop for Windows' disclaimer might bother some, I believe Docker for Windows is not meant for production use cases anyway.
Additionally, publish this mapping for Jenkins agent-master communication -p 50000:50000
Related
I'm using Jenkins on Docker on my local Mac Machine.
And I'm running another Docker on ubuntu VirtualBox. So now, there are 2 docker machines. one is on my mac machine and one is on my Ubuntu VirtualBox machine. I'm running Jenkins on Mac Docker. Now in the Jenkins pipeline, I want to build an image on my ubuntu machine.
I've configured Jenkins docker cloud and in the docker host URL, it is connected to the ubuntu docker-machine.
But while building a new image, I'm getting the error. Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
I've tried even adding ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock
at /lib/systemd/system/docker.service
WHen i check ps -aux,
Can someone please help me out?
help is appreciated.
First personally if I had a setup like that I would not bother connecting to the remote docker and would just install a Jenkins agent on the ubuntu machine and make it talk to the Jenkins master.
But if you want to do it they way you have it set up right now we a Jenkins talking from inside out one docker host into another docker host I suggest looking into the following:
Your Jenkins master and the ubuntu machine a very isolated they might as well just be on different machines not even in the same room. Unix domain sockets, the ones that are identified by unix://* are made for communicating within a single local OS kernel, trying to bridge them into remote machine will lead to disaster.
So the only way Jenkins could communicate to the remote host is via a remote protocol like TCP. Most of the time when you install docker with the default settings it doesn't even listen to TCP at all, mostly for security reasons.
First thing you should do is to configure a docker inside of the ubuntu machine to listen on TCP port and accept connections from remote hosts. You can use netstat -nat to see if anything is listening on TCP 4243. When things are configured correctly you see the line that stats with 0.0.0.0:4243 or something like that in the output of the nestat
Second you need to make sure your the firewalls/iptables/netfilter configuration on the Ubuntu host lets in connections from outside. A good test to try is to telnet <ubuntu-ip> 4243 from a terminal session on your Mac.
Then you need to make sure you that docker networking is configured correctly so that connections from the inside of the container that is running Jenkins end up on your ubuntu box. To test you need to exec -it into your jenkins container and repeat the telnet test. On modern linuxes telnet is usually not installed, so you can use curl -vvv which will always end up with an error, so just look at the verbose output to see if the error because things cannot communicate (timeout, connection reset etc) or the error occurs because your curl tried to talk HTTP to docker and got gibberish response. In the later case you can consider things to be set up correctly.
Finally you need to tell Jenkins Docker to communicate to the remote docker via TCP. Usually that is given on the command line to your docker run, docker ps, docker exec
I've configured it by defining the slave label in my Jenkins Pipeline.
Jenkins agents run on a variety of different environments such as physical machines, virtual machines, Kubernetes clusters, and Docker images.
In your Jenkins Pipeline or In your JenkinsFile, you've to set the agent accordingly to what you're using either using Docker image or any virtual machine.
Also Thank you so much #Vlad, all the things you told me, were really helpful.
I installed Docker for windows on my work computer. When I try to login to docker hub I get an "unknown error" error. Adding a working corporate proxy doesn't help either.
Following the recommendations described in this article https://mandie.net/2017/12/10/docker-for-windows-behind-a-corporate-web-proxy-tips-and-tricks/ also did not solve the problem.
Docker Desktop lets you configure HTTP/HTTPS Proxy Settings and automatically propagates these to Docker. For example, if you set your proxy settings to http://proxy.example.com Docker uses this proxy when pulling containers.
Your proxy settings, however, will not be propagated into the containers you start. If you wish to set the proxy settings for your containers, you need to define environment variables for them, just like you would do on Linux, for example:
> docker run -e HTTP_PROXY=https://proxy.example.com:3128 alpine env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=b7edf988b2b5
TERM=xterm
HOME=/root
HTTP_PROXY=http://proxy.example.com:3128
For more information on setting environment variables for running containers, Set environment variables
I use px proxy (https://github.com/genotrance/px) and configure docker proxy to use it running on my machine (in my case http://127.0.0.1:3131/).
Then docker login, on the command line, works as well as pull and push requests.
Docker "sign in" in the graphical interface always returns "unknown error". Using the command line everything else works fine.
I'm seeing problems mounting local volumes when running docker on Windows 10. The problems only appear when I have my company VPN enabled.
C:\Users\matt> docker run --rm -v d:/tmp:/data alpine ls /data
my_local_test_file.txt
When connected to VPN, I get this:
C:\Users\matt> docker run --rm -v d:/tmp:/data alpine ls /data
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: error while creating mount source path '/host_mnt/d/tmp': mkdir /host_mnt/d: file exists.
Docker version is 17.12.0-ce-win47
I believe the problem is that docker uses the network when mounting local volumes, and the VPN routes ALL network traffic via the VPN gateway, so docker can't see the local drive.
Is there a workaround for this?
I'm aware I could run docker within a linux VM, or use docker toolbox. Neither of those are particularly good.
Is there another possible workaround?
the VPN routes ALL network traffic via the VPN gateway
You're probably right, in which case all traffic routed from Docker client to Docker daemon will also be through the VPN. When you use Docker CLI on Windows, it will connect to the Docker daemon which is accessible through the network. Using a VPN may disrupt this mechanism.
I think what's happening is:
When VPN is disabled, you use the Docker daemon on your machine and everything works
When VPN is enabled, another Docker daemon is used either because your VPN redirect traffic addressed to your Docker host (127.0.0.1 by default or set via -H flag or DOCKER_HOST env variable). This means that somehow this IP or host exists on your VPN network and there is a Docker daemon listening on it (which is kind of odd admittedly, it may be risky to use that daemon)
If that's really happening, you'll certainly see different output from docker ps -a, docker images, etc. because you are connecting to different daemons. (the daemon accessible through your VPN is actually being owned by someone else, you'd better not use it!)
What you can do:
Do not route 127.0.0.1 (or whatever is configured as Docker host) through your VPN
Action to take will depend on the VPN software you are using, or you can add route directly on your windows machine (here is a good article on the subject)
Find out your IP when VPN is enabled and configure Daemon to listen to this IP
When your VPN is enabled, run ipconfig /all and find the interface used by your VPN and it's IP address, for example 10.142.0.12 (you can compare output before/after enabling VPN to identify which one it is)
Configure your Docker daemon to listen this IP address and restart it. Either use the UI, or on Windows config file is located at %programdata%\docker\config\daemon.json by default, you need to specify "hosts": ["10.142.0.12", "127.0.0.1"] for example (see docs for details)
Configure Docker host to 10.142.0.12 when VPN is enabled, either by setting environment variable DOCKER_HOST=10.142.0.12 or with client docker -H 10.142.0.12 <cmd>
/!\ Security note: this may present a security issue as anyone knowing your IP on the VPN network will be able to use the Daemon on your machine
Hope this helps. I am not a Windows expert so I was not able to give details on Windows-related issues, but feel free to ask details if needed.
new to docker and docker swarm. Trying docker and docker swarm both.
initially i had started a docker daemon and was able to connect it on http port i.e. 2375. I had installed docker colud plugin in jenkins and added http://daemon-IP:2375 and was able to create containers. well it creates a container, does my build inside it and destroys the container.
My Query is, will i be able to connect to docker swarm on http port, the same way i a am connecting to a standalone docker daemon ? is there any documentation on it. or the my understanding about the swarm is wrong.
please suggest.
Thanks
Yeah you can connect to a remote host the same way you are doing via the Unix Socket. People very often forget that docker is a client-server architecture and your "docker run..." commands are basically just commands issued by the docker client.
If you set certain environment variables:
DOCKER_HOST=tcp:ip.address.of.host:port
DOCKER_TLS_VERIFY=1
DOCKER_CERTS=/directory/where/certs/are
(The last two are optional for TLS connections, which I would highly recommend. You'd have to setup https://docs.docker.com/engine/security/https/ which is recommended for a production environment)
Once you've set your DOCKER_HOST environment variable, if you issue a docker command and get a response, it will be from the remote host if everything is setup correctly.
I setup both of my mesos-master and mesos-slave on a standalone server. E.g. To start my mesos-slave, I used this command:
sudo bin/mesos-slave.sh --master=zk://<IP address of server>:2181/mesos --log_dir=/var/log/mesos --containerizers=docker,mesos
What I am trying to figure out is how the containerizer on Mesos is implemented with just --containerizers=docker,mesos.
Will it be able to automatically detect whether Docker is installed on the mesos-slave? If it is, which tcp port will it normally get? port 4243 or 2375?
Mesos will try to autodetect docker by running docker version. You can specific an absolute path for the docker executable by passing the --docker=/path/to/docker flag to the slave. There are other docker-specific flags for the slave, like --docker_sandbox_directory, --docker_remove_delay, and --docker_stop_timeout. For more details on those, see https://mesos.apache.org/documentation/latest/configuration/
Mesos currently uses the docker command-line interface locally from the slave node, not via the remote API, so I don't think the docker port is relevant here.