I try to setup unit testing agains database with docker. I use library, that runs database image during tests. Docs says:
Testcontainers will try to connect to a Docker daemon using the
following strategies in order:
Environment variables:
DOCKER_HOST (this should be set to an
HTTP/HTTPS connection rather than a unix socket at present)
DOCKER_TLS_VERIFY
DOCKER_CERT_PATH
Defaults:
DOCKER_HOST=https://localhost:2376
DOCKER_TLS_VERIFY=1
DOCKER_CERT_PATH=~/.docker
If Docker Machine is installed, the docker machine environment for the
first machine found. Docker Machine needs to be on the PATH for this
to succeed.
I have docker installed in Ubuntu 16.04, but dont understund how to configure docker daemon to use with this testing library. Any helps?
Based on the doc snippet you included:
(this should be set to an
HTTP/HTTPS connection rather than a unix socket at present)
it sounds like this software doesn't support communicating with a Docker daemon over a unix socket, which is the default method the regular docker CLI will attempt to connect to docker when DOCKER_HOST is unset.
You will need to configure your locally running daemon to listen on a tcp socket in addition to the default /var/run/docker.sock unix socket.
You can set this up manually by generating TLS certificates and putting them in place on both the client and the server. Details on this procedure can be found in the Protect the Docker daemon socket article in Docker's official docs.
Related
I'm using Jenkins on Docker on my local Mac Machine.
And I'm running another Docker on ubuntu VirtualBox. So now, there are 2 docker machines. one is on my mac machine and one is on my Ubuntu VirtualBox machine. I'm running Jenkins on Mac Docker. Now in the Jenkins pipeline, I want to build an image on my ubuntu machine.
I've configured Jenkins docker cloud and in the docker host URL, it is connected to the ubuntu docker-machine.
But while building a new image, I'm getting the error. Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
I've tried even adding ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock
at /lib/systemd/system/docker.service
WHen i check ps -aux,
Can someone please help me out?
help is appreciated.
First personally if I had a setup like that I would not bother connecting to the remote docker and would just install a Jenkins agent on the ubuntu machine and make it talk to the Jenkins master.
But if you want to do it they way you have it set up right now we a Jenkins talking from inside out one docker host into another docker host I suggest looking into the following:
Your Jenkins master and the ubuntu machine a very isolated they might as well just be on different machines not even in the same room. Unix domain sockets, the ones that are identified by unix://* are made for communicating within a single local OS kernel, trying to bridge them into remote machine will lead to disaster.
So the only way Jenkins could communicate to the remote host is via a remote protocol like TCP. Most of the time when you install docker with the default settings it doesn't even listen to TCP at all, mostly for security reasons.
First thing you should do is to configure a docker inside of the ubuntu machine to listen on TCP port and accept connections from remote hosts. You can use netstat -nat to see if anything is listening on TCP 4243. When things are configured correctly you see the line that stats with 0.0.0.0:4243 or something like that in the output of the nestat
Second you need to make sure your the firewalls/iptables/netfilter configuration on the Ubuntu host lets in connections from outside. A good test to try is to telnet <ubuntu-ip> 4243 from a terminal session on your Mac.
Then you need to make sure you that docker networking is configured correctly so that connections from the inside of the container that is running Jenkins end up on your ubuntu box. To test you need to exec -it into your jenkins container and repeat the telnet test. On modern linuxes telnet is usually not installed, so you can use curl -vvv which will always end up with an error, so just look at the verbose output to see if the error because things cannot communicate (timeout, connection reset etc) or the error occurs because your curl tried to talk HTTP to docker and got gibberish response. In the later case you can consider things to be set up correctly.
Finally you need to tell Jenkins Docker to communicate to the remote docker via TCP. Usually that is given on the command line to your docker run, docker ps, docker exec
I've configured it by defining the slave label in my Jenkins Pipeline.
Jenkins agents run on a variety of different environments such as physical machines, virtual machines, Kubernetes clusters, and Docker images.
In your Jenkins Pipeline or In your JenkinsFile, you've to set the agent accordingly to what you're using either using Docker image or any virtual machine.
Also Thank you so much #Vlad, all the things you told me, were really helpful.
can i run docker socket over ssh?
i'm trying to run unix:///var/run/docker.sock but i'm getting the error "Is daemon service running?, Cannnot connect to daemon service"
Jenkins master and the ubuntu machine a very isolated they might as well just be on different machines not even in the same room. Unix domain sockets, the ones that are identified by unix://* are made for communicating within a single local OS kernel, trying to bridge them into remote machine will lead to disaster.
how can i use Docker sock over ssh?
stephen proposed a solution but i find this one more adequate to your use case.
you can simply use
ssh xxx "docker run yyy"
or you can use env variables :
be sure that you have ssh key authentification active
and call all your docker commands with this env variable defined :
DOCKER_HOST=remoteservername
docker will use ssh connection to run commands
you can also use -H (works the same)
see more here
https://betterprogramming.pub/docker-tips-access-the-docker-daemon-via-ssh-97cd6b44a53
I am a newbie to docker. When I go through docker tutorial, I saw that "Docker client can communicate with more than one daemon". What does that mean exactly?
By default, the Docker daemon listens on a Unix socket, /var/run/docker.sock. However, Docker can also be configured to listen on a TCP socket. In fact, it is often configured this way on Mac and Windows systems because Docker is actually running inside a virtual machine and the default Docker socket is not available on the host filesystem.
Because there are different ways of connecting to Docker, you must be able to configure the Docker client to connect to a Docker daemon at a specific location. You can do this using the DOCKER_HOST environment variable. You can point this at a network location:
export DOCKER_HOST=tcp://192.168.99.101:2376
Or at an alternate socket location:
export DOCKER_HOST=unix:///tmp/docker.sock
If you have Docker configured to listen for tcp connections, you can use the Docker client on a single machine to communicate with Docker on multiple hosts (but if you decide to do something like this, read through "Protect the Docker daemon socket").
Per the Docker Documentation,
The Docker client can communicate with more than one daemon.
This means that the command-line utility docker can connect to different services that run in the background,
Docker uses a client-server architecture. The Docker client talks to the Docker daemon, which does the heavy lifting of building, running, and distributing your Docker containers.
So for example, you could configure the daemon to run on a separate machine and connect to it from your workstation.
new to docker and docker swarm. Trying docker and docker swarm both.
initially i had started a docker daemon and was able to connect it on http port i.e. 2375. I had installed docker colud plugin in jenkins and added http://daemon-IP:2375 and was able to create containers. well it creates a container, does my build inside it and destroys the container.
My Query is, will i be able to connect to docker swarm on http port, the same way i a am connecting to a standalone docker daemon ? is there any documentation on it. or the my understanding about the swarm is wrong.
please suggest.
Thanks
Yeah you can connect to a remote host the same way you are doing via the Unix Socket. People very often forget that docker is a client-server architecture and your "docker run..." commands are basically just commands issued by the docker client.
If you set certain environment variables:
DOCKER_HOST=tcp:ip.address.of.host:port
DOCKER_TLS_VERIFY=1
DOCKER_CERTS=/directory/where/certs/are
(The last two are optional for TLS connections, which I would highly recommend. You'd have to setup https://docs.docker.com/engine/security/https/ which is recommended for a production environment)
Once you've set your DOCKER_HOST environment variable, if you issue a docker command and get a response, it will be from the remote host if everything is setup correctly.
I setup both of my mesos-master and mesos-slave on a standalone server. E.g. To start my mesos-slave, I used this command:
sudo bin/mesos-slave.sh --master=zk://<IP address of server>:2181/mesos --log_dir=/var/log/mesos --containerizers=docker,mesos
What I am trying to figure out is how the containerizer on Mesos is implemented with just --containerizers=docker,mesos.
Will it be able to automatically detect whether Docker is installed on the mesos-slave? If it is, which tcp port will it normally get? port 4243 or 2375?
Mesos will try to autodetect docker by running docker version. You can specific an absolute path for the docker executable by passing the --docker=/path/to/docker flag to the slave. There are other docker-specific flags for the slave, like --docker_sandbox_directory, --docker_remove_delay, and --docker_stop_timeout. For more details on those, see https://mesos.apache.org/documentation/latest/configuration/
Mesos currently uses the docker command-line interface locally from the slave node, not via the remote API, so I don't think the docker port is relevant here.