Is there a command that I can evaluate (eval $COMMAND) or an environment variable that I can inspect to get the URI of the locally running docker host? I need to be this an expression to evaluate at runtime, since I need to use it in a script intended to run on hosts where docker engine may not be running on the standard ports.
Thanks.
I am not sure I have fully understood your question but hopefully this may shed some light.
The docker command is just a REST client. By default the client connects to the unix-socket /var/run/docker.sock to send requests to the docker daemon.
You can practically achieve the same thing by running curl:
curl --unix-socket /var/run/docker.sock http:/localhost/version
Now if your docker daemon was configured for remote access, you probably have the URI information in the docker.service service file. See this post
How do I find the Docker REST API URL?.
Hope that helps
Hey you can use ps command on the host and grep docker like below
>ps aux | grep docker
root 1729 0.0 0.3 2222936 56288 ? Ssl May08 8:05 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
then you can get which port(s) or sock it is linked with
Related
I have a group of docker containers running on a host (172.16.0.1). Because of restrictions of the size of the host running the docker containers, I'm trying to set up an auto-test framework on a different host (172.16.0.2). I need my auto-test framework to be able to access the docker containers. I've looked over the docker documentation and I don't see anything that says how to do this.
Is it possible to run a docker exec and point it to the docker host? I was hoping to do something like the following but there isn't an option to specify the host.:
docker exec -h 172.16.0.1 -it my_container bash
Should I be using a different command?
Thank you!
Not sure why there is need of doing docker exec remotely. But anyways it is achievable.
You need to make sure your docker daemon on your host where your containers are running is listening on a socket.
Something like this:
# Running docker daemon which listens on tcp socket
$ sudo dockerd -H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375
Now interact with the docker daemon remotely from external VM using:
$ docker -H tcp://<machine-ip>:2375 exec -it my-container bash
OR
$ export DOCKER_HOST="tcp://<machine-ip>:2375"
$ docker exec -it my-container bash
Note: Exposing docker socket publicly in your network has some serious security risks. Although there are other ways to expose it on encrypted HTTPS socket or over the ssh protocol.
Please go through these docs carefully, before attempting anything:
https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-socket-option
https://docs.docker.com/engine/security/https/
If you have SSH on both machines you can easily execute commands on remote daemon like that:
docker -H "ssh://username#remote_host" <your normal docker command>
# for example:
docker -H "ssh://username#remote_host" exec ...
docker -H "ssh://username#remote_host" ps
# and so on
Another way to do the same is to store -H key value into DOCKER_HOST environment variable:
export DOCKER_HOST=ssh://username#remote_host
# now you can talk to remote daemon with your regular commands
# these will be executed on remote host:
docker ps
docker exec ...
Without SSH you can make Docker listen for TCP. This will require you to make some preparations to maintain security. This guide walks through creating certificates and some basic usage. After that you will have somewhat similar usage:
docker --tlsverify --tlscacert=ca.pem --tlscert=cert.pem --tlskey=key.pem \
-H=172.16.0.1:2376
At last you can use docker context to save external hosts and their configuration. Using context allows you to communicate with various remote hosts with ease by using --context <name> option. Read context documentation here.
I installed docker on an Ubuntu 16.04 following the official directions and am successfully running the registry as a container. I want to remote connect into another container, so I try:
docker -H tcp://1.2.3.4:2375 exec -it 19f36d1bdfaf /bin/bash
And I get an error:
error during connect: Post
http://1.2.3.4:2375/v1.29/containers/19f36d1bdfaf/exec: dial tcp
1.2.3.4:2375: connectex: No connection could be made because the target machine actively refused it.
Why am I getting this error and how do I resolve it?
The docker.json file has contents:
{
"hosts": [
"tcp://0.0.0.0:2375",
"npipe://"
]
}
When I view the services it looks like the daemon is not listening on tcp://0.0.0.0:2375 as I would expect (this is just for testing, I'm going to secure this once I can get it actually working):
UPDATE:
Got it to partially work by creating a daemon.json file (a copy of docker.json), then running:
sudo dockerd
The problem with this is that now the client does not work: docker info results in an error:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
UPDATE and SOLUTION:
Andreas' answer helped me realize the daemon was not being run properly. Looking at that screenshot earlier in this post, the docker daemon was not being launched with the right -H option. I then found this Github issue which solved the problem. So all that was needed was:
Edit the $DOCKER_OPTS variable in /etc/default/docker: DOCKER_OPTS="-H tcp://0.0.0.0:2375" (note that this is not very secure, it's just for testing)
Edit the /lib/systemd/system/docker.service file by adding a line under [Service] for the EnvironmentFile: EnvironmentFile=-/etc/default/docker then update the ExecStart line: ExecStart=/usr/bin/dockerd $DOCKER_OPTS -H fd://
Restart the service sudo service docker restart
Restart the daemon with systemctl daemon-reload
Note that I did not add a daemon.json file -- I left the existing docker.json file.
By default the docker daemon isn't exposed to the outside world for security reasons. You can of course change this setting when starting your docker daemon with the -H flag.
To test it out, you can simply start your daemon manually (be sure to stop the service before). Assuming 1.2.3.4 is the ip of the host running the daemon you want to connect to.
<path to>/dockerd -H tcp://1.2.3.4:2375
Or you bind it to all network interfaces:
<path to>/dockerd -H tcp://0.0.0.0:2375
You can provide more than one -H option here to not disable the unix socket when binding to the tcp socket. For details on the daemon binding options, please see the docs (Bind Docker to another host/port or a Unix socket).
To have this permanently, you can configure your daemon startup settings in a daemon.json file where you can also specify an array of hosts. Please see the docs (Configure the Docker daemon) and Linux configuration File for this, too.
{
"hosts": [
"tcp://0.0.0.0:2375",
"unix:///var/run/docker.sock"
]
}
You can provide a list of entries for hosts, so your daemon can listen to tcp and the unix socket at the same time.
Please be aware that by just binding to tcp 0.0.0.0 anyone that is able to reach your machine is also able to start containers remotely and thus is almost able to do anything on your system like with a really bad root user password. You should only do this for testing or in an environment that is isolated / firewalled correctly.
Andreas' answer helped me realize the daemon was not being run properly. Looking at that screenshot earlier in this post, the docker daemon was not being launched with the right -H option. I then found this Github issue which solved the problem. So all that was needed was:
Edit the $DOCKER_OPTS variable in /etc/default/docker: DOCKER_OPTS="-H tcp://0.0.0.0:2375" (note that this is not very secure, it's just for testing)
Edit the /lib/systemd/system/docker.service file by adding a line under [Service] for the EnvironmentFile: EnvironmentFile=-/etc/default/docker then update the ExecStart line: ExecStart=/usr/bin/dockerd $DOCKER_OPTS -H fd://
Restart the service sudo service docker restart
Note that I did not add a daemon.json file -- I left the existing docker.json file.
From the document in the following link: https://github.com/hyperledger/fabric/blob/master/docs/dev-setup/install.md
we've got to know that we should do
Make sure that the Docker daemon initialization includes the options
-H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock
However, should we initialize the docker every time we restart the blockchain server?
In addition, I conduct the following command:
nohup docker daemon -g /data/docker -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock&
What does -g /data/docker mean?
The options you are passing to docker do the following:
-g /data/docker: this changes the runtime directory from /var/lib/docker to the one you've provided
-H tcp://0.0.0.0:2375: this tells docker to listen on all network interfaces to port 2375, unencrypted. Caution: this allows anyone with network access to your machine to have full root access, a firewall or isolated machine is required for security.
-H unit:///var/run/docker.sock: this tells docker to process commands from any user with access to this socket, typically restricted to root and members of the "docker" group.
I'm not familiar with the blockchain install, but typically a docker engine can restart the containers contained within, so it should not be required to restart, only start it on boot as a service.
I have installed the Docker build step plugin for Jenkins.
The documentation is telling me:
Name : Choose a name for this Docker cloud provider
Docker URL: The URL to use to access your Docker server API (e.g: http://172.16.42.43:4243)
How can I find my URL to the REST API (I have Docker installed on my host)?
If you are on Linux and need to connect to Docker API on the local machine, its URL is probably unix:///var/run/docker.sock, like it is mentioned in documentation: Develop with Docker Engine SDKs and API
By default the Docker daemon listens on unix:///var/run/docker.sock and the client must have root access to interact with the daemon. If a group named docker exists on your system, docker applies ownership of the socket to the group.
This might be helpful if you are connecting to Docker from a JetBrains IDE.
Here are two approaches.
How do I access the Docker REST API remotely?
Warning: After this setup your Docker REST API port (in this case 1111) is exposed to remote
access.
Here is how I enabled it on Ubuntu 16.04 (Xenial Xerus).
Edit the docker service file (it is better to avoid directly editing /lib/systemd/system/docker.service as it will be replaced on upgrades)
sudo systemctl edit docker.service
Add the following content
[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// -H tcp://0.0.0.0:1111
For docker 18+, the content is a bit different:
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock -H tcp://0.0.0.0:1111
Save the modified file. Here I used port 1111, but any free port can be used.
Make sure the Docker service notices the modified configuration:
systemctl daemon-reload
Restart the Docker service:
sudo service docker restart
Test
curl http://localhost:1111/version
See the result
{"Version":"17.05.0-ce","ApiVersion":"1.29","MinAPIVersion":"1.12","GitCommit":"89658be","GoVersion":"go1.7.5","Os":"linux","Arch":"amd64","KernelVersion":"4.15.0-20-generic","BuildTime":"2017-05-04T22:10:54.638119411+00:00"}
Now you can use the REST API.
How do I access the Docker REST API through a socket (from localhost)?
Connect the internal Unix socket somewhat like this,
Using curl
curl --unix-socket /var/run/docker.sock http:/localhost/version
And here is how to do it using PHP
$fs = fsockopen('/var/run/docker.sock');
fwrite($fs, "GET / HTTP/1.1\r\nHOST: http:/images/json\r\n\r\n");
while (!feof($fs)) {
print fread($fs,256);
}
In PHP 7 you can use curl_setopt with the CURLOPT_UNIX_SOCKET_PATH option.
It depends on your host, but look for /etc/default/docker or /var/lib/boot2docker/profile (for Docker Machine hosts using a boot2docker VM).
You will see the port used by the docker daemon, for instance:
DOCKER_OPTS="-H unix:// -H tcp://0.0.0.0:2375"
^^^^^
Then get the IP address of the machine hosting your Docker daemon.
(With a Docker Machine created host, that would be: docker-machine ip <yourmachine>.)
The URL to use is the combination of those the IP address and the port.
If you are on windows:
npipe:////./pipe/docker_engine
source: https://docs.docker.com/docker-for-windows/faqs/#how-do-i-connect-to-the-remote-docker-engine-api
I have installed Ubuntu and Docker. I am trying to launch Raik container:
$ DOCKER_RIAK_AUTOMATIC_CLUSTERING=1 DOCKER_RAIK_CLUSTER_SIZE=5 DOCKER_RIAK_BACKEND=leveldb make start-cluster ./bin/start
and get the error message:
It looks like the environment variable DOCKER_HOST has not been set.
The Riak cluster cannot be started unless this has been set
appropriately. For example:
export DOCKER_HOST="tcp://127.0.0.1:2375"
If I set
export DOCKER_HOST="tcp://127.0.0.1:2375"
all my other containers stop working and said, that can not find the Docker daemon.
It looks like my Docker damon use other than 2375 port. How can I check it ?
By default, the docker daemon will use the unix socket unix:///var/run/docker.sock (you can check this is the case for you by doing a sudo netstat -tunlp and note that there is no docker daemon process listening on any ports). It's recommended to keep this setting for security reasons but it sounds like Riak requires the daemon to be running on a TCP socket.
To start the docker daemon with a TCP socket that anybody can connect to, use the -H option:
sudo docker -H 0.0.0.0:2375 -d &
Warning: This means machines that can talk to the daemon through that TCP socket can get root access to your host machine.
Related docs:
http://basho.com/posts/technical/running-riak-in-docker/
https://docs.docker.com/install/linux/linux-postinstall/#configure-where-the-docker-daemon-listens-for-connections
Prepare extra configuration file. Create a file named /etc/systemd/system/docker.service.d/docker.conf. Inside the file docker.conf, paste below content:
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock
Note that if there is no directory like docker.service.d or a file named docker.conf then you should create it.
Restart Docker. After saving this file, reload the configuration by systemctl daemon-reload and restart Docker by systemctl restart docker.service.
Check your Docker daemon. After restarting docker service, you can see the port in the output of systemctl status docker.service
like /usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock.
Hope this may help
Thank you!
Reference docs of docker: https://docs.docker.com/install/linux/linux-postinstall/#configure-where-the-docker-daemon-listens-for-connections
There are 2 ways in configuring the docker daemon port
1) Configuring at /etc/default/docker file:
DOCKER_OPTS="-H tcp://127.0.0.1:5000 -H unix:///var/run/docker.sock"
2) Configuring at /etc/docker/daemon.json:
{
"debug": true,
"hosts": ["tcp://127.0.0.1:5000", "unix:///var/run/docker.sock"]
}
If the docker default socket is not configured Docker will wait for infinite period.i.e
Waiting for /var/run/docker.sock
Waiting for /var/run/docker.sock
Waiting for /var/run/docker.sock
Waiting for /var/run/docker.sock
Waiting for /var/run/docker.sock
NOTE : BUT DON'T CONFIGURE IN BOTH THE CONFIGURATION FILES, the following error may occur :
Waiting for /var/run/docker.sock
unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: hosts: (from flag: [tcp://127.0.0.1:5000 unix:///var/run/docker.sock], from file: tcp://127.0.0.1:5000)
The reason for adding both the user port[ tcp://127.0.0.1:5000] and default docker socket[unix:///var/run/docker.sock] is that the user port enables the access to the docker APIs whereas the default socket enables the CLI. In case the default port[unix:///var/run/docker.sock] is not mentioned in /etc/default/docker file the following error may occur:
# docker ps
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
This error is not because that the docker is not running, but because of default docker socket is not enabled.
Once the configuration is enabled restart the docker service and verify the docker port is enabled or not:
# netstat -tunlp | grep -i 5000
tcp 0 0 127.0.0.1:5000 0.0.0.0:* LISTEN 31661/dockerd
Applicable for Docker Version 17.04, may vary with different versions of docker.
Since I also had the same problem of "How to detect a docker daemon port" however I had on OSX and after little digging in I found the answer. I thought to share the answer here for people coming from osx.
If you visit known-issues from docker for mac and github issue, you will find that by default the docker daemon only listens on unix socket /var/run/docker.sock and not on tcp. The default port for docker is 2375 (unencrypted) and 2376(encrypted) communication over tcp(although you can choose any other port).
On OSX its not straight forward to run the daemon on tcp port. To do this one way is to use socat container to redirect the Docker API exposed on the unix domain socket to the host port on OSX.
docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 127.0.0.1:2375:2375 bobrik/socat TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock
and then
export DOCKER_HOST=tcp://localhost:2375
However for local client on mac os you don't need to export DOCKER_HOST variable to test the api.
If you run ps -aux | grep dockerd you should see the endpoints it is running on.
Try add -H tcp://0.0.0.0:2375(at end of Execstart line) instead of -H 0.0.0.0:2375.