The question title is the specific problem I am trying to solve. But even more simply, is it at all possible to list all tasks in a service together with the node IPs running them?
docker service ps will list task IDs together with the hostname of the node the task is running on. No other node identifier is provided such as ID or IP.
But I am using Vagrant to manage VMs and without a hostname configured, all host names are named the same ("vagrant"). This makes it very hard for me to figure out exactly which node is running the task!
This is important because I have to manually delete unused images or risk having the machines crash in the future when there's no more disk space. So figuring out where a task ran is the first step of this process lol.
For you Vagrant users, I changed my hostname quite easily in the Vagrantfile using the config.vm.hostname option. But of course, the question is still totally legit.
I can manually run docker images or docker ps on each node to figure out which node store the expected image and/or is currently running which container (the container name would be a concatenation of the task ID and task name, separated with a dot). But this is cumbersome.
I could also list all nodes with their IDs using docker node ls and then headhunt the task for each node, for example by using docker node ps 7b ("7b" is the first two letters in one of my node IDs). But this is cumbersome too and at best, I will "only" learn the node ID and not the IP.
But, I can find the IP using a node ID with a command like this: docker inspect 7b --format '{{.Status.Addr}}'. So getting at the IP directly is not a strict requirement and for a moment - when I understood this - I thought finding a node ID for a given task ID is going to be much easier!
But no. Even this seems to be impossible? As noted earlier, docker service ps does not give me the node ID. The docs for the command says that the placeholder .Name should give me the "Node ID" but this is wrong.
Until this moment I must have tried a billion different hacks.
One thing in particular that I find disturbing is that the docs for the docker node ps command explicitly states that it can be used to "list tasks running on one or more nodes" (emphasize mine). But if I do this: docker node ps vagrant I get an error message that the hostname is ambiguous because it resolves to more than one node! lol isn't that funny (even without using hostnames I have not gotten this command to work for listing tasks on multiple nodes). Not that it matters, because docker node ps just like docker service ps does not even output node IDs and the docs for both these commands lie about being able to do so.
So there you have it. I am left wondering if there's something right in front of me that I have missed or is the entire world relying on unique hostnames? It feels like I have to miss something obvious. Surely this oh so popular product Docker must be able to provide a way to find a node ID or node IP given a task ID? hmm.
I am running Docker version 17.06.2.
This gives me the node ID, given a <task ID>:
docker inspect <task ID> --format '{{.NodeID}}'
Use the node ID to get the node IP:
docker inspect <node ID> --format '{{.Status.Addr}}'
Or, all in one compressed line:
docker inspect -f '{{.Status.Addr}}' $(docker inspect -f '{{.NodeID}}' <task ID>)
As a bonus, the MAC address:
docker inspect -f '{{.NetworkSettings.Networks.ingress.MacAddress}}' $(docker inspect -f '{{.Status.ContainerStatus.ContainerID}}' <task ID>)
Related
I've just done my first ever Docker deployment, when i run this command to see the status of recent processes...
docker ps -a
I get this output
My question is; what are those name referring to?
Those are random names generated for each container you are running.
You can see those names at pkg/namesgenerator/names-generator.go.
// Docker, starting from 0.7.x, generates names from notable scientists and hackers.
// Please, for any amazing man that you add to the list, consider adding an equally amazing woman to it, and vice versa.
right = [...]string{
// Muhammad ibn Jābir al-Ḥarrānī al-Battānī was a founding father of astronomy. https://en.wikipedia.org/wiki/Mu%E1%B8%A5ammad_ibn_J%C4%81bir_al-%E1%B8%A4arr%C4%81n%C4%AB_al-Batt%C4%81n%C4%AB
"albattani",
// Frances E. Allen, became the first female IBM Fellow in 1989. In 2006, she became the first female recipient of the ACM's Turing Award. https://en.wikipedia.org/wiki/Frances_E._Allen
"allen",
...
You could have fixed name by adding --name to your docker run commands.
That practice was introduced in commit 0d29244 by Michael Crosby (crosbymichael) for docker 0.6.5 in Oct. 2013:
Add -name for docker run
Remove docker link
Do not add container id as default name
Create an auto generated container name if not specified at runtime.
Solomon Hykes (shykes) evolved that practice in docker 0.7 with commit 5d6ef317:
New collection of random names for 0.7:
mood + famous inventor.
Eg. 'sad-tesla' or 'naughty-turing'
As VonC already wrote, "those are random names generated for each container you are running". So why should you use custom names?
docker run [image-name] -[container-name]
docker run wildfly-8.1 -mywildfly
well if you want to stop/kill/run/inspect or do anything with your container, you can use your name to do so:
docker kill mydocker
docker run mydocker
Otherwise you have to do docker ps all the time and check the id (since you will not remember those made-up custom names).
One important sidenote, if you assign custom name to docker, it will be forever assigned to that specific container, which means even if you kill the container, it is not running anymore, you cannot reuse the name.
If you run
docker ps -a
you can see the old one is still there, but stopped. You can't re-use a container name until the previous container has been removed by docker rm.
To remove containers older than some custom time, use the command:
$ docker ps -a | grep 'weeks ago' | awk '{print $1}' | xargs
--no-run-if-empty docker rm
More on removing containers here.
I'm using docker-compose up --scale to create multiple versions of the same container. As a result I end up with containers named container_foo_1, container_foo_2 etc.
Does docker support any kind of glob / wildcard matching on container names in it's command line tools? What I want to do is this:
docker inspect container_foo_*
What I'm doing right now in the short term is just using:
docker-inspect container_foo_{1,2} (using bash brace expansion)
but I'd love if there was a way where I didn't know how many containers there were / what the numbers were ahead of time.
You can use the argument --filter | -f at docker ps with docker inspect.
Usage: docker ps --filter key=value,
where value accept regular expressions.
The currently supported filters are:
id Container’s ID
name Container’s name
label An arbitrary string representing either a key or a key-value pair. Expressed as or =
exited An integer representing the container’s exit code. Only useful with --all.
status One of created, restarting, running, removing, paused, exited, or dead
ancestor Filters containers which share a given image as an ancestor. Expressed as * [:], , or image#digest
before or since Filters containers created before or after a given container ID or name
volume Filters running containers which have mounted a given volume or bind mount.
network Filters running containers connected to a given network.
publish or expose Filters containers which publish or expose a given port. Expressed as <port>[/<proto>] or <startport-endport>/[<proto>]
health Filters containers based on their healthcheck status. One of starting, healthy, unhealthy or none.
isolation Windows daemon only. One of default, process, or hyperv.
is-task Filters containers that are a “task” for a service. Boolean option (true or false)
Ex: docker inspect $(docker ps --filter name=^/server --quiet)
References:
Filtering
How to filter docker ps by exact name?
On the local host, I can remove an image using either docker image rm or docker rmi.
What if my current host is a manager node in a Docker swarm and I wish to cascade this operation throughout the swarm?
When I first created the Docker service, the image was pulled down on each node in the swarm. Removing the service did not remove the image and all nodes retain a copy of the image.
It feels natural that if there's a way to "push" an image out to all the nodes then there should be an equally natural way to remove them too without having to SSH into every single machine :'( Plus, this is a real problem. Sooner or later the nodes are bound to have no more disk space!
AFAIK there is no such option as of now. Each node is responsible of its own cleanup. There is a command docker system prune -f that you can use to clear container data.
But tagged images can be deleted using docker rmi only. See below issues
https://github.com/moby/moby/issues/24079
This is doable. Create host entries in /etc/hosts on your manager node, like this
1.1.1.1 node01
1.1.1.2 node02
1.1.1.3 node03
Then run
for i in {01..03}; do ssh host$i "docker rmi $(docker images -q)"; done
Warning: this command will remove all images on all nodes, listed in /etc/hosts.
I've just done my first ever Docker deployment, when i run this command to see the status of recent processes...
docker ps -a
I get this output
My question is; what are those name referring to?
Those are random names generated for each container you are running.
You can see those names at pkg/namesgenerator/names-generator.go.
// Docker, starting from 0.7.x, generates names from notable scientists and hackers.
// Please, for any amazing man that you add to the list, consider adding an equally amazing woman to it, and vice versa.
right = [...]string{
// Muhammad ibn Jābir al-Ḥarrānī al-Battānī was a founding father of astronomy. https://en.wikipedia.org/wiki/Mu%E1%B8%A5ammad_ibn_J%C4%81bir_al-%E1%B8%A4arr%C4%81n%C4%AB_al-Batt%C4%81n%C4%AB
"albattani",
// Frances E. Allen, became the first female IBM Fellow in 1989. In 2006, she became the first female recipient of the ACM's Turing Award. https://en.wikipedia.org/wiki/Frances_E._Allen
"allen",
...
You could have fixed name by adding --name to your docker run commands.
That practice was introduced in commit 0d29244 by Michael Crosby (crosbymichael) for docker 0.6.5 in Oct. 2013:
Add -name for docker run
Remove docker link
Do not add container id as default name
Create an auto generated container name if not specified at runtime.
Solomon Hykes (shykes) evolved that practice in docker 0.7 with commit 5d6ef317:
New collection of random names for 0.7:
mood + famous inventor.
Eg. 'sad-tesla' or 'naughty-turing'
As VonC already wrote, "those are random names generated for each container you are running". So why should you use custom names?
docker run [image-name] -[container-name]
docker run wildfly-8.1 -mywildfly
well if you want to stop/kill/run/inspect or do anything with your container, you can use your name to do so:
docker kill mydocker
docker run mydocker
Otherwise you have to do docker ps all the time and check the id (since you will not remember those made-up custom names).
One important sidenote, if you assign custom name to docker, it will be forever assigned to that specific container, which means even if you kill the container, it is not running anymore, you cannot reuse the name.
If you run
docker ps -a
you can see the old one is still there, but stopped. You can't re-use a container name until the previous container has been removed by docker rm.
To remove containers older than some custom time, use the command:
$ docker ps -a | grep 'weeks ago' | awk '{print $1}' | xargs
--no-run-if-empty docker rm
More on removing containers here.
I run a specific docker image for the first time:
docker run [OPTIONS] image [CMD]
Some of the options I supply include --link (link with other containers) and -p (expose ports)
I noticed that if I kill that container and simply do docker start <container-id>, Docker honors all the options that I specified during the run command including the links and ports.
Is this behavior explicitly documented and can I always count on the start command to reincarnate the container with all the options I supplied in the run command?
Also, I noticed that killing/starting a container which is linked to another container updates the upstream container's /etc/hosts file automatically:
A--(link)-->B (A has an entry in /etc/hosts for B)
If I kill B, B will normally get a new IP address. I notice that when i start B, the entry for B in A's /etc/hosts file is automatically updated... This is very nice.
I read here that --link does not handle container restarts... Has this been updated recently? If not, why am I seeing this behavior?
(Im using Docker version 1.7.1, build 786b29d)
Yes, things work as you describe :)
You can rely on the behaviour of docker start as it doesn't really "reincarnate" your container; it was always there on disk, just in a stopped state. It will also retain any changes to files, but changes in RAM, such as process state, will be lost. (Note that kill doesn't remove a container, it just stops it with a SIGKILL rather than a SIGTERM, use docker rm to truly remove a container).
Links are now updated when a container changes IP address due to a restart. This didn't use to be the case. However, that's not what the linked question is about - they are discussing whether you can replace a container with a new container of the same name and have links still work. This isn't possible, but that scenario will be covered by the new networking functionality and "service" objects which is currently in the Docker experimental channel.