Run docker commands using glob / wildcard on container names - docker

I'm using docker-compose up --scale to create multiple versions of the same container. As a result I end up with containers named container_foo_1, container_foo_2 etc.
Does docker support any kind of glob / wildcard matching on container names in it's command line tools? What I want to do is this:
docker inspect container_foo_*
What I'm doing right now in the short term is just using:
docker-inspect container_foo_{1,2} (using bash brace expansion)
but I'd love if there was a way where I didn't know how many containers there were / what the numbers were ahead of time.

You can use the argument --filter | -f at docker ps with docker inspect.
Usage: docker ps --filter key=value,
where value accept regular expressions.
The currently supported filters are:
id Container’s ID
name Container’s name
label An arbitrary string representing either a key or a key-value pair. Expressed as or =
exited An integer representing the container’s exit code. Only useful with --all.
status One of created, restarting, running, removing, paused, exited, or dead
ancestor Filters containers which share a given image as an ancestor. Expressed as * [:], , or image#digest
before or since Filters containers created before or after a given container ID or name
volume Filters running containers which have mounted a given volume or bind mount.
network Filters running containers connected to a given network.
publish or expose Filters containers which publish or expose a given port. Expressed as <port>[/<proto>] or <startport-endport>/[<proto>]
health Filters containers based on their healthcheck status. One of starting, healthy, unhealthy or none.
isolation Windows daemon only. One of default, process, or hyperv.
is-task Filters containers that are a “task” for a service. Boolean option (true or false)
Ex: docker inspect $(docker ps --filter name=^/server --quiet)
References:
Filtering
How to filter docker ps by exact name?

Related

Remove multiple Docker containers based on creation time

I have many Docker containers both running and exited. I only want to keep the containers that were created before today/some specified time -- I would like to remove all containers created today. Easiest way to approach this?
Out of the box on all OS there is the possibility to remove all containers younger than a given one:
docker rm -f $(docker ps -a --filter 'since=<containername>' --format "{{.ID}}")
so the container given in since will be kept, but all newer ones not. Maybe that suits your use case.
If you really need a period of time there will be some bash magic doing that. But specify your needs exactly then..
In detail:
docker rm: removing one or more containers
-f: force running containers to stop
docker ps -a: listing all containers
--filter 'since=..' filtering containers created since given with all details
--format "{{.ID}}": filtering ID-column only

How to list the volume ids associated with the containers created from docker-compose.yaml?

I have multiple containers running in my machine. Each of them belongs to a docker-compose.yaml file and they create different volumes.
Now I'd like to delete the volumes of containers created from a specific docker-compose.yaml. I can not find anything online.
docker ps -a, docker ps -aq, docker volume ls => None of this gives an association.
I'd like to delete the volumes [...] created from a specific docker-compose.yaml
docker-compose down -v deletes containers, networks, and volumes. (Without -v it keeps volumes for future use, since they're presumed to contain user data that's hard to recreate.) You don't need to know the specific Docker volume ID for this to work.
There are equivalent docker-compose commands to run most common tasks. These are generally driven off of the service name (block heading) in the docker-compose.yml, so you don't need to know or specify the Docker name of a container or network.

List docker containers by App using command line tool

How can I list docker containers by application? For example, assume that I have app1 (that has c1, c2, c3) and app2 (that has c4, c5) deployed. I can see c1, c2 and c3 are under app1 and c4 and c5 are under app2 using docker dashboard. But, if I do "docker ps", it returns a flat list of containers, I have no way to know which containers belong to which app? Below is an example.
As can be seen in docker dashboard on the left side, there are two applications, nginx-gohttp-master and demo, deployed in the docker environment. The first app has two containers and the second app has four containers, and the two apps are independent of each other. However, I only see a flat list of containers on the right side as listed by "docker ps". I would like to know how many apps are deployed and which containers belong to which app. How can I do that using a docker command line tool?
Thanks
By default, docker-compose adds some labels including com.docker.compose.project which contains the name of the compose app/project. You can get that output using docker ps with the --format option like so:
$ docker ps --format 'table {{.Label "com.docker.compose.project"}} {{.ID}} {{.Names}} {{.Command}} {{.Status}}'
project CONTAINER ID NAMES COMMAND STATUS
app1 1fd93328e69b app1_nginx_1 "/docker-entrypoint.…" Up 3 minutes
app2 9fe81391b961 app2_nginx_1 "/docker-entrypoint.…" Up 2 minutes
app3 d28fca61a184 app3_nginx_1 "/docker-entrypoint.…" Up 1 minutes
You can see some of the other labels that docker-compose applies here.
Check out filters
Filter running containers
docker ps --filter "name=nginx-gohttp-master"
Filter all containers (running + stopped)
docker ps -a --filter "name=nginx-gohttp-master"
Apparently there's no need for a wildcard character *, because = is not a direct exact-match string equality but a sub-string search.
For an exact match (which is not what you want in your question) you need to pipe the result into something else as discussed here.

Use docker system prune to filter networks

I want to use the command docker system prune to remove all unused containers and images, but I want network with a certain driver to be kept alive.
If I run docker system prune --filter "driver!=foo" I get the following result: Invalid filter 'driver!'.
When I use docker network ls I can filtering a specific driver using docker network ls --filter driver=foo.
How can I filter a certain driver in docker system prune?
The filter you are trying to apply is not supported. As a workaround, you can use label when creating the network:
$ docker network create --label=mynet your_network_name_here
and then use:
docker system prune --filter "label!=mynet"
The docs mention that docker system prune supports 2 filter cases:
Filtering
The currently supported filters are:
until (<timestamp>) - only remove containers, images, and networks created before given timestamp
label (label=<key>, label=<key>=<value>, label!=<key>, or label!=<key>=<value>) - only remove containers, images, networks, and volumes with (or without, in case label!=... is used) the specified labels.
Example:
$ docker network create test-1
13ce668830472bb33d2d2b4be5b3236b59df4ea5d20f571e5ba04359ea3617f1
$ docker network create test-2
46a4bb3021250667fd26bf8d76bd06f789c7cf8149ab698b1cec40e1c23d34ad
$ docker network create test-3
a4c2f507e767dfef12684ce451e855a88225c44fcadfdbed7c7cd4502fd0cdd2
$ docker network create --label=mynet test-4
41e9d3c38c1de54e6bb4e25784d73ef7bfbe3e848208a130eb2564c46670a330
$ docker system prune --filter "label!=mynet"
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all dangling images
- all dangling build cache
- Elements to be pruned will be filtered with:
- label={"label!":{"mynet":true}}
Are you sure you want to continue? [y/N] y
Deleted Networks:
test-1
test-2
test-3
Total reclaimed space: 0B
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
3ca2cec4eda2 bridge bridge local
dc9d8ffaed44 host host local
d21298cd02e3 none null local
41e9d3c38c1d test-4 bridge local

In a Docker Swarm, how do I find which node IP is running a given task?

The question title is the specific problem I am trying to solve. But even more simply, is it at all possible to list all tasks in a service together with the node IPs running them?
docker service ps will list task IDs together with the hostname of the node the task is running on. No other node identifier is provided such as ID or IP.
But I am using Vagrant to manage VMs and without a hostname configured, all host names are named the same ("vagrant"). This makes it very hard for me to figure out exactly which node is running the task!
This is important because I have to manually delete unused images or risk having the machines crash in the future when there's no more disk space. So figuring out where a task ran is the first step of this process lol.
For you Vagrant users, I changed my hostname quite easily in the Vagrantfile using the config.vm.hostname option. But of course, the question is still totally legit.
I can manually run docker images or docker ps on each node to figure out which node store the expected image and/or is currently running which container (the container name would be a concatenation of the task ID and task name, separated with a dot). But this is cumbersome.
I could also list all nodes with their IDs using docker node ls and then headhunt the task for each node, for example by using docker node ps 7b ("7b" is the first two letters in one of my node IDs). But this is cumbersome too and at best, I will "only" learn the node ID and not the IP.
But, I can find the IP using a node ID with a command like this: docker inspect 7b --format '{{.Status.Addr}}'. So getting at the IP directly is not a strict requirement and for a moment - when I understood this - I thought finding a node ID for a given task ID is going to be much easier!
But no. Even this seems to be impossible? As noted earlier, docker service ps does not give me the node ID. The docs for the command says that the placeholder .Name should give me the "Node ID" but this is wrong.
Until this moment I must have tried a billion different hacks.
One thing in particular that I find disturbing is that the docs for the docker node ps command explicitly states that it can be used to "list tasks running on one or more nodes" (emphasize mine). But if I do this: docker node ps vagrant I get an error message that the hostname is ambiguous because it resolves to more than one node! lol isn't that funny (even without using hostnames I have not gotten this command to work for listing tasks on multiple nodes). Not that it matters, because docker node ps just like docker service ps does not even output node IDs and the docs for both these commands lie about being able to do so.
So there you have it. I am left wondering if there's something right in front of me that I have missed or is the entire world relying on unique hostnames? It feels like I have to miss something obvious. Surely this oh so popular product Docker must be able to provide a way to find a node ID or node IP given a task ID? hmm.
I am running Docker version 17.06.2.
This gives me the node ID, given a <task ID>:
docker inspect <task ID> --format '{{.NodeID}}'
Use the node ID to get the node IP:
docker inspect <node ID> --format '{{.Status.Addr}}'
Or, all in one compressed line:
docker inspect -f '{{.Status.Addr}}' $(docker inspect -f '{{.NodeID}}' <task ID>)
As a bonus, the MAC address:
docker inspect -f '{{.NetworkSettings.Networks.ingress.MacAddress}}' $(docker inspect -f '{{.Status.ContainerStatus.ContainerID}}' <task ID>)

Resources