I don't understand why
docker network ls | cut -f2
has no effect on the output of docker network ls. The output seems to be delimited by tabs. I would expect
NETWORK ID NAME DRIVER SCOPE
5e2e0d7c6daf bridge bridge local
5012d9100982 host host local
9574504d9579 test_default bridge local
0f7083dc1db7 php_fpm bridge local
to output
NAME
bridge
host
test_default
php_fpm
The default delimiter of cut is tab, and only one character allowed in cut, so it cannot meet your requirements, suggest to use awk:
docker network ls | awk -F' +' '{print $2}'
Sample output:
NAME
bridge
dashboard_default
host
none
Related
I have a simple query in Prometheus like:
irate(node_network_receive_bytes_total{job="job-name",device=~"br-.*",name="a-name"}[5m])
The Legend contains: {{device}} received
On the Panel i see my metric graphics and to the right a legend list with all network "devices" by it's name like:
br-a539a330b41b received
br-6c596943b7c8 received
br-dc406d831d4e received
[...]
If I ask docker network on that computer I can get a name list like:
ubuntu#ip-10-278-0-45:~$ docker network ls
NETWORK ID NAME DRIVER SCOPE
0a9c74974c7f bridge bridge local
6c596943b7c8 a-service_default bridge local
dc406d831d4e b-service_default bridge local
a539a330b41b c-service_default bridge local
68a058a0a38b host host local
0b101d111154 none null local
[...]
Within a simple bash snipped as follows I could translate the current given Panel Legends to it's better understandable service names.
docker network ls | grep 6c596943b7c8 | awk '{print $2}' | sed 's/_default//'
Is there a simple way to translate the current Legend {{device}} to something more readable like the names found in docker network ls?
So I get a legend like:
a-service received
b-service received
c-service received
[...]
Or is there an other ''simple'' way, to translate such uuid like bridge network names?
One ''other'' way could be to "manipulate" the node_exporter output.
You can use label_map function with the network_id -> network_name mapping. For example:
label_map(
irate(node_network_receive_bytes_total{job="job-name",device=~"br-.*",name="a-name"}[5m]),
"device",
"br-6c596943b7c8", "a-service",
"br-dc406d831d4e", "b-service",
"br-a539a330b41b", "c-service",
)
The label_map function is available in VictoriaMetrics. I hope it will be added to Prometheus soon.
The problem
I used inet:gethostbyname(Hoststr) in a docker environment with a couple containers for over a year without problems. Due to non recoverable read errors on the SSD which -- according to the provider -- is perfectly fine I was forced to reinstall from scratch with a CentOS image.
After restore I get a crash which boils down to
3> inet:gethostbyname("www").
{ok,{hostent,"www",
["www"],
inet,4,
[{10,0,1,205},{10,0,1,180}]}}
obviously because I get 2 IPs.
getaddr works fine:
4> inet:getaddr("www", inet).
{ok,{10,0,1,205}}
Question
I can replace gethostbyname with getaddr, no problem, but I would like to know why I get 2 IPs in the first place and how this misbehavior could creep in.
PHP does just fine:
id=$(docker ps -a | grep "vx_www.1\." | grep -v "xited" | awk '{print $1}') && docker exec -it $id ash
php > echo gethostbyname('www');
10.0.1.205
Is it a docker problem?
The problem may lie on the docker side, as both addresses ping.
It gets even more interesting (from the host):
/ # ip a | grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"
127.0.0.1
10.0.1.205
10.0.1.255
172.18.0.13
172.18.255.255
10.0.8.33
10.0.8.255
10.0.0.162
10.0.0.255
10.0.9.19
10.0.9.255
This should show addresses from within the docker system, if I understand correctly, and the latter address is not in the list. But it is somewhere:
# docker inspect sa3oizstflg3 | grep "10.0.1"
"Addr": "10.0.1.180/24"
What is a VirtualIP?
Actually I get this address with the ID of www as VirtualIP
together with a bunch of others
"VirtualIPs": [
{
"NetworkID": "y3rf5yes97yjs1rfzpzf4aou8",
"Addr": "10.0.0.148/24"
},
{
"NetworkID": "swagio8bzavl2bf5u5bivmt13",
"Addr": "10.0.1.180/24"
},
{
"NetworkID": "tnmpad21shpvsps6fps5m4own",
"Addr": "10.0.8.5/24"
},
{
"NetworkID": "mz9aogx7jxit8x2bflgpyh5lh",
"Addr": "10.0.9.2/24"
}
The same container listens to 2 different IPs
Taking a different container with PHP on board, I get the second address given by inet:gethostbyname("www") for the same container, so both seem to be correct and usable:
# id=$(docker ps -a | grep "vx_wsm.1\." | grep -v "xited" | awk '{print $1}') && docker exec -it $id ash
/ # php -a
Interactive shell
php > echo gethostbyname('www');
10.0.1.180
Now I am confused. Anybody knows what is happening here?
inet:gethostbyname seems to be not wrong but more correct, then.
Erlang question
As an addendum: I am not that proficient in Erlang. In my code it reads:
get_ip_web(Web) -> % e.g. www = > 100.0.1.226
[case X of $, -> $.; _ -> X end || X <- lists:flatten(io_lib:format("~p",element(6, element(2, inet:gethostbyname(Web))))), X=/=${, X=/=$}].
How to rewrite this fine piece of code to pick one of the two addresses, but working also with only one result?
Well, this is mostly an academic question as I didn't understand this one-liner at all initially -- my comment was incomprehensible. This is no longer the case, but I still struggle with constructs easy to handle in other languages, especially if there is no routine for a long time.
For your information: I replaced the one-liner above with this much simpler one-liner:
inet:ntoa(element(2,inet:getaddr(Web, inet)))
Erlang:
From the code, inet:getaddr/2 just gets the first ip from inet:getaddrs/2, which in turn gets them from gethostbyname.
You should use the hostent record instead of element (only if you go with functions that return a hostent, I'd rather use the getaddr like you did in the end, it's actually the same code):
-module(test).
-include_lib("kernel/include/inet.hrl").
-export([get_ip/0]).
get_ip() ->
{ok, #hostent{h_addr_list = [IP|_]}} = inet:gethostbyname("www.stackoverflow.com"),
inet:ntoa(IP).
Docker:
If you run ip -a in the host, you'll get only the ips for the host, not the ones used by containers. Usually the host has an IP in each of the bridges that make the different docker networks, so it's in the same range as the containers.
Regarding the several ips, I don't have experience with docker swarm (which it seems that you're using), only with kubernetes.
My guess is that you have a service exposed in several docker networks that shares the name (www) with the container that implements it. Thus, in the network of the www container you have DNS both resolve www to the container IP and the service virtualIP address. Maybe you can find the virtual ips in iptables/nftables?
How do I get a list of all the containers in a user-defined docker network?
I would like to get all the commit hashes of every container for a specific user-defined docker network.
docker network inspect \
-f '{{ range $key, $value := .Containers }}{{ printf "%s\n" $key}}{{ end }}' \
<network-id>
will give a newline delimited list of the container hashes belonging to the network-id network.
You can adjust the printf format to get other syntax (comma-separated, tab-separated, container id+name, etc). For example:
$ docker network inspect \
-f '{{ range $key, $value := .Containers }}{{printf "%s: %s\n" $key .Name}}{{ end }}' \
web-ingress
0b7f82ad7535ac7c1a454aaf0a5df6b87547d3cb9d751d0e0e1b4952a849b11b: mon_grafana.1.v5i0ea6nv12k0h3eo8egglg6n
1cce723642af2ce9382c7d46cca868d4674b4645d07458eeed9e928d29a4fb1f: mon_prometheus.1.lsfdcig6uhqfbbv7n07irpj3j
1f4840710f77fa1e02bd3e95581139b9f3c13fe4c857ce6ac44bfbdae4916920: mon_alertmanager.1.kwvw7kqsfpi9qzpmqdrd85yc7
2efea443fee41fd5dbca714145ca6ff95d91be9c60a469be597aadfaca90914d: mon_unsee.1.3lg8qgnvshibklnypzt5rw95s
6fdb893488f6e56766501e763d4c60196ae12a22ee9bd204d84fe324331714e8: mon_dashboard.1.r6b6nhatwmk88y5ncgagnaxh4
lb-web-ingress: web-ingress-endpoint
You can list all networks with:
docker network ls
And inspect one network to see its hosts (containers)
docker network inspect <network-id>
Inspecting that a container, you can see which network it is connected to:
docker inspect <container-id>
More info in the docs: https://docs.docker.com/engine/userguide/networking/work-with-networks/#create-networks
Another way using pure bash and no python - easier to remember for me
docker network inspect YOUR_NETWORK_ID | grep Name | tail -n +2 | cut -d':' -f2 | tr -d ',"'
Sample output
container1
container2
A simple command to list the names of the container using a custom network is:
docker inspect network-name | grep Name
This command will list the name of the network and containers connected to it like below:
"Name": "network-name",
"Name": "Container1",
"Name": "Container2",
Solution can be simple as below as well. To list name of the container only
docker network inspect NETWORK ID | grep "Name" | awk -F":" '{print $2}' | tr -d ',"'
This is my bash command using python's json parsing
docker network inspect YOUR_NETWORK_ID \
| python -c "import sys, json; print([v['Name'] for k,v in json.load(sys.stdin)[0]['Containers'].items()])"
Sample output
[u'container1', u'container2']
I found that when a container has been brought up with an attribute of
network_mode: "service: <some service name>"
that the 'docker inspect' or 'docker network inspect ' commands are not going to give you the information as detailed above.
What you will see instead is something like this:
"NetworkMode": "container: <some container id representing the service>"
Allow me to contribute with another way of obtaining the result, still using docker network inspect, but providing other possibilities in terms of data inspection and output format.
You list the available networks for Docker:
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
a1682358dfa3 bridge bridge local
c6910d1da750 host host local
c9790e542025 none null local
9b265ecb4042 web_default bridge local
You inspect the network, with output in JSON, accessing the Containers object + prettifying the JSON:
$ docker network inspect web_default -f '{{json .Containers}}' | python3 -m json.tool
{
"5d05c2606b022b0c9bc480f138b8f8f0e382dca4b1decb1c27181a66e641803f": {
"Name": "django",
"EndpointID": "b6bf6b1dc5437f2359fb550368a5bc6529cd80fa5b9ece3c752eecef6627ea57",
"MacAddress": "02:42:ac:12:00:03",
"IPv4Address": "172.18.0.3/16",
"IPv6Address": ""
},
"b971715bd0b3560831b03e86bcf2151ed04a60a46ad90b24f1f2265617360344": {
"Name": "postgres",
"EndpointID": "5883f465e235c66327271c1dfd05372f02012efa78928dd447c5444b0468647c",
"MacAddress": "02:42:ac:12:00:02",
"IPv4Address": "172.18.0.2/16",
"IPv6Address": ""
}
}
You can even redirect the command output to a for further inspection:
$ docker network inspect web_default -f '{{json .Containers}}' | python3 -m json.tool > web_default_containers.json
Our internal network has the range 172.20.0.0/16 reserved for internal purposes and docker uses the 172 range by default for its internal networking. I can reset the bridge to live in 192.168 by providing the bip setting to the daemon:
➜ ~ sudo cat /etc/docker/daemon.json
{
"bip": "192.168.2.1/24"
}
➜ ~ ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.2.1 netmask 255.255.255.0 broadcast 0.0.0.0
However, when creating new custom networks via docker network create or by defining them in the networks sections of the docker-compose.yaml these are still created in 172, thus eventually clashing with 172.20:
➜ ~ docker network create foo
610fd0b7ccde621f87d40f8bcbed1699b22788b70a75223264bb14f7e63f5a87
➜ ~ docker network inspect foo | grep Subnet
"Subnet": "172.17.0.0/16",
➜ ~ docker network create foo1
d897eab31b2c558517df7fb096fab4af9a4282c286fc9b6bb022be7382d8b4e7
➜ ~ docker network inspect foo1 | grep Subnet
"Subnet": "172.18.0.0/16",
I understand I can provide the subnet value to docker network create, but I rather want all such subnets created under 192.168.*.
How can one configure dockerd to do this automatically?
For anyone who found this question. Now it is possible.
$ docker -v
Docker version 18.06.0-ce, build 0ffa825
Edit or create config file for docker daemon:
# nano /etc/docker/daemon.json
Add lines:
{
"default-address-pools":
[
{"base":"10.10.0.0/16","size":24}
]
}
Restart dockerd:
# service docker restart
Check the result:
$ docker network create foo
$ docker network inspect foo | grep Subnet
"Subnet": "10.10.1.0/24"
It works for docker-compose too.
Your "bip": "192.168.2.1/24" works for bridge0 only. It means that any container which run without --network will use this default network.
I have a Docker Compose v2 file which starts a container. I locally run a service on port 3001. I want to reach this service from the Docker container.
The Docker Compose file looks like this:
version: '2'
services:
my-thingy:
image: my-image:latest
#network_mode: host #DOES not help
environment:
- THE_HOST_I_WANT_TO_CONNECT_TO=http://127.0.0.1:3001
ports:
- "3010:3010"
Now, how can I reach THE_HOST_I_WANT_TO_CONNECT_TO?
What I tried is:
Setting network_mode to host. This did not work. 127.0.0.1 could not be reached.
I can also see that I can reach the host from the container if I use the local IP of the host. A quick hack would be to use something like ifconfig | grep broadcast | awk '{print $2}' to obtain the IP and substitute that in Docker Compose. Since this IP can change on reconnect and different setups can have different ifconfig results, I am looking for a better solution.
I've used another hack/workarkound from comments in the docker issue #1143. Seems to Work For Me™ for the time being... Specifically, I've added the following lines in my Dockerfile:
# - net-tools contains netstat, used to discover IP of Docker host server.
# NOTE: the netstat trick is to make Docker host server accessible
# from inside Docker container under name 'dockerhost'. Unfortunately,
# as of 2016.10, there's no official/robust way to do this when Docker host
# has no public IP/DNS entry. What is used here is built based on:
# - https://github.com/docker/docker/issues/1143#issuecomment-39364200
# - https://github.com/docker/docker/issues/1143#issuecomment-46105218
# See also:
# - http://stackoverflow.com/q/38936738/98528
# - https://github.com/docker/docker/issues/8395#issuecomment-200808798
# - https://github.com/docker/docker/issues/23177
RUN apt-get update && apt-get install -y net-tools
CMD (netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2" dockerhost"}' >> /etc/hosts) && \
...old CMD...
With this, I can use dockerhost as the name of the host where Docker is installed. As mentioned above, this is based on:
https://github.com/docker/docker/issues/1143#issuecomment-39364200
(...) One way is to rely on the fact that the Docker host is reachable through the address of the Docker bridge, which happens to be the default gateway for the container. In other words, a clever parsing of ip route ls | grep ^default might be all you need in that case. Of course, it relies on an implementation detail (the default gateway happens to be an IP address of the Docker host) which might change in the future. (...)
https://github.com/docker/docker/issues/1143#issuecomment-46105218
(...) A lot of people like us are looking for a little tidbit like this
netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}'
where netstat -nr means:
Netstat prints information about the Linux networking subsystem.
(...)
--route , -r
Display the kernel routing tables.
(...)
--numeric , -n
Show numerical addresses instead of trying to determine symbolic host, port or user names.
This is a known issue with Docker Compose: see Document how to connect to Docker host from container #1143. The suggested solution of a dockerhost entry in /etc/hosts is not implemented.
I went for the solution with a shell variable as also suggested in a comment by amcdl on the issue:
Create a LOCAL_XX_HOST variable: export LOCAL_XX_HOST="http://$(ifconfig en0 inet | grep "inet " | awk -F'[: ]+' '{ print $2 }'):3001".
Then, for example, refer to this variable in docker-compose like this:
my-thingy:
image: my-image:latest
environment:
- THE_HOST_I_WANT_TO_CONNECT_TO=${LOCAL_XX_HOST}