I have a few micro-services running in docker containers (One service in each container).
How do I find out the network usage between every two pair of docker containers? So that I can make a graph such that I have containers as vertices and on edges I have the amount of bytes transmitted/received.
I used cAdvisor, but it gives me the overall network usage of each container.
Start with docker stats:
$ docker stats
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
a645ca0d8feb wp_wordpress_1 0.00% 135.5MiB / 15.55GiB 0.85% 12.2MB / 5.28MB 2.81MB / 9.12MB 11
95e1649c5b79 wp_db_1 0.02% 203MiB / 15.55GiB 1.28% 6.44MB / 3.11MB 0B / 1.08GB 30
Related
I am running command docker stats <container_id> > performance.txt over a period of 1 hour during multi-user testing. Some stats like memory, CPU increase, then normalize. But, it is with NET I/O value that it kept on increasing.
At the start, the O/P was:
NAME CPU % MEM USAGE / LIMIT NET I/O BLOCK I/O PIDS
my-service 0.10% 5.63GiB / 503.6GiB 310MB / 190MB 0B / 0B 80
NAME CPU % MEM USAGE / LIMIT NET I/O BLOCK I/O PIDS
my-service 0.20% 5.63GiB / 503.6GiB 310MB / 190MB 0B / 0B 80
After 1 hour, it is:
NAME CPU % MEM USAGE / LIMIT NET I/O BLOCK I/O PIDS
my-service 116.26% 11.54GiB / 503.6GiB 891MB / 523MB 0B / 0B 89
NAME CPU % MEM USAGE / LIMIT NET I/O BLOCK I/O PIDS
my-service 8.52% 11.54GiB / 503.6GiB 892MB / 523MB 0B / 0B 89
As above, the value of NET I/O is always increasing. What can it probably mean?
Docker documentation says it is the input received and output given by the container. If so, then why is it increasing? Is there some issue with the image running in the container?
NET I/O is a cumulative counter. It only goes up (when your app receives and sends data).
https://docs.docker.com/engine/reference/commandline/stats/
Column name
Description
NET I/O
The amount of data the container has sent and received over its network interface
So it's accumulated over time. Unlike, say, CPU % which is how much CPU the container is using right now.
The docker stats command returns a live data stream for running containers.
It's considered the total amount of data passed over the network since the container started. From the definition of stream:
computing
a continuous flow of data or instructions,
It doesn't say that explicitly but you can obviously infer this by the term continuous or stream. Perhaps the documentation could be made a bit more clear in that respect.
I want multiple running containers to have mutually exclusive resources with each other. For example, when there are CPU cores from id0 to id63, if 32 CPU cores are allocated to each container, the CPU cores assigned to them are mutually exclusive. In addition, when the host has 16GB of RAM, we want to allocate 8GB to each container so that one container does not affect the memory usage of another container.
Is there good way to do this?
I think all you need is to just limit container resources. This way you can ensure that no container uses more than X cores and/or Y RAM. To limit CPU usage to 1 core add --cpus=1.0 to your docker run command. To limit RAM to 8 gigabytes add -m=8g. Putting it altogether:
docker run --rm --cpus=1 -m=8g debian:buster cat /dev/stdout
And if your look at docker stats you will see that memory is limited (no indication for CPU though):
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
8d9a33b00950 funny_shirley 0.00% 1MiB / 8GiB 0.10% 6.7kB / 0B 0B / 0B 1
Read more in the docs.
CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
48c16e180af6 0.20% 91.48MiB / 31.31GiB 0.29% 3.86kB / 0B 85.3MB / 0B 33
f734efe5a249 0.00% 472KiB / 31.31GiB 0.00% 3.97kB / 0B 12.3kB / 0B 1
165a7b031093 0.00% 480KiB / 31.31GiB 0.00% 9.49kB / 0B 3.66MB / 0B 1
Does anyone know how to get resource consumption of a specific Docker container within its running environment?
Outside of a container, we can get it easily by typing a command "docker stats". Besides, if I try to get resource consumption inside a container, it will get the consumption (RAM, CPU) of the physical computer which the container runs on.
Another option is using 'htop' command, but it does not show the result exactly compared to 'docker stats' command.
If you want the processes consumption inside the container, you can go into the container and monitor the processes.
docker exec -it <container-name> watch ps -aux
Notice that after running the above command, the container doesn't know about any docker processes running.
I have run my docker container containing RabbitMQ instance. Ia have used docker run command with three parameters (among others):
-m 300m
--kernel-memory="300m"
--memory-swap="400m"
docker stats shows:
> CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O
> 1f50929f8e4e 0.40% 126.8 MB / 349.2 MB 36.30% 908.2 kB / 1.406 MB 24.69 MB / 1.114 MB
I have expected that RabbitMQ will see only 300MB of RAM memory, but high watermark visible on Rabbit UI shows 5,3GB. My host has 8GB available so, probably RabbitMQ read memory size from host.
I am trying to get the stats of the containers that are running inside the kubernetes nodes via docker stats command. But unfortunately I am getting all the values as "0" for all the pod containers.
CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
7dc5af9923b2 0.00% 0 B / 0 B 0.00% 0 B / 0 B 0 B / 0 B 0
I did the same with containers that I brought up manually via docker run command in the same node and I am getting proper values for those containers.
CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
8be93c039a25 0.12% 133.3 MB / 3.892 GB 3.43% 0 B / 648 B 2.208 MB / 0 B 0
Is there any specific method to get the stats fot the pod containers other than this?
Note: docker version is 1.11.2 and kube version is 1.3.7
I resolved this one. I used the kubelet API to get the metrics for the node and well as for individual containers. The following API will return the metrics for a pod container.
http://<nodeIP>:10255/stats/<podName>/<containerName> - POST
This post that was suggested in the comments was much helpful.
This document has some more APIs to collect the metrics.