I want ask 2 question about docker stats
for example
NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
container_1 1.52% 11.72MiB / 7.388GiB 0.15% 2.99GB / 372MB 9.4MB / 0B 9
in this situation net i/o statement 2.99GB / 372MB
how much time reflected in that?
for one hour? or all of time?
and how can check docker container network traffic for an hour or minute?
i would appreciate if you any other advice.
thank you
This blog explains the network io of the docker stats command
Displays total bytes received (RX) and transmitted (TX).
If you need finer grained access, the blog also suggests to use the network pseudo files on your host system.
$ CONTAINER_PID=`docker inspect -f '{{ .State.Pid }}' $CONTAINER_ID`
$ cat /proc/$CONTAINER_PID/net/dev
To your second part: I'm not aware of any build-in method to get the traffic over the specific period, others might correct me. I think the easiest solution is to poll one of the two interfaces and calculate the differences yourself.
Related
I am running command docker stats <container_id> > performance.txt over a period of 1 hour during multi-user testing. Some stats like memory, CPU increase, then normalize. But, it is with NET I/O value that it kept on increasing.
At the start, the O/P was:
NAME CPU % MEM USAGE / LIMIT NET I/O BLOCK I/O PIDS
my-service 0.10% 5.63GiB / 503.6GiB 310MB / 190MB 0B / 0B 80
NAME CPU % MEM USAGE / LIMIT NET I/O BLOCK I/O PIDS
my-service 0.20% 5.63GiB / 503.6GiB 310MB / 190MB 0B / 0B 80
After 1 hour, it is:
NAME CPU % MEM USAGE / LIMIT NET I/O BLOCK I/O PIDS
my-service 116.26% 11.54GiB / 503.6GiB 891MB / 523MB 0B / 0B 89
NAME CPU % MEM USAGE / LIMIT NET I/O BLOCK I/O PIDS
my-service 8.52% 11.54GiB / 503.6GiB 892MB / 523MB 0B / 0B 89
As above, the value of NET I/O is always increasing. What can it probably mean?
Docker documentation says it is the input received and output given by the container. If so, then why is it increasing? Is there some issue with the image running in the container?
NET I/O is a cumulative counter. It only goes up (when your app receives and sends data).
https://docs.docker.com/engine/reference/commandline/stats/
Column name
Description
NET I/O
The amount of data the container has sent and received over its network interface
So it's accumulated over time. Unlike, say, CPU % which is how much CPU the container is using right now.
The docker stats command returns a live data stream for running containers.
It's considered the total amount of data passed over the network since the container started. From the definition of stream:
computing
a continuous flow of data or instructions,
It doesn't say that explicitly but you can obviously infer this by the term continuous or stream. Perhaps the documentation could be made a bit more clear in that respect.
Good morning all,
In the process of trying to train myself in Docker, I'm having trouble.
I created a docker container from a wordpress image, via docker compose.
[root#vps672971 ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
57bb123aa365 wordpress:latest "docker-entrypoint.s…" 16 hours ago Up 2 0.0.0.0:8001->80/tcp royal-by-jds-wordpress-container
I would like to allocate more memory to this container, however after the execution of the following command, the information returned by docker stats are not correct.
docker container update --memory 3GB --memory-swap 4GB royal-by-jds-wordpress-container
docker stats
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
57bb123aa365 royal-by-jds-wordpress-container 0.01% 9.895MiB / 1.896GiB 0.51% 2.68kB / 0B 0B / 0B 6
I also tried to request API engine to retrieve information about my container, but the limitation displayed is not correct either.
curl --unix-socket /var/run/docker.sock http:/v1.21/containers/royal-by-jds-wordpress-container/stats
[...]
"memory_stats":{
"usage":12943360,
"max_usage":12955648,
"stats":{},
"limit":2035564544
},
[...]
It seems that the modification of the memory allocated to the container didn't work.
Anyone have an idea?
Thank you in advance.
Maxence
Here's the scenario:
On a Debian GNU/Linux 9 (stretch) VM I have two containers running. The day before yesterday I got a warning from the monitoring that the memory usage is relatively high. After looking at the VM it could be determined that not the containers but Docker daemon needs them. htop
After a restart of the service I noticed a new increase of memory demand after two days. See graphic.
RAM + Swap overview
Is there a known memory leak for this version?
Docker version
Memory development (container) after 2 days:
Container 1 is unchanged
Container 2 increased from 21.02MiB to 55MiB
Memory development (VM) after 2 days:
The MEM increased on the machine from 273M (after reboot) to 501M
dockerd
- after restart 1.3% MEM%
- 2 days later 6.0% of MEM%
Monitor your containers to see if their memory usage changes over time:
> docker stats
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
623104d00e43 hq 0.09% 81.16MiB / 15.55GiB 0.51% 6.05kB / 0B 25.5MB / 90.1kB 3
We saw a similar issue and it seems to have been related to the gcplogs logging driver. We saw the problem on docker 19.03.6 and 19.03.9 (the most up-to-date that we can easily use).
Switching back to using a log forwarding container (e.g. logspout) resolved the issue for us.
I have a setup of 10 docker containers from different immages in a swarm on 3 machines. I need to monitor and log network latency / packet delays between each container. Is there a right tool for it?
I can implement something like
while true; for host in "${my_hosts[*]}"; do ping -c 1 "$host" > latency.log; done done
and launch it on each machine, tailing latency.log to monitor like Prometheus. But it feels like reinvensing a square wheel.
I hope i understand what you need , Im implementing something like this myself .
I tested netdata with prometheus and graphana and metricbeat\filebeat with elastic and kibana
we choose to use elastic (ELK stack) because in the same DB you can handle metrics and textual data .
hope i gave you some directions .
What I have at the end is a setup that:
Shares hosts between containers by volume,
Measures latency feeding hosts to fping,
Writes fping output to log file,
Serves this log file to Prometheus by mtail.
I've implemented wrapper around fping to let it work with mtail:
#!/usr/bin/env bash
# It wraps `fping -lDe` to give output for multiple hosts one line at time (for `mtail` parser)
# Default `fping -lDe` behavior produce X lines at time where X = number of hosts to ping
# This waits for hosts file with `# docker` section as described in usage guide
echo "Measuing time delays to docker hosts from '$1'"
# take hostnames after '# docker' comment line
hosts=$(cat $1 | sed -n '/# docker/,$p' | sed 1,1d)
trap "exit" INT # exit loop by SIGINT
# start `fping` and write it's output to stdout line by line
stdbuf -oL fping -lDe $hosts |
while IFS= read -r line
do
echo $line
done
And there is mtail parser for the log file:
gauge delay by host
gauge loss by host
# [<epoch_time>] <host> : [2], 84 bytes, <delay> ms (0.06 avg, 0% loss)
/\[(?P<epoch_time>\d+\.\d+)\] (?P<host>[\w\d\.]*) : \[\d+\], \d+ \w+, (?P<delay>\d+\.\d+) ms \(\d+\.\d+ avg, (?P<loss>\d+)% loss\)/ {
delay[$host]=$delay
loss[$host]=$loss
}
Now you can add fping and mtail to your images to let it serve delays and losses metrics for Prometheus.
References:
mtail: https://github.com/google/mtail
fping: https://fping.org/
I have been working with Docker for a while now, I have installed docker and launched a container using
docker run -it --cpuset-cpus=0 ubuntu
When I log into the docker console and run
grep processor /proc/cpuinfo | wc -l
It shows 3 which are the number of cores I have on my host machine.
Any idea on how to restrict the resources to the container and how to verify the restrictions??
The issue has been already raised in #20770. The file /sys/fs/cgroup/cpuset/cpuset.cpus reflects the correct output.
The cpuset-cpus is taking effect however is not being reflected in /proc/cpuinfo
docker inspect <container_name>
will give the details of the container launched u have to check for "CpusetCpus" in there and then u will find the details.
Containers aren't complete virtual machines. Some kernel resources will still appear as they do on the host.
In this case, --cpuset-cpus=0 modifies the resources the container cgroup has access to which is available in /sys/fs/cgroup/cpuset/cpuset.cpus. Not what the VM and container have in /proc/cpuinfo.
One way to verify is to run the stress-ng tool in a container:
Using 1 cpu will be pinned at 1 core (1 / 3 cores in use, 100% or 33% depending on what tool you use):
docker run --cpuset-cpus=0 deployable/stress -c 3
This will use 2 cores (2 / 3 cores, 200%/66%):
docker run --cpuset-cpus=0,2 deployable/stress -c 3
This will use 3 ( 3 / 3 cores, 300%/100%):
docker run deployable/stress -c 3
Memory limits are another area that don't appear in kernel stats
$ docker run -m 64M busybox free -m
total used free shared buffers cached
Mem: 3443 2500 943 173 261 1858
-/+ buffers/cache: 379 3063
Swap: 1023 0 1023
yamaneks answer includes the github issue.
it should be in double quotes --cpuset-cpus="", --cpuset-cpus="0" means it make use of cpu0.