This question already has answers here:
Check Resources Used by each Docker Container
(4 answers)
Closed 2 years ago.
How to see resource usage by docker containers?
I am using docker-compose version 2.1 and 2.4.
I would like to see running containers and their resources usage: RAM, CPU.
docker-compose ps returns containers, images, but not resources.
$ docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------------------------------------------------------------------
service-1 /app.sh Up (healthy) 8001/tcp
service-2 /app.sh Up (healthy) 0.0.0.0:80->4000/tcp
kafka /etc/confluent/docker/run Up 0.0.0.0:29092->29092/tcp, 0.0.0.0:7071->7071/tcp, 9092/tcp
mongo /init.sh Up (unhealthy) 0.0.0.0:27017->27017/tcp
zookeeper /etc/confluent/docker/run Up 2181/tcp, 2888/tcp, 0.0.0.0:32181->32181/tcp, 3888/tcp
You can see container resource usages with docker stats command.
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
95f5cdca9700 service-1 0.01% 35.39MiB / 381.5MiB 9.28% 141kB / 78kB 19.7MB / 0B 3
df49fdac6009 service-2 10.43% 278.6MiB / 381.5MiB 73.02% 126MB / 183MB 435MB / 0B 151
9dff501eb008 kafka 12.62% 348.1MiB / 572.2MiB 60.83% 610MB / 188MB 223MB / 0B 224
c00000a1d397 zookeeper 21.27% 313.7MiB / 381.5MiB 82.23% 600MB / 220MB 209MB / 0B 281
6d2d5000bb2e mongo 14.56% 325.8MiB / 381.5MiB 85.39% 531MB / 211MB 259MB / 0B 245
Related
The goal is to run two containers of publisher-app. One container should be mapped to port 8080 on the host machine, and the other on 8081. Here is the docker-compose:
publisher_app:
ports:
- "8080-8081:8080"
environment:
server.port: 8080
deploy:
mode: replicated
replicas: 2
Two containers are created, but as I understand, both ports are assigned to the first one, and the second one produces this error: Ports are not available: listen tcp 0.0.0.0:8081: bind: address already in use.
Here is the output of docker ps -a:
6c7067b4ebee spring-boot-rest-kafka_publisher_app "java -jar /app.jar" 33 seconds ago Up 28 seconds 0.0.0.0:8080->8080/tcp, 0.0.0.0:8081->8080/tcp spring-boot-rest-kafka_publisher_app_2
70828ba8f370 spring-boot-rest-kafka_publisher_app "java -jar /app.jar" 33 seconds ago Created spring-boot-rest-kafka_publisher_app_1
Docker engine version: 20.10.11
Docker compose version: 2.2.1
How to handle this case? Your help will be very appreciated.
Here is the source code: https://github.com/aleksei17/springboot-rest-kafka-mysql/blob/master/docker-compose.yml
tried locally on Windows 10 and failed similarly, both with v2 and with v2 disabled.
It seems like a compose issue
when tried on arch: amd64 fedora based linux distro with package manager installed docker and manually installing docker-compose 1.29.2 (using the official guide for linux) worked:
compose file:
version : "3"
services:
web:
image: "nginx:latest"
ports:
- "8000-8020:80"
docker command:
docker-compose up --scale web=5
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b304d397b2cd nginx:latest "/docker-entrypoint.…" 14 seconds ago Up 7 seconds 0.0.0.0:8004->80/tcp, :::8004->80/tcp testdir_web_4
a8c6f177a6e6 nginx:latest "/docker-entrypoint.…" 14 seconds ago Up 7 seconds 0.0.0.0:8003->80/tcp, :::8003->80/tcp testdir_web_3
b1abe53e7d7d nginx:latest "/docker-entrypoint.…" 14 seconds ago Up 8 seconds 0.0.0.0:8002->80/tcp, :::8002->80/tcp testdir_web_2
ead91e9df671 nginx:latest "/docker-entrypoint.…" 14 seconds ago Up 9 seconds 0.0.0.0:8001->80/tcp, :::8001->80/tcp testdir_web_5
65ffd6a87715 nginx:latest "/docker-entrypoint.…" 24 seconds ago Up 21 seconds 0.0.0.0:8000->80/tcp, :::8000->80/tcp testdir_web_1
Good morning all,
In the process of trying to train myself in Docker, I'm having trouble.
I created a docker container from a wordpress image, via docker compose.
[root#vps672971 ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
57bb123aa365 wordpress:latest "docker-entrypoint.s…" 16 hours ago Up 2 0.0.0.0:8001->80/tcp royal-by-jds-wordpress-container
I would like to allocate more memory to this container, however after the execution of the following command, the information returned by docker stats are not correct.
docker container update --memory 3GB --memory-swap 4GB royal-by-jds-wordpress-container
docker stats
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
57bb123aa365 royal-by-jds-wordpress-container 0.01% 9.895MiB / 1.896GiB 0.51% 2.68kB / 0B 0B / 0B 6
I also tried to request API engine to retrieve information about my container, but the limitation displayed is not correct either.
curl --unix-socket /var/run/docker.sock http:/v1.21/containers/royal-by-jds-wordpress-container/stats
[...]
"memory_stats":{
"usage":12943360,
"max_usage":12955648,
"stats":{},
"limit":2035564544
},
[...]
It seems that the modification of the memory allocated to the container didn't work.
Anyone have an idea?
Thank you in advance.
Maxence
I am trying to deploy the application on multiple instances. On master node. after deployed application running the only master node. cannot deploy service different node in the docker swarm cluster.
here my docker-compose file
version: "3"
services:
mydb:
image: localhost:5000/mydb-1
environment:
TZ: "Asia/Colombo"
ports:
- 9042:9042
volumes:
- /root/data/cdb:/var/lib/cassandra
- /root/logs/cdb:/var/log/cassandra
command docker service scale mydb-1_mydb=5
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7fxxxxxxxx7 localhost:5000/mydb-1:latest "docker-entrypoint.sh" 16 seconds ago Up 5 seconds 7000-7001/tcp, 7199/tcp, 9042/tcp, 9160/tcp mydb-1_mydb.2.q77i258vn2xynlgein9s7tdpb
34fcxxxx14bd localhost:5000/mydb-1:latest "docker-entrypoint.sh" 16 seconds ago Up 4 seconds 7000-7001/tcp, 7199/tcp, 9042/tcp, 9160/tcp mydb-1_mydb.1.s2mzitj8yzb0zo7spd3dmpo1j
9axxxx1efb localhost:5000/mydb-1:latest "docker-entrypoint.sh" 16 seconds ago Up 8 seconds 7000-7001/tcp, 7199/tcp, 9042/tcp, 9160/tcp mydb-1_mydb.3.zgyev3p4qdg7hf7h67oeedutr
f14xxxee59 localhost:5000/mydb-1:latest "docker-entrypoint.sh" 16 seconds ago Up 2 seconds 7000-7001/tcp, 7199/tcp, 9042/tcp, 9160/tcp mydb-1_mydb.4.r0themodonzzr1izdbnppd5bi
e3xxx16d localhost:5000/mydb-1:latest "docker-entrypoint.sh" 16 seconds ago Up 6 seconds 7000-7001/tcp, 7199/tcp, 9042/tcp, 9160/tcp mydb-1_mydb.5.bdebi4
all running only master-node. Does anyone know the issue?
Your image appears to be locally built with a name that cannot be resolved in other nodes (localhost:5000/mydb-1). In swarm, images should be pushed to a registry, and that registry needs to be accessible by all nodes. You can run your own registry service on your own node, there's a docker image, or you can push to docker hub. If the registry is private, you also need to perform a docker login on the node running the stack deploy and include registry credentials in that deploy, e.g.
docker stack deploy -c compose.yml --with-registry-auth stack-name
Thanks. I find the issue and fixed.
volumes:
- /root/data/cdb:/var/lib/cassandra
- /root/logs/cdb:/var/log/cassandra
If you bind mount a host path into your service’s containers, the path must exist on every swarm node.
docker service scale zkr_zkr=2
after scale-up service running my node
root#beta-node-1:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f9bxxx15861 localhost:5000/zookeeper:latest "/docker-entrypoint.…" 40 minutes ago Up 40 minutes 2181/tcp, 2888/tcp, 3888/tcp, 8080/tcp zkr_zkr.3.qpr8qp5y
01dxxxx64bc localhost:5000/zookeeper:latest "/docker-entrypoint.…" 40 minutes ago Up 40 minutes 2181/tcp, 2888/tcp, 3888/tcp, 8080/tcp zkr_zkr.1.g2uee5j
A Confluent Kafka instance is running via docker-compose on my Debian 9 server. I followed this tutorial to get it up and running. However, Control Center is shutting down periodically.
sudo docker-compose ps gives the following output:
control-center /etc/confluent/docker/run Exit 1
The rest of the Confluent services stay up and running.
When checking docker logs (sudo docker-compose logs) I can see that it is spamming the following error:
control-center | INFO [Consumer clientId=_confluent-controlcenter-5-3-0-1-9de26cca-62ca-42d6-9d46-86731fc8109a-StreamThread-5-restore-consumer, groupId=null] Unsubscribed all topics or patterns and assigned partitions (org.apache.kafka.clients.consumer.KafkaConsumer)
EDIT: discovered some more logs:
control-center | [2019-08-30 23:10:02,304] INFO [Consumer clientId=_confluent-controlcenter-5-3-0-1-39ae65e2-457c-4696-b592-504fe320038e-StreamThread-3-consumer, groupId=_confluent-controlcenter-5-3-0-1] Group coordinator broker:29092 (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery
control-center | [2019-08-30 22:38:39,291] INFO [Consumer clientId=_confluent-controlcenter-5-3-0-1-39ae65e2-457c-4696-b592-504fe320038e-StreamThread-8-consumer, groupId=_confluent-controlcenter-5-3-0-1] Attempt to heartbeat failed since group is rebalancing (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
EDIT 2: memory available to docker containers:
NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
ksql-datagen 0.00% 3.312MiB / 7.792GiB 0.04% 18.2kB / 2.11kB 92.8MB / 65.5kB 1
control-center 0.00% 0B / 0B 0.00% 0B / 0B 0B / 0B 0
ksql-cli 0.00% 484KiB / 7.792GiB 0.01% 19.6kB / 0B 41kB / 0B 1
ksql-server 0.36% 136MiB / 7.792GiB 1.70% 39.8MB / 34.5MB 210MB / 147kB 30
rest-proxy 0.12% 107.2MiB / 7.792GiB 1.34% 22.2kB / 2.41kB 72.6MB / 81.9kB 28
connect 0.60% 1.571GiB / 7.792GiB 20.16% 124MB / 110MB 1.04GB / 81.9kB 36
schema-registry 0.20% 176.8MiB / 7.792GiB 2.22% 40.2MB / 38.4MB 93.7MB / 156kB 32
broker 7.59% 621MiB / 7.792GiB 7.78% 573MB / 791MB 171MB / 335MB 73
zookeeper 0.10% 80.9MiB / 7.792GiB 1.01% 9.56MB / 8.99MB 38.4MB / 410kB 21
System memory (command: free):
total used free shared buff/cache available
Mem: 8366596096 6770286592 160227328 219533312 1436082176 1099268096
Swap: 34356588544 2301014016 32055574528
Any ideas how to fix this?
This error comes up when the memory allocated is very less.
If you are using Docker Desktop then increase the memory . Go to Docker Desktop->Dashboard->Settings->Preferences->Resources->Advanced and increase the memory and you will be all set.
I have docker in version 18.03.1-ce that support command docker system df. His output:
Images space usage:
REPOSITORY TAG IMAGE ID CREATED ago SIZE SHARED SIZE UNIQUE SiZE CONTAINERS
registry.gitlab.com/precisesale/app latest b7833546c2cf About an hour ago ago 252.1MB 123.8MB 128.4MB 1
healthdiary/app latest 565c6d3906e6 2 days ago ago 312.2MB 123.8MB 188.4MB 1
mongo latest f93ff881751f 5 days ago ago 367.6MB 0B 367.6MB 2
nginx latest b175e7467d66 6 weeks ago ago 108.9MB 0B 108.9MB 1
jwilder/docker-gen latest 8959ee34c769 2 months ago ago 19.91MB 4.148MB 15.77MB 1
jrcs/letsencrypt-nginx-proxy-companion latest 17939ceb7a52 2 months ago ago 86.86MB 4.148MB 82.71MB 1
Containers space usage:
CONTAINER ID IMAGE COMMAND LOCAL VOLUMES SIZE CREATED ago STATUS NAMES
c20dc3438552 healthdiary/app "./entrypoint.sh nod…" 0 0B 8 minutes ago ago Up 8 minutes healthdiary_app_1
bf8c4307dcbb mongo:latest "docker-entrypoint.s…" 1 0B 8 minutes ago ago Up 8 minutes healthdiary_mongo_1
47fced8d18fe registry.gitlab.com/precisesale/app "./entrypoint.sh nod…" 0 0B 9 minutes ago ago Up 9 minutes precisesale_app_1
597d97d5c1fa mongo:latest "docker-entrypoint.s…" 1 0B 9 minutes ago ago Up 9 minutes precisesale_db_1
b5bb14faa910 jwilder/docker-gen "/usr/local/bin/dock…" 0 0B 7 hours ago ago Up 19 minutes nginx-gen
8eee2bee084a nginx "nginx -g 'daemon of…" 0 2B 7 hours ago ago Up 19 minutes nginx-web
6b8b0cd5d938 jrcs/letsencrypt-nginx-proxy-companion "/bin/bash /app/entr…" 0 1.66kB 7 hours ago ago Up 19 minutes nginx-letsencrypt
Local Volumes space usage:
VOLUME NAME LINKS SIZE
0a40fac6ca98e776dad972c8193362a51a485b3305979e58996545d97310a3c7 1 0B
929b0b88849ad4d390efd4666e6a0e5f82e0e6dd34f7a09f609de90b190e6148 1 0B
Build cache usage: 0B
Even if I do not take into account savings from shared space from two first containers summary size is 1147.5 MB
But if I measure size of docker overlay2 on disc by du I get
du -hs /var/lib/docker/overlay2/
2.7G /var/lib/docker/overlay2/
Where is reason of difference in size of containers measured by docker system df and du?
I was wondering the same thing some time ago.
It’s not a bug, it’s a feature :-)
du -sh /var/lib/docker/overlay2
is not showing objective value because merge folders have been mounted using overlay driver and du output is not actual disk allocation size.
You can see the actual disk allocation size by examining only diff folders like:
du -shc /var/lib/docker/overlay2/*/diff
You can test this in your environment like this:
run
df -h /dev/sd*
du -shc /var/lib/docker/overlay2/*/diff
du -sh /var/lib/docker/overlay2
Now start 20 centos containers and observe what has change:
for i in {1..20}; do docker run -itd centos bash; done
df -h /dev/sd*
du -shc /var/lib/docker/overlay2/*/diff
du -sh /var/lib/docker/overlay2
You can see that the actual disk allocation (df command) is just cca 200MB more than before, but “du” on whole folder outputs 4.2G allocation.
“du” on “diff” folders shows 212M what is correct.
This is how Docker works and what makes it great!