I want to deploy 100 containers to one of my linux pc.
If not use docker, the memory is ok for that 100 services.
But if use docker, I see next:
25504 0.0 0.0 10796 5060 ? Sl 12:41 0:00 containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/779ab7bff4bed93d2912437e58e9b830c2f72ef2d8f3b4a5ba31e6d0db8e72b2 -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
That means for one container, containerd-shim will cost 5M memory, the worse thing is: for every new container there will be one containerd-shim process. This means for 100 containers, there will be additional 5M * 100 = 500M memory consume.
Is above expected? Any chance for me to try to reduce the 500M?
steps to use runc
-Extract json from Docker image:
mkdir my_container
cd my_container
mkdir rootfs
docker export $(docker create busybox) | tar -C rootfs -xvf -
docker-runc spec
-start the container
runc run container-name
Source
PS : you may also try to add the run time to Docker daemon:
docker daemon --add-runtime "oci=/usr/local/sbin/runc"
Related
I have debian 10.5 host with docker running in rootless mode (followed this guide : https://docs.docker.com/engine/security/rootless/)
When there is a power cut (I don't have a UPS), my debian 10.5 VM starts automatically when power is restored.
Everything works fine except my docker daemon. There's no error with the service:
systemctl --user status docker
● docker.service - Docker Application Container Engine (Rootless)
Loaded: loaded (/home/dockerprod/.config/systemd/user/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2022-01-08 12:04:40 +04; 4min 43s ago
Docs: https://docs.docker.com
Main PID: 770 (rootlesskit)
CGroup: /user.slice/user-1001.slice/user#1001.service/docker.service
├─770 rootlesskit --net=vpnkit --mtu=1500 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run /
├─805 /proc/self/exe --net=vpnkit --mtu=1500 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/ru
├─816 vpnkit --ethernet /tmp/rootlesskit308973386/vpnkit-ethernet.sock --mtu 1500 --host-ip 0.0.0.0
├─896 dockerd --experimental --storage-driver=vfs
└─936 containerd --config /run/user/1001/docker/containerd/containerd.toml --log-level info
But the containers did not start for some reason.
I am not sure what logs to look at.
sudo journalctl -u docker.service
returns nothing
If I restart the host, the containers start as normal. So I always need to restart the host after a power cut which is not ideal when I am not at home.
Any idea what to look at?
Maybe a clue; my docker lib folder (where containers and images are stored) is on another HDD mounted automatically in /etc/fstab
Maybe after a power cut upon reboot, docker daemon is started before the HDD on which the docker lib folder is mounted? Does not know if this makes sense.
edit:
I moved the mounting command in /etc/fstab of HDD on which the docker lib is on to the top.
Does not solve the issue.
Another note, /lib/docker/containers/ is empty after the power cut
If I restart the host, /lib/docker/containers/ contains the containers again..
I have run free -h and see that I have 29G of swap space.
total used free shared buff/cache available
Mem: 15G 6.9G 8.8G 17M 223M 8.9G
Swap: 29G 2.0M 29G
I have also enabled 100 swappiness.
$ sudo sysctl vm.swappiness=100
vm.swappiness = 100
$ cat /proc/sys/vm/swappiness
100
However, docker build --memory-swap=20g does not appear to use the swap space. This is the output of htop throughout the docker build.
1 [|||||||||||||||| 18.7%]
2 [||||||| 7.3%]
3 [|||||||||||||||||||||| 26.5%]
4 [||||||||||||||| 18.0%]
Mem[||||||||||||||||||||||||||||||||||| 6.47G/15.9G]
Swp[| 2.00M/29.6G]
This is the docker build command:
docker build --build-arg NODE_OPTIONS="--max-old-space-size=325" \
--memory=600m --memory-swap=20g \
--cpu-period=100000 --cpu-quota=50000 \
--no-cache --tag farm_app_image:latest --file Dockerfile .
The docker build appears to be running out of RAM, because the build's internal process (NodeJS) runs out of heap space and crashes. Also, immediately before the crash the memory is maxed:
shaun#DESKTOP-5T629JB:/mnt/c/Users/bigfo$ docker ps -q | xargs docker stats --no-stream
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
66bdf8efb492 charming_maxwell 51.72% 562.2MiB / 600MiB 93.70% 46.8MB / 1.53MB 277MB / 230MB 94
Why is it running out of RAM without using the swap space? How can we make it use the available swap space?
May be you should try to run it with --privileged flag.
docker run -ti --privileged yourimage
But make sure that you know what you are doing.
You should also read docker-tips-privilaged-flag
1) I use next to start a container:
docker run --name test -idt python:3 python -m http.server
2) Then, I try to validate memory usage like next:
a)
root#shubuntu1:~# ps aux | grep "python -m http.server"
root 17416 3.0 0.2 27368 19876 pts/0 Ss+ 17:11 0:00 python -m http.server
b)
root#shubuntu1:~# docker exec -it test ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.9 0.2 27368 19876 pts/0 Ss+ 09:11 0:00 python -m http.
c)
root#shubuntu1:~# docker stats --no-stream test
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
d72f2ece6816 test 0.01% 12.45MiB / 7.591GiB 0.16% 3.04kB / 0B 0B / 0B 1
You can see from both docker host & docker container, we could see python -m http.server consume 19876/1024=19.1289MB memory (RSS), but from docker stats, I find 12.45MB, why it show container memory consume even less than the PID1 process in container?
rss RSS resident set size, the non-swapped physical memory that a task has used (in kiloBytes). (alias rssize, rsz).
MEM USAGE / LIMIT the total memory the container is using, and the total amount of memory it is allowed to use
Fisrt, docker run -d busybox sleep 2000
Seconde, ps aux | grep {container id} to find the pid;
Third, pstree -paAl {pid} shows the nine strange docker-containe;
What does these containes mean? 5214-5222
Fedora 29, docker.
I don't know how to format the output, but you can try it on your pc.I
only know the 5213 docker-containe is my container...
[emacsliu#localhost ~]$ pstree -aAlp 5213
docker-containe,5213
a82c3cf17ed83570758a80ff84fc9f6ff9e649b3407f4f0bc371f2e3aa5f351e /var/run/docker/libcontainerd/a82c3cf17ed83570758a80ff84fc9f6ff9e649b340
7f4f0bc371f2e3aa5f351e /usr/libexec/docker/docker-runc-current
|-sleep,5232 2000
|-{docker-containe},5214
|-{docker-containe},5215
|-{docker-containe},5216
|-{docker-containe},5217
|-{docker-containe},5218
|-{docker-containe},5219
|-{docker-containe},5220
|-{docker-containe},5221
-{docker-containe},5222
docker-containe is actually abbreviated docker-containerd-shim (or containerd-shim on some systems) and it is Docker component. Use other tools, which will show a full process name. For example htop tree view:
Ref:
http://alexander.holbreich.org/docker-components-explained/
dockerd vs docker-containerd vs docker-runc vs docker-containerd-ctr vs docker-containerd-shim
I would like to run a docker container that requires a lot of memory on a machine that doesn't have much RAM. I have been trying to increase the swap space available for the container to no avail. Here is the last command I tried:
docker run -d -m 1000M --memory-swap=10000M --name=my_container my_image
Following these tips on how to check memory metrics I found the following:
$ boot2docker ssh
docker#boot2docker:~$ cat /sys/fs/cgroup/memory/docker/35af5a072751c7af80ce7a255a01ab3c14b3ee0e3f15341f7bb22a777091c67b/memory.stat
cache 454656
rss 65015808
rss_huge 29360128
mapped_file 208896
writeback 0
swap 0
pgpgin 31532
pgpgout 22702
pgfault 49372
pgmajfault 0
inactive_anon 28672
active_anon 65183744
inactive_file 241664
active_file 16384
unevictable 0
hierarchical_memory_limit 1048576000
hierarchical_memsw_limit 10485760000
total_cache 454656
total_rss 65015808
total_rss_huge 29360128
total_mapped_file 208896
total_writeback 0
total_swap 0
total_pgpgin 31532
total_pgpgout 22702
total_pgfault 49372
total_pgmajfault 0
total_inactive_anon 28672
total_active_anon 65183744
total_inactive_file 241664
total_active_file 16384
total_unevictable 0
Is it possible to run a container that requires 5G of memory on a machine that only has 4G of physical memory?
This GitHub issue was very helpful in figuring out how to increase the swap space available in the boot2docker-vm. Adapting it to my situation I used the following commands to ssh into the boot2docker-vm and set up a new swapfile:
boot2docker ssh
export SWAPFILE=/mnt/sda1/swapfile
sudo dd if=/dev/zero of=$SWAPFILE bs=1024 count=4194304
sudo mkswap $SWAPFILE
sudo chmod 600 $SWAPFILE
sudo swapon $SWAPFILE
exit