I am able to run containers fine with this combination.
But I noticed - there is no /etc/docker directory on the linux side and when I do ps -eF I get this. I was expecting dockerd and container processes as children of dockerd
rookie#MAIBENBEN-PC:/mnt/c/Users/rookie$ ps -eF
UID PID PPID C SZ RSS PSR STIME TTY TIME CMD
root 1 0 0 223 580 6 04:07 ? 00:00:00 /init
root 98 1 0 223 80 5 04:07 ? 00:00:00 /init
root 99 98 0 223 80 5 04:07 ? 00:00:00 /init
rookie 100 99 0 191067 43220 0 04:07 pts/0 00:00:00 docker serve --address unix:///home/rookie/.docker/run/d
root 101 98 0 0 0 1 04:07 ? 00:00:00 [init] <defunct>
root 103 98 0 223 80 7 04:07 ? 00:00:00 /init
root 104 103 0 384463 28888 0 04:07 pts/1 00:00:00 /mnt/wsl/docker-desktop/docker-desktop-proxy --distro-na
root 142 1 0 223 80 4 05:17 ? 00:00:00 /init
root 143 142 0 223 80 6 05:17 ? 00:00:00 /init
rookie 144 143 0 2509 5048 2 05:17 pts/2 00:00:00 -bash
rookie 221 144 0 2654 3264 7 05:21 pts/2 00:00:00 ps -eF
Your Ubuntu session (and all WSL2 sessions) are set up as docker clients, but the actual docker daemon is running in a separate WSL session named "docker-desktop".
I generally recommend leaving this instance alone, as it is auto-configured and managed by Docker Desktop, but if you really want to take a look, run:
wsl -d docker-desktop
... from PowerShell, CMD, or Windows Start/Run.
Note that this instance is running BusyBox, so some commands will be different than you expect. For instance, the -F argument is not valid for ps.
You'll see dockerd and the associated containerd processes here.
There's also a separate image, docker-desktop-data, but it is not bootable (there is no init in it). If you want to see the filesystem, at least, you can wsl --export it and examine the tar file that is created. I wrote up an answer on Super User with details a few months ago.
Related
I am using composer to run some system workers on the docker container, which is normally started with the www-data user on remote servers.
When I run them on the docker container they are started by the root user which is not correct, because the www-data user can not stop them from the browser app.
composer.json
...
"require": {
...
},
"scripts": {
"worker:start": [
"php path/to/the/script"
],
},
...
Start the script on the docker container
composer worker:start
And top results
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 267152 36396 29584 S 0.0 0.2 0:00.12 php-fpm
91 root 20 0 19132 4216 2208 S 0.0 0.0 0:00.04 sendmail-mta
103 www-data 20 0 267152 8952 2136 S 0.0 0.1 0:00.00 php-fpm
104 www-data 20 0 267152 8952 2136 S 0.0 0.1 0:00.00 php-fpm
154 root 20 0 2528 580 488 S 0.0 0.0 0:00.00 timeout
156 root 20 0 124460 56344 27900 S 0.0 0.4 0:00.18 php
157 root 20 0 2528 576 484 S 0.0 0.0 0:00.00 timeout
159 root 20 0 124460 55484 28224 S 0.0 0.3 0:00.19 php
160 root 20 0 2528 584 488 S 0.0 0.0 0:00.00 timeout
161 root 20 0 129012 61356 28020 S 0.0 0.4 0:00.27 php
162 root 20 0 4100 3428 2920 S 0.0 0.0 0:00.02 bash
168 root 20 0 7016 3360 2820 T 0.0 0.0 0:00.02 top
189 root 20 0 2528 576 484 S 0.0 0.0 0:00.00 timeout
191 root 20 0 124460 54948 27436 S 0.0 0.3 0:00.17 php
192 root 20 0 2528 576 484 S 0.0 0.0 0:00.00 timeout
194 root 20 0 122280 54548 28080 S 0.0 0.3 0:00.15 php
195 root 20 0 2528 640 548 S 0.0 0.0 0:00.00 timeout
196 root 20 0 128968 60336 27972 S 0.0 0.4 0:00.23 php
197 root 20 0 7016 3352 2812 R 0.0 0.0 0:00.00 top
As you see, only php-fpm proccess is run with www-data user.
How to configure docker container to run all PHP processes as www-data user instead root?
The reason FPM is running with that user is because it's written in the FPM config file. So it doesn't run as the root user, but as the user in the config file.
For example, somewhere in one of your FPM config files are settings simular to the below:
[www]
user = www-data
group = www-data
Composer doesn't seem to do this. At least not by default or with its current configuration.
I suggest generally switching the user in the docker container, for security purposes. Put this at the end of your Dockerfile.
USER www-data
This is good security practice and should also fix your problem.
EDIT: First post, I'm trying to get some formatting...
I want to mount a host directory into a container directory so I can get container-created files back into the host. I've investigated at least a dozen examples with no luck. As near as I can tell, the following should work.
C:\tmp>ls -al jmeter
total 0
drwxrwxrwx 1 0 0 0 May 22 19:25 .
drwxrwxrwx 1 0 0 0 May 22 19:36 ..
C:\tmp>docker run -v /tmp/jmeter:/tmp/jmeter -it ubuntu bash
root#62a046b1dd74:/# ls -al /tmp/jmeter
total 4
drwxr-xr-x 2 root root 40 May 23 02:00 .
drwxrwxrwt 1 root root 4096 May 23 02:00 ..
root#62a046b1dd74:/# touch /tmp/jmeter/bob.txt
root#62a046b1dd74:/# ls -al /tmp/jmeter
total 4
drwxr-xr-x 2 root root 60 May 23 02:01 .
drwxrwxrwt 1 root root 4096 May 23 02:00 ..
-rw-r--r-- 1 root root 0 May 23 02:01 bob.txt
root#62a046b1dd74:/# exit
exit
C:\tmp>ls -al jmeter</b>
total 0
drwxrwxrwx 1 0 0 0 May 22 19:25 .
drwxrwxrwx 1 0 0 0 May 22 19:36 ..
C:\tmp>
My expectation is that /tmp/jmeter/bob.txt would exist on localhost.
FWIW, localhost is Windows 10 here, but I have the same problem in a github action, which I believe is Linux.
I have launched several docker containers and using docker stats, I have verified that one of them increases the consumption of ram memory since it starts until it is restarted.
My question is if there is any way to verify where such consumption comes from within the docker container. There is some way to check the consumption inside the container, something of the docker stats style but for the inside of the container.
Thanks for your cooperation.
Not sure if it's what you are asking for, but here's an example:
(Before your start):
Run a test container docker run --rm -it ubuntu
Install stress by typing apt-get update and apt-get install stress
Run stress --vm-bytes $(awk '/MemAvailable/{printf "%d\n", $2 * 0.9;}' < /proc/meminfo)k --vm-keep -m 1 (it will start consuming memory)
1. with top
If you go to a new terminal you can type docker container exec -it <your container name> top and you will get something like the following:
(notice that the %MEM usage of PID 285 is 68.8%)
docker container exec -it dreamy_jang top
top - 12:46:04 up 22 min, 0 users, load average: 1.48, 1.55, 1.12
Tasks: 4 total, 2 running, 2 sleeping, 0 stopped, 0 zombie
%Cpu(s): 20.8 us, 0.8 sy, 0.0 ni, 78.4 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 6102828 total, 150212 free, 5396604 used, 556012 buff/cache
KiB Swap: 1942896 total, 1937508 free, 5388 used. 455368 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
285 root 20 0 4209376 4.007g 212 R 100.0 68.8 6:56.90 stress
1 root 20 0 18500 3148 2916 S 0.0 0.1 0:00.09 bash
274 root 20 0 36596 3072 2640 R 0.0 0.1 0:00.21 top
284 root 20 0 8240 1192 1116 S 0.0 0.0 0:00.00 stress
2. with ps aux
Again, from a new terminal you type docker container exec -it <your container name> ps aux
(notice that the %MEM usage of PID 285 is 68.8%)
docker container exec -it dreamy_jang ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 18500 3148 pts/0 Ss 12:25 0:00 /bin/bash
root 284 0.0 0.0 8240 1192 pts/0 S+ 12:39 0:00 stress --vm-byt
root 285 99.8 68.8 4209376 4201300 pts/0 R+ 12:39 8:53 stress --vm-byt
root 286 0.0 0.0 34400 2904 pts/1 Rs+ 12:48 0:00 ps aux
My source for this stress thing is from this question: How to fill 90% of the free memory?
I have docker-compose.yml like this:
version: '3'
services:
mysql:
image: mysql
volumes:
- data:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=$ROOT_PASSWORD
volumes:
data:
And my mount point looks like: /var/lib/docker/volumes/some_app/_data and I want to access data from that mount point and I'm not sure how to do it on Windows machine. Maybe I can create some additional container which can pass data from docker virtual machine to my directory?
When I'm mounting folder like this:
volumes:
- ./data:/var/lib/mysql
to use my local directory - I had no success because of permissions issue. And read that "right way" is using docker volumes.
UPD: MySQL container it's just example. I want to use such behaviour for my codebase and use docker foe local development.
For Linux containers under Windows, docker runs actually over a Linux virtual machine, so your named volume is a mapping of a local directory in that VM to a directory in the container.
So what you got as /var/lib/docker/volumes/some_app/_data is a directory inside that VM. To inspect it you can:
docker run --rm -it -v /:/vm-root alpine:edge ls -l /vm-root/var/lib/docker/volumes/some_app/_data
total 188476
-rw-r----- 1 999 ping 56 Jun 4 04:49 auto.cnf
-rw------- 1 999 ping 1675 Jun 4 04:49 ca-key.pem
-rw-r--r-- 1 999 ping 1074 Jun 4 04:49 ca.pem
-rw-r--r-- 1 999 ping 1078 Jun 4 04:49 client-cert.pem
-rw------- 1 999 ping 1679 Jun 4 04:49 client-key.pem
-rw-r----- 1 999 ping 1321 Jun 4 04:50 ib_buffer_pool
-rw-r----- 1 999 ping 50331648 Jun 4 04:50 ib_logfile0
-rw-r----- 1 999 ping 50331648 Jun 4 04:49 ib_logfile1
-rw-r----- 1 999 ping 79691776 Jun 4 04:50 ibdata1
-rw-r----- 1 999 ping 12582912 Jun 4 04:50 ibtmp1
drwxr-x--- 2 999 ping 4096 Jun 4 04:49 mysql
drwxr-x--- 2 999 ping 4096 Jun 4 04:49 performance_schema
-rw------- 1 999 ping 1679 Jun 4 04:49 private_key.pem
-rw-r--r-- 1 999 ping 451 Jun 4 04:49 public_key.pem
-rw-r--r-- 1 999 ping 1078 Jun 4 04:49 server-cert.pem
-rw------- 1 999 ping 1675 Jun 4 04:49 server-key.pem
drwxr-x--- 2 999 ping 12288 Jun 4 04:49 sys
That is running an auxiliar container which has mounted the hole root filesystem of that VM / into the container dir /vm-root.
To get some file run the container with some command in background (tail -f /dev/null in my case), then you can use docker cp:
docker run --name volume-holder -d -it -v /:/vm-root alpine:edge tail -f /dev/null
docker cp volume-holder:/vm-root/var/lib/docker/volumes/volumes_data/_data/public_key.pem .
If you want a transparent SSH to that VM, it seems that is not supported yet, as of Jun-2017. Here a docker staff member said that.
I want to monitor docker containers log from non-root user(td-agent) and on host server,
sudo chmod o+rx /var/lib/docker
sudo find /var/lib/docker/containers/ -type d -exec chmod o+rx {} \;
sudo find /var/lib/docker/containers/ -type f -exec chmod o+r {} \;
But containers directory rollback 600 and each container directory keep 600.
# find /var/lib/docker/containers -ls
143142 4 drwx------ 4 root root 4096 Aug 14 12:01 /var/lib/docker/containers
146027 4 drwx------ 2 root root 4096 Aug 14 12:00 /var/lib/docker/containers/145efa73652aad14e1706e8fcd1597ccbbb49fd756047f3931270b46fe01945d
146031 4 -rw-r--r-- 1 root root 190 Aug 14 12:00 /var/lib/docker/containers/145efa73652aad14e1706e8fcd1597ccbbb49fd756047f3931270b46fe01945d/hostconfig.json
146046 4 -rw-r--r-- 1 root root 13 Aug 14 12:00 /var/lib/docker/containers/145efa73652aad14e1706e8fcd1597ccbbb49fd756047f3931270b46fe01945d/hostname
146047 4 -rw-r--r-- 1 root root 174 Aug 14 12:00 /var/lib/docker/containers/145efa73652aad14e1706e8fcd1597ccbbb49fd756047f3931270b46fe01945d/hosts
146030 4 -rw-r--r-- 1 root root 3305 Aug 14 12:00 /var/lib/docker/containers/145efa73652aad14e1706e8fcd1597ccbbb49fd756047f3931270b46fe01945d/config.json
146049 4 -rw------- 1 root root 1853 Aug 14 12:00 /var/lib/docker/containers/145efa73652aad14e1706e8fcd1597ccbbb49fd756047f3931270b46fe01945d/145efa73652aad14e1706e8fcd1597ccbbb49fd756047f3931270b46fe01945d-json.log
146050 4 drwx------ 2 root root 4096 Aug 14 12:01 /var/lib/docker/containers/f09796f978ef5bab1449d2d10d400228eb76376579e7e33c615313eeed53f370
146054 4 -rw-r--r-- 1 root root 190 Aug 14 12:01 /var/lib/docker/containers/f09796f978ef5bab1449d2d10d400228eb76376579e7e33c615313eeed53f370/hostconfig.json
146056 4 -rw-r--r-- 1 root root 13 Aug 14 12:01 /var/lib/docker/containers/f09796f978ef5bab1449d2d10d400228eb76376579e7e33c615313eeed53f370/hostname
146057 4 -rw-r--r-- 1 root root 174 Aug 14 12:01 /var/lib/docker/containers/f09796f978ef5bab1449d2d10d400228eb76376579e7e33c615313eeed53f370/hosts
146053 4 -rw-r--r-- 1 root root 3286 Aug 14 12:01 /var/lib/docker/containers/f09796f978ef5bab1449d2d10d400228eb76376579e7e33c615313eeed53f370/config.json
146058 4 -rw------- 1 root root 1843 Aug 14 12:01 /var/lib/docker/containers/f09796f978ef5bab1449d2d10d400228eb76376579e7e33c615313eeed53f370/f09796f978ef5bab1449d2d10d400228eb76376579e7e33c615313eeed53f370-json.log
How to monitor this each json.log? or any other good monitoring way?
logspout is another way to collect containerslogs. I'm not sure this is the best solution, but it is very interesting and consistent way to collect containers logs.
You just need to run logspout container. This container has a feature that send docker containers' logs to other syslog server. (or you can use HTTP api also. see repository)
# (172.17.42.1 is host ip address)
$ docker run -v=/var/run/docker.sock:/tmp/docker.sock progrium/logspout syslog://172.17.42.1:5140
And fluentd that is running on host can handle these logs through syslog protocal. Below is td-agent.conf example. It receive logs from syslog protocal and send them to elasticsearch server. (check this example project)
<source>
type syslog
port 5140
bind 0.0.0.0
tag syslog.udp
format /^(?<time>.*?) (?<container_id>.*?) (?<container_name>.*?): (?<message>.*?)$/
time_format %Y-%m-%dT%H:%M:%S%z
</source>
<match syslog.**>
index_name <ES_INDEX_NAME>
type_name <ES_TYPE_NAME>
type elasticsearch
host <ES_HOST>
port <ES_PORT>
flush_interval 3s
</match>
As I discussed in detail in this answer that the OP never acknowledged whatsoever, I find the best approach is to configure the applications running within the container to log messages to syslog, and mount the host's syslog socket to the container.
docker run -v /dev/log:/dev/log ...
Downside of this approach is that if the syslog daemon on the host is restarted, the container will lose it's socket since the daemon recreates the socket at restart.
A fix for this would be to add another socket (in rsyslog this can be done using the imuxsock module). Create the additional socket in some known directory, then bind mount the directory instead of /dev/log directly. The additional socket will also be removed when rsyslog restarts, but will be recreated and available to the application in the directory following the restart.
One easy way to deal with this issue is to mount host's /sys/fs/cgroup into a Docker container that's running in_docker_metrics. See https://github.com/bdehamer/docker-librato
Sematext Docker Agent (open-source, github) can do this for you. You won't need td-agent. SDA will collect logs, but also events and metrics. See https://github.com/sematext/sematext-agent-docker and
https://sematext.com/docker