I am running nginxinc/nginx-unprivileged:stable-alpine docker image on RHEL 8.8 server. when docker container starts its creating directory and file with umask 0027.
But my docker 20.10.17 daemon running with Umask of 0022. my server default umask is 0027 this I can't change due to security requirements.
# systemd-analyze dump |egrep -i 'docker|umask'
ReferencedBy: docker.service (destination-file)
UMask: 0022
Here is inside container file system permission on RHEL 8 server.
# ls -l
total 76
drwxr-x--- 1 root root 4096 Jun 16 21:57 app
drwxr-x--- 1 root root 4096 Jun 16 21:57 bin
drwxr-x--- 5 root root 360 Jun 17 20:18 dev
drwxr-x--- 1 root root 4096 Jun 16 21:57 docker-entrypoint.d
-rwxr-x--- 1 root root 1202 Jun 16 21:57 docker-entrypoint.sh
drwxr-x--- 1 root root 4096 Jun 17 20:18 etc
drwxr-x--- 2 root root 4096 Jun 16 21:57 home
drwxrwxrwt 1 root root 4096 Jun 16 21:57 tmp
drwxr-x--- 1 root root 4096 Jun 16 21:57 usr
drwxr-x--- 1 root root 4096 Jun 16 21:57 var
Here is inside container file system permission on windows machine with same docker iamge.
ls -l
drwxr-xr-x 2 root root 4096 May 23 16:51 bin
drwxr-xr-x 5 root root 360 Jun 17 18:39 dev
drwxr-xr-x 1 root root 4096 Jun 16 10:36 docker-entrypoint.d
-rwxr-xr-x 1 root root 1202 Jun 16 10:36 docker-entrypoint.sh
drwxr-xr-x 1 root root 4096 Jun 17 18:39 etc
drwxr-xr-x 2 root root 4096 May 23 16:51 home
drwxr-xr-x 1 root root 4096 May 23 16:51 usr
drwxr-xr-x 1 root root 4096 May 23 16:51 var
How can I make docker container file system created with umask of 0022?
Thanks
when docker container starts
That means you need to build your own image, based on nginxinc/nginx-unprivileged:stable-alpine, with a new entry point like:
#!/bin/sh
# entrypoint.sh
umask 022
# ... other first-time setup ...
exec "$#"
See "Change umask in docker containers" for more details, but the idea remains the same.
I run a Fedora distribution. My /home is an encrypted volume.
When mounting files or directories as volume with docker using the -v option or through the volumes directory in a docker-compose.yml file I get very weird permissions on the file :
$ touch ~/test
$ docker run -v /home/jkr/test:/test -it --rm nginx bash
$ root#65fcd1754a1d:/# ls -la /
ls: cannot access '/test': Permission denied
total 84
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 .
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 ..
-rwxr-xr-x. 1 root root 0 Aug 30 20:29 .dockerenv
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 bin
drwxr-xr-x. 2 root root 4096 Jun 13 10:30 boot
drwxr-xr-x. 5 root root 360 Aug 30 20:29 dev
drwxr-xr-x. 1 root root 4096 Aug 17 11:46 docker-entrypoint.d
-rwxrwxr-x. 1 root root 1202 Aug 17 11:45 docker-entrypoint.sh
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 etc
drwxr-xr-x. 2 root root 4096 Jun 13 10:30 home
drwxr-xr-x. 1 root root 4096 Aug 17 11:46 lib
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 lib64
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 media
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 mnt
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 opt
dr-xr-xr-x. 490 root root 0 Aug 30 20:29 proc
drwx------. 2 root root 4096 Aug 16 00:00 root
drwxr-xr-x. 3 root root 4096 Aug 16 00:00 run
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 sbin
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 srv
dr-xr-xr-x. 13 root root 0 Aug 30 20:16 sys
-?????????? ? ? ? ? ? test
drwxrwxrwt. 1 root root 4096 Aug 17 11:46 tmp
drwxr-xr-x. 1 root root 4096 Aug 16 00:00 usr
drwxr-xr-x. 1 root root 4096 Aug 16 00:00 var
Notice the permission of the /test file inside the docker
-?????????? ? ? ? ? ? test
If I create this test file on a non encrypted llvm volume e.g. in /opt/ or /tmp then the permission is correct.
Is there any way to mount docker volume on an encrypted partition ?
- means its regular file , when this file is created then which user are use this permission maybe this user is deleted but file is exists thats why ?? is showing
I'm getting status code 500 on my Dockerized Flask server.
I bashed into the container to check the logs:
docker exec -ti container_name /bin/bash
in /var/log I found:
root#b80b0c02fd18:/var/log# ls -al
total 224
drwxr-xr-x 1 root root 4096 Oct 13 21:02 .
drwxr-xr-x 1 root root 4096 Oct 12 07:00 ..
-rw-r--r-- 1 root root 9052 Oct 20 20:50 alternatives.log
drwxr-xr-x 1 root root 4096 Oct 20 20:49 apt
-rw-rw---- 1 root utmp 0 Oct 12 07:00 btmp
-rw-r--r-- 1 root root 164661 Oct 20 20:50 dpkg.log
-rw-r--r-- 1 root root 3232 Oct 12 07:00 faillog
-rw-rw-r-- 1 root utmp 29492 Oct 12 07:00 lastlog
-rw-rw-r-- 1 root utmp 0 Oct 12 07:00 wtmp
I couldn't cat or nano the files faillog nor lastlog so I don't know if the files
are relevant.
Where do I find access log or error log for containerized Flask server ?
docker logs <your container_name> to see the logs of the container
You can also find the logs for the flask app at /var/log/daemon.log
When I add a remote interpreter from one of my docker-compose, it doesn't seem to succeed and doesn't show any packages in the dialog. When I add an interpreter to the debugger it says:
python packaging tools not found.
Then if i click on install packaging tools, error displayed:
ERROR: for dockeryard_pycharm_helpers_1
Cannot start service pycharm_helpers: network not found
Starting dockeryard_postgres_1 ...
Starting dockeryard_nginx_1 ...
Starting dockeryard_redis_1 ...
Starting dockeryard_postgres_1 ...
Starting dockeryard_nginx_1 ...
Starting dockeryard_pycharm_helpers_1
Starting dockeryard_redis_1
Starting dockeryard_worker_1 ...
Starting dockeryard_worker_1
Starting dockeryard_pycharm_helpers_1
ERROR: for dockeryard_pycharm_helpers_1 Cannot start service pycharm_helpers: network not found
ERROR: for pycharm_helpers Cannot start service pycharm_helpers: network not found
[31m
ERROR [0m:
Note, this interpreter was already in use and I was able to connect remotely with PyCharm, but I have added and eventually removed a custom network to the container.
As explained in Configuring Remote Python Interpreters - "When a remote Python interpreter is added, at first the PyCharm helpers are copied to the remote host". And my guess something went wrong since network was updated in the docker-compose.
From what I understand from the error message, when PyCharm starts interpreter it tries to use/find that network c7b0cc277c94ba5f58f6e72dcbab1ba24794e72422e839a83ea6102d08c40452.
I don't see that network listed anywhere when I run:
$ docker network inspect dockeryard_default
So PyCharm stores it somewhere and not been updated with the change.
I have tried to remove interpreter (using PyCharm dialog) and add it back - same result.
How can I get rid of this network and make PyCharm to be able to debug again?
Thanks.
Was having a near identical error and was able to get past it. I did two things though I'm uncertain as to which was the actual solution:
Made sure the mappings were correct under both (a) Preferences -> Project -> Project Interpreter -> Path mappings and (b) Run -> Edit Configurations -> <Your_Configuration> -> Path mappings
Removed/deleted any containers that looked to be related to PyCharm (believe this is more than likely what solved things).
Hope this helps. PyCharm docker-compose seems to work for some and be a real PITA for others.
One other note. I downgraded from PyCharm 2018 to 2017.3 as there's known docker bugs in 2018.
EDIT: And it would seem a docker-compose down from CLI reintroduces the error -_-
TLDR:
The {project_name}_pycharm_helpers_{pycharm_build_number} volume has been removed or is corrupted.
To repopulate it, run:
docker volume rm {project_name}_pycharm_helpers_{pycharm_build_number}
docker run -v {project_name}_pycharm_helpers_{pycharm_build_number}:/opt/.pycharm_helpers pycharm_helpers:{pycharm_build_number}
The pycharm_build_number can be found in the about section of your pycharm (mac OS: Pycharm > About)
Long story
I struggled a lot with PyCharm suddenly not finding the helpers any more or any related bugs, sometimes because I was clearing my containers or volumes. For instance, running
docker rm -f `docker container ps -aq`
docker volume rm $(docker volume ls -q)
will almost surely get pycharm into troubles.
AFAIK about how PyCharm works, there is:
a PyCharm base image named pycharm_helpers with tag corresponding to your pycharm build number (for example: PY-202.7660.27)
the first time you create docker related things, PyCharm creates volumes that get data from this image for later use in your containers. For instance, after a first attempt at running a remote docker-compose interpreter, I see the newly created myproject_pycharm_helpers_PY-202.7660.27 volume when doing docker volume ls.
when running the docker interpreter, PyCharm adds this volume into the /opt/.pycharm_helpers directory by adding at some point a -v myproject_pycharm_helpers_PY-202.7660.27:/opt/.pycharm_helpers to your command. For instance using docker-compose, you can see the addition of the -f /Users/clementwalter/Library/Caches/JetBrains/PyCharm2020.2/tmp/docker-compose.override.1508.yml and when you actually look into this file you see:
version: "3.8"
services:
local:
command:
- "python"
- "/opt/.pycharm_helpers/pydev/pydevconsole.py"
- "--mode=server"
- "--port=55824"
entrypoint: ""
environment:
PYCHARM_MATPLOTLIB_INTERACTIVE: "true"
PYTHONPATH: "/opt/project:/opt/.pycharm_helpers/pycharm_matplotlib_backend:/opt/.pycharm_helpers/pycharm_display:/opt/.pycharm_helpers/third_party/thriftpy:/opt/.pycharm_helpers/pydev"
PYTHONUNBUFFERED: "1"
PYTHONIOENCODING: "UTF-8"
PYCHARM_MATPLOTLIB_INDEX: "0"
PYCHARM_HOSTED: "1"
PYCHARM_DISPLAY_PORT: "63342"
IPYTHONENABLE: "True"
volumes:
- "/Users/clementwalter/Documents/myproject:/opt/project:rw"
- "pycharm_helpers_PY-202.7660.27:/opt/.pycharm_helpers"
working_dir: "/opt/project"
volumes:
pycharm_helpers_PY-202.7660.27: {}
You get into troubles when this volume is not correctly populated anymore.
Fortunately the docker volume documentation has a section "Populate a volume using a container" which is exactly what PyCharm does under the hood.
For the record you can check the content of the pycharm_helpers image:
$ docker run -it pycharm_helpers:PY-202.7660.27 sh
/opt/.pycharm_helpers #
you end up into the pycharm_helpers directory and find all the helpers here:
/opt/.pycharm_helpers # ls -la
total 5568
drwxr-xr-x 21 root root 4096 Dec 17 16:38 .
drwxr-xr-x 1 root root 4096 Dec 17 11:07 ..
-rw-r--r-- 1 root root 274 Dec 17 11:07 Dockerfile
drwxr-xr-x 5 root root 4096 Dec 17 16:38 MathJax
-rw-r--r-- 1 root root 2526 Sep 16 11:14 check_all_test_suite.py
-rw-r--r-- 1 root root 3194 Sep 16 11:14 conda_packaging_tool.py
drwxr-xr-x 2 root root 4096 Dec 17 16:38 coverage_runner
drwxr-xr-x 3 root root 4096 Dec 17 16:38 coveragepy
-rw-r--r-- 1 root root 11586 Sep 16 11:14 docstring_formatter.py
drwxr-xr-x 4 root root 4096 Dec 17 16:38 epydoc
-rw-r--r-- 1 root root 519 Sep 16 11:14 extra_syspath.py
drwxr-xr-x 3 root root 4096 Dec 17 16:38 generator3
-rw-r--r-- 1 root root 8 Sep 16 11:14 icon-robots.txt
-rw-r--r-- 1 root root 3950 Sep 16 11:14 packaging_tool.py
-rw-r--r-- 1 root root 1490666 Sep 16 11:14 pip-20.1.1-py2.py3-none-any.whl
drwxr-xr-x 2 root root 4096 Dec 17 16:38 pockets
drwxr-xr-x 3 root root 4096 Dec 17 16:38 profiler
-rw-r--r-- 1 root root 863 Sep 16 11:14 py2ipnb_converter.py
drwxr-xr-x 3 root root 4096 Dec 17 16:38 py2only
drwxr-xr-x 3 root root 4096 Dec 17 16:38 py3only
drwxr-xr-x 7 root root 4096 Dec 17 16:38 pycharm
drwxr-xr-x 4 root root 4096 Dec 17 16:38 pycharm_display
drwxr-xr-x 3 root root 4096 Dec 17 16:38 pycharm_matplotlib_backend
-rw-r--r-- 1 root root 103414 Sep 16 11:14 pycodestyle.py
drwxr-xr-x 24 root root 4096 Dec 17 16:38 pydev
drwxr-xr-x 9 root root 4096 Dec 17 16:38 python-skeletons
drwxr-xr-x 2 root root 4096 Dec 17 16:38 rest_runners
-rw-r--r-- 1 root root 583493 Sep 16 11:14 setuptools-44.1.1-py2.py3-none-any.whl
-rw-r--r-- 1 root root 29664 Sep 16 11:14 six.py
drwxr-xr-x 3 root root 4096 Dec 17 16:38 sphinxcontrib
-rw-r--r-- 1 root root 128 Sep 16 11:14 syspath.py
drwxr-xr-x 3 root root 4096 Dec 17 16:38 third_party
drwxr-xr-x 3 root root 4096 Dec 17 16:38 tools
drwxr-xr-x 5 root root 4096 Dec 17 16:38 typeshed
-rw-r--r-- 1 root root 3354133 Sep 16 11:14 virtualenv-16.7.10-py2.py3-none-any.whl
to make these helpers available again, following the docker documentation, you have to fix the volume. To do so:
docker rm {project_name}_pycharm_helpers_{pycharm_build}
docker run -v {project_name}_pycharm_helpers_{pycharm_build}:"/opt/.pycharm_helpers" pycharm_helpers:{tag}
et voilà
If you're still seeing this in PyCharm 2020.2 then do this:
close PyCharm
try #peterc's suggestion:
docker ps -a | grep -i pycharm | awk '{print $1}' | xargs docker rm
launch PyCharm again
The option invalidate cache -> Clear downloaded shared indexes will also repopulate the Pycharm volumes. (At least in 2021.1)
I have a simple example set up, running a centos or ubuntu image I've discovered that all my symlinks inside of a mounted volume are broken.
given the directory structure
testsyms
real
--one
--twoHundred
--three
syms
--one
--twoHundred
--three
and using the following docker command to start my container
docker run -ti -v $HOME/testsyms/:$HOME/testsyms -w $HOME/testsyms
I then do the following
inside the container :
[root#96b9af1cd545 testsyms]# ls -l **/*
-rw-r--r-- 1 501 games 0 Jan 8 06:00 real/one
-rw-r--r-- 1 501 games 0 Jan 8 06:03 real/three
-rw-r--r-- 1 501 games 0 Jan 8 06:00 real/twoHundred
lrwxr-xr-x 1 501 games 11 Jan 8 06:00 syms/one -> l/one
lrwxr-xr-x 1 501 games 19 Jan 8 06:03 syms/three -> ../real/three
lrwxr-xr-x 1 501 games 18 Jan 8 06:01 syms/twoHundred -> l/twoHundred
outside the container :
tam#tam-osx:testsyms$ ls -l **/*
-rw-r--r-- 1 tam staff 0 Jan 7 23:00 real/one
-rw-r--r-- 1 tam staff 0 Jan 7 23:03 real/three
-rw-r--r-- 1 tam staff 0 Jan 7 23:00 real/twoHundred
lrwxr-xr-x 1 tam staff 11 Jan 7 23:00 syms/one -> ../real/one
lrwxr-xr-x 1 tam staff 19 Jan 7 23:03 syms/three -> /Users../real/three
lrwxr-xr-x 1 tam staff 18 Jan 7 23:01 syms/twoHundred -> ../real/twoHundred
I created the links one and twoHundred outside the container while I created link three inside the container. inside the container links one and TwoHundred are broken. outside the container link three is broken as you should be able to see from the above outputs.
UPDATE--
Base on the comments I tried to ssh into the docker machine and found that the links are both correct, and incorrect. Doing some digging I find that I have my shared folder Users in 2 places. I have a /Users directory and I have a /mnt/hgfs/Users directory. Here is the output of each directory
/Users/ :
docker#default:/mnt/hgfs$ ls -l /Users/boger/testsyms/**/*
-rw-r--r-- 1 501 20 0 Jan 8 06:00 /Users/boger/testsyms/real/one
-rw-r--r-- 1 501 20 0 Jan 8 06:03 /Users/boger/testsyms/real/three
-rw-r--r-- 1 501 20 0 Jan 8 06:00 /Users/boger/testsyms/real/twoHundred
lrwxr-xr-x 1 501 20 11 Jan 8 06:00 /Users/boger/testsyms/syms/one -> l/one
lrwxr-xr-x 1 501 20 19 Jan 8 06:03 /Users/boger/testsyms/syms/three -> ../real/three
lrwxr-xr-x 1 501 20 18 Jan 8 06:01 /Users/boger/testsyms/syms/twoHundred -> l/twoHundred
/mnt/hgfs/Users/ :
docker#default:/mnt/hgfs$ ls -l /mnt/hgfs/Users/boger/testsyms/**/*
-rw-r--r-- 1 501 20 0 Jan 8 06:00 /mnt/hgfs/Users/boger/testsyms/real/one
-rw-r--r-- 1 501 20 0 Jan 8 06:03 /mnt/hgfs/Users/boger/testsyms/real/three
-rw-r--r-- 1 501 20 0 Jan 8 06:00 /mnt/hgfs/Users/boger/testsyms/real/twoHundred
lrwxr-xr-x 1 501 20 11 Jan 8 06:00 /mnt/hgfs/Users/boger/testsyms/syms/one -> ../real/one
lrwxr-xr-x 1 501 20 19 Jan 8 06:03 /mnt/hgfs/Users/boger/testsyms/syms/three -> /Users../real/three
lrwxr-xr-x 1 501 20 18 Jan 8 06:01 /mnt/hgfs/Users/boger/testsyms/syms/twoHundred -> ../real/twoHundred
its worth noting they have the same pattern as what I showed about inside and outside the container. Below is my config for the vm for the shared folders
sharedFolder0.present = "true"
sharedFolder0.enabled = "true"
sharedFolder0.readAccess = "true"
sharedFolder0.writeAccess = "true"
sharedFolder0.hostPath = "/Users"
sharedFolder0.guestName = "Users"
sharedFolder0.expiration = "never"
sharedFolder0.followSymlinks = “TRUE”
sharedFolder.maxNum = "1"
To work around this it turns out I just need to mount a different folder. I tried starting docker with -v /mnt/hgfs/Users/... and it works without any issues. I would really like to know what I can do to set up my vm so this isn't a problem down the road for other developers on my team though. Is my best option really to just ignore the broken directory and mount a new one ?