I am running nginxinc/nginx-unprivileged:stable-alpine docker image on RHEL 8.8 server. when docker container starts its creating directory and file with umask 0027.
But my docker 20.10.17 daemon running with Umask of 0022. my server default umask is 0027 this I can't change due to security requirements.
# systemd-analyze dump |egrep -i 'docker|umask'
ReferencedBy: docker.service (destination-file)
UMask: 0022
Here is inside container file system permission on RHEL 8 server.
# ls -l
total 76
drwxr-x--- 1 root root 4096 Jun 16 21:57 app
drwxr-x--- 1 root root 4096 Jun 16 21:57 bin
drwxr-x--- 5 root root 360 Jun 17 20:18 dev
drwxr-x--- 1 root root 4096 Jun 16 21:57 docker-entrypoint.d
-rwxr-x--- 1 root root 1202 Jun 16 21:57 docker-entrypoint.sh
drwxr-x--- 1 root root 4096 Jun 17 20:18 etc
drwxr-x--- 2 root root 4096 Jun 16 21:57 home
drwxrwxrwt 1 root root 4096 Jun 16 21:57 tmp
drwxr-x--- 1 root root 4096 Jun 16 21:57 usr
drwxr-x--- 1 root root 4096 Jun 16 21:57 var
Here is inside container file system permission on windows machine with same docker iamge.
ls -l
drwxr-xr-x 2 root root 4096 May 23 16:51 bin
drwxr-xr-x 5 root root 360 Jun 17 18:39 dev
drwxr-xr-x 1 root root 4096 Jun 16 10:36 docker-entrypoint.d
-rwxr-xr-x 1 root root 1202 Jun 16 10:36 docker-entrypoint.sh
drwxr-xr-x 1 root root 4096 Jun 17 18:39 etc
drwxr-xr-x 2 root root 4096 May 23 16:51 home
drwxr-xr-x 1 root root 4096 May 23 16:51 usr
drwxr-xr-x 1 root root 4096 May 23 16:51 var
How can I make docker container file system created with umask of 0022?
Thanks
when docker container starts
That means you need to build your own image, based on nginxinc/nginx-unprivileged:stable-alpine, with a new entry point like:
#!/bin/sh
# entrypoint.sh
umask 022
# ... other first-time setup ...
exec "$#"
See "Change umask in docker containers" for more details, but the idea remains the same.
I run a Fedora distribution. My /home is an encrypted volume.
When mounting files or directories as volume with docker using the -v option or through the volumes directory in a docker-compose.yml file I get very weird permissions on the file :
$ touch ~/test
$ docker run -v /home/jkr/test:/test -it --rm nginx bash
$ root#65fcd1754a1d:/# ls -la /
ls: cannot access '/test': Permission denied
total 84
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 .
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 ..
-rwxr-xr-x. 1 root root 0 Aug 30 20:29 .dockerenv
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 bin
drwxr-xr-x. 2 root root 4096 Jun 13 10:30 boot
drwxr-xr-x. 5 root root 360 Aug 30 20:29 dev
drwxr-xr-x. 1 root root 4096 Aug 17 11:46 docker-entrypoint.d
-rwxrwxr-x. 1 root root 1202 Aug 17 11:45 docker-entrypoint.sh
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 etc
drwxr-xr-x. 2 root root 4096 Jun 13 10:30 home
drwxr-xr-x. 1 root root 4096 Aug 17 11:46 lib
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 lib64
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 media
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 mnt
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 opt
dr-xr-xr-x. 490 root root 0 Aug 30 20:29 proc
drwx------. 2 root root 4096 Aug 16 00:00 root
drwxr-xr-x. 3 root root 4096 Aug 16 00:00 run
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 sbin
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 srv
dr-xr-xr-x. 13 root root 0 Aug 30 20:16 sys
-?????????? ? ? ? ? ? test
drwxrwxrwt. 1 root root 4096 Aug 17 11:46 tmp
drwxr-xr-x. 1 root root 4096 Aug 16 00:00 usr
drwxr-xr-x. 1 root root 4096 Aug 16 00:00 var
Notice the permission of the /test file inside the docker
-?????????? ? ? ? ? ? test
If I create this test file on a non encrypted llvm volume e.g. in /opt/ or /tmp then the permission is correct.
Is there any way to mount docker volume on an encrypted partition ?
- means its regular file , when this file is created then which user are use this permission maybe this user is deleted but file is exists thats why ?? is showing
I've Docker Container on Ubuntu and trying to start it that was initially setup by a professional developer (I'm a newbie with docker/container thing)
While starting the container using docker start 16e5e9280bfe -a I get this error
bash: startup.sh: No such file or directory
here is the list of containers
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
16e5e9280bfe 9fc1df773d19 "bash startup.sh" 9 months ago Exited (127) 53 seconds ago mystifying_kapitsa
The Command bash startup.sh has a problem that the docker is not able to find the startup.sh file (probably it gets deleted) and I dont know where this file should be created (need your help in this regards)
I tried to edit the config.v2.json file (removing the CMD and ARG parameter) under /var/lib/docker/containers/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39/ but when I start the container, it get's overwritten by the system and startup.sh appears again in the file
Incase you need to know I've many overlays
drwx------ 3 root root 4096 Mar 15 09:02 09a7c19f2a8e478b75d8362915c9f324fca4a0a02e24637a1e636017ac94306d/
drwx------ 3 root root 4096 Mar 15 09:02 1c8a6778fe9c2285a0fb9497cf033f70c04a16d66e644f93e4d332b7f68e1b5a/
drwx------ 3 root root 4096 Jul 14 2019 1d0c600d79c41bf9b5554fa928a297ac3a359fbff1c8ef903c40809b913ea640/
drwx------ 3 root root 4096 Jul 14 2019 407310e8d2aefdb76bd01fa1675eef1b7512885532703c9f658c14fb9bd16b5d/
drwx------ 3 root root 4096 Mar 15 09:02 63a54717c69963a50921d0c63931674c8547ad032a13de29f585df956e3e8fa5/
drwx------ 3 root root 4096 Feb 27 2018 65a84394f71682bcf7eb92bb77e78525dc594b8688726e2fc2e125fe7c762f4d/
drwx------ 3 root root 4096 Mar 15 09:02 68b4eb941a39414a13e1d20d623f1fa65951cf688e3ef221e3ce7ebb4a3bb5a1/
drwx------ 3 root root 4096 Oct 19 2019 758f8ce11982261aae7c2200e421952f155742cbee4421d73c2ee822b6a44d6d/
drwx------ 3 root root 4096 Mar 4 2018 7cfe53f80077d076c046ffe12bb189c0ae8397c02879e4ab82dfb2970708ff7b/
drwx------ 3 root root 4096 Feb 27 2018 7ea1bc5aab7bdbe917daa8ab70f02a1bbdd5fef3ccd7b5865d5f0b65cf188168/
drwx------ 3 root root 4096 Feb 27 2018 8726b7fc216e2caf30bc6bdfd67aac681c076fe016a3078093a327de0eb86f71/
drwx------ 3 root root 4096 Oct 19 2019 a051306523973e4bb6942c9d9bb58d39fe55e5a4d8ba69bd907285d321f8c361/
drwx------ 4 root root 4096 Mar 15 12:06 a1af7c75c5d4bdd231d5494618851ba1226adf91879e7091cf03313d8b97b89a/
drwx------ 4 root root 4096 Mar 15 07:26 a1af7c75c5d4bdd231d5494618851ba1226adf91879e7091cf03313d8b97b89a-init/
drwx------ 3 root root 4096 Mar 15 09:02 aeea6b5c888be7896a298965b7163ea14343e3bf4bb5ccb8cd2a839cba66e62d/
drwx------ 3 root root 4096 Oct 19 2019 b44ca2240ee9a220eca0598a2f747ad1dfeb439019363189cbec85fb69a74775/
drwx------ 3 root root 4096 Feb 27 2018 bb8c3313a4e30681ac71c8e0279ed72ea94d4fbcb1f6cf6144ac98a238e3df34/
drwx------ 3 root root 4096 Mar 15 09:02 cdbbda1e3e039677378745b5e0a971fabc78d7ca37c6b3c15da45a54037da57b/
drwx------ 3 root root 4096 Feb 27 2018 d1edbc1173ed75f9fc4b800893975bdf3c6f2440f8483fb9e5acb817f19a7e45/
drwx------ 3 root root 4096 Feb 27 2018 e8b6a178f59cfa58f9821b555fbe28ae25ee64a22525271c5d8507dbaa41d553/
drwx------ 3 root root 4096 Feb 27 2018 eb14f3c333daad51203a19145d00d484862c2443f02ad711a28b9bad3bbdf08e/
drwx------ 3 root root 4096 Jul 14 2019 f598570a29c2ba8a452969191bb362431d62ef33a81da9bd4ac2aabaac2027da/
drwx------ 3 root root 4096 Feb 27 2018 ffe7c7b1cc80b55698a1e7bc355fe48595dc07502a59f261821de01fcbc59f49/
This is the config.v2.json File - you may want to check please
{"StreamConfig":{},"State":{"Running":false,"Paused":false,"Restarting":false,"OOMKilled":false,"RemovalInProgress":false,"Dead":false,"Pid":0,"ExitCode":127,"Error":"","StartedAt":"2021-03-15T10:49:57.080556832Z","FinishedAt":"2021-03-15T10:49:57.114871213Z","Health":null},"ID":"16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39","Created":"2020-05-31T17:01:37.405644454Z","Managed":false,"Path":"bash","Args":["startup.sh"],"Config":{"Hostname":"16e5e9280bfe","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"ExposedPorts":{"21/tcp":{},"22/tcp":{},"443/tcp":{},"80/tcp":{}},"Tty":true,"OpenStdin":true,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","FFMPEG_VERSION=3.3.6","FDKAAC_VERSION=0.1.5","LAME_VERSION=3.99.5","LIBASS_VERSION=0.13.7","OGG_VERSION=1.3.2","OPENCOREAMR_VERSION=0.1.4","OPUS_VERSION=1.2","OPENJPEG_VERSION=2.1.2","THEORA_VERSION=1.1.1","VORBIS_VERSION=1.3.5","VPX_VERSION=1.7.0","X264_VERSION=20170226-2245-stable","X265_VERSION=2.3","XVID_VERSION=1.3.4","FREETYPE_VERSION=2.5.5","FRIBIDI_VERSION=0.19.7","FONTCONFIG_VERSION=2.12.4","LIBVIDSTAB_VERSION=1.1.0","KVAZAAR_VERSION=1.2.0","SRC=/usr/local","LD_LIBRARY_PATH=/usr/local/lib"],"Cmd":["startup.sh"],"Image":"9fc1df773d19","Volumes":{"/var/log/":{}},"WorkingDir":"/var/www/html","Entrypoint":["bash"],"OnBuild":null,"Labels":{}},"Image":"sha256:9fc1df773d198694f22f33c823ea8a05db78dcc7ea787ffafdc6ee95008bcbab","NetworkSettings":{"Bridge":"","SandboxID":"e32693eb6d1f685f8a77187c9f9713558d49248bc47ab6b8a97045ad37856a3e","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"1dfe0ae53916827fbc1a6fe18387a7653f48cdc445b823cc3d42cce04a8ac242","EndpointID":"","Gateway":"","IPAddress":"","IPPrefixLen":0,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"","DriverOpts":null,"IPAMOperational":false}},"Service":null,"Ports":null,"SandboxKey":"/var/run/docker/netns/e32693eb6d1f","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"IsAnonymousEndpoint":true,"HasSwarmEndpoint":false},"LogPath":"/var/lib/docker/containers/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39-json.log","Name":"/mystifying_kapitsa","Driver":"overlay","OS":"linux","MountLabel":"","ProcessLabel":"","RestartCount":0,"HasBeenStartedBefore":true,"HasBeenManuallyStopped":false,"MountPoints":{"/var/log":{"Source":"","Destination":"/var/log","RW":true,"Name":"5154e45a0f7d2fe5ce97d406f496474f9247e9413e4fc3ce076b59a45014c60c","Driver":"local","Type":"volume","Spec":{},"SkipMountpointCreation":false},"/var/www/demo":{"Source":"/var/www/demo","Destination":"/var/www/demo","RW":true,"Name":"","Driver":"","Type":"bind","Propagation":"rprivate","Spec":{"Type":"bind","Source":"/var/www/demo","Target":"/var/www/demo"},"SkipMountpointCreation":false},"/var/www/html":{"Source":"/var/www/html","Destination":"/var/www/html","RW":true,"Name":"","Driver":"","Type":"bind","Propagation":"rprivate","Spec":{"Type":"bind","Source":"/var/www/html","Target":"/var/www/html/"},"SkipMountpointCreation":false}},"SecretReferences":null,"ConfigReferences":null,"AppArmorProfile":"docker-default","HostnamePath":"/var/lib/docker/containers/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39/hostname","HostsPath":"/var/lib/docker/containers/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39/hosts","ShmPath":"","ResolvConfPath":"/var/lib/docker/containers/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39/resolv.conf","SeccompProfile":"","NoNewPrivileges":false}
I'm using Ubuntu 18.04.3 LTS
Thank you!
You need to find where is the startup script in the overlays of the image, container and volumes that are attached. You will need to find the last version (last layer) where the file was changed last and modify it there.
However, this is not to be done by someone who do not have in-depth understanding of the docker-engine.
But you don't need to do all this things just to enter in a stopped+failed container.
If a you need the access to the container, what you can do is to create image from the container
docker commit mystifying_kapitsa my_container:latest
Once you have the image, you can use it as a base for new images or, you can create new container from the image like this:
docker exec -ti --name i_am_in my_container:latest /bin/bash
This will give you a shell inside the image from the container and all data that ware in the mystifying_kapitsa will be in.
I'm new to docker and started with simple examples, but get "Permission denied":
Gyro#Helper:~$ sudo docker run prakhar1989/static-site sh -c 'echo foo > delete.me;echo bar >> delete.me;ls -l;type cat;ls -l /bin/cat;echo -n "I am ";whoami;cat delete.me'
total 72
drwxr-xr-x 2 root root 4096 Dec 4 2015 bin
drwxr-xr-x 2 root root 4096 Aug 26 2015 boot
-rw-r--r-- 1 root root 8 Feb 15 15:38 delete.me
drwxr-xr-x 5 root root 340 Feb 15 15:38 dev
drwxr-xr-x 1 root root 4096 Feb 15 15:38 etc
drwxr-xr-x 2 root root 4096 Aug 26 2015 home
drwxr-xr-x 9 root root 4096 Nov 27 2014 lib
drwxr-xr-x 2 root root 4096 Dec 4 2015 lib64
drwxr-xr-x 2 root root 4096 Dec 4 2015 media
drwxr-xr-x 2 root root 4096 Dec 4 2015 mnt
drwxr-xr-x 2 root root 4096 Dec 4 2015 opt
dr-xr-xr-x 271 root root 0 Feb 15 15:38 proc
drwx------ 2 root root 4096 Dec 4 2015 root
drwxr-xr-x 3 root root 4096 Dec 4 2015 run
drwxr-xr-x 2 root root 4096 Dec 4 2015 sbin
drwxr-xr-x 2 root root 4096 Dec 4 2015 srv
dr-xr-xr-x 13 root root 0 Dec 26 11:32 sys
drwxrwxrwt 1 root root 4096 Dec 16 2015 tmp
drwxr-xr-x 1 root root 4096 Jan 3 2016 usr
drwxr-xr-x 1 root root 4096 Dec 16 2015 var
-rwxr-xr-x 1 root root 69 Jan 3 2016 wrapper.sh
cat is /bin/cat
-rwxr-xr-x 1 root root 51912 Mar 14 2015 /bin/cat
I am root
cat: delete.me: Permission denied
The original problem was Permission denied on wrapper.sh so I played around a bit to get more information on the problem.
The code and result above tells me that I can create a file named delete.me, append contents, see the file via ls, can check that cat is /bin/cat and has execute permission. I am logged in into the container as user root, but cat won't obey.
Is it a bug?
docker info gives:
Gyro#Helper:~$ sudo docker info
Containers: 38
Running: 0
Paused: 0
Stopped: 38
Images: 5
Server Version: 18.09.9
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc version: N/A
init version: fec3683
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.0.0-32-generic
Operating System: Ubuntu Core 16
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.08GiB
Name: x260
ID: JCNS:55GU:FMFK:GJGF:JQ2P:IMSA:JOW2:JT5L:VQOB:QG4C:2NU5:Z6DR
Docker Root Dir: /var/snap/docker/common/var-lib-docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
Thanks to the help of #JohnKugelman it works now.
Key thing is not to call
sudo snap install docker
but
sudo apt install docker.io
Gyro#Helper:~$ sudo snap remove docker
docker removed
Gyro#Helper:~$ sudo docker run prakhar1989/static-site sh -c 'echo foo > delete.me;echo bar >> delete.me;ls -l;type cat;ls -l /bin/cat;echo -n "I am ";whoami;cp delete.me newfile.del;ls -alrt'
sudo: docker: command not found
Gyro#Helper:~$ docker
Command 'docker' not found, but can be installed with:
sudo snap install docker # version 18.09.9, or
See 'snap info docker' for additional versions.
Gyro#Helper:~$ sudo apt install docker.io
Reading package lists... Done
(stuff deleted)
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for ureadahead (0.100.0-21) ...
7[0;51r8[1A[JGyro#Helper:~$
Gyro#Helper:~$ sudo docker run prakhar1989/static-site sh -c 'echo foo > delete.me;echo bar >> delete.me;ls -l;type cat;ls -l /bin/cat;echo -n "I am ";whoami;cp delete.me newfile.del;ls -alrt'
Unable to find image 'prakhar1989/static-site:latest' locally
latest: Pulling from prakhar1989/static-site
(download progress deleted)
total 72
drwxr-xr-x 2 root root 4096 Dec 4 2015 bin
drwxr-xr-x 2 root root 4096 Aug 26 2015 boot
-rw-r--r-- 1 root root 8 Feb 15 17:16 delete.me
drwxr-xr-x 5 root root 340 Feb 15 17:16 dev
drwxr-xr-x 1 root root 4096 Feb 15 17:16 etc
drwxr-xr-x 2 root root 4096 Aug 26 2015 home
drwxr-xr-x 9 root root 4096 Nov 27 2014 lib
drwxr-xr-x 2 root root 4096 Dec 4 2015 lib64
drwxr-xr-x 2 root root 4096 Dec 4 2015 media
drwxr-xr-x 2 root root 4096 Dec 4 2015 mnt
drwxr-xr-x 2 root root 4096 Dec 4 2015 opt
dr-xr-xr-x 292 root root 0 Feb 15 17:16 proc
drwx------ 2 root root 4096 Dec 4 2015 root
drwxr-xr-x 3 root root 4096 Dec 4 2015 run
drwxr-xr-x 2 root root 4096 Dec 4 2015 sbin
drwxr-xr-x 2 root root 4096 Dec 4 2015 srv
dr-xr-xr-x 13 root root 0 Dec 26 11:32 sys
drwxrwxrwt 1 root root 4096 Dec 16 2015 tmp
drwxr-xr-x 1 root root 4096 Jan 3 2016 usr
drwxr-xr-x 1 root root 4096 Dec 16 2015 var
-rwxr-xr-x 1 root root 69 Jan 3 2016 wrapper.sh
cat is /bin/cat
-rwxr-xr-x 1 root root 51912 Mar 14 2015 /bin/cat
I am root
total 84
drwxr-xr-x 9 root root 4096 Nov 27 2014 lib
drwxr-xr-x 2 root root 4096 Aug 26 2015 home
drwxr-xr-x 2 root root 4096 Aug 26 2015 boot
drwxr-xr-x 2 root root 4096 Dec 4 2015 srv
drwxr-xr-x 3 root root 4096 Dec 4 2015 run
drwx------ 2 root root 4096 Dec 4 2015 root
drwxr-xr-x 2 root root 4096 Dec 4 2015 opt
drwxr-xr-x 2 root root 4096 Dec 4 2015 mnt
drwxr-xr-x 2 root root 4096 Dec 4 2015 media
drwxr-xr-x 2 root root 4096 Dec 4 2015 lib64
drwxr-xr-x 2 root root 4096 Dec 4 2015 bin
drwxr-xr-x 2 root root 4096 Dec 4 2015 sbin
drwxrwxrwt 1 root root 4096 Dec 16 2015 tmp
drwxr-xr-x 1 root root 4096 Dec 16 2015 var
-rwxr-xr-x 1 root root 69 Jan 3 2016 wrapper.sh
drwxr-xr-x 1 root root 4096 Jan 3 2016 usr
dr-xr-xr-x 13 root root 0 Dec 26 11:32 sys
drwxr-xr-x 1 root root 4096 Feb 15 17:16 etc
-rwxr-xr-x 1 root root 0 Feb 15 17:16 .dockerenv
dr-xr-xr-x 290 root root 0 Feb 15 17:16 proc
drwxr-xr-x 5 root root 340 Feb 15 17:16 dev
-rw-r--r-- 1 root root 8 Feb 15 17:16 delete.me
-rw-r--r-- 1 root root 8 Feb 15 17:16 newfile.del
drwxr-xr-x 1 root root 4096 Feb 15 17:16 ..
drwxr-xr-x 1 root root 4096 Feb 15 17:16 .
Gyro#Helper:~$ sudo docker run prakhar1989/static-site sh -c 'echo foo > delete.me;echo bar >> delete.me;ls -l;type cat;ls -l /bin/cat;echo -n "I am ";whoami;cat delete.me;ls -alrt'
total 72
drwxr-xr-x 2 root root 4096 Dec 4 2015 bin
drwxr-xr-x 2 root root 4096 Aug 26 2015 boot
-rw-r--r-- 1 root root 8 Feb 15 17:17 delete.me
drwxr-xr-x 5 root root 340 Feb 15 17:17 dev
drwxr-xr-x 1 root root 4096 Feb 15 17:17 etc
drwxr-xr-x 2 root root 4096 Aug 26 2015 home
drwxr-xr-x 9 root root 4096 Nov 27 2014 lib
drwxr-xr-x 2 root root 4096 Dec 4 2015 lib64
drwxr-xr-x 2 root root 4096 Dec 4 2015 media
drwxr-xr-x 2 root root 4096 Dec 4 2015 mnt
drwxr-xr-x 2 root root 4096 Dec 4 2015 opt
dr-xr-xr-x 291 root root 0 Feb 15 17:17 proc
drwx------ 2 root root 4096 Dec 4 2015 root
drwxr-xr-x 3 root root 4096 Dec 4 2015 run
drwxr-xr-x 2 root root 4096 Dec 4 2015 sbin
drwxr-xr-x 2 root root 4096 Dec 4 2015 srv
dr-xr-xr-x 13 root root 0 Dec 26 11:32 sys
drwxrwxrwt 1 root root 4096 Dec 16 2015 tmp
drwxr-xr-x 1 root root 4096 Jan 3 2016 usr
drwxr-xr-x 1 root root 4096 Dec 16 2015 var
-rwxr-xr-x 1 root root 69 Jan 3 2016 wrapper.sh
cat is /bin/cat
-rwxr-xr-x 1 root root 51912 Mar 14 2015 /bin/cat
I am root
foo
bar
total 80
drwxr-xr-x 9 root root 4096 Nov 27 2014 lib
drwxr-xr-x 2 root root 4096 Aug 26 2015 home
drwxr-xr-x 2 root root 4096 Aug 26 2015 boot
drwxr-xr-x 2 root root 4096 Dec 4 2015 srv
drwxr-xr-x 3 root root 4096 Dec 4 2015 run
drwx------ 2 root root 4096 Dec 4 2015 root
drwxr-xr-x 2 root root 4096 Dec 4 2015 opt
drwxr-xr-x 2 root root 4096 Dec 4 2015 mnt
drwxr-xr-x 2 root root 4096 Dec 4 2015 media
drwxr-xr-x 2 root root 4096 Dec 4 2015 lib64
drwxr-xr-x 2 root root 4096 Dec 4 2015 bin
drwxr-xr-x 2 root root 4096 Dec 4 2015 sbin
drwxrwxrwt 1 root root 4096 Dec 16 2015 tmp
drwxr-xr-x 1 root root 4096 Dec 16 2015 var
-rwxr-xr-x 1 root root 69 Jan 3 2016 wrapper.sh
drwxr-xr-x 1 root root 4096 Jan 3 2016 usr
dr-xr-xr-x 13 root root 0 Dec 26 11:32 sys
drwxr-xr-x 1 root root 4096 Feb 15 17:17 etc
-rwxr-xr-x 1 root root 0 Feb 15 17:17 .dockerenv
dr-xr-xr-x 289 root root 0 Feb 15 17:17 proc
drwxr-xr-x 5 root root 340 Feb 15 17:17 dev
-rw-r--r-- 1 root root 8 Feb 15 17:17 delete.me
drwxr-xr-x 1 root root 4096 Feb 15 17:17 ..
drwxr-xr-x 1 root root 4096 Feb 15 17:17 .
Gyro#Helper:~$ sudo docker run --rm prakhar1989/static-site
Nginx is running...
I have a docker compose file that contains the below volume mapping.
volumes:
- /opt/cloudera/parcels/SPARK2/lib/spark2:/opt/cloudera/parcels/SPARK2/lib/spark2
The contents of this directory are:
rwxr-xr-x 13 root root 247 Nov 30 16:39 .
drwxr-xr-x 3 root root 20 Jan 9 2018 ..
drwxr-xr-x 2 root root 4096 Jan 9 2018 bin
drwxr-xr-x 2 root root 39 Jan 9 2018 cloudera
lrwxrwxrwx 1 root root 16 Jan 9 2018 conf -> /etc/spark2/conf ***
drwxr-xr-x 5 root root 50 Jan 9 2018 data
drwxr-xr-x 4 root root 29 Jan 9 2018 examples
drwxr-xr-x 2 root root 8192 May 22 2018 jars
drwxr-xr-x 2 root root 204 Jan 9 2018 kafka-0.10
drwxr-xr-x 2 root root 201 Jan 9 2018 kafka-0.9
-rw-r--r-- 1 root root 17881 Jan 9 2018 LICENSE
drwxr-xr-x 2 root root 4096 Jan 9 2018 licenses
-rw-r--r-- 1 root root 24645 Jan 9 2018 NOTICE
drwxr-xr-x 6 root root 204 Jan 9 2018 python
-rw-r--r-- 1 root root 3809 Jan 9 2018 README.md
-rw-r--r-- 1 root root 313 Jan 9 2018 RELEASE
drwxr-xr-x 2 root root 4096 Jan 9 2018 sbin
lrwxrwxrwx 1 root root 20 Jan 9 2018 work -> /var/run/spark2/work
drwxr-xr-x 2 root root 52 Jan 9 2018 yarn
Of note is the starred conf directory, which itself is a series of symbolic links which eventually point to to the /etc/spark2/conf.cloudera.spark2_on_yarn folder that contains:
drwxr-xr-x 3 root root 194 Nov 30 16:39 .
drwxr-xr-x 3 root root 54 Nov 12 14:45 ..
-rw-r--r-- 1 root root 13105 Sep 16 03:07 classpath.txt
-rw-r--r-- 1 root root 20 Sep 16 03:07 __cloudera_generation__
-rw-r--r-- 1 root root 148 Sep 16 03:07 __cloudera_metadata__
-rw-r--r-- 1 ember 10000 2060 Nov 30 16:33 envars.test
-rw-r--r-- 1 root root 951 Sep 16 03:07 log4j.properties
-rw-r--r-- 1 root root 1837 Sep 16 03:07 spark-defaults.conf
-rw-r--r-- 1 root root 2331 Sep 16 03:07 spark-env.sh
drwxr-xr-x 2 root root 242 Sep 16 03:07 yarn-conf
When mapping the spark2 directory, only the yarn-conf subfolder shows up, the spark-env.sh file and other files are absent.
Is it the series of symbolic links that is causing these files to be absent? If so, do I need to explicitly set a mapping for every single folder in order to get all of the necessary dependencies to appear? I was under the impression that docker-compose volumes would recursively mount all files/folders under a particular directory.
The bind mount should faithfully reproduce the contents of the host: conf inside the container should be a symbolic link to /etc/spark2/conf. The container may or may not have anything at that path, but Docker doesn't recursively search the bind-mounted tree and try to do anything special with symlinks.
Are you trying to use docker run -v to "install" a Spark distribution in your container? You might be better off building a standalone Docker image with the software you want, and then using a bind mount to only inject the config files. That could look something like
docker run \
-v /etc/spark2/conf:/spark/conf \
-v $PWD/spark:/spark/work \
mysparkimage
Possible duplication of this question. In short, symlinks don't work very well inside docker containers.