I run a Fedora distribution. My /home is an encrypted volume.
When mounting files or directories as volume with docker using the -v option or through the volumes directory in a docker-compose.yml file I get very weird permissions on the file :
$ touch ~/test
$ docker run -v /home/jkr/test:/test -it --rm nginx bash
$ root#65fcd1754a1d:/# ls -la /
ls: cannot access '/test': Permission denied
total 84
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 .
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 ..
-rwxr-xr-x. 1 root root 0 Aug 30 20:29 .dockerenv
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 bin
drwxr-xr-x. 2 root root 4096 Jun 13 10:30 boot
drwxr-xr-x. 5 root root 360 Aug 30 20:29 dev
drwxr-xr-x. 1 root root 4096 Aug 17 11:46 docker-entrypoint.d
-rwxrwxr-x. 1 root root 1202 Aug 17 11:45 docker-entrypoint.sh
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 etc
drwxr-xr-x. 2 root root 4096 Jun 13 10:30 home
drwxr-xr-x. 1 root root 4096 Aug 17 11:46 lib
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 lib64
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 media
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 mnt
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 opt
dr-xr-xr-x. 490 root root 0 Aug 30 20:29 proc
drwx------. 2 root root 4096 Aug 16 00:00 root
drwxr-xr-x. 3 root root 4096 Aug 16 00:00 run
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 sbin
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 srv
dr-xr-xr-x. 13 root root 0 Aug 30 20:16 sys
-?????????? ? ? ? ? ? test
drwxrwxrwt. 1 root root 4096 Aug 17 11:46 tmp
drwxr-xr-x. 1 root root 4096 Aug 16 00:00 usr
drwxr-xr-x. 1 root root 4096 Aug 16 00:00 var
Notice the permission of the /test file inside the docker
-?????????? ? ? ? ? ? test
If I create this test file on a non encrypted llvm volume e.g. in /opt/ or /tmp then the permission is correct.
Is there any way to mount docker volume on an encrypted partition ?
- means its regular file , when this file is created then which user are use this permission maybe this user is deleted but file is exists thats why ?? is showing
Sorry I post my question here as I permanently get an error on the sphinxsearch forum where it belongs to.
I have set min_prefix_len = 3.
According to http://sphinxsearch.com/docs/current/conf-min-prefix-len.html I expect a match for
val*
but I do not even get a match for
vale*
however I get all matches for
valeg*
valegr*
valegro*
What is it that I do not understand?
I'm sorry, now it works like it should. Possible explanation is an error in the index. While exploring manticore as an alternative to sphinx, I reindexed and by chance noticed that rotation did not occur although the indexer said so:
rotating indices: successfully sent SIGHUP to searchd (pid=8).
But looking at the disk proved that this was not the case:
# ls -latrh /d/data/sphinx/*sm_en*
-rw-r--r-- 1 root root 0 Jul 29 04:43 /d/data/sphinx/lng_sm_en.spm
-rw-r--r-- 1 root root 31K Jul 29 04:43 /d/data/sphinx/lng_sm_en.spa
-rw-r--r-- 1 root root 0 Jul 29 04:43 /d/data/sphinx/lng_sm_en.spk
-rw-r--r-- 1 root root 607K Jul 29 04:43 /d/data/sphinx/lng_sm_en.spi
-rw-r--r-- 1 root root 5.7M Jul 29 04:43 /d/data/sphinx/lng_sm_en.spd
-rw-r--r-- 1 root root 11M Jul 29 04:43 /d/data/sphinx/lng_sm_en.spp
-rw-r--r-- 1 root root 30M Jul 29 04:43 /d/data/sphinx/lng_sm_en.sps
-rw-r--r-- 1 root root 823 Jul 29 04:43 /d/data/sphinx/lng_sm_en.sph
-rw-r--r-- 1 root root 38K Jul 29 04:43 /d/data/sphinx/lng_sm_en.spe
-rw------- 1 root root 0 Jul 29 04:43 /d/data/sphinx/lng_sm_en.spl
-rw-r--r-- 1 root root 0 Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spm
-rw-r--r-- 1 root root 31K Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spa
-rw-r--r-- 1 root root 0 Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spk
-rw-r--r-- 1 root root 11M Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spp
-rw-r--r-- 1 root root 607K Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spi
-rw-r--r-- 1 root root 5.7M Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spd
-rw-r--r-- 1 root root 823 Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.sph
-rw-r--r-- 1 root root 30M Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.sps
-rw-r--r-- 1 root root 38K Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spe
-rw------- 1 root root 0 Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spl
So I deleted these files and indexed again. Probably this fixed the issue, as rotating works again as well.
As the filedates show, this problem must have been lingering for 2 weeks. Now the dates are correct, too.
I've Docker Container on Ubuntu and trying to start it that was initially setup by a professional developer (I'm a newbie with docker/container thing)
While starting the container using docker start 16e5e9280bfe -a I get this error
bash: startup.sh: No such file or directory
here is the list of containers
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
16e5e9280bfe 9fc1df773d19 "bash startup.sh" 9 months ago Exited (127) 53 seconds ago mystifying_kapitsa
The Command bash startup.sh has a problem that the docker is not able to find the startup.sh file (probably it gets deleted) and I dont know where this file should be created (need your help in this regards)
I tried to edit the config.v2.json file (removing the CMD and ARG parameter) under /var/lib/docker/containers/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39/ but when I start the container, it get's overwritten by the system and startup.sh appears again in the file
Incase you need to know I've many overlays
drwx------ 3 root root 4096 Mar 15 09:02 09a7c19f2a8e478b75d8362915c9f324fca4a0a02e24637a1e636017ac94306d/
drwx------ 3 root root 4096 Mar 15 09:02 1c8a6778fe9c2285a0fb9497cf033f70c04a16d66e644f93e4d332b7f68e1b5a/
drwx------ 3 root root 4096 Jul 14 2019 1d0c600d79c41bf9b5554fa928a297ac3a359fbff1c8ef903c40809b913ea640/
drwx------ 3 root root 4096 Jul 14 2019 407310e8d2aefdb76bd01fa1675eef1b7512885532703c9f658c14fb9bd16b5d/
drwx------ 3 root root 4096 Mar 15 09:02 63a54717c69963a50921d0c63931674c8547ad032a13de29f585df956e3e8fa5/
drwx------ 3 root root 4096 Feb 27 2018 65a84394f71682bcf7eb92bb77e78525dc594b8688726e2fc2e125fe7c762f4d/
drwx------ 3 root root 4096 Mar 15 09:02 68b4eb941a39414a13e1d20d623f1fa65951cf688e3ef221e3ce7ebb4a3bb5a1/
drwx------ 3 root root 4096 Oct 19 2019 758f8ce11982261aae7c2200e421952f155742cbee4421d73c2ee822b6a44d6d/
drwx------ 3 root root 4096 Mar 4 2018 7cfe53f80077d076c046ffe12bb189c0ae8397c02879e4ab82dfb2970708ff7b/
drwx------ 3 root root 4096 Feb 27 2018 7ea1bc5aab7bdbe917daa8ab70f02a1bbdd5fef3ccd7b5865d5f0b65cf188168/
drwx------ 3 root root 4096 Feb 27 2018 8726b7fc216e2caf30bc6bdfd67aac681c076fe016a3078093a327de0eb86f71/
drwx------ 3 root root 4096 Oct 19 2019 a051306523973e4bb6942c9d9bb58d39fe55e5a4d8ba69bd907285d321f8c361/
drwx------ 4 root root 4096 Mar 15 12:06 a1af7c75c5d4bdd231d5494618851ba1226adf91879e7091cf03313d8b97b89a/
drwx------ 4 root root 4096 Mar 15 07:26 a1af7c75c5d4bdd231d5494618851ba1226adf91879e7091cf03313d8b97b89a-init/
drwx------ 3 root root 4096 Mar 15 09:02 aeea6b5c888be7896a298965b7163ea14343e3bf4bb5ccb8cd2a839cba66e62d/
drwx------ 3 root root 4096 Oct 19 2019 b44ca2240ee9a220eca0598a2f747ad1dfeb439019363189cbec85fb69a74775/
drwx------ 3 root root 4096 Feb 27 2018 bb8c3313a4e30681ac71c8e0279ed72ea94d4fbcb1f6cf6144ac98a238e3df34/
drwx------ 3 root root 4096 Mar 15 09:02 cdbbda1e3e039677378745b5e0a971fabc78d7ca37c6b3c15da45a54037da57b/
drwx------ 3 root root 4096 Feb 27 2018 d1edbc1173ed75f9fc4b800893975bdf3c6f2440f8483fb9e5acb817f19a7e45/
drwx------ 3 root root 4096 Feb 27 2018 e8b6a178f59cfa58f9821b555fbe28ae25ee64a22525271c5d8507dbaa41d553/
drwx------ 3 root root 4096 Feb 27 2018 eb14f3c333daad51203a19145d00d484862c2443f02ad711a28b9bad3bbdf08e/
drwx------ 3 root root 4096 Jul 14 2019 f598570a29c2ba8a452969191bb362431d62ef33a81da9bd4ac2aabaac2027da/
drwx------ 3 root root 4096 Feb 27 2018 ffe7c7b1cc80b55698a1e7bc355fe48595dc07502a59f261821de01fcbc59f49/
This is the config.v2.json File - you may want to check please
{"StreamConfig":{},"State":{"Running":false,"Paused":false,"Restarting":false,"OOMKilled":false,"RemovalInProgress":false,"Dead":false,"Pid":0,"ExitCode":127,"Error":"","StartedAt":"2021-03-15T10:49:57.080556832Z","FinishedAt":"2021-03-15T10:49:57.114871213Z","Health":null},"ID":"16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39","Created":"2020-05-31T17:01:37.405644454Z","Managed":false,"Path":"bash","Args":["startup.sh"],"Config":{"Hostname":"16e5e9280bfe","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"ExposedPorts":{"21/tcp":{},"22/tcp":{},"443/tcp":{},"80/tcp":{}},"Tty":true,"OpenStdin":true,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","FFMPEG_VERSION=3.3.6","FDKAAC_VERSION=0.1.5","LAME_VERSION=3.99.5","LIBASS_VERSION=0.13.7","OGG_VERSION=1.3.2","OPENCOREAMR_VERSION=0.1.4","OPUS_VERSION=1.2","OPENJPEG_VERSION=2.1.2","THEORA_VERSION=1.1.1","VORBIS_VERSION=1.3.5","VPX_VERSION=1.7.0","X264_VERSION=20170226-2245-stable","X265_VERSION=2.3","XVID_VERSION=1.3.4","FREETYPE_VERSION=2.5.5","FRIBIDI_VERSION=0.19.7","FONTCONFIG_VERSION=2.12.4","LIBVIDSTAB_VERSION=1.1.0","KVAZAAR_VERSION=1.2.0","SRC=/usr/local","LD_LIBRARY_PATH=/usr/local/lib"],"Cmd":["startup.sh"],"Image":"9fc1df773d19","Volumes":{"/var/log/":{}},"WorkingDir":"/var/www/html","Entrypoint":["bash"],"OnBuild":null,"Labels":{}},"Image":"sha256:9fc1df773d198694f22f33c823ea8a05db78dcc7ea787ffafdc6ee95008bcbab","NetworkSettings":{"Bridge":"","SandboxID":"e32693eb6d1f685f8a77187c9f9713558d49248bc47ab6b8a97045ad37856a3e","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"1dfe0ae53916827fbc1a6fe18387a7653f48cdc445b823cc3d42cce04a8ac242","EndpointID":"","Gateway":"","IPAddress":"","IPPrefixLen":0,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"","DriverOpts":null,"IPAMOperational":false}},"Service":null,"Ports":null,"SandboxKey":"/var/run/docker/netns/e32693eb6d1f","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"IsAnonymousEndpoint":true,"HasSwarmEndpoint":false},"LogPath":"/var/lib/docker/containers/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39-json.log","Name":"/mystifying_kapitsa","Driver":"overlay","OS":"linux","MountLabel":"","ProcessLabel":"","RestartCount":0,"HasBeenStartedBefore":true,"HasBeenManuallyStopped":false,"MountPoints":{"/var/log":{"Source":"","Destination":"/var/log","RW":true,"Name":"5154e45a0f7d2fe5ce97d406f496474f9247e9413e4fc3ce076b59a45014c60c","Driver":"local","Type":"volume","Spec":{},"SkipMountpointCreation":false},"/var/www/demo":{"Source":"/var/www/demo","Destination":"/var/www/demo","RW":true,"Name":"","Driver":"","Type":"bind","Propagation":"rprivate","Spec":{"Type":"bind","Source":"/var/www/demo","Target":"/var/www/demo"},"SkipMountpointCreation":false},"/var/www/html":{"Source":"/var/www/html","Destination":"/var/www/html","RW":true,"Name":"","Driver":"","Type":"bind","Propagation":"rprivate","Spec":{"Type":"bind","Source":"/var/www/html","Target":"/var/www/html/"},"SkipMountpointCreation":false}},"SecretReferences":null,"ConfigReferences":null,"AppArmorProfile":"docker-default","HostnamePath":"/var/lib/docker/containers/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39/hostname","HostsPath":"/var/lib/docker/containers/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39/hosts","ShmPath":"","ResolvConfPath":"/var/lib/docker/containers/16e5e9280bfea319c5094cddb4b2da71b6e461be824b119c32817e281d282f39/resolv.conf","SeccompProfile":"","NoNewPrivileges":false}
I'm using Ubuntu 18.04.3 LTS
Thank you!
You need to find where is the startup script in the overlays of the image, container and volumes that are attached. You will need to find the last version (last layer) where the file was changed last and modify it there.
However, this is not to be done by someone who do not have in-depth understanding of the docker-engine.
But you don't need to do all this things just to enter in a stopped+failed container.
If a you need the access to the container, what you can do is to create image from the container
docker commit mystifying_kapitsa my_container:latest
Once you have the image, you can use it as a base for new images or, you can create new container from the image like this:
docker exec -ti --name i_am_in my_container:latest /bin/bash
This will give you a shell inside the image from the container and all data that ware in the mystifying_kapitsa will be in.
I'm getting status code 500 on my Dockerized Flask server.
I bashed into the container to check the logs:
docker exec -ti container_name /bin/bash
in /var/log I found:
root#b80b0c02fd18:/var/log# ls -al
total 224
drwxr-xr-x 1 root root 4096 Oct 13 21:02 .
drwxr-xr-x 1 root root 4096 Oct 12 07:00 ..
-rw-r--r-- 1 root root 9052 Oct 20 20:50 alternatives.log
drwxr-xr-x 1 root root 4096 Oct 20 20:49 apt
-rw-rw---- 1 root utmp 0 Oct 12 07:00 btmp
-rw-r--r-- 1 root root 164661 Oct 20 20:50 dpkg.log
-rw-r--r-- 1 root root 3232 Oct 12 07:00 faillog
-rw-rw-r-- 1 root utmp 29492 Oct 12 07:00 lastlog
-rw-rw-r-- 1 root utmp 0 Oct 12 07:00 wtmp
I couldn't cat or nano the files faillog nor lastlog so I don't know if the files
are relevant.
Where do I find access log or error log for containerized Flask server ?
docker logs <your container_name> to see the logs of the container
You can also find the logs for the flask app at /var/log/daemon.log
I have a docker compose file that contains the below volume mapping.
volumes:
- /opt/cloudera/parcels/SPARK2/lib/spark2:/opt/cloudera/parcels/SPARK2/lib/spark2
The contents of this directory are:
rwxr-xr-x 13 root root 247 Nov 30 16:39 .
drwxr-xr-x 3 root root 20 Jan 9 2018 ..
drwxr-xr-x 2 root root 4096 Jan 9 2018 bin
drwxr-xr-x 2 root root 39 Jan 9 2018 cloudera
lrwxrwxrwx 1 root root 16 Jan 9 2018 conf -> /etc/spark2/conf ***
drwxr-xr-x 5 root root 50 Jan 9 2018 data
drwxr-xr-x 4 root root 29 Jan 9 2018 examples
drwxr-xr-x 2 root root 8192 May 22 2018 jars
drwxr-xr-x 2 root root 204 Jan 9 2018 kafka-0.10
drwxr-xr-x 2 root root 201 Jan 9 2018 kafka-0.9
-rw-r--r-- 1 root root 17881 Jan 9 2018 LICENSE
drwxr-xr-x 2 root root 4096 Jan 9 2018 licenses
-rw-r--r-- 1 root root 24645 Jan 9 2018 NOTICE
drwxr-xr-x 6 root root 204 Jan 9 2018 python
-rw-r--r-- 1 root root 3809 Jan 9 2018 README.md
-rw-r--r-- 1 root root 313 Jan 9 2018 RELEASE
drwxr-xr-x 2 root root 4096 Jan 9 2018 sbin
lrwxrwxrwx 1 root root 20 Jan 9 2018 work -> /var/run/spark2/work
drwxr-xr-x 2 root root 52 Jan 9 2018 yarn
Of note is the starred conf directory, which itself is a series of symbolic links which eventually point to to the /etc/spark2/conf.cloudera.spark2_on_yarn folder that contains:
drwxr-xr-x 3 root root 194 Nov 30 16:39 .
drwxr-xr-x 3 root root 54 Nov 12 14:45 ..
-rw-r--r-- 1 root root 13105 Sep 16 03:07 classpath.txt
-rw-r--r-- 1 root root 20 Sep 16 03:07 __cloudera_generation__
-rw-r--r-- 1 root root 148 Sep 16 03:07 __cloudera_metadata__
-rw-r--r-- 1 ember 10000 2060 Nov 30 16:33 envars.test
-rw-r--r-- 1 root root 951 Sep 16 03:07 log4j.properties
-rw-r--r-- 1 root root 1837 Sep 16 03:07 spark-defaults.conf
-rw-r--r-- 1 root root 2331 Sep 16 03:07 spark-env.sh
drwxr-xr-x 2 root root 242 Sep 16 03:07 yarn-conf
When mapping the spark2 directory, only the yarn-conf subfolder shows up, the spark-env.sh file and other files are absent.
Is it the series of symbolic links that is causing these files to be absent? If so, do I need to explicitly set a mapping for every single folder in order to get all of the necessary dependencies to appear? I was under the impression that docker-compose volumes would recursively mount all files/folders under a particular directory.
The bind mount should faithfully reproduce the contents of the host: conf inside the container should be a symbolic link to /etc/spark2/conf. The container may or may not have anything at that path, but Docker doesn't recursively search the bind-mounted tree and try to do anything special with symlinks.
Are you trying to use docker run -v to "install" a Spark distribution in your container? You might be better off building a standalone Docker image with the software you want, and then using a bind mount to only inject the config files. That could look something like
docker run \
-v /etc/spark2/conf:/spark/conf \
-v $PWD/spark:/spark/work \
mysparkimage
Possible duplication of this question. In short, symlinks don't work very well inside docker containers.