Sorry I post my question here as I permanently get an error on the sphinxsearch forum where it belongs to.
I have set min_prefix_len = 3.
According to http://sphinxsearch.com/docs/current/conf-min-prefix-len.html I expect a match for
val*
but I do not even get a match for
vale*
however I get all matches for
valeg*
valegr*
valegro*
What is it that I do not understand?
I'm sorry, now it works like it should. Possible explanation is an error in the index. While exploring manticore as an alternative to sphinx, I reindexed and by chance noticed that rotation did not occur although the indexer said so:
rotating indices: successfully sent SIGHUP to searchd (pid=8).
But looking at the disk proved that this was not the case:
# ls -latrh /d/data/sphinx/*sm_en*
-rw-r--r-- 1 root root 0 Jul 29 04:43 /d/data/sphinx/lng_sm_en.spm
-rw-r--r-- 1 root root 31K Jul 29 04:43 /d/data/sphinx/lng_sm_en.spa
-rw-r--r-- 1 root root 0 Jul 29 04:43 /d/data/sphinx/lng_sm_en.spk
-rw-r--r-- 1 root root 607K Jul 29 04:43 /d/data/sphinx/lng_sm_en.spi
-rw-r--r-- 1 root root 5.7M Jul 29 04:43 /d/data/sphinx/lng_sm_en.spd
-rw-r--r-- 1 root root 11M Jul 29 04:43 /d/data/sphinx/lng_sm_en.spp
-rw-r--r-- 1 root root 30M Jul 29 04:43 /d/data/sphinx/lng_sm_en.sps
-rw-r--r-- 1 root root 823 Jul 29 04:43 /d/data/sphinx/lng_sm_en.sph
-rw-r--r-- 1 root root 38K Jul 29 04:43 /d/data/sphinx/lng_sm_en.spe
-rw------- 1 root root 0 Jul 29 04:43 /d/data/sphinx/lng_sm_en.spl
-rw-r--r-- 1 root root 0 Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spm
-rw-r--r-- 1 root root 31K Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spa
-rw-r--r-- 1 root root 0 Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spk
-rw-r--r-- 1 root root 11M Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spp
-rw-r--r-- 1 root root 607K Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spi
-rw-r--r-- 1 root root 5.7M Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spd
-rw-r--r-- 1 root root 823 Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.sph
-rw-r--r-- 1 root root 30M Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.sps
-rw-r--r-- 1 root root 38K Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spe
-rw------- 1 root root 0 Jul 31 00:36 /d/data/sphinx/lng_sm_en.new.spl
So I deleted these files and indexed again. Probably this fixed the issue, as rotating works again as well.
As the filedates show, this problem must have been lingering for 2 weeks. Now the dates are correct, too.
Related
Our deployment model is that we create containers on the fly with docker-java-api, some of these containers use heavily rocksdb database. The files of the DB are on the host, like:
ls -lrt /mnt/data/rocksdb
-rw-r--r-- 1 root root 8374 Nov 28 15:32 fileA
-rw-r--r-- 1 root root 0 Nov 28 15:32 fileB
-rw-r--r-- 1 root root 37 Nov 28 15:32 ....
-rw-r--r-- 1 root root 16 Nov 28 15:32 fileC
-rw-r--r-- 1 root root 19646 Nov 28 15:32 ..
-rw-r--r-- 1 root root 22500 Nov 28 15:32 .... etc
/mnt/data/rocksdb gets mounted onto a container with app that uses the DB heavily.
What I notice is that after starting the container, ownership of the files changes to:
ls -lrnt /mnt/data/rocksdbdata/
total 84092
-rw-r--r-- 1 999 999 8374 Nov 28 15:32 fileA
-rw-r--r-- 1 999 999 0 Nov 28 15:32 LOCK
-rw-r--r-- 1 999 999 37 Nov 28 15:32 fileB
-rw-r--r-- 1 999 999 16 Nov 28 15:32 fileC
-rw-r--r-- 1 999 999 19646 Nov 28 15:32 ...
-rw-r--r-- 1 999 999 22500 Nov 28 15:32 .....etc
User with these UID:GID (999:999) is docker.
Can you tell me why is this happening?
It needs a long explanation. Long story short, docker is changing the mounted file permissions because of access with root permissions.
For more details, please look at this answer;
Docker changes owner of local files mounted as volume
I run a Fedora distribution. My /home is an encrypted volume.
When mounting files or directories as volume with docker using the -v option or through the volumes directory in a docker-compose.yml file I get very weird permissions on the file :
$ touch ~/test
$ docker run -v /home/jkr/test:/test -it --rm nginx bash
$ root#65fcd1754a1d:/# ls -la /
ls: cannot access '/test': Permission denied
total 84
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 .
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 ..
-rwxr-xr-x. 1 root root 0 Aug 30 20:29 .dockerenv
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 bin
drwxr-xr-x. 2 root root 4096 Jun 13 10:30 boot
drwxr-xr-x. 5 root root 360 Aug 30 20:29 dev
drwxr-xr-x. 1 root root 4096 Aug 17 11:46 docker-entrypoint.d
-rwxrwxr-x. 1 root root 1202 Aug 17 11:45 docker-entrypoint.sh
drwxr-xr-x. 1 root root 4096 Aug 30 20:29 etc
drwxr-xr-x. 2 root root 4096 Jun 13 10:30 home
drwxr-xr-x. 1 root root 4096 Aug 17 11:46 lib
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 lib64
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 media
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 mnt
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 opt
dr-xr-xr-x. 490 root root 0 Aug 30 20:29 proc
drwx------. 2 root root 4096 Aug 16 00:00 root
drwxr-xr-x. 3 root root 4096 Aug 16 00:00 run
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 sbin
drwxr-xr-x. 2 root root 4096 Aug 16 00:00 srv
dr-xr-xr-x. 13 root root 0 Aug 30 20:16 sys
-?????????? ? ? ? ? ? test
drwxrwxrwt. 1 root root 4096 Aug 17 11:46 tmp
drwxr-xr-x. 1 root root 4096 Aug 16 00:00 usr
drwxr-xr-x. 1 root root 4096 Aug 16 00:00 var
Notice the permission of the /test file inside the docker
-?????????? ? ? ? ? ? test
If I create this test file on a non encrypted llvm volume e.g. in /opt/ or /tmp then the permission is correct.
Is there any way to mount docker volume on an encrypted partition ?
- means its regular file , when this file is created then which user are use this permission maybe this user is deleted but file is exists thats why ?? is showing
I created a Docker image.
Its repository name is docker-hello-world and its Docker Image ID is e127d87570f9.
I understood that it should be placed in /var/lib/docker/overlay2/, but when I open this folder this is what I see:
$ sudo ls -alF /var/lib/docker/overlay2/
total 40
drwx------ 10 root root 4096 Dec 28 17:28 ./
drwx--x--x 14 root root 4096 Dec 28 01:35 ../
drwx------ 3 root root 4096 Dec 28 01:35 0a6ee9708ef2529b105ae04ee9279877f23a3408d81ea58433e34f9e278fd90e/
drwx------ 4 root root 4096 Dec 28 17:28 17c9c50edb4ecc6ffd47a688720dfe1b7f5d407ac9eb8f2b0ec5e62eccf9400f/
drwx------ 4 root root 4096 Dec 28 01:36 5fd838043c830a61c6efeeec62c1c1cca9d500556faca5870c526b0a34766a63/
drwx------ 4 root root 4096 Dec 28 01:35 5fd838043c830a61c6efeeec62c1c1cca9d500556faca5870c526b0a34766a63-init/
drwx------ 4 root root 4096 Dec 28 17:28 820dc021f10e31423aaba6ada86fbc06a03aa531d277c5de598e101dd2cd9881/
drwx------ 3 root root 4096 Dec 28 17:28 cfc4379bcf9885bd305fae0f85368941b863bb7533db4d088ffcd397b86f2182/
drwx------ 4 root root 4096 Dec 28 17:28 fa4ffb5887a1cbde6b3e58505119043ec891754bb3d797f1aedf68292c52477b/
drwx------ 2 root root 4096 Dec 28 17:28 l/
Do you know where to look for the image file?
I have a docker compose file that contains the below volume mapping.
volumes:
- /opt/cloudera/parcels/SPARK2/lib/spark2:/opt/cloudera/parcels/SPARK2/lib/spark2
The contents of this directory are:
rwxr-xr-x 13 root root 247 Nov 30 16:39 .
drwxr-xr-x 3 root root 20 Jan 9 2018 ..
drwxr-xr-x 2 root root 4096 Jan 9 2018 bin
drwxr-xr-x 2 root root 39 Jan 9 2018 cloudera
lrwxrwxrwx 1 root root 16 Jan 9 2018 conf -> /etc/spark2/conf ***
drwxr-xr-x 5 root root 50 Jan 9 2018 data
drwxr-xr-x 4 root root 29 Jan 9 2018 examples
drwxr-xr-x 2 root root 8192 May 22 2018 jars
drwxr-xr-x 2 root root 204 Jan 9 2018 kafka-0.10
drwxr-xr-x 2 root root 201 Jan 9 2018 kafka-0.9
-rw-r--r-- 1 root root 17881 Jan 9 2018 LICENSE
drwxr-xr-x 2 root root 4096 Jan 9 2018 licenses
-rw-r--r-- 1 root root 24645 Jan 9 2018 NOTICE
drwxr-xr-x 6 root root 204 Jan 9 2018 python
-rw-r--r-- 1 root root 3809 Jan 9 2018 README.md
-rw-r--r-- 1 root root 313 Jan 9 2018 RELEASE
drwxr-xr-x 2 root root 4096 Jan 9 2018 sbin
lrwxrwxrwx 1 root root 20 Jan 9 2018 work -> /var/run/spark2/work
drwxr-xr-x 2 root root 52 Jan 9 2018 yarn
Of note is the starred conf directory, which itself is a series of symbolic links which eventually point to to the /etc/spark2/conf.cloudera.spark2_on_yarn folder that contains:
drwxr-xr-x 3 root root 194 Nov 30 16:39 .
drwxr-xr-x 3 root root 54 Nov 12 14:45 ..
-rw-r--r-- 1 root root 13105 Sep 16 03:07 classpath.txt
-rw-r--r-- 1 root root 20 Sep 16 03:07 __cloudera_generation__
-rw-r--r-- 1 root root 148 Sep 16 03:07 __cloudera_metadata__
-rw-r--r-- 1 ember 10000 2060 Nov 30 16:33 envars.test
-rw-r--r-- 1 root root 951 Sep 16 03:07 log4j.properties
-rw-r--r-- 1 root root 1837 Sep 16 03:07 spark-defaults.conf
-rw-r--r-- 1 root root 2331 Sep 16 03:07 spark-env.sh
drwxr-xr-x 2 root root 242 Sep 16 03:07 yarn-conf
When mapping the spark2 directory, only the yarn-conf subfolder shows up, the spark-env.sh file and other files are absent.
Is it the series of symbolic links that is causing these files to be absent? If so, do I need to explicitly set a mapping for every single folder in order to get all of the necessary dependencies to appear? I was under the impression that docker-compose volumes would recursively mount all files/folders under a particular directory.
The bind mount should faithfully reproduce the contents of the host: conf inside the container should be a symbolic link to /etc/spark2/conf. The container may or may not have anything at that path, but Docker doesn't recursively search the bind-mounted tree and try to do anything special with symlinks.
Are you trying to use docker run -v to "install" a Spark distribution in your container? You might be better off building a standalone Docker image with the software you want, and then using a bind mount to only inject the config files. That could look something like
docker run \
-v /etc/spark2/conf:/spark/conf \
-v $PWD/spark:/spark/work \
mysparkimage
Possible duplication of this question. In short, symlinks don't work very well inside docker containers.
As stated in this document, docker container's runtime matrix should be available at /sys/fs/cgroup/memory/lxc/<longid>/.
However, if I am using boot2docker image, I can only find a folder named docker under /sys/fs/cgroup/memory/ with some matrices.
$ ls /sys/fs/cgroup/memory/ -l
total 0
-rw-r--r-- 1 root root 0 Mar 25 08:39 cgroup.clone_children
--w--w--w- 1 root root 0 Mar 25 08:39 cgroup.event_control
-rw-r--r-- 1 root root 0 Mar 25 08:39 cgroup.procs
-r--r--r-- 1 root root 0 Mar 25 08:39 cgroup.sane_behavior
drwxr-xr-x 2 root root 0 Mar 25 07:41 docker/
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.failcnt
--w------- 1 root root 0 Mar 25 08:39 memory.force_empty
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.kmem.failcnt
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.kmem.limit_in_bytes
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.kmem.max_usage_in_bytes
-r--r--r-- 1 root root 0 Mar 25 08:39 memory.kmem.slabinfo
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.kmem.tcp.failcnt
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.kmem.tcp.limit_in_bytes
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.kmem.tcp.max_usage_in_bytes
-r--r--r-- 1 root root 0 Mar 25 08:39 memory.kmem.tcp.usage_in_bytes
-r--r--r-- 1 root root 0 Mar 25 08:39 memory.kmem.usage_in_bytes
-rw-r--r-- 1 root root 0 Mar 25 04:00 memory.limit_in_bytes
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.max_usage_in_bytes
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.memsw.failcnt
-rw-r--r-- 1 root root 0 Mar 25 04:00 memory.memsw.limit_in_bytes
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.memsw.max_usage_in_bytes
-r--r--r-- 1 root root 0 Mar 25 08:39 memory.memsw.usage_in_bytes
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.move_charge_at_immigrate
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.oom_control
---------- 1 root root 0 Mar 25 08:39 memory.pressure_level
-rw-r--r-- 1 root root 0 Mar 25 04:00 memory.soft_limit_in_bytes
-r--r--r-- 1 root root 0 Mar 25 08:39 memory.stat
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.swappiness
-r--r--r-- 1 root root 0 Mar 25 08:39 memory.usage_in_bytes
-rw-r--r-- 1 root root 0 Mar 25 08:39 memory.use_hierarchy
-rw-r--r-- 1 root root 0 Mar 25 08:39 notify_on_release
-rw-r--r-- 1 root root 0 Mar 25 08:39 release_agent
-rw-r--r-- 1 root root 0 Mar 25 08:39 tasks
How can I get back the missing LXC matrices in boot2docker image?