I have a Rails application that indexes classifeds ads. I'm using thinking-sphinx.
Yesterday, I realized that my ads are not getting indexed anymore. I'm a beginner in Rails so I don't know exactly what is happening here...
We can see the issue in searchd.query.log file :
[Sun May 19 22:19:46.432 2013] 0.000 sec [all/5/ext 0 (0,1000000000)] [ad_core]
[Sun May 19 22:19:46.446 2013] 0.000 sec [all/5/ext 0 (0,1000000000)] [ad_core]
[Sun May 19 22:19:46.455 2013] 0.000 sec [all/4/ext **21** (0,1000000000)] [ad_core]
[Sun May 19 22:19:46.590 2013] 0.000 sec [all/5/ext 0 (0,25)] [ad_core]
[Mon May 20 01:21:50.385 2013] 0.000 sec [all/5/ext 0 (0,1000000000)] [ad_core]
[Mon May 20 01:21:50.400 2013] 0.000 sec [all/5/ext 0 (0,1000000000)] [ad_core]
[Mon May 20 01:21:50.409 2013] 0.000 sec [all/4/ext **21** (0,1000000000)] [ad_core]
[Mon May 20 01:21:50.633 2013] 0.000 sec [all/5/ext 0 (0,25)] [ad_core]
[Mon May 20 02:13:12.492 2013] 0.000 sec [all/5/ext 0 (0,1000000000)] [ad_core]
[Mon May 20 02:13:12.508 2013] 0.000 sec [all/5/ext 0 (0,1000000000)] [ad_core]
[Mon May 20 02:13:12.521 2013] 0.000 sec [all/4/ext 0 (0,1000000000)] [ad_core]
[Mon May 20 02:13:12.780 2013] 0.000 sec [all/5/ext 0 (0,25)] [ad_core]
[Mon May 20 02:13:21.341 2013] 0.000 sec [all/5/ext 0 (0,1000000000)] [ad_core]
[Mon May 20 02:13:21.453 2013] 0.000 sec [all/5/ext 0 (0,25)] [ad_core]
[Mon May 20 02:13:36.183 2013] 0.000 sec [all/5/ext 0 (0,1000000000)] [ad_core]
[Mon May 20 02:13:36.197 2013] 0.000 sec [all/5/ext 0 (0,1000000000)] [ad_core]
[Mon May 20 02:13:36.206 2013] 0.000 sec [all/4/ext 0 (0,1000000000)] [ad_core]
After 2AM, the 21 ads are not taken into account anymore ...
Could you please advice on how to track the problem ?
Is there any simple solution to maybe initialize the indexation ?
please find please what command "rake ts:index" returns me :
Generating Configuration to /home/kar/www/myapp/releases/20130520063222/config/development.sphinx.conf
Sphinx 2.0.4-release (r3135)
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)
using config file '/home/kar/www/myapp/releases/20130520063222/config/**development**.sphinx.conf'...
indexing index 'ad_core'...
WARNING: collect_hits: mem_limit=0 kb too low, increasing to 17152 kb
collected 0 docs, 0.0 MB
total 0 docs, 0 bytes
total 0.012 sec, 0 bytes/sec, 0.00 docs/sec
skipping non-plain index 'ad'...
total 1 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
total 5 writes, 0.000 sec, 0.1 kb/call avg, 0.0 msec/call avg
rotating indices: succesfully sent SIGHUP to searchd (pid=13500).
I think I have several problems here, the first one is that I'm taking development index file instead of production... I have to review my deployment procedure...
Related
When creating a dataset you are required to either upload the csv or xml file containing the data or link to the data file located on a server. When I attempt to upload a csv or xml file the upload process starts but the page eventually crash with the following message.
Server Error
An internal server error occurred
also see below error in /var/log/apache2/ckan_default.error.log
[Mon Aug 10 19:08:18.934269 2015] [:error] [pid 1243] 2015-08-10 19:08:18,934 INFO [ckan.lib.base] / render time 0.169 seconds
[Mon Aug 10 19:08:21.217534 2015] [:error] [pid 1243] 2015-08-10 19:08:21,217 INFO [ckan.lib.base] /api/i18n/en render time 0.002 seconds
[Mon Aug 10 19:08:23.832089 2015] [:error] [pid 1244] 2015-08-10 19:08:23,831 INFO [ckan.lib.base] /user/admin render time 0.413 seconds
[Mon Aug 10 19:08:24.430716 2015] [:error] [pid 1243] 2015-08-10 19:08:24,430 INFO [ckan.lib.base] /api/i18n/en render time 0.002 seconds
[Mon Aug 10 19:08:26.858892 2015] [:error] [pid 1243] 2015-08-10 19:08:26,858 WARNI [ckan.lib.maintain] Function get_action() in module ckan.lib.helpers has been deprecated and will be removed in a later release of ckan. helpers.get_action() is deprecated and will be removed in a future version of CKAN. Instead, please use the extra_vars param to render() in your controller to pass results from action functions to your templates.
[Mon Aug 10 19:08:27.080089 2015] [:error] [pid 1243] 2015-08-10 19:08:27,079 WARNI [ckan.lib.maintain] Function get_action() in module ckan.lib.helpers has been deprecated and will be removed in a later release of ckan. helpers.get_action() is deprecated and will be removed in a future version of CKAN. Instead, please use the extra_vars param to render() in your controller to pass results from action functions to your templates.
[Mon Aug 10 19:08:27.190495 2015] [:error] [pid 1243] 2015-08-10 19:08:27,190 INFO [ckan.lib.base] /dataset/grocery-prices render time 0.551 seconds
[Mon Aug 10 19:08:27.664450 2015] [:error] [pid 1244] 2015-08-10 19:08:27,664 INFO [ckan.lib.base] /api/i18n/en render time 0.002 seconds
[Mon Aug 10 19:08:30.847326 2015] [:error] [pid 1243] 2015-08-10 19:08:30,847 INFO [ckan.lib.base] /dataset/edit/grocery-prices render time 0.405 seconds
[Mon Aug 10 19:08:31.373743 2015] [:error] [pid 1244] 2015-08-10 19:08:31,373 INFO [ckan.lib.base] /api/i18n/en render time 0.002 seconds
[Mon Aug 10 19:08:34.934102 2015] [:error] [pid 1243] 2015-08-10 19:08:34,933 INFO [ckan.lib.base] /dataset/new render time 0.091 seconds
[Mon Aug 10 19:08:35.451324 2015] [:error] [pid 1244] 2015-08-10 19:08:35,451 INFO [ckan.lib.base] /dataset/new_resource/grocery-prices render time 0.412 seconds
[Mon Aug 10 19:08:36.271926 2015] [:error] [pid 1244] 2015-08-10 19:08:36,271 INFO [ckan.lib.base] /api/i18n/en render time 0.002 seconds
[Mon Aug 10 19:08:49.310932 2015] [:error] [pid 1244] 2015-08-10 19:08:49,310 INFO [ckan.lib.base] /api/2/util/resource/format_autocomplete render time 0.006 seconds
[Mon Aug 10 19:09:11.244287 2015] [:error] [pid 1243] /usr/lib/ckan/default/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py:79: SAWarning: Usage of the 'related attribute set' operation is not currently supported within the execution stage of the flush process. Results may not be consistent. Consider using alternative event listeners or connection-level operations instead.
[Mon Aug 10 19:09:11.244325 2015] [:error] [pid 1243] sess._flush_warning("related attribute set")
[Mon Aug 10 19:09:11.518274 2015] [:error] [pid 1243] 2015-08-10 19:09:11,518 INFO [ckan.lib.base] /dataset/new_resource/grocery-prices render time 0.176 seconds
[Mon Aug 10 19:09:11.975593 2015] [:error] [pid 1244] 2015-08-10 19:09:11,975 INFO [ckan.lib.base] /api/i18n/en render time 0.002 seconds
Not sure what caused this error and the logs do not provide enough information. The file is not deposited to the dataset directory location.
Using CKAN 2.3 on a Ubuntu 14.04 server running apache2
I have been getting a few bug reports for an access violation that occurs while registering Canvas classes at startup, more specifically performing an #IntfClear during TCustomDX10Context.CheckDevice.
Full bugreport:
date/time : 2014-06-04, 18:05:43, 594ms
computer name : <...>
user name : <...>
registered owner : <...>
operating system : Windows 7 x64 Service Pack 1 build 7601
system language : English
system up time : 54 minutes 48 seconds
program up time : 99 milliseconds
processors : 4x Intel(R) Core(TM) i5 CPU 661 # 3.33GHz
physical memory : 5055/8055 MB (free/total)
free disk space : (C:) 91.84 GB
display mode : 1920x1080, 32 bit
process id : $18c0
allocated memory : 71.76 MB
executable : <...>.exe
exec. date/time : 2014-05-29 22:36
version : 3.2.1.2
compiled with : Delphi XE3
madExcept version : 4.0.6
callstack crc : $077d2d6c, $53160fd9, $53160fd9
exception number : 1
exception class : EAccessViolation
exception message : Zugriffsverletzung bei Adresse 077D2D6C. Lesen von Adresse 000000AE.
main thread ($d0):
077d2d6c +000 ???
0040e028 +010 <...>.exe System 191 +0 #IntfClear
0096af25 +0cd <...>.exe FMX.Context.DX10 TCustomDX10Context.CheckDevice
00976559 +005 <...>.exe FMX.Canvas.D2D RegisterCanvasClasses
00990c6c +000 <...>.exe FMX.Platform.Win 3404 +0 TPlatformWin.RegisterCanvasClasses
009d6312 +056 <...>.exe FMX.Types TCanvasManager.GetDefaultCanvas
009d62aa +00e <...>.exe FMX.Types TCanvasManager.CreateFromWindow
00a0574e +026 <...>.exe FMX.Forms TCustomForm.CreateHandle
009fff91 +0b9 <...>.exe FMX.Forms TCommonCustomForm.InitializeNewForm
00a054fa +006 <...>.exe FMX.Forms TCustomForm.InitializeNewForm
009ffce4 +094 <...>.exe FMX.Forms TCommonCustomForm.Create
00a056c6 +016 <...>.exe FMX.Forms TCustomForm.Create
009fb947 +057 <...>.exe FMX.Forms TApplication.CreateForm
009fbb0b +05f <...>.exe FMX.Forms TApplication.RealCreateForms
00988861 +011 <...>.exe FMX.Platform.Win 461 +1 TPlatformWin.Run
009fe821 +0d1 <...>.exe FMX.Forms TApplication.Run
00beccf4 +2c0 <...>.exe <...> 106 +39 initialization
75863388 +010 kernel32.dll BaseThreadInitThunk
thread $186c:
778a0156 +0e ntdll.dll NtWaitForMultipleObjects
76e715f1 +fa KERNELBASE.dll WaitForMultipleObjectsEx
758619f3 +89 kernel32.dll WaitForMultipleObjectsEx
75330864 +00 USER32.dll MsgWaitForMultipleObjectsEx
75330b64 +1a USER32.dll MsgWaitForMultipleObjects
004a42e9 +0d <...>.exe madExcept CallThreadProcSafe
004a434e +32 <...>.exe madExcept ThreadExceptFrame
75863388 +10 kernel32.dll BaseThreadInitThunk
>> created by main thread ($d0) at:
734878e1 +00 gdiplus.dll
thread $1288: <priority:1>
7789f8ca +0e ntdll.dll NtWaitForSingleObject
76e714a5 +92 KERNELBASE.dll WaitForSingleObjectEx
7586118f +3e kernel32.dll WaitForSingleObjectEx
75861143 +0d kernel32.dll WaitForSingleObject
004a42e9 +0d <...>.exe madExcept CallThreadProcSafe
004a434e +32 <...>.exe madExcept ThreadExceptFrame
75863388 +10 kernel32.dll BaseThreadInitThunk
>> created by main thread ($d0) at:
54377390 +00 nvwgf2um.dll
thread $28c:
778a1f3f +0b ntdll.dll NtWaitForWorkViaWorkerFactory
75863388 +10 kernel32.dll BaseThreadInitThunk
thread $1bcc:
778a1f3f +0b ntdll.dll NtWaitForWorkViaWorkerFactory
75863388 +10 kernel32.dll BaseThreadInitThunk
modules:
00400000 <...>.exe 3.2.1.2 C:\<...>
10000000 nvspcap.dll 10.11.15.0 C:\Windows\system32
53c10000 nvwgf2um.dll 9.18.13.2723 C:\Windows\system32
603d0000 AcGenral.DLL 6.1.7601.18128 C:\Windows\AppPatch
606d0000 atkdx11disp.dll 7.14.10.305 C:\Windows\system32
60700000 d3d10_1core.dll 6.2.9200.16492 C:\Windows\system32
60750000 d3d10_1.dll 6.2.9200.16492 C:\Windows\system32
60780000 security.dll 6.1.7600.16385 C:\Windows\system32
60790000 FaultRep.dll 6.1.7601.17514 C:\Windows\system32
607f0000 sfc_os.DLL 6.1.7600.16385 C:\Windows\system32
60800000 sfc.dll 6.1.7600.16385 C:\Windows\system32
6ed00000 d3d11.dll 6.2.9200.16570 C:\Windows\system32
706d0000 dxgi.dll 6.2.9200.16492 C:\Windows\system32
70ca0000 propsys.dll 7.0.7601.17514 C:\Windows\system32
70e00000 d3d9.dll 6.1.7601.17514 C:\Windows\system32
70fd0000 nvapi.dll 9.18.13.2723 C:\Windows\system32
71a50000 comctl32.dll 6.10.7601.17514 C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_6.0.7601.17514_none_41e6975e2bd6f2b2
71bf0000 winmm.dll 6.1.7601.17514 C:\Windows\system32
72150000 samcli.dll 6.1.7601.17514 C:\Windows\system32
73210000 MSACM32.dll 6.1.7600.16385 C:\Windows\system32
73230000 d3d8thk.dll 6.1.7600.16385 C:\Windows\system32
73440000 gdiplus.dll 6.1.7601.18120 C:\Windows\WinSxS\x86_microsoft.windows.gdiplus_6595b64144ccf1df_1.1.7601.18120_none_72d2e82386681b36
73890000 apphelp.dll 6.1.7601.17514 C:\Windows\system32
738e0000 MPR.dll 6.1.7600.16385 C:\Windows\system32
73ca0000 wsock32.dll 6.1.7600.16385 C:\Windows\system32
73f20000 SECUR32.DLL 6.1.7601.18443 C:\Windows\system32
74090000 dwmapi.dll 6.1.7600.16385 C:\Windows\system32
74100000 UxTheme.dll 6.1.7600.16385 C:\Windows\system32
742e0000 comctl32.dll 5.82.7601.18201 C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_5.82.7601.18201_none_ec80f00e8593ece5
74f90000 profapi.dll 6.1.7600.16385 C:\Windows\system32
74fa0000 USERENV.dll 6.1.7601.17514 C:\Windows\system32
750b0000 ntmarta.dll 6.1.7600.16385 C:\Windows\system32
750e0000 sophos_detoured.dll 10.3.3.128 C:\Program Files (x86)\Sophos\Sophos Anti-Virus
75130000 winspool.drv 6.1.7601.17514 C:\Windows\system32
75190000 version.dll 6.1.7600.16385 C:\Windows\system32
752a0000 CRYPTBASE.dll 6.1.7600.16385 C:\Windows\syswow64
752b0000 SspiCli.dll 6.1.7601.18443 C:\Windows\syswow64
75310000 USER32.dll 6.1.7601.17514 C:\Windows\syswow64
75410000 iertutil.dll 11.0.9600.17041 C:\Windows\syswow64
75630000 SETUPAPI.dll 6.1.7601.17514 C:\Windows\syswow64
757d0000 PSAPI.DLL 6.1.7600.16385 C:\Windows\syswow64
757e0000 api-ms-win-downlevel-normaliz-l1-1-0.dll 6.2.9200.16492 C:\Windows\syswow64
757f0000 NSI.dll 6.1.7600.16385 C:\Windows\syswow64
75800000 WLDAP32.dll 6.1.7601.17514 C:\Windows\syswow64
75850000 kernel32.dll 6.1.7601.18409 C:\Windows\syswow64
759f0000 DEVOBJ.dll 6.1.7601.17621 C:\Windows\syswow64
75a10000 RPCRT4.dll 6.1.7601.18205 C:\Windows\syswow64
75b00000 WININET.dll 11.0.9600.17041 C:\Windows\syswow64
75cc0000 shell32.dll 6.1.7601.18429 C:\Windows\syswow64
76910000 WS2_32.dll 6.1.7601.17514 C:\Windows\syswow64
769b0000 api-ms-win-downlevel-advapi32-l1-1-0.dll 6.2.9200.16492 C:\Windows\syswow64
769c0000 MSCTF.dll 6.1.7600.16385 C:\Windows\syswow64
76a90000 comdlg32.dll 6.1.7601.17514 C:\Windows\syswow64
76b10000 api-ms-win-downlevel-shlwapi-l1-1-0.dll 6.2.9200.16492 C:\Windows\syswow64
76b50000 msvcrt.dll 7.0.7601.17744 C:\Windows\syswow64
76c00000 api-ms-win-downlevel-ole32-l1-1-0.dll 6.2.9200.16492 C:\Windows\syswow64
76c10000 ole32.dll 6.1.7601.17514 C:\Windows\syswow64
76d70000 normaliz.DLL 6.1.7600.16385 C:\Windows\syswow64
76d80000 CLBCatQ.DLL 2001.12.8530.16385 C:\Windows\syswow64
76e10000 CFGMGR32.dll 6.1.7601.17621 C:\Windows\syswow64
76e40000 sechost.dll 6.1.7600.16385 C:\Windows\SysWOW64
76e60000 KERNELBASE.dll 6.1.7601.18409 C:\Windows\syswow64
76eb0000 IMM32.DLL 6.1.7601.17514 C:\Windows\system32
76f10000 USP10.dll 1.626.7601.18009 C:\Windows\syswow64
76fb0000 WINTRUST.dll 6.1.7601.18205 C:\Windows\syswow64
76fe0000 api-ms-win-downlevel-version-l1-1-0.dll 6.2.9200.16492 C:\Windows\syswow64
76ff0000 CRYPT32.dll 6.1.7601.18277 C:\Windows\syswow64
77110000 MSASN1.dll 6.1.7601.17514 C:\Windows\syswow64
77120000 ADVAPI32.dll 6.1.7601.18247 C:\Windows\syswow64
771c0000 oleaut32.dll 6.1.7601.17676 C:\Windows\syswow64
77250000 SHLWAPI.dll 6.1.7601.17514 C:\Windows\syswow64
772b0000 api-ms-win-downlevel-user32-l1-1-0.dll 6.2.9200.16492 C:\Windows\syswow64
772c0000 GDI32.dll 6.1.7601.18275 C:\Windows\syswow64
77350000 urlmon.dll 11.0.9600.17041 C:\Windows\syswow64
77850000 LPK.dll 6.1.7601.18177 C:\Windows\syswow64
77880000 ntdll.dll 6.1.7601.18247 C:\Windows\SysWOW64
processes:
0000 Idle 0 0 0
0004 System 0 0 0
0120 smss.exe 0 0 0
01c8 csrss.exe 0 0 0
0210 wininit.exe 0 0 0
0224 csrss.exe 1 0 0
0250 services.exe 0 0 0
0260 lsass.exe 0 0 0
0268 lsm.exe 0 0 0
02d4 winlogon.exe 1 0 0
02f4 svchost.exe 0 0 0
0330 ATKFUSService.exe 0 0 0
0344 nvvsvc.exe 0 0 0
0360 nvSCPAPISvr.exe 0 0 0
038c svchost.exe 0 0 0
03c8 atiesrxx.exe 0 0 0
03f8 svchost.exe 0 0 0
014c svchost.exe 0 0 0
0298 svchost.exe 0 0 0
0228 svchost.exe 0 0 0
0454 audiodg.exe 0 0 0
04ac SavService.exe 0 0 0
04b8 atieclxx.exe 1 0 0
04c8 nvxdsync.exe 1 0 0
04d0 nvvsvc.exe 1 0 0
0614 ATKFastUserSwitching.exe 1 18 11 normal
0660 dwm.exe 1 18 2 high
0678 explorer.exe 1 356 266 normal
0790 svchost.exe 0 0 0
07e4 svchost.exe 0 0 0
0490 spoolsv.exe 0 0 0
082c taskhost.exe 1 25 23 normal
08d0 NvTmru.exe 1 9 4 normal C:\Program Files (x86)\NVIDIA Corporation\NVIDIA Update Core
0924 NvBackend.exe 1 9 4 normal C:\Program Files (x86)\NVIDIA Corporation\Update Core
0944 ANIWConnService.exe 0 0 0
0980 ANIWZCSdS.exe 0 0 0
099c RtlService.exe 0 0 0
09dc flux.exe 1 22 13 normal C:\Users\<...>\AppData\Local\FluxSoftware\Flux
09f4 SpotifyWebHelper.exe 1 9 3 normal C:\Users\<...>\AppData\Roaming\Spotify\Data
0a3c nvtray.exe 1 81 5 normal
0a74 BCUService.exe 0 0 0
0aac FABS.exe 0 0 0
0ae0 RtWLan.exe 1 0 0
0b48 NvNetworkService.exe 0 0 0
0ba8 nvstreamsvc.exe 0 0 0
0610 spotify.exe 1 222 81 normal C:\Users\<...>\AppData\Roaming\Spotify
046c razerhid.exe 1 9 5 normal C:\Program Files (x86)\Razer\Arctosa
09c8 BCU.exe 1 9 4 normal C:\Program Files (x86)\DeviceVM\Browser Configuration Utility
0b9c jusched.exe 1 9 2 normal C:\Program Files (x86)\Common Files\Java\Java Update
0b84 PnkBstrA.exe 0 0 0
0910 PnkBstrB.exe 0 0 0
0920 SAVAdminService.exe 0 0 0
09d4 ALsvc.exe 0 0 0
0c70 swc_service.exe 0 0 0
0c98 svchost.exe 0 0 0
0cac swi_service.exe 0 0 0
0cec TurboBoost.exe 0 0 0
0d30 ALMon.exe 1 47 40 normal C:\Program Files (x86)\Sophos\AutoUpdate
0d60 WLIDSVC.EXE 0 0 0
0e04 WLIDSVCM.EXE 0 0 0
0efc WmiPrvSE.exe 0 0 0
0ff0 SearchIndexer.exe 0 0 0
0af0 svchost.exe 0 0 0
11b8 nvstreamsvc.exe 1 0 0
11cc conhost.exe 1 0 0
127c wmpnetwk.exe 0 0 0
130c svchost.exe 0 0 0
11d0 svchost.exe 0 0 0
1304 SpotifyHelper.exe 1 52 1 normal C:\Users\<...>\AppData\Roaming\Spotify\Data
1134 SpotifyHelper.exe 1 33 1 normal C:\Users\<...>\AppData\Roaming\Spotify\Data
15c8 SpotifyHelper.exe 1 13 3 normal C:\Users\<...>\AppData\Roaming\Spotify\Data
17a8 dllhost.exe 0 0 0
151c LMS.exe 0 0 0
14b0 svchost.exe 0 0 0
1018 UNS.exe 0 0 0
059c SpotifyHelper.exe 1 142 1 normal C:\Users\<...>\AppData\Roaming\Spotify\Data
08d8 SpotifyHelper.exe 1 11 1 normal C:\Users\<...>\AppData\Roaming\Spotify\Data
0abc chrome.exe 1 375 53 normal C:\Users\<...>\AppData\Local\Google\Chrome\Application
0eb4 chrome.exe 1 12 5 normal C:\Users\<...>\AppData\Local\Google\Chrome\Application
12d8 chrome.exe 1 11 1 normal C:\Users\<...>\AppData\Local\Google\Chrome\Application
0da8 chrome.exe 1 719 1 normal C:\Users\<...>\AppData\Local\Google\Chrome\Application
10bc chrome.exe 1 11 1 normal C:\Users\<...>\AppData\Local\Google\Chrome\Application
1114 chrome.exe 1 9 1 normal C:\Users\<...>\AppData\Local\Google\Chrome\Application
0968 chrome.exe 1 11 1 normal C:\Users\<...>\AppData\Local\Google\Chrome\Application
1220 chrome.exe 1 10 1 normal C:\Users\<...>\AppData\Local\Google\Chrome\Application
1974 jucheck.exe 1 12 6 normal C:\Program Files (x86)\Common Files\Java\Java Update
0778 chrome.exe 1 251 1 below normal C:\Users\<...>\AppData\Local\Google\Chrome\Application
1144 WINWORD.EXE 1 474 118 normal C:\Program Files (x86)\Microsoft Office\Office14
0cc4 OSPPSVC.EXE 0 0 0
17b8 splwow64.exe 1 4 2 normal
1410 SearchProtocolHost.exe 0 0 0
1258 SearchFilterHost.exe 0 0 0 idle
18e0 chrome.exe 1 209 1 below normal C:\Users\<...>\AppData\Local\Google\Chrome\Application
18c0 <...>.exe 1 36 34 normal C:\<...>
cpu registers:
eax = 00000000
ebx = 00000000
ecx = 00000000
edx = 00000008
esi = 0018fca4
edi = 077d2d6c
eip = 077d2d6c
esp = 0018fc60
ebp = 077d2444
stack dump:
0018fc60 a4 fc 18 00 44 24 7d 07 - 6c 2d 7d 07 44 24 00 06 ....D$}.l-}.D$..
0018fc70 2c 63 de 6e 48 81 7d 07 - 6c 2d 7d 07 00 00 00 00 ,c.nH.}.l-}.....
0018fc80 01 00 00 00 00 00 00 00 - ac fc 18 00 42 00 d2 6e ............B..n
0018fc90 6c 2d 7d 07 01 00 00 00 - 58 25 7d 07 44 24 7d 07 l-}.....X%}.D$}.
0018fca0 3c 27 7d 07 3c 27 7d 07 - 3c a6 d0 6e c0 fc 18 00 <'}.<'}.<..n....
0018fcb0 1b ff d1 6e 6c 2d 7d 07 - 44 24 7d 07 a0 00 00 00 ...nl-}.D$}.....
0018fcc0 cc fc 18 00 43 25 d1 6e - 3c 27 7d 07 e0 fc 18 00 ....C%.n<'}.....
0018fcd0 6d 6d d1 6e fc 26 7d 07 - 00 00 00 00 3c 24 7d 07 mm.n.&}.....<$}.
0018fce0 ec fc 18 00 f1 26 d1 6e - 10 24 7d 07 f8 fc 18 00 .....&.n.$}.....
0018fcf0 52 c1 d4 6e 3c 24 7d 07 - 14 fd 18 00 5f 39 d1 6e R..n<$}....._9.n
0018fd00 10 24 7d 07 04 34 d1 6e - 10 24 7d 07 66 38 d1 6e .$}..4.n.$}.f8.n
0018fd10 10 24 7d 07 28 fd 18 00 - 5f 34 d1 6e 00 00 00 00 .$}.(..._4.n....
0018fd20 16 00 00 00 90 05 00 00 - 34 fd 18 00 c5 25 d1 6e ........4....%.n
0018fd30 10 24 7d 07 40 fd 18 00 - d1 f5 d4 6e 3c 27 7d 07 .$}.#......n<'}.
0018fd40 84 fd 18 00 2b e0 40 00 - 74 27 7d 07 80 fd 18 00 ....+.#.t'}.....
0018fd50 2a af 96 00 60 fd 18 00 - 9c 98 40 00 84 fd 18 00 *...`.....#.....
0018fd60 6c fd 18 00 9c 98 40 00 - 84 fd 18 00 94 fd 18 00 l.....#.........
0018fd70 9c 98 40 00 84 fd 18 00 - a0 00 00 00 00 00 75 60 ..#...........u`
0018fd80 00 00 00 00 bc fd 18 00 - 5e 65 97 00 71 0c 99 00 ........^e..q...
0018fd90 15 63 9d 00 10 fe 18 00 - 9c 98 40 00 bc fd 18 00 .c........#.....
disassembling:
[...]
0040e01c jz loc_40e02c
0040e01e mov dword ptr [eax], 0
0040e024 push eax
0040e025 push edx
0040e026 mov eax, [edx]
0040e028 > call dword ptr [eax+8]
0040e02b pop eax
0040e02c ret
This is by no means happening on all machines but seems to be a very rare issue.
EDIT:
Following the suggestion of #Graymatter I've been able to establish an SSCCE.
program SSCCE;
{$APPTYPE CONSOLE}
{$R *.res}
uses System.SysUtils, FMX.Context.DX10;
begin
try
TCustomDX10Context.CheckDevice;
WriteLn('Everything went fine, guess we have to dig deeper.');
except
on E: Exception do Writeln(E.ClassName, ': ', E.Message);
end;
Write('Press enter to exit.');
ReadLn;
end.
The output produced is Zugriffsverletzung bei Adresse 05C1006C. Lesen von Adresse 06F29ED3 Any suggestions as to what to do about that ?
Here is my problem:
top - 11:32:47 up 22:20, 2 users, load average: 0.03, 0.72, 1.27
Tasks: 112 total, 1 running, 110 sleeping, 1 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 8193844k total, 7508292k used, 685552k free, 80636k buffers
Swap: 2102456k total, 15472k used, 2086984k free, 7070220k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
28555 root 20 0 57424 38m 1492 S 0 0.5 0:06.38 bash
28900 root 20 0 39488 7732 3176 T 0 0.1 0:03.12 python
28553 root 20 0 72132 5052 2600 S 0 0.1 0:00.22 sshd
28859 root 20 0 70588 3424 2584 S 0 0.0 0:00.06 sshd
29404 root 20 0 70448 3320 2600 S 0 0.0 0:00.06 sshd
28863 root 20 0 42624 2188 1472 S 0 0.0 0:00.02 sftp-server
29406 root 20 0 19176 1984 1424 S 0 0.0 0:00.00 bash
2854 root 20 0 115m 1760 488 S 0 0.0 5:37.02 rsyslogd
29410 root 20 0 19064 1400 1016 R 0 0.0 0:05.14 top
3111 ntp 20 0 22484 604 460 S 0 0.0 10:26.79 ntpd
3134 proftpd 20 0 64344 452 280 S 0 0.0 6:29.16 proftpd
2892 root 20 0 49168 356 232 S 0 0.0 0:31.58 sshd
1 root 20 0 27388 284 132 S 0 0.0 0:01.38 init
3121 root 20 0 4308 248 172 S 0 0.0 0:16.48 mdadm
As you can see 7.5 GB of memory is used, but there is no process that use it.
How it can be, and how to fix this?
Thanks for answer.
www.linuxatemyram.com
It's too good of a site to ruin by copy/pasting the entire contents here.
in order to see all process you can use that command:
ps aux
and then try to sort with different filters
ps faux
Hope that helps.
If your system starts using the swap file - then you have high memory load. Depends on the file system, programs that you use - linux system may allocate all of your system memory - but that doesn't mean that they are using it.
Lots of ubuntu and debian servers that we use have free memory 32 or 64 mb but don't use swap.
I'm not Linux-gure however, so please someone to correct me if I'm wrong :)
I don't have a Linux box handy to experiment, but it looks like you can sort top's output with interactive commands, so you could bring the biggest memory users to the top. Check the man page and experiment.
Update: In the version of top I have (procps 3.2.7), you can hit "<" and ">" to change the field it's sorting by. Doesn't actually say what field it is, you have to look at how the display is changing. It's not hard once you experiment a little.
However, Arrowmaster's point (that it's probably being used for cache) is a better answer. Use "free" to see how much is being used.
I had a similar problem. I was running Raspbian on a Pi B+ with a TP-Link USB Wireless LAN stick connected. The stick caused a problem which resulted in nearly all memory being consumed on system start (around 430 of 445 MB). Just like in your case, the running processes did not consume that much memory. When I removed the stick and rebooted everything was fine, just 50 MB memory consumption.
I have a Rails 2.3.8 app hosted and running on slicehost (256M). I am not familiar at all with the back-end, I basically followed the steps from the slicehost tutorials to install Apache. The memory usage being very high, I then changed my Apache conf file to reduce the MaxClient number to 10... but my slice is still swapping.
Here is what the memory usage I get after just a few clicks on my site:
top - 23:57:12 up 28 min, 2 users, load average: 0.43, 0.54, 0.30
Tasks: 79 total, 1 running, 78 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.1%us, 0.1%sy, 0.0%ni, 97.8%id, 0.1%wa, 0.0%hi, 0.0%si, 2.0%st
Mem: 262364k total, 258656k used, 3708k free, 260k buffers
Swap: 524280k total, 262772k used, 261508k free, 6328k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4004 web-app 20 0 178m 72m 1888 S 0 28.4 0:04.38 ruby1.8
4001 web-app 20 0 172m 61m 1932 S 0 24.2 0:02.72 ruby1.8
3941 root 20 0 164m 57m 1672 S 0 22.5 0:21.44 ruby
3990 web-app 20 0 209m 21m 1696 S 0 8.4 0:18.00 ruby1.8
3950 web-app 20 0 165m 7464 1548 S 0 2.8 0:20.40 ruby1.8
3684 mysql 20 0 224m 6504 2084 S 0 2.5 0:14.34 mysqld
3938 root 20 0 53632 3048 1036 S 1 1.2 0:01.50 starling
3839 root 20 0 243m 1456 1248 S 0 0.6 0:00.34 apache2
3897 www-data 20 0 243m 1452 1072 S 0 0.6 0:00.04 apache2
3894 www-data 20 0 243m 1368 1008 S 0 0.5 0:00.04 apache2
3895 www-data 20 0 243m 1220 960 S 0 0.5 0:00.02 apache2
3888 root 20 0 46520 1204 1100 S 0 0.5 0:02.29 ruby1.8
3866 root 20 0 17648 1184 896 S 0 0.5 0:00.08 bash
3896 www-data 20 0 243m 1180 952 S 0 0.4 0:00.00 apache2
3964 www-data 20 0 243m 1164 956 S 0 0.4 0:00.02 apache2
3892 www-data 20 0 243m 1132 956 S 0 0.4 0:00.00 apache2
3948 www-data 20 0 243m 1132 956 S 0 0.4 0:00.00 apache2
3962 www-data 20 0 243m 1132 956 S 0 0.4 0:00.02 apache2
3963 www-data 20 0 243m 1132 956 S 0 0.4 0:00.00 apache2
3965 www-data 20 0 243m 1080 888 S 0 0.4 0:00.00 apache2
3887 root 20 0 89008 960 796 S 0 0.4 0:00.00 ApplicationPool
I'm not sure what to do next... I could upgrade to a larger slice but for now I have almost no traffic on this app, so I think it's more a problem with my configuration or maybe my code?
Any concrete recommendations would be welcome!
Thanks
It looks like your rails app is using all your available memory. I would recommend three things:
Upgrade the memory on your server. 256MB is not very much for a Rails app. Going to 512 may alleviate your problem. If that solves it, you then need to consider the additional cost ($18/mo) vs how much time it will take to track down performance issues.
Profile your application to figure out which requests are consuming the most memory. This is likely going to be places where you're finding a lot of records and possibly including some associated tables too. There are a couple of tools out there to help you narrow down possible trouble areas. I've used oink but there are definitely others. Once you figure out where the problems are, you can make some tweaks to try and reduce the memory usage.
Assuming you're using Passenger with Apache, you can reduce the number of concurrent requests in the Passenger config file. This might be useful for that https://serverfault.com/questions/15350/running-ruby-on-rails-app-on-apache-passenger-to-much-memory
In short, 256MB is tight for a Rails application. You did not really give any specifics on how you are running rails, but I assume you are using Apache with the Passenger module. The Passenger module can be configured on how many instances it keeps running. You have 4 ruby instances running under the web-app account. I guess those come from Passenger. In the configuration, you can limit how many instances Passenger starts. This will reduce the memory requirements.
On the other hand, when working with only 256MB, and when you are only hosting 1 rails application, it might be better to go for another setup. The setup that I used myself before was an Nginx web server, and a mongrel cluster with 2 mongrels (on 192MB, and application was only for testing purposes). Basically that means that at any one time, you can process 2 (and only 2) rails requests in parallel. The setup is maybe a bit harder than Apache+Passenger, but definitely not difficult. I think that is a more performant solution when you stick with the 256MB.
I have a rails app (2.3.5) running on a VPS with 4 cores # 2 GHz and 4GB memory. I am running nginx (0.7.61) and phusion passenger(2.2.14) on Ruby Enterprise (1.8.7-2010.01) with the max pool size set at 30. My problem is that it seems as if every ruby process that is executing a rails request runs at near 100% cpu. If I run TOP they drop off every time the display refreshes so they are not getting hung, but they are still running at 100%.
Is there any way I can bring this down? Or at least figure out what portion of code is spiking the CPU? Is this a normal behavior?
Here is the TOP output:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2427 psadmin 25 0 91904 76m 2696 R 100 1.9 739:05.96 Rails: /var/www/apps/main_rails_app/current
3457 psadmin 25 0 98180 82m 2532 R 100 2.0 711:21.91 Rails: /var/www/apps/main_rails_app/current
2415 psadmin 25 0 93952 77m 2708 R 99 1.9 727:49.31 Rails: /var/www/apps/main_rails_app/current
3455 psadmin 25 0 99204 83m 2528 R 69 2.0 726:04.70 Rails: /var/www/apps/main_rails_app/current
2791 psadmin 16 0 98044 81m 2492 S 31 2.0 0:10.16 Rails: /var/www/apps/main_rails_app/current
8034 psadmin 15 0 8160 3656 1772 S 1 0.1 0:35.39 nginx: worker process
8035 psadmin 15 0 8324 3696 1732 S 0 0.1 0:31.34 nginx: worker process
2588 psadmin 15 0 197m 183m 2712 S 0 4.5 1:02.16 Rails: /var/www/apps/main_rails_app/current
Thanks!
Edit: Tried strace with follow forks as mentioned below. This is the output that is dumped over and over:
sudo strace -f -p 3455
clock_gettime(CLOCK_MONOTONIC, {394577, 508326476}) = 0
select(0, [], [], [], {0, 0}) = 0 (Timeout)
--- SIGVTALRM (Virtual timer expired) # 0 (0) ---
sigreturn()
check your logs for suspicious behavior. In general rails does suck a bunch of cpu though...you could also try pointing strace at the offending pids.