Any ideas why Thinking Sphinx Rake tasks are not running? - ruby-on-rails

Question

I'm finding that Thinking Sphinx sometimes errors out when I try to run its Rake tasks. Sometimes the tasks work fine, and sometimes I get errors like the one below.
I'm running the tasks as a normal user, not root. Not using sudo.
In the example below, searchd is still running after the Rake task fails.
I've also seen similar errors with ts:rebuild.
Has anyone else seen this?
I'm using Sphinx 0.9.9-release (r2117)
I'm using Thinking Sphinx 1.4.4 instead of the latest version because I'm on Rails 2.3.11.
$ rake ts:stop RAILS_ENV=production
(in /var/www/blog_app/releases/20110425053509)
rake aborted!
execution expired
(See full trace by running task with --trace)
[ethan#apphost current]$ rake ts:stop RAILS_ENV=production --trace
(in /var/www/blog_app/releases/20110425053509)
** Invoke ts:stop (first_time)
** Invoke thinking_sphinx:stop (first_time)
** Invoke thinking_sphinx:app_env (first_time)
** Execute thinking_sphinx:app_env
** Invoke environment (first_time)
** Execute environment
** Execute thinking_sphinx:stop
rake aborted!
execution expired
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/1.8/timeout.rb:60
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/thinking-sphinx-1.4.4/lib/thinking_sphinx/tasks.rb:58
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `execute'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `execute'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:597:in `invoke_with_call_chain'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/1.8/monitor.rb:242:in `synchronize'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:607:in `invoke_prerequisites'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:604:in `each'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:604:in `invoke_prerequisites'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:596:in `invoke_with_call_chain'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/1.8/monitor.rb:242:in `synchronize'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2051:in `invoke_task'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `each'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2023:in `top_level'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2001:in `run'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in `run'
/opt/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/rake-0.8.7/bin/rake:31
/opt/ruby/bin/rake:19:in `load'
/opt/ruby/bin/rake:19

Answer 1

I believe this error is caused when you have a search daemon already running.
Try
ps | grep searchd
if that gives you something like this...
73470 ttys002 0:00.03 searchd --pidfile --config /Users/dwickwire/Projects/production/config/development.sphinx.conf
81907 ttys002 0:00.00 grep searchd
then kill the searchd
sudo kill -9 process_id
process_id in my case would be 73470
Then try rake ts:rebuild again.

Answer 2

I tried command 'ps | grep searchd' to get process ID, but I didn't get it.
Then I tried command 'pgrep searchd', it gives me process ID.
I killed that process and ran command 'rake ts:rebuild'. It worked.
I blogged about the entire experience here (disclaimer: I'm the author).

Answer 3

In thinking_sphinx 1.4.4 there is a 5 second timeout on rake ts:stop
If rake ts:stop takes close to or longer than 5 seconds, ruby will throw an exception.
I'm using the thinking_sphinx 1.3.18 in the meantime. (there is no timeout on 1.3.18).
Also I have submitted a patch to the github repository and the timeout will be configurable with a sphinx.yml configuration variable stop_timeout.
NOTE: rake ts:rebuild == rake ts:stop; rake ts:index; rake ts:start

Answer 4

I guess after deploy by capistrano you didn't run rake thinking_sphinx:configure
it would generate sphinx config file in config/production.sphinx.conf
try add pidfile to config/sphinx.yml
production:
...
pid_file: "/tmp/searchd.pid"
it will make sure to find pid of searchd to kill
and added following code to config/deploy.rb
namespace :my_tasks do
task :generate_sphinx_config, :roles => [:web] do
run "cd #{latest_release} && RAILS_ENV=#{rails_env} bundle exec rake thinking_sphinx:configure"
end
end
....
after :deploy, "my_tasks:generate_sphinx_config"
to generate sphinx config file in each release folder

Answer 5

The Problem
This problem occurs if your searchd crashed or you killed it manually. On Windows it doesn't overwrite your searchd.<environment>.pid, so still the old PID is saved and ThinkingSphinx doesn't check the real running processes in your taskmanager. So all these commands (rake ts:restart, rake ts:rebuild, ...) won't work, rake will always abort and you get the error rake aborted, ... searchd already running
The solution
Delete your path-to/your-app/log/searchd.<environment>.pid.
<environment> may be development or production, depending on your current stage.
Hope this answer will help the googler's.