I got a simple and free heroku app running where I added redis/sidekiq in order to send out mails in the background. I set everything up and once I start the worker, I get following error message in a loop:
Error fetching job: ERR max number of clients reached
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:114:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:95:in `block in connect'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:280:in `with_reconnect'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:93:in `connect'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:351:in `ensure_connected'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:208:in `block in process'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:293:in `logging'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:207:in `process'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:113:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:196:in `block in call_with_timeout'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:267:in `with_socket_timeout'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:195:in `call_with_timeout'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis.rb:1097:in `block in _bpop'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis.rb:57:in `block in synchronize'
/app/vendor/ruby-2.2.2/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis.rb:57:in `synchronize'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis.rb:1094:in `_bpop'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis.rb:1139:in `brpop'
/app/vendor/bundle/ruby/2.2.0/gems/redis-namespace-1.5.2/lib/redis/namespace.rb:393:in `call_with_namespace'
/app/vendor/bundle/ruby/2.2.0/gems/redis-namespace-1.5.2/lib/redis/namespace.rb:290:in `method_missing'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/fetch.rb:35:in `block in retrieve_work'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq.rb:84:in `block in redis'
/app/vendor/bundle/ruby/2.2.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:64:in `block (2 levels) in with'
/app/vendor/bundle/ruby/2.2.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in `handle_interrupt'
/app/vendor/bundle/ruby/2.2.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in `block in with'
/app/vendor/bundle/ruby/2.2.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `handle_interrupt'
/app/vendor/bundle/ruby/2.2.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `with'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq.rb:81:in `redis'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/fetch.rb:35:in `retrieve_work'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:85:in `get_one'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:95:in `fetch'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:78:in `process_one'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:67:in `run'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/util.rb:16:in `watchdog'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/util.rb:24:in `block in safe_thread'
sidekiq.yml:
---
:queues:
- default
- mailers
sidekiq.rb:
Sidekiq.configure_server do |config|
config.redis = { :namespace => "mynamespace" }
end
Sidekiq.configure_client do |config|
config.redis = { :namespace => "mynamespace" }
end
require "sidekiq/web"
Sidekiq::Web.app_url = "/"
Sidekiq::Web.use(Rack::Auth::Basic, "Application") do |username, password|
username == ENV.fetch("SIDEKIQ_WEB_USERNAME") &&
password == ENV.fetch("SIDEKIQ_WEB_PASSWORD")
end
Anyone an idea what's wrong here? If you need any further info on the config files or something, let me know.
Changing sidekiq.yml to
---
:queues:
- default
- mailers
:concurrency: 5
did the trick. Setting concurrency to 9 did not for some reason even though heroku allows 10 for the redis nano addon.
This was the best solution that I found. If you are using the free heroku plan and the free redis nano plan. All the connections have to add up to 10 or less. To make sure that the connections add up check out this blog post. http://manuelvanrijn.nl/blog/2012/11/13/sidekiq-on-heroku-with-redistogo-nano/
He explains how everything adds up.
The writer also has a calculator that will help to calculate what the configuration should be for your particular server, sidekiq etc. It also tells you where the information needs to be added.
I was stuck on this too hope it saves somebody some time :)
Related
I have an Expenditure model:
class Expenditure < ApplicationRecord
multi_tenant :company
after_commit :related_reindex
def related_reindex
ExpenditureRelatedReindex.perform_async(id)
end
end
Here is my worker expenditure_related_reindex.rb:
class ExpenditureRelatedReindex
include Sidekiq::Worker
sidekiq_options :queue => :critical, :retry => true, :backtrace => true
def perform(record_id)
e = Expenditure.find(record_id)
MultiTenant.with(e.company) do
return unless e
e.expenditure_items.each(&:reindex)
e.children&.each(&:reindex)
end
end
end
The reindex can take some time so I want these to spin off to SideKiq. I have sime multi tenant code I should mention but I don't think it's the issue. After the record is updated I get:
NameError: uninitialized constant ExpenditureRelatedReindex
Did you mean? ExpenditureItemsHelper
from sidekiq/processor.rb:268:in `const_get'
from sidekiq/processor.rb:268:in `constantize'
from sidekiq/processor.rb:132:in `block (5 levels) in dispatch'
from sidekiq/rails.rb:43:in `block in call'
from active_support/execution_wrapper.rb:87:in `wrap'
from active_support/reloader.rb:73:in `block in wrap'
from active_support/execution_wrapper.rb:87:in `wrap'
from active_support/reloader.rb:72:in `wrap'
from sidekiq/rails.rb:42:in `call'
from sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
from sidekiq/processor.rb:257:in `stats'
from sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
from sidekiq/job_logger.rb:13:in `call'
from sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
from sidekiq/job_retry.rb:78:in `global'
from sidekiq/processor.rb:124:in `block in dispatch'
from sidekiq/logger.rb:10:in `with'
from sidekiq/job_logger.rb:33:in `prepare'
from sidekiq/processor.rb:123:in `dispatch'
from sidekiq/processor.rb:162:in `process'
from sidekiq/processor.rb:78:in `process_one'
from sidekiq/processor.rb:68:in `run'
from sidekiq/util.rb:15:in `watchdog'
from sidekiq/util.rb:24:in `block in safe_thread'
I have even taken out the entire 'perform' block of code i.e. the worker does nothing to confirm I don't have some sort of regressive call etc. I confirm that my other workers fire and process just fine. Checked for obvious typos - banging my head against the wall at this point.
UPDATE
Ok - I have confirmed one thing - if I add any new workers with any name this triggers the same error. I even rebooted the entire production server to confirm the whole code was reloaded etc.
It ended up being Redis (or at least my fix was related). Found this post:
https://github.com/mperham/sidekiq/issues/2834#issuecomment-184800981
and it was a Redis namespace conflict. My server does have elasticsearch running also so that makes sense. I am not sure why the old workers run but the new ones failed. My fix looks like this:
config.redis = {
url: ENV['REDIS_URL'],
namespace: "some_namespace_different_for_each_app"
}
You also need the redis-namespace gem BTW
By executing the whenever --update-crontab command, the following error returns:
/usr/lib/ruby/vendor_ruby/whenever/cron.rb:132:in `parse_as_string': Couldn't parse: 31557600 (ArgumentError)
from /usr/lib/ruby/vendor_ruby/whenever/cron.rb:112:in `parse_time'
from /usr/lib/ruby/vendor_ruby/whenever/cron.rb:50:in `time_in_cron_syntax'
from /usr/lib/ruby/vendor_ruby/whenever/cron.rb:42:in `output'
from /usr/lib/ruby/vendor_ruby/whenever/cron.rb:36:in `block (2 levels) in output'
from /usr/lib/ruby/vendor_ruby/whenever/cron.rb:35:in `each'
from /usr/lib/ruby/vendor_ruby/whenever/cron.rb:35:in `block in output'
from /usr/lib/ruby/vendor_ruby/whenever/cron.rb:34:in `each'
from /usr/lib/ruby/vendor_ruby/whenever/cron.rb:34:in `output'
from /usr/lib/ruby/vendor_ruby/whenever/job_list.rb:151:in `block (2 levels) in cron_jobs'
from /usr/lib/ruby/vendor_ruby/whenever/job_list.rb:147:in `each'
from /usr/lib/ruby/vendor_ruby/whenever/job_list.rb:147:in `block in cron_jobs'
from /usr/lib/ruby/vendor_ruby/whenever/job_list.rb:146:in `each'
from /usr/lib/ruby/vendor_ruby/whenever/job_list.rb:146:in `cron_jobs'
from /usr/lib/ruby/vendor_ruby/whenever/job_list.rb:65:in `generate_cron_output'
from /usr/lib/ruby/vendor_ruby/whenever.rb:10:in `cron'
from /usr/lib/ruby/vendor_ruby/whenever/command_line.rb:54:in `whenever_cron'
from /usr/lib/ruby/vendor_ruby/whenever/command_line.rb:106:in `updated_crontab'
from /usr/lib/ruby/vendor_ruby/whenever/command_line.rb:35:in `run'
from /usr/lib/ruby/vendor_ruby/whenever/command_line.rb:6:in `execute'
from /usr/bin/whenever:41:in `<main>'
My schedule.rb file:
every 1.year do
rake "maintenances:renew"
end
Any ideas?
If you're having problems with 1.year method, you can use the two following alternatives.
1 - Shortcuts
every :year do
rake "maintenances:renew"
end
2 - Expressions
every '0 0 1 12 *' do
rake "maintenances:renew"
end
If you want to learn more about these options, check the official documentation.
To test expressions, I recommend crontab.guru
I”m using Rails 4.2.7 and I have several Tor gems installed.
gem 'tor'
gem 'tor_requests'
gem 'tor-privoxy'
gem 'net-telnet'
I started my Tor browser (running on Mac El Capitan) and I want to periodically (every 20th request) change the IP address of where my TOR web requests originate. So I tried this
agent = TorPrivoxy::Agent.new '127.0.0.1', '', {8118 => 9151} do |agent|
sleep 1
puts "New IP is #{agent.ip}"
end
However, this results in the below error. I’m confused about how else I need to configure things so that I can make the above work.
Error during processing: Failed to open TCP connection to 127.0.0.1:8118 (general SOCKS server failure)
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:879:in `block in connect'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:878:in `connect'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:858:in `start'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:700:in `start'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:631:in `connection_for'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:994:in `request'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/mechanize-2.7.5/lib/mechanize/http/agent.rb:274:in `fetch'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/mechanize-2.7.5/lib/mechanize.rb:464:in `get'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/tor-privoxy-0.1.1/lib/tor-privoxy/agent.rb:38:in `ip'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:57:in `block in get_content'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/tor-privoxy-0.1.1/lib/tor-privoxy/agent.rb:11:in `initialize'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:55:in `new'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:55:in `rescue in get_content'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:50:in `get_content'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:14:in `get_url'
/Users/davea/Documents/workspace/myproject/app/services/onlinerr_race_finder_service.rb:41:in `get_race_list'
/Users/davea/Documents/workspace/myproject/app/services/abstract_race_finder_service.rb:26:in `process_data'
/Users/davea/Documents/workspace/myproject/app/services/run_crawlers_service.rb:18:in `block in run_all_crawlers'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/delegation.rb:46:in `each'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/delegation.rb:46:in `each'
/Users/davea/Documents/workspace/myproject/app/services/run_crawlers_service.rb:5:in `run_all_crawlers'
(irb):2:in `irb_binding'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/workspace.rb:87:in `eval'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/workspace.rb:87:in `evaluate'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/context.rb:380:in `evaluate'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:489:in `block (2 levels) in eval_input'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:623:in `signal_status'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:486:in `block in eval_input'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/ruby-lex.rb:246:in `block (2 levels) in each_top_level_statement'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/ruby-lex.rb:232:in `loop'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/ruby-lex.rb:232:in `block in each_top_level_statement'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/ruby-lex.rb:231:in `catch'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/ruby-lex.rb:231:in `each_top_level_statement'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:485:in `eval_input'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:395:in `block in start'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:394:in `catch'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:394:in `start'
How do I programmatically force an IP address change using Ruby/Tor?
Edit:Here is what I included in my helper file and below is the error that results
require 'rubygems'
$:.unshift "./tor/lib"
require 'tor'
…
cookie_file = '/Users/davea/Library/Application Support/TorBrowser-Data/Tor/control_auth_cookie'
file = File.open(cookie_file, 'rb')
cookie = file.read # read contents of auth cookie to string
file.close
Tor::Controller.connect(:port => 9150, :cookie => cookie) do |tor|
tor.signal('NEWNYM') # send NEWNYM signal (gets new IP)
end
The error that results in the rails console …
Error during processing: undefined method `signal' for #<Tor::Controller:0x007fe044b1e550>
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:87:in `block in get_content'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/tor-0.1.2/lib/tor/control.rb:38:in `connect'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:86:in `rescue in get_content'
Edit 2:
After adding this to my Gemfile file
gem 'tor', :git => 'https://github.com/dryruby/tor.rb.git'
I ran the code above, and now got the error …
Error during processing: end of file reached
/Users/davea/.rvm/gems/ruby-2.3.0/bundler/gems/tor.rb-08e589d17196/lib/tor/control.rb:301:in `readline'
/Users/davea/.rvm/gems/ruby-2.3.0/bundler/gems/tor.rb-08e589d17196/lib/tor/control.rb:301:in `read_reply'
/Users/davea/.rvm/gems/ruby-2.3.0/bundler/gems/tor.rb-08e589d17196/lib/tor/control.rb:194:in `authenticate'
/Users/davea/.rvm/gems/ruby-2.3.0/bundler/gems/tor.rb-08e589d17196/lib/tor/control.rb:282:in `send_command'
/Users/davea/.rvm/gems/ruby-2.3.0/bundler/gems/tor.rb-08e589d17196/lib/tor/control.rb:269:in `signal'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:90:in `block in get_content'
To change your IP in Tor, you need to connect directly to the controller (it uses a telnet-like line based command/response protocol.
It looks like you're trying to connect through Privoxy which isn't necessary. The controller only allows local connections by default.
Tor Browser's Tor config enforces authentication uses the cookie method, without changing any options in torrc you need to read the contents of the auth cookie to connect.
I was able to change the IP with the following code:
require 'rubygems'
$:.unshift "./tor/lib"
require 'tor'
# the cookie file contains a "password" for authentication, 32 random bytes
cookie_file = '/home/me/tor-browser_en-US/Browser/TorBrowser/Data/Tor/control_auth_cookie'
file = File.open(cookie_file, 'rb')
cookie = file.read # read contents of auth cookie to string
file.close
Tor::Controller.connect(:port => 9151, :cookie => cookie) do |tor|
p tor.signal('NEWNYM') # send NEWNYM signal (gets new IP)
end
tor.signal returns a string, which should read "250 OK" if the NEWNYM command was successful. There is some internal rate limiting which prevents this from being run too often (like every 10 seconds I think) which shouldn't be a problem for you.
EDIT: The 0.1.2 (current release) of this Gem doesn't include the signal method from the master branch. You'll need to use the latest code or replicate their code instead of calling signal.
I'm working with a Rails 3.2 application that has a mysql database and a number of models that are being indexed in Solr.
Here's what's happening:
I am running the following command:
RAILS_ENV=development bundle exec rake sunspot:reindex[1000] --trace
After indexing about 12% of the 4 million records (although it's a different percentage every time), the process inevitably bombs out with the following error and stack trace:
rake aborted!
getaddrinfo: nodename nor servname provided, or not known
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:762:in `initialize'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:762:in `open'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:762:in `block in connect'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/timeout.rb:54:in `timeout'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/timeout.rb:99:in `timeout'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:762:in `connect'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:755:in `do_start'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:744:in `start'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:1284:in `request'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/connection.rb:15:in `execute'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/solr_instrumentation.rb:14:in `block in execute_with_as_instrumentation'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/activesupport-3.2.13/lib/active_support/notifications.rb:123:in `block in instrument'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/activesupport-3.2.13/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/activesupport-3.2.13/lib/active_support/notifications.rb:123:in `instrument'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/solr_instrumentation.rb:12:in `execute_with_as_instrumentation'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:167:in `execute'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:161:in `send_and_receive'
(eval):2:in `post'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:67:in `update'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:87:in `add'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/indexer.rb:106:in `add_documents'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/indexer.rb:30:in `add'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/session.rb:91:in `index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/session_proxy/retry_5xx_session_proxy.rb:17:in `method_missing'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot.rb:184:in `index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/searchable.rb:261:in `block (2 levels) in solr_index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/searchable.rb:365:in `solr_benchmark'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/searchable.rb:260:in `block in solr_index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/activerecord-3.2.13/lib/active_record/relation/batches.rb:72:in `find_in_batches'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/activerecord-3.2.13/lib/active_record/querying.rb:8:in `find_in_batches'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/searchable.rb:259:in `solr_index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/searchable.rb:203:in `solr_reindex'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/tasks.rb:64:in `block (3 levels) in <top (required)>'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/class_set.rb:16:in `each'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/class_set.rb:16:in `each'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/tasks.rb:63:in `block (2 levels) in <top (required)>'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:246:in `call'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:246:in `block in execute'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:241:in `each'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:241:in `execute'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:184:in `block in invoke_with_call_chain'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:177:in `invoke_with_call_chain'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:170:in `invoke'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:143:in `invoke_task'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `block (2 levels) in top_level'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `each'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `block in top_level'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:110:in `run_with_threads'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:95:in `top_level'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:73:in `block in run'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:160:in `standard_exception_handling'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:70:in `run'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/bin/rake:33:in `<top (required)>'
/Users/tchapin/.rbenv/versions/1.9.3-p392/bin/rake:23:in `load'
/Users/tchapin/.rbenv/versions/1.9.3-p392/bin/rake:23:in `<main>'
The app is running in development mode at localhost:3000, and solr is running at localhost:8982. Here's my solr.rake file:
Rake::Task['sunspot:reindex'].enhance ['sunspot:scope_models_for_index']
Rake::Task['sunspot:solr:reindex'].enhance ['sunspot:scope_models_for_index']
namespace 'sunspot' do
task :scope_models_for_index => :environment do
require 'rsolr/error'
Dir.glob(Rails.root.join('app/models/**/*.rb')).each { |path| require path }
# Add the GC
commit_extension = Module.new do
def commit
GC.start
super
end
end
Sunspot.extend commit_extension
# Set all the models default scopes the index scope
Sunspot.searchable.each do |model|
model.class_eval do
default_scope ->{ sunspot_index } if model.respond_to?(:sunspot_index)
end
end
end
end
Anyone know what might be causing this error, or how to fix it?
Apparently the problem is due to an exception: SocketError.
The exception occurs in the file /Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/connection.rb:
def execute client, request_context
h = http request_context[:uri], request_context[:proxy], request_context[:read_timeout], request_context[:open_timeout]
request = setup_raw_request request_context
request.body = request_context[:data] if request_context[:method] == :post and request_context[:data]
begin
response = h.request request
charset = response.type_params["charset"]
{:status => response.code.to_i, :headers => response.to_hash, :body => force_charset(response.body, charset)}
rescue Errno::ECONNREFUSED => e
raise(Errno::ECONNREFUSED.new(request_context.inspect))
# catch the undefined closed? exception -- this is a confirmed ruby bug
rescue NoMethodError
$!.message == "undefined method `closed?' for nil:NilClass" ?
raise(Errno::ECONNREFUSED.new) :
raise($!)
end
end
It's not the prettiest fix, and it's obviously not a solution for production code, but I was able to temporarily solve the problem by adding a rescue handler for the SocketError exception, like so:
def execute client, request_context
h = http request_context[:uri], request_context[:proxy], request_context[:read_timeout], request_context[:open_timeout]
request = setup_raw_request request_context
request.body = request_context[:data] if request_context[:method] == :post and request_context[:data]
retries = 10
begin
response = h.request request
charset = response.type_params["charset"]
{:status => response.code.to_i, :headers => response.to_hash, :body => force_charset(response.body, charset)}
rescue Errno::ECONNREFUSED => e
raise(Errno::ECONNREFUSED.new(request_context.inspect))
# catch the undefined closed? exception -- this is a confirmed ruby bug
rescue NoMethodError
$!.message == "undefined method `closed?' for nil:NilClass" ?
raise(Errno::ECONNREFUSED.new) :
raise($!)
rescue SocketError => e
puts e
if retries > 0
puts "SocketError! Retrying connection after 1 second..."
retries -= 1
sleep(1)
retry
else
puts "SocketError: Not responding after 10 retries! Giving up!")
exit
end
end
end
Additionally, it looks like my local copy of Solr was having some issues with running out of memory. I solved the memory problem by updating my sunspot.yml file accordingly:
development:
solr:
hostname: 127.0.0.1
port: 8982
min_memory: 512M
max_memory: 2G
log_level: INFO
# open_timeout: 3
# read_timeout: 3
I have an error that appears to be connected to these two links:
https://groups.google.com/forum/#!msg/mongoid/L6ESKP8QDpg/gmEyqejdAEEJ
https://github.com/mongoid/moped/pull/59
See console log below. I am running with Mongoid/MongoHQ, when I create an object (apparently any database object) I get this "need to login" error. Note however that the database object IS created and is in the database after the error.
Is there some way I can fix this myself, or perhaps it's something that MongoHQ needs to fix?
$ heroku run console
irb(main):004:0> Request.create!(:from_user => User.first, :to_user => User.first)
Moped::Errors::OperationFailure: The operation: #<Moped::Protocol::Command
#length=69
#request_id=7
#response_to=0
#op_code=2004
#flags=[]
#full_collection_name="admin.$cmd"
#skip=0
#limit=-1
#selector={:getlasterror=>1, :safe=>true}
#fields=nil>
failed with error "need to login"
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:74:in `block in command'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:521:in `[]'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:521:in `block (3 levels) in flush'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:520:in `map'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:520:in `block (2 levels) in flush'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:123:in `ensure_connected'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:516:in `block in flush'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:531:in `logging'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:515:in `flush'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:286:in `pipeline'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/session/context.rb:58:in `block in insert'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/session/context.rb:109:in `block in with_node'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/cluster.rb:150:in `block in with_primary'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:168:in `ensure_primary'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/cluster.rb:149:in `with_primary'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/session/context.rb:108:in `with_node'
... 13 levels...
from /app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
from /app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_save_callbacks'
from /app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/callbacks.rb:100:in `run_callbacks'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/persistence/insertion.rb:23:in `prepare'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/persistence/operations/insert.rb:26:in `persist'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/persistence.rb:50:in `insert'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/persistence.rb:251:in `block in create!'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/threaded/lifecycle.rb:173:in `_creating'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/persistence.rb:249:in `create!'
from (irb):4
from /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.8/lib/rails/commands/console.rb:47:in `start'
from /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.8/lib/rails/commands/console.rb:8:in `start'
from /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.8/lib/rails/commands.rb:41:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'irb(main):005:0>
I am having the exact same problem. This is not exactly a solution but you can work around the problem by setting the 'safe' option in mongoid.yml to false. It would look similar to this:
production:
sessions:
default:
uri: <%= ENV['MONGOHQ_URL'] %>
options:
skip_version_check: true
safe: false
It turns out the problem is that MongoHQ upgraded their MongoDB and it introduced an incompatibility with mongoid's stable branch.
Chris Winslett fixed the bug and durran pulled it into mongoid master. But it was not included in the stable branch when I posted this stackoverflow.
Solutions:
I solved the problem by updating my gemfile to pull mongoid from github.
durran has now included it in stable-1.2.0 so you should get the fix if you gem update.