I”m using Rails 4.2.7 and I have several Tor gems installed.
gem 'tor'
gem 'tor_requests'
gem 'tor-privoxy'
gem 'net-telnet'
I started my Tor browser (running on Mac El Capitan) and I want to periodically (every 20th request) change the IP address of where my TOR web requests originate. So I tried this
agent = TorPrivoxy::Agent.new '127.0.0.1', '', {8118 => 9151} do |agent|
sleep 1
puts "New IP is #{agent.ip}"
end
However, this results in the below error. I’m confused about how else I need to configure things so that I can make the above work.
Error during processing: Failed to open TCP connection to 127.0.0.1:8118 (general SOCKS server failure)
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:879:in `block in connect'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:878:in `connect'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:858:in `start'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:700:in `start'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:631:in `connection_for'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:994:in `request'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/mechanize-2.7.5/lib/mechanize/http/agent.rb:274:in `fetch'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/mechanize-2.7.5/lib/mechanize.rb:464:in `get'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/tor-privoxy-0.1.1/lib/tor-privoxy/agent.rb:38:in `ip'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:57:in `block in get_content'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/tor-privoxy-0.1.1/lib/tor-privoxy/agent.rb:11:in `initialize'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:55:in `new'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:55:in `rescue in get_content'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:50:in `get_content'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:14:in `get_url'
/Users/davea/Documents/workspace/myproject/app/services/onlinerr_race_finder_service.rb:41:in `get_race_list'
/Users/davea/Documents/workspace/myproject/app/services/abstract_race_finder_service.rb:26:in `process_data'
/Users/davea/Documents/workspace/myproject/app/services/run_crawlers_service.rb:18:in `block in run_all_crawlers'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/delegation.rb:46:in `each'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/delegation.rb:46:in `each'
/Users/davea/Documents/workspace/myproject/app/services/run_crawlers_service.rb:5:in `run_all_crawlers'
(irb):2:in `irb_binding'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/workspace.rb:87:in `eval'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/workspace.rb:87:in `evaluate'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/context.rb:380:in `evaluate'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:489:in `block (2 levels) in eval_input'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:623:in `signal_status'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:486:in `block in eval_input'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/ruby-lex.rb:246:in `block (2 levels) in each_top_level_statement'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/ruby-lex.rb:232:in `loop'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/ruby-lex.rb:232:in `block in each_top_level_statement'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/ruby-lex.rb:231:in `catch'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb/ruby-lex.rb:231:in `each_top_level_statement'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:485:in `eval_input'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:395:in `block in start'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:394:in `catch'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/irb.rb:394:in `start'
How do I programmatically force an IP address change using Ruby/Tor?
Edit:Here is what I included in my helper file and below is the error that results
require 'rubygems'
$:.unshift "./tor/lib"
require 'tor'
…
cookie_file = '/Users/davea/Library/Application Support/TorBrowser-Data/Tor/control_auth_cookie'
file = File.open(cookie_file, 'rb')
cookie = file.read # read contents of auth cookie to string
file.close
Tor::Controller.connect(:port => 9150, :cookie => cookie) do |tor|
tor.signal('NEWNYM') # send NEWNYM signal (gets new IP)
end
The error that results in the rails console …
Error during processing: undefined method `signal' for #<Tor::Controller:0x007fe044b1e550>
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:87:in `block in get_content'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/tor-0.1.2/lib/tor/control.rb:38:in `connect'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:86:in `rescue in get_content'
Edit 2:
After adding this to my Gemfile file
gem 'tor', :git => 'https://github.com/dryruby/tor.rb.git'
I ran the code above, and now got the error …
Error during processing: end of file reached
/Users/davea/.rvm/gems/ruby-2.3.0/bundler/gems/tor.rb-08e589d17196/lib/tor/control.rb:301:in `readline'
/Users/davea/.rvm/gems/ruby-2.3.0/bundler/gems/tor.rb-08e589d17196/lib/tor/control.rb:301:in `read_reply'
/Users/davea/.rvm/gems/ruby-2.3.0/bundler/gems/tor.rb-08e589d17196/lib/tor/control.rb:194:in `authenticate'
/Users/davea/.rvm/gems/ruby-2.3.0/bundler/gems/tor.rb-08e589d17196/lib/tor/control.rb:282:in `send_command'
/Users/davea/.rvm/gems/ruby-2.3.0/bundler/gems/tor.rb-08e589d17196/lib/tor/control.rb:269:in `signal'
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:90:in `block in get_content'
To change your IP in Tor, you need to connect directly to the controller (it uses a telnet-like line based command/response protocol.
It looks like you're trying to connect through Privoxy which isn't necessary. The controller only allows local connections by default.
Tor Browser's Tor config enforces authentication uses the cookie method, without changing any options in torrc you need to read the contents of the auth cookie to connect.
I was able to change the IP with the following code:
require 'rubygems'
$:.unshift "./tor/lib"
require 'tor'
# the cookie file contains a "password" for authentication, 32 random bytes
cookie_file = '/home/me/tor-browser_en-US/Browser/TorBrowser/Data/Tor/control_auth_cookie'
file = File.open(cookie_file, 'rb')
cookie = file.read # read contents of auth cookie to string
file.close
Tor::Controller.connect(:port => 9151, :cookie => cookie) do |tor|
p tor.signal('NEWNYM') # send NEWNYM signal (gets new IP)
end
tor.signal returns a string, which should read "250 OK" if the NEWNYM command was successful. There is some internal rate limiting which prevents this from being run too often (like every 10 seconds I think) which shouldn't be a problem for you.
EDIT: The 0.1.2 (current release) of this Gem doesn't include the signal method from the master branch. You'll need to use the latest code or replicate their code instead of calling signal.
Related
I'm a new Ruby developer. I would like to send some concurrent requests (maximum 3 requests) at same time to get data. I know there are a lot of Http libs in Ruby that support for sending request. In the end, I choose HTTParty, however, it comes to some error when I try to send request.
response_data = Http::SearchFlightService.search(url, options)
The url value:
http://booknow.jetstar.com/Search.aspx?culture=vi-VN
And value of options params:
{:body=>{"search-origin01"=>"", "search-destination01"=>"", "ControlGroupSearchView$ButtonSubmit"=>"", "__VIEWSTATE"=>"", "undefined"=>"", "children"=>"0", "infants"=>"0", "ControlGroupSearchView$AvailabilitySearchInputSearchView$DropDownListPassengerType_INFANT"=>"0", "ControlGroupSearchView$AvailabilitySearchInputSearchView$DropDownListPassengerType_CHD"=>"0", "datedepart-01"=>"19/06/2016", "datereturn-01"=>"20/06/2016", "adults"=>"1", "ControlGroupSearchView$AvailabilitySearchInputSearchView$DropDownListMarketDay1"=>"19", "ControlGroupSearchView$AvailabilitySearchInputSearchView$DropDownListPassengerType_ADT"=>"1", "ControlGroupSearchView$AvailabilitySearchInputSearchView$DropDownListMarketDay2"=>"20", "ControlGroupSearchView$AvailabilitySearchInputSearchView$DropDownListMarketMonth1"=>"2016-06", "ControlGroupSearchView$AvailabilitySearchInputSearchView$DropDownListMarketMonth2"=>"2016-06", "ControlGroupSearchView$AvailabilitySearchInputSearchView$TextBoxMarketOrigin1"=>"SGN", "ControlGroupSearchView$AvailabilitySearchInputSearchView$DropDownListFareTypes"=>"I", "_pe_39b5379c652b_9df496572198"=>"null", "travel-indicator"=>"on", "ControlGroupSearchView$AvailabilitySearchInputSearchView$RadioButtonMarketStructure"=>"RoundTrip", "ControlGroupSearchView$AvailabilitySearchInputSearchView$TextBoxMarketDestination1"=>"HAN", "pageToken"=>"sLkmnwXwAsY=", "culture"=>"vi-VN", "locale"=>"vi-VN", "currencyPicker"=>"VND", "ControlGroupSearchView$AvailabilitySearchInputSearchView$DropDownListCurrency"=>"VND"}, :headers=>{"Accept-Encoding"=>"gzip, deflate", "Content-type"=>"application/x-www-form-urlencoded"}}
My Http::SearchFlightService
class Http::SearchFlightService
include HTTParty
ssl_version :SSLv3
def self.search(url, options)
post(url, options)
end
end
It showed error
/Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/protocol.rb:153:in `read_nonblock': Connection reset by peer (Errno::ECONNRESET)
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/protocol.rb:153:in `rbuf_fill'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/protocol.rb:104:in `read'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http/response.rb:399:in `read'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http/response.rb:289:in `block in read_body_0'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http/response.rb:260:in `inflater'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http/response.rb:279:in `read_body_0'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http/response.rb:201:in `read_body'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http/response.rb:226:in `body'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http/response.rb:163:in `reading_body'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1420:in `block in transport_request'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1411:in `catch'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1411:in `transport_request'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1384:in `request'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1377:in `block in request'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:853:in `start'
from /Users/Dona/.rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1375:in `request'
Please give me some advice. Thanks
httparty seems to have some bugs. Had same Errno::ECONNRESET: Connection reset by peer. Fixed by replacing httparty call to net/http
require 'net/http'
JSON.parse Net::HTTP.get(URI.parse(url))
I got a simple and free heroku app running where I added redis/sidekiq in order to send out mails in the background. I set everything up and once I start the worker, I get following error message in a loop:
Error fetching job: ERR max number of clients reached
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:114:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:95:in `block in connect'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:280:in `with_reconnect'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:93:in `connect'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:351:in `ensure_connected'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:208:in `block in process'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:293:in `logging'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:207:in `process'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:113:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:196:in `block in call_with_timeout'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:267:in `with_socket_timeout'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis/client.rb:195:in `call_with_timeout'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis.rb:1097:in `block in _bpop'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis.rb:57:in `block in synchronize'
/app/vendor/ruby-2.2.2/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis.rb:57:in `synchronize'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis.rb:1094:in `_bpop'
/app/vendor/bundle/ruby/2.2.0/gems/redis-3.2.2/lib/redis.rb:1139:in `brpop'
/app/vendor/bundle/ruby/2.2.0/gems/redis-namespace-1.5.2/lib/redis/namespace.rb:393:in `call_with_namespace'
/app/vendor/bundle/ruby/2.2.0/gems/redis-namespace-1.5.2/lib/redis/namespace.rb:290:in `method_missing'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/fetch.rb:35:in `block in retrieve_work'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq.rb:84:in `block in redis'
/app/vendor/bundle/ruby/2.2.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:64:in `block (2 levels) in with'
/app/vendor/bundle/ruby/2.2.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in `handle_interrupt'
/app/vendor/bundle/ruby/2.2.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:63:in `block in with'
/app/vendor/bundle/ruby/2.2.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `handle_interrupt'
/app/vendor/bundle/ruby/2.2.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `with'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq.rb:81:in `redis'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/fetch.rb:35:in `retrieve_work'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:85:in `get_one'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:95:in `fetch'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:78:in `process_one'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/processor.rb:67:in `run'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/util.rb:16:in `watchdog'
/app/vendor/bundle/ruby/2.2.0/gems/sidekiq-4.0.1/lib/sidekiq/util.rb:24:in `block in safe_thread'
sidekiq.yml:
---
:queues:
- default
- mailers
sidekiq.rb:
Sidekiq.configure_server do |config|
config.redis = { :namespace => "mynamespace" }
end
Sidekiq.configure_client do |config|
config.redis = { :namespace => "mynamespace" }
end
require "sidekiq/web"
Sidekiq::Web.app_url = "/"
Sidekiq::Web.use(Rack::Auth::Basic, "Application") do |username, password|
username == ENV.fetch("SIDEKIQ_WEB_USERNAME") &&
password == ENV.fetch("SIDEKIQ_WEB_PASSWORD")
end
Anyone an idea what's wrong here? If you need any further info on the config files or something, let me know.
Changing sidekiq.yml to
---
:queues:
- default
- mailers
:concurrency: 5
did the trick. Setting concurrency to 9 did not for some reason even though heroku allows 10 for the redis nano addon.
This was the best solution that I found. If you are using the free heroku plan and the free redis nano plan. All the connections have to add up to 10 or less. To make sure that the connections add up check out this blog post. http://manuelvanrijn.nl/blog/2012/11/13/sidekiq-on-heroku-with-redistogo-nano/
He explains how everything adds up.
The writer also has a calculator that will help to calculate what the configuration should be for your particular server, sidekiq etc. It also tells you where the information needs to be added.
I was stuck on this too hope it saves somebody some time :)
I'm working with a Rails 3.2 application that has a mysql database and a number of models that are being indexed in Solr.
Here's what's happening:
I am running the following command:
RAILS_ENV=development bundle exec rake sunspot:reindex[1000] --trace
After indexing about 12% of the 4 million records (although it's a different percentage every time), the process inevitably bombs out with the following error and stack trace:
rake aborted!
getaddrinfo: nodename nor servname provided, or not known
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:762:in `initialize'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:762:in `open'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:762:in `block in connect'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/timeout.rb:54:in `timeout'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/timeout.rb:99:in `timeout'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:762:in `connect'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:755:in `do_start'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:744:in `start'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/net/http.rb:1284:in `request'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/connection.rb:15:in `execute'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/solr_instrumentation.rb:14:in `block in execute_with_as_instrumentation'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/activesupport-3.2.13/lib/active_support/notifications.rb:123:in `block in instrument'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/activesupport-3.2.13/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/activesupport-3.2.13/lib/active_support/notifications.rb:123:in `instrument'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/solr_instrumentation.rb:12:in `execute_with_as_instrumentation'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:167:in `execute'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:161:in `send_and_receive'
(eval):2:in `post'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:67:in `update'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:87:in `add'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/indexer.rb:106:in `add_documents'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/indexer.rb:30:in `add'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/session.rb:91:in `index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/session_proxy/retry_5xx_session_proxy.rb:17:in `method_missing'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot.rb:184:in `index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/searchable.rb:261:in `block (2 levels) in solr_index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/searchable.rb:365:in `solr_benchmark'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/searchable.rb:260:in `block in solr_index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/activerecord-3.2.13/lib/active_record/relation/batches.rb:72:in `find_in_batches'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/activerecord-3.2.13/lib/active_record/querying.rb:8:in `find_in_batches'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/searchable.rb:259:in `solr_index'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/searchable.rb:203:in `solr_reindex'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/tasks.rb:64:in `block (3 levels) in <top (required)>'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/class_set.rb:16:in `each'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot-2.0.0/lib/sunspot/class_set.rb:16:in `each'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/sunspot_rails-2.0.0/lib/sunspot/rails/tasks.rb:63:in `block (2 levels) in <top (required)>'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:246:in `call'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:246:in `block in execute'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:241:in `each'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:241:in `execute'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:184:in `block in invoke_with_call_chain'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:177:in `invoke_with_call_chain'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:170:in `invoke'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:143:in `invoke_task'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `block (2 levels) in top_level'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `each'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `block in top_level'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:110:in `run_with_threads'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:95:in `top_level'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:73:in `block in run'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:160:in `standard_exception_handling'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:70:in `run'
/Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rake-10.0.4/bin/rake:33:in `<top (required)>'
/Users/tchapin/.rbenv/versions/1.9.3-p392/bin/rake:23:in `load'
/Users/tchapin/.rbenv/versions/1.9.3-p392/bin/rake:23:in `<main>'
The app is running in development mode at localhost:3000, and solr is running at localhost:8982. Here's my solr.rake file:
Rake::Task['sunspot:reindex'].enhance ['sunspot:scope_models_for_index']
Rake::Task['sunspot:solr:reindex'].enhance ['sunspot:scope_models_for_index']
namespace 'sunspot' do
task :scope_models_for_index => :environment do
require 'rsolr/error'
Dir.glob(Rails.root.join('app/models/**/*.rb')).each { |path| require path }
# Add the GC
commit_extension = Module.new do
def commit
GC.start
super
end
end
Sunspot.extend commit_extension
# Set all the models default scopes the index scope
Sunspot.searchable.each do |model|
model.class_eval do
default_scope ->{ sunspot_index } if model.respond_to?(:sunspot_index)
end
end
end
end
Anyone know what might be causing this error, or how to fix it?
Apparently the problem is due to an exception: SocketError.
The exception occurs in the file /Users/tchapin/.rbenv/versions/1.9.3-p392/lib/ruby/gems/1.9.1/gems/rsolr-1.0.9/lib/rsolr/connection.rb:
def execute client, request_context
h = http request_context[:uri], request_context[:proxy], request_context[:read_timeout], request_context[:open_timeout]
request = setup_raw_request request_context
request.body = request_context[:data] if request_context[:method] == :post and request_context[:data]
begin
response = h.request request
charset = response.type_params["charset"]
{:status => response.code.to_i, :headers => response.to_hash, :body => force_charset(response.body, charset)}
rescue Errno::ECONNREFUSED => e
raise(Errno::ECONNREFUSED.new(request_context.inspect))
# catch the undefined closed? exception -- this is a confirmed ruby bug
rescue NoMethodError
$!.message == "undefined method `closed?' for nil:NilClass" ?
raise(Errno::ECONNREFUSED.new) :
raise($!)
end
end
It's not the prettiest fix, and it's obviously not a solution for production code, but I was able to temporarily solve the problem by adding a rescue handler for the SocketError exception, like so:
def execute client, request_context
h = http request_context[:uri], request_context[:proxy], request_context[:read_timeout], request_context[:open_timeout]
request = setup_raw_request request_context
request.body = request_context[:data] if request_context[:method] == :post and request_context[:data]
retries = 10
begin
response = h.request request
charset = response.type_params["charset"]
{:status => response.code.to_i, :headers => response.to_hash, :body => force_charset(response.body, charset)}
rescue Errno::ECONNREFUSED => e
raise(Errno::ECONNREFUSED.new(request_context.inspect))
# catch the undefined closed? exception -- this is a confirmed ruby bug
rescue NoMethodError
$!.message == "undefined method `closed?' for nil:NilClass" ?
raise(Errno::ECONNREFUSED.new) :
raise($!)
rescue SocketError => e
puts e
if retries > 0
puts "SocketError! Retrying connection after 1 second..."
retries -= 1
sleep(1)
retry
else
puts "SocketError: Not responding after 10 retries! Giving up!")
exit
end
end
end
Additionally, it looks like my local copy of Solr was having some issues with running out of memory. I solved the memory problem by updating my sunspot.yml file accordingly:
development:
solr:
hostname: 127.0.0.1
port: 8982
min_memory: 512M
max_memory: 2G
log_level: INFO
# open_timeout: 3
# read_timeout: 3
I have an error that appears to be connected to these two links:
https://groups.google.com/forum/#!msg/mongoid/L6ESKP8QDpg/gmEyqejdAEEJ
https://github.com/mongoid/moped/pull/59
See console log below. I am running with Mongoid/MongoHQ, when I create an object (apparently any database object) I get this "need to login" error. Note however that the database object IS created and is in the database after the error.
Is there some way I can fix this myself, or perhaps it's something that MongoHQ needs to fix?
$ heroku run console
irb(main):004:0> Request.create!(:from_user => User.first, :to_user => User.first)
Moped::Errors::OperationFailure: The operation: #<Moped::Protocol::Command
#length=69
#request_id=7
#response_to=0
#op_code=2004
#flags=[]
#full_collection_name="admin.$cmd"
#skip=0
#limit=-1
#selector={:getlasterror=>1, :safe=>true}
#fields=nil>
failed with error "need to login"
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:74:in `block in command'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:521:in `[]'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:521:in `block (3 levels) in flush'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:520:in `map'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:520:in `block (2 levels) in flush'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:123:in `ensure_connected'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:516:in `block in flush'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:531:in `logging'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:515:in `flush'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:286:in `pipeline'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/session/context.rb:58:in `block in insert'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/session/context.rb:109:in `block in with_node'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/cluster.rb:150:in `block in with_primary'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/node.rb:168:in `ensure_primary'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/cluster.rb:149:in `with_primary'
from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.2.0/lib/moped/session/context.rb:108:in `with_node'
... 13 levels...
from /app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
from /app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_save_callbacks'
from /app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/callbacks.rb:100:in `run_callbacks'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/persistence/insertion.rb:23:in `prepare'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/persistence/operations/insert.rb:26:in `persist'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/persistence.rb:50:in `insert'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/persistence.rb:251:in `block in create!'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/threaded/lifecycle.rb:173:in `_creating'
from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.4/lib/mongoid/persistence.rb:249:in `create!'
from (irb):4
from /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.8/lib/rails/commands/console.rb:47:in `start'
from /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.8/lib/rails/commands/console.rb:8:in `start'
from /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.8/lib/rails/commands.rb:41:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'irb(main):005:0>
I am having the exact same problem. This is not exactly a solution but you can work around the problem by setting the 'safe' option in mongoid.yml to false. It would look similar to this:
production:
sessions:
default:
uri: <%= ENV['MONGOHQ_URL'] %>
options:
skip_version_check: true
safe: false
It turns out the problem is that MongoHQ upgraded their MongoDB and it introduced an incompatibility with mongoid's stable branch.
Chris Winslett fixed the bug and durran pulled it into mongoid master. But it was not included in the stable branch when I posted this stackoverflow.
Solutions:
I solved the problem by updating my gemfile to pull mongoid from github.
durran has now included it in stable-1.2.0 so you should get the fix if you gem update.
I'm trying to use the twitter gem in a rails 3 app but keep getting the following error:
TypeError: can't convert Pathname into String
Here's the code I'm trying to run:
class Tweet < ActiveRecord::Base
def self.test_tweet
Twitter.user_timeline("sferik").first.text
end
end
Here's the full error message:
TypeError: can't convert Pathname into String
from /home/shane/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:658:in `initialize'
from /home/shane/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:658:in `new'
from /home/shane/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:658:in `connect'
from /home/shane/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:637:in `do_start'
from /home/shane/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:626:in `start'
from /home/shane/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:1168:in `request'
from /home/shane/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:888:in `get'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/faraday-0.7.6/lib/faraday/adapter/net_http.rb:59:in `call'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/faraday-0.7.6/lib/faraday/response.rb:8:in `call'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/faraday-0.7.6/lib/faraday/response.rb:8:in `call'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/faraday-0.7.6/lib/faraday/response.rb:8:in `call'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/faraday-0.7.6/lib/faraday/request/url_encoded.rb:14:in `call'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/faraday-0.7.6/lib/faraday/request/multipart.rb:13:in `call'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/twitter-2.1.1/lib/twitter/request/multipart_with_file.rb:17:in `call'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/twitter-2.1.1/lib/twitter/request/phoenix.rb:13:in `call'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/faraday-0.7.6/lib/faraday/connection.rb:210:in `run_request'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/twitter-2.1.1/lib/twitter/request.rb:23:in `request'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/twitter-2.1.1/lib/twitter/request.rb:11:in `get'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/twitter-2.1.1/lib/twitter/client/timelines.rb:208:in `user_timeline'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/twitter-2.1.1/lib/twitter.rb:17:in `method_missing'
from /home/shane/projects/bv_data/app/models/tweet.rb:5:in `test_tweet'
from (irb):1
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/railties-3.1.2/lib/rails/commands/console.rb:45:in `start'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/railties-3.1.2/lib/rails/commands/console.rb:8:in `start'
from /home/shane/.rvm/gems/ruby-1.9.2-p290#rails312/gems/railties-3.1.2/lib/rails/commands.rb:40:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'
I get this error even when running the same code from the console:
Twitter.user_timeline("sferik").first.text
I created a blank test app with the same version of rails and the twitter gem and it runs no problem so I'm not sure what the problem is.
As I had this working before in the app I went back through my commits until I could find a point were it was working and what I did to break it.
So in case anyone happens to be in the same position what I had added was a fix_ssl.rb file in config/initializers which was to address a previous SSL connection issue. Removing this file allowed the Twitter gem to work correctly again.
config/initializers/fix_ssl.rb
require 'open-uri'
require 'net/https'
module Net
class HTTP
alias_method :original_use_ssl=, :use_ssl=
def use_ssl=(flag)
self.ca_file = Rails.root.join('lib/ca-bundle.crt')
self.verify_mode = OpenSSL::SSL::VERIFY_PEER
self.original_use_ssl = flag
end
end
end