I'm fetching some weather data from an online xml doc using Nokogiri, and I would like to set up a timeout for graceful recovery in case the source can't be reached...
My google searches show several possible methods for open-uri and Net::HTTP, but none specific to Nokogiri. My attempts to use those methods are failing (not too surprisingly):
begin
currentloc = ("http://api.wunderground.com/auto/wui/geo/WXCurrentObXML/index.xml?query=" + #destination.weatherloc)
currentloc.read_timeout = 10 #
doc = Nokogiri::XML(open(currentloc))
rescue Timeout::Error
return "Current weather for this location not available: request timed out."
end
returns "NoMethodError", and:
begin
currentloc = ("http://api.wunderground.com/auto/wui/geo/WXCurrentObXML/index.xml?query=" + #destination.weatherloc)
doc = Nokogiri::XML(open(currentloc), :read_timeout => 10)
rescue Timeout::Error
return "Current weather for this location not available: request timed out."
end
returns "TypeError can't convert Hash into String"
Does Nokogiri support this kind of method (and if so... how?), or should I be looking at some other solution?
Thanks.
You can use the timeout module :
require 'open-uri'
require 'nokogiri'
require 'timeout'
begin
timeout(10) do
result = Nokogiri::XML(open(currentloc))
end
rescue Timeout::Error
return "Current weahter..."
end
Related
I am using appsignal gem to track if there is an error processing in my app.
This case i do call external API using faraday.
def truck_information(req_params)
response = #conn.post('truck/info') do |req|
req.headers['Content-Type'] = 'application/json'
req.body = req_params
end
return JSON.parse(response.body) if response_successful?(response)
response_error(response)
end
def response_successful?(response)
response.status == 200
end
def response_error(response)
err = NctError, "Code: #{response.status}, response: #{response.body}"
Appsignal.set_error(err)
raise NctError, I18n.t('error_messages.ppob.server_error')
end
my truck_information is used to call external api. and if success i will parse it to json. but if error i will call response_error method parser to create custom class error (NctError) and i want to send to appsignal to show the error without breaking the application process.
But when i was tested it, it doesn't send to appsignal. How to do send error to appsignal, even if it doesn't crash a request? because i need to track the error.
Thank you
You could try Appsignal.send_error
Appsignal.send_error(err)
If above doesn't work either, then set_error and send_error may only work with Exception:
def response_error(response)
raise NctError, I18n.t('error_messages.ppob.server_error')
rescue => e
Appsignal.send_error(e) do |transaction|
transaction.params = { code: response.status, response: response.body }
end
end
I want to send a
"Keep alive from client"
message every 30 seconds for my websocket connection. Here's what the code that I have in my websocket initializer looks like:
ws = WebSocket::Client::Simple.connect 'wss://bitcoin.toshi.io/'
ws.on :message do |msg|
rawJson = msg.data
message_response = JSON.parse(rawJson)
end
ws.on :open do
ws.send "{\"subscribe\":\"blocks\"}"
end
ws.on :close do |e|
puts "WEBSOCKET HAS CLOSED #{e}"
exit 1
end
ws.on :error do |e|
puts "WEBSOCKET ERROR #{e}"
end
Without any sort of 'keep alive', the connect closes in about 45 seconds. How should I send the 'heart-beat' packet? It seems that the connection is closed by their server, not mine.
You can use Websocket Eventmachine Client gem to send hearbeat:
require 'websocket-eventmachine-client'
EM.run do
ws = WebSocket::EventMachine::Client.connect(:uri => 'wss://bitcoin.toshi.io/')
puts ws.comm_inactivity_timeout
ws.onopen do
puts "Connected"
end
ws.onmessage do |msg, type|
puts "Received message: #{msg}"
end
ws.onclose do |code, reason|
puts "Disconnected with status code: #{code}"
end
EventMachine.add_periodic_timer(15) do
ws.send "{}"
end
end
You can setup timer for EventMachine with EM::add_periodic_timer(interval_in_seconds), and then send your heartbeat with it.
You can use the auto-ping feature (its default and can't be turned off) if you're using Iodine's Websocket client:
require 'iodine/http'
# prevents the Iodine's server from running
Iodine.protocol = :timer
# starts Iodine while the script is still running
Iodine.force_start!
# set pinging to a 40 seconds interval.
Iodine::Http::Websockets.default_timeout = 40
settings = {}
# set's the #on_open event callback.
settings[:on_open] = Proc.new do
write 'sending this connection string.'
end
# set's the #on_message(data) event callback.
settings[:on_message] = Proc.new { |data| puts "Received message: #{data}" }
# connects to the websocket
Iodine::Http.ws_connect 'ws://localhost:8080', settings
It's a fairly basic client, but also easy to manage.
EDIT
Iodine also includes some cookie and custom header's support, as now seen in Iodine's documentation. So it's possible to use different authentication techniques (authentication headers or cookies).
I just created this little currency converter for my Rails 4 app:
module Currency
def self.get_exchange_rate(from_curr = "EUR", to_curr = "USD")
if from_curr == to_curr
result = 1
else
begin
amount = 1
url = "http://www.google.com/finance/converter?a=#{amount}&from=#{from_curr}&to=#{to_curr}"
doc = Nokogiri::HTML(open(url))
result_span = doc.css('span.bld').text
result = result_span.tr('^0-9.', '')
rescue => e
puts e
result = 1
end
end
result
end
end
I haven't done this a lot, so my question would be: How can I deal with the (unlikely) event that Google Currency times out or is not available for some reason?
In that case I would like my result to be 1. How can this be achieved?
Thanks for any suggestions.
You can simply rescue any error that Nokogiri might raise (or OpenURI) like so:
require 'nokogiri'
require 'open-uri'
def currency(a)
Nokogiri::HTML(open(a))
rescue => e
puts e
1 # default value when error is raised
end
puts currency('https://www.somedomainthatdoesntexist.com')
That will print the backtrace of the error and then return the number 1
getaddrinfo: nodename nor servname provided, or not known
1
For a custom timeout actions you can use Timeout module
http://ruby-doc.org/stdlib-2.1.2/libdoc/timeout/rdoc/Timeout.html
It'd be worth looking at this answer to see how to get nokogiri to timeout : Adjusting timeouts for Nokogiri connections
The question makes use of the timeout module which is an option but I think it is better to get the connection to time out as per the accepted answer.
All that will mean that you'll get a Timeout::Error exception raised if the call times out which you then need to handle:
begin
doc = ... use nokogiri ...
result_span = doc.css('span.bld').text
result = result_span.tr('^0-9.', '')
rescue Timeout::Error
result = 1
end
result
The following code
purchase = #order.authorize_payment(#credit_card, options)
is_success = purchase.success?
if is_success
...
else
flash[:notice] = "!! " + purchase.message + "" +
purchase.params['missingField'].to_s
redirect_to :action => :payment, :id => #order.id
end
results in "!! Failed with 500 Internal Server Error" in my flash[:notice]. There is no stacktrace, no webserver error, all that I know is that purchase.message is populated and purchase.success? is false.
I am really at a loss to figure out how to troubleshoot this. I think it might be an ssl requirement, but I can't either see the soap request, or test basic connectivity with cybersource (my payment gateway).
I establish my gateway with this code (after config.after_initialize do):
ActiveMerchant::Billing::Base.mode = :production # :test
ActiveMerchant::Billing::CreditCard.require_verification_value = false
ActiveMerchant::Billing::CyberSourceGateway.wiredump_device = File.new(File.join([Rails.root, "log", "cybersource.log"]), "a") # doesn't work (!)
# we need to open an external file to get the password
mypassphrase = File.open('/var/www/foo/shared/passphrase.txt').read
OrderTransaction.gateway = ActiveMerchant::Billing::CyberSourceGateway.new(:login => 'vxxxxxxx',
:password => mypassphrase.to_s,
:test => false,
:vat_reg_number => 'your VAT registration number',
# sets the states/provinces where you have a physical presense for tax purposes
:nexus => "GA OH",
# don‘t want to use AVS so continue processing even if AVS would have failed
:ignore_avs => true,
# don‘t want to use CVV so continue processing even if CVV would have failed
:ignore_cvv => true,
:money_format => :dollars
)
Can I see the soap request? Are there ways to test part of this? Any help greatly appreciated.
Best,
Tim
ActiveMerchant::Billing::CyberSourceGateway.logger = your_logger
So, late response but...
I've done a good amount of work with the Cybersource gateway, and the only way to see the SOAP request/response of the cybersource gateway currently is to open up the gem and edit it.
If you modify the commit method of lib/active_merchant/billing/gateways/cybersource.rb, you can do something like this:
def commit(request, options)
puts "*** POSTING TO: #{test? ? TEST_URL : LIVE_URL}"
request = build_request(request, options)
puts "*** POSTING:"
puts request
begin
post_response = ssl_post(test? ? TEST_URL : LIVE_URL, request)
rescue ActiveMerchant::ResponseError => e
puts "ERROR!"
puts e.response
end
puts post_response
It would be nice if there was a way to get that response without going through that hassle, I'll see if there's a way to pass that information up through the response object that's returned and add it to my fork.
I'm testing out a piece of code to ping a bunch of websites I own on a regular basis, to make sure they're up.
I'm using rails and so far I have this hideous test action that I'm using to try it out (see below).
The problem though, is that sometimes it works, and other times it won't ... sometimes it runs through the code just fine, other times, it seems to completely ignore the begin/rescue block ...
a. I need help figuring out what the problem is
b. And refactoring this to make it look respectable.
Your help is much appreciated.
edit 1: Here is the updated code, sorry it took so long, pastie.org was down since yesterday http://pastie.org/927201
Its still doing the same thing ... skipping the begin block (because it only updates up_check_time) ... however if one of the sites times out, it actually updates everything (check_msg, code etc) correctly ... confusing, yeah?
require 'net/http'
require 'uri'
def ping
#sites = NewsSource.all
#sites.each do |site|
if site.uri and !site.uri.empty?
uri = URI.parse(site.uri)
response = nil
path = uri.path.blank? ? '/' : uri.path
path = uri.query.blank? ? path : "#{path}?#{uri.query}"
begin
Net::HTTP.start(uri.host, uri.port) {|http|
http.open_timeout = 30
http.read_timeout = 30
response = http.head(path)
}
if response.code.eql?('200') or response.code.eql?('301') or response.code.eql?('302')
site.up = true
else
site.up = false
end
site.up_check_msg = response.message
site.up_check_code = response.code
rescue Errno::EBADF
rescue Timeout::Error
site.up = false
site.up_check_msg = 'timeout'
site.up_check_code = '408'
end
site.up_check_time = 0.seconds.ago
site.save
end
end
end
You currently have an empty rescue block for Errno::EBADF so if that exception is raised then you will not be setting site.up to false.
Also, a couple of other minor improvements:
Instead of if site.uri and !site.uri.empty? you can use:
next if site.uri.nil? or site.uri.empty?
to skip that iteration of the each loop and avoid indenting the code by an additional level.
And:
if response.code.eql?('200') or response.code.eql?('301') or response.code.eql?('302')
site.up = true
else
site.up = false
end
can be written more concisely:
site.up = ['200', '301', '302'].include? response.code
If you tidy up the code with some of these tips then it might help narrow down the problem.
Here's a snippet from one of my programs, maybe it helps:
urls.each_with_index do |url, idx|
print "Processing URL #%04d: " % (idx+1)
uri = URI.parse(url)
response = nil
begin
Net::HTTP.start(uri.host, uri.port) do |http|
response = http.head(uri.path.size > 0 ? uri.path : "/")
end
rescue => e
puts "#{e.message} - #{url}"
next
end
# handle redirects
if response.is_a?(Net::HTTPRedirection)
new_uri = URI.parse(response['location'])
puts "URI redirects to #{new_uri}"
next
end
puts case response.code
when '200' then ...
when '404' then ...
else ...
end
end
The only thing that I can think of is that you are getting some other exception in your begin block. Since you are only explicitly rescuing Errno::EBADF, Timeout::Error it would appear that your begin and rescue got skipped. You might be able to verify this by getting rid of Errno::EBADF, Timeout::Error and just having a plain rescue, then put the following in your rescue block
logger.info(">>Exception was: "+$!)
Then look in your logs to see what exceptions you are getting.
If you are monitoring your servers why not use Nagios? it's free and also has some Ruby support, Here and Here.
EDIT:
Ruby GEM: http://hobodave.com/2010/01/10/simple-nagios-probes-in-ruby/