code to ping websites works sometimes - ruby-on-rails

I'm testing out a piece of code to ping a bunch of websites I own on a regular basis, to make sure they're up.
I'm using rails and so far I have this hideous test action that I'm using to try it out (see below).
The problem though, is that sometimes it works, and other times it won't ... sometimes it runs through the code just fine, other times, it seems to completely ignore the begin/rescue block ...
a. I need help figuring out what the problem is
b. And refactoring this to make it look respectable.
Your help is much appreciated.
edit 1: Here is the updated code, sorry it took so long, pastie.org was down since yesterday http://pastie.org/927201
Its still doing the same thing ... skipping the begin block (because it only updates up_check_time) ... however if one of the sites times out, it actually updates everything (check_msg, code etc) correctly ... confusing, yeah?
require 'net/http'
require 'uri'
def ping
#sites = NewsSource.all
#sites.each do |site|
if site.uri and !site.uri.empty?
uri = URI.parse(site.uri)
response = nil
path = uri.path.blank? ? '/' : uri.path
path = uri.query.blank? ? path : "#{path}?#{uri.query}"
begin
Net::HTTP.start(uri.host, uri.port) {|http|
http.open_timeout = 30
http.read_timeout = 30
response = http.head(path)
}
if response.code.eql?('200') or response.code.eql?('301') or response.code.eql?('302')
site.up = true
else
site.up = false
end
site.up_check_msg = response.message
site.up_check_code = response.code
rescue Errno::EBADF
rescue Timeout::Error
site.up = false
site.up_check_msg = 'timeout'
site.up_check_code = '408'
end
site.up_check_time = 0.seconds.ago
site.save
end
end
end

You currently have an empty rescue block for Errno::EBADF so if that exception is raised then you will not be setting site.up to false.
Also, a couple of other minor improvements:
Instead of if site.uri and !site.uri.empty? you can use:
next if site.uri.nil? or site.uri.empty?
to skip that iteration of the each loop and avoid indenting the code by an additional level.
And:
if response.code.eql?('200') or response.code.eql?('301') or response.code.eql?('302')
site.up = true
else
site.up = false
end
can be written more concisely:
site.up = ['200', '301', '302'].include? response.code
If you tidy up the code with some of these tips then it might help narrow down the problem.

Here's a snippet from one of my programs, maybe it helps:
urls.each_with_index do |url, idx|
print "Processing URL #%04d: " % (idx+1)
uri = URI.parse(url)
response = nil
begin
Net::HTTP.start(uri.host, uri.port) do |http|
response = http.head(uri.path.size > 0 ? uri.path : "/")
end
rescue => e
puts "#{e.message} - #{url}"
next
end
# handle redirects
if response.is_a?(Net::HTTPRedirection)
new_uri = URI.parse(response['location'])
puts "URI redirects to #{new_uri}"
next
end
puts case response.code
when '200' then ...
when '404' then ...
else ...
end
end

The only thing that I can think of is that you are getting some other exception in your begin block. Since you are only explicitly rescuing Errno::EBADF, Timeout::Error it would appear that your begin and rescue got skipped. You might be able to verify this by getting rid of Errno::EBADF, Timeout::Error and just having a plain rescue, then put the following in your rescue block
logger.info(">>Exception was: "+$!)
Then look in your logs to see what exceptions you are getting.

If you are monitoring your servers why not use Nagios? it's free and also has some Ruby support, Here and Here.
EDIT:
Ruby GEM: http://hobodave.com/2010/01/10/simple-nagios-probes-in-ruby/

Related

How to catch a Rack RangeError in Rails 6

I have a Rails 6 app to which users can upload CSV files. Rails/Rack imposes a limit in the number of params that can be included in a request, and I've set this to a size larger than likely submissions to my app. However, I would like to return a friendly response if a too-large file is uploaded.
It looks like I need to add some custom middleware, to catch and rescue the error, but I can't get the code to work - the basic error is still raised without my rescue block being called.
The error from the server is:
Rack app error handling request { POST /[PATH_TO]/datasets }
#<RangeError: exceeded available parameter key space>
The code in my app/middleware/catch_errors.rb file is basically taken from a previous SO answer, where someone was catching ActionDispatch::ParamsParser::ParseError in JSON, but with my own code in the rescue block (which I realise may not work properly in this context, but that's not the issue right now):
class CatchErrors
def initialize(_app)
#app = _app
end
def call(_env)
begin
#app.call(_env)
rescue RangeError => _error
_error_output = "There were too many fields in the data you submitted: #{_error}"
if env['HTTP_ACCEPT'] =~ /application\/html/
Rails.logger.error("Caught RangeError: #{_error}")
flash[:error_title] = 'Too many fields in your data'
flash[:error_detail1] = _error_output
render 'static_pages/error', status: :bad_request
elsif env['HTTP_ACCEPT'] =~ /application\/json/
return [
:bad_request, { "Content-Type" => "application/json" },
[ { status: :bad_request, error: _error_output }.to_json ]
]
else
raise _error
end
end
end
end
I'm loading it in config.application.rb like this:
require_relative '../app/middleware/catch_errors'
...
config.middleware.use CatchErrors
I'm resetting the size limit for testing in app/initializers/rack.rb like this:
if Rack::Utils.respond_to?("key_space_limit=")
Rack::Utils.key_space_limit = 1
end
Any help gratefully received!
First, execute command to see all middlewares:
bin/rails middleware
config.middleware.use place your middleware at the bottom of the middleware stack. Because of that it can not catch error. Try to place it at the top:
config.middleware.insert_before 0, CatchErrors
Another point to mention, may be you will need to config.middleware.move_after or even config.middleware.delete some middleware. For instance, while tinkering I needed to place:
config.middleware.move_after CatchErrors, Rack::MiniProfiler

How to deal with timeout in currency conversion service?

I just created this little currency converter for my Rails 4 app:
module Currency
def self.get_exchange_rate(from_curr = "EUR", to_curr = "USD")
if from_curr == to_curr
result = 1
else
begin
amount = 1
url = "http://www.google.com/finance/converter?a=#{amount}&from=#{from_curr}&to=#{to_curr}"
doc = Nokogiri::HTML(open(url))
result_span = doc.css('span.bld').text
result = result_span.tr('^0-9.', '')
rescue => e
puts e
result = 1
end
end
result
end
end
I haven't done this a lot, so my question would be: How can I deal with the (unlikely) event that Google Currency times out or is not available for some reason?
In that case I would like my result to be 1. How can this be achieved?
Thanks for any suggestions.
You can simply rescue any error that Nokogiri might raise (or OpenURI) like so:
require 'nokogiri'
require 'open-uri'
def currency(a)
Nokogiri::HTML(open(a))
rescue => e
puts e
1 # default value when error is raised
end
puts currency('https://www.somedomainthatdoesntexist.com')
That will print the backtrace of the error and then return the number 1
getaddrinfo: nodename nor servname provided, or not known
1
For a custom timeout actions you can use Timeout module
http://ruby-doc.org/stdlib-2.1.2/libdoc/timeout/rdoc/Timeout.html
It'd be worth looking at this answer to see how to get nokogiri to timeout : Adjusting timeouts for Nokogiri connections
The question makes use of the timeout module which is an option but I think it is better to get the connection to time out as per the accepted answer.
All that will mean that you'll get a Timeout::Error exception raised if the call times out which you then need to handle:
begin
doc = ... use nokogiri ...
result_span = doc.css('span.bld').text
result = result_span.tr('^0-9.', '')
rescue Timeout::Error
result = 1
end
result

rescue from connection reset by peer error and retry

I am hitting an external service which does some password encryption and returns couple of things.
Now if I want to generate 50 passwords we run this function in a loop 50 times
def self.encrypt_password(password)
retries = 2
uri = URI
params = Base64.encode64(password)
uri.query = URI.encode("Source=#{params}")
begin
retries.times.each do
res = Net::HTTP.get_response(uri)
if res.is_a?(Net::HTTPSuccess)
obj = JSON.parse(res.body)
pw = Base64.decode64(obj["Data"])
ps = Base64.decode64(obj["Key"])
pws = Iconv.iconv('ascii', 'utf-16', pws)
return pwe,pws[0]
end
end
rescue
raise "Error generating pws: #{$!}"
end
end
But the problem, i am encountering is that there are occasions when the service just returns the following error in the middle of a loop and exits:
"Connection reset by Peer error"
My question is how do I rescue from that error and retry a few times without breaking the flow of the program?
Or can someone recommend alternate solutions to my problem?
NOTE: I am using ruby on rails 2 and ruby 1.8.x
Ruby has the retry method, that can be used in the rescue clause.
It just runs the current method again, so you can use a counter to limit the number of retries:
def self.encrypt_password(password)
retries = 2
uri = URI
params = Base64.encode64(password)
uri.query = URI.encode("Source=#{params}")
retries.times.each do
res = Net::HTTP.get_response(uri)
if res.is_a?(Net::HTTPSuccess)
obj = JSON.parse(res.body)
pw = Base64.decode64(obj["Data"])
ps = Base64.decode64(obj["Key"])
pws = Iconv.iconv('ascii', 'utf-16', pws)
return pwe,pws[0]
end
end
rescue SomeExceptionType
if retries > 0
retries -= 1
retry
else
raise "Error generating pws: #{$!}"
end
end
end

How to specify a read timeout for a Net::HTTP::Post.new request in Ruby 2

I have a post happening to a rails application from a ruby script. The script creates a variable request as
request = Net::HTTP::Post.new(url.path)
which is then used as follows
request.content_type = "application/json"
request.body = JSON.generate( params )
response = Net::HTTP.start(url.host, url.port) {|http| http.request(request)}
There is quite a lot of processing happening on the server side, and I'm getting a Net::ReadTimeout error
I tried to specify a timeout period
request.read_timeout = 500
as per this stackoverflow answer but I got a
undefined method `read_timeout=' for #<Net::HTTP::Post POST> (NoMethodError)
error. I assume that I'm missing something simple somewhere. All clues gratefully received
Technical info:
Ruby 2.0.0p247
Rails 4.0.0
Windows 7 32 bit ruby
Solved via this stackoverflow answer
I've changed my
response = Net::HTTP.start(url.host, url.port) {|http| http.request(request)}
line to be
response = Net::HTTP.start(url.host, url.port, :read_timeout => 500) {|http| http.request(request)}
and this seems to have got around this problem.
The read_timeout is available with a plain Net::HTTP object:
url = URI.parse('http://google.com')
http = Net::HTTP.new(url.host, url.port)
http.read_timeout = 5 # seconds
http.request_post(url.path, JSON.generate(params)) do |response|
# do something with response
p response
end
One thing to keep in mind is that if read_timeout is set to a small value such that a timeout does occur...Net::HTTP will "helpfully" retry the request. For a slow HTTP server, a timeout error may not be raised to the code calling Net::HTTP until 2x the read_timeout value.
This certainly was not the behavior I expected.
More info on this topic and how possible solutions differ for Ruby < 2.5 and >= 2.5 may be found here:
https://stackoverflow.com/a/59186209/5299483
I catch both OpenTimeout and ReadTimeout and it's work. test in Ruby:2.6.5
def ping(host, port)
begin
url = URI.parse("http://#{host}:#{port}/ping")
req = Net::HTTP::Get.new(url.to_s)
# setting both OpenTimeout and ReadTimeout
res = Net::HTTP.start(url.host, url.port, :open_timeout => 3, :read_timeout => 3) {|http|
http.request(req)
}
if JSON.parse(res.body)["ok"]
# return true
STDERR.puts "#{host}:#{port} is reachable"
else
STDERR.puts "#{host}:#{port} is NOT reachable"
end
rescue Net::ReadTimeout => exception
STDERR.puts "#{host}:#{port} is NOT reachable (ReadTimeout)"
rescue Net::OpenTimeout => exception
STDERR.puts "#{host}:#{port} is NOT reachable (OpenTimeout)"
end
end
ping("#{ENV['FIRST_HOST']}", 2345)
ping("#{ENV['SECOND_HOST']}", 2345)
If anyone is still facing timeout setting issue and Net::HTTP timeout not working as expected, then you may follow below approach as well:
begin
Timeout::timeout(10) {
####
## YOUR REQUEST CODE WILL BE HERE
####
}
rescue
408
end

Rails/unicode issue

I have a bit of my Ruby/Rails (Ruby 2.0.0p195, Rails 3.2.13) project that works as a proxy; that is, you pass it a URL, it goes out and fetches the page, and presents it to you. This generally works as expected, but it seems to munge certain characters (such as è).
A simplified version of the controller is this:
class HomeController < ApplicationController
def geoproxy
require 'net/http'
require 'timeout'
rawurl = CGI::unescape(params[:url])
fixedurl = rawurl.gsub('\\', '%5C') # Escape backslashes... why oh why???!?
r = nil;
status = 200
content_type = ''
begin
Timeout::timeout(15) { # Time, in seconds
if request.get? then
res = Net::HTTP.get_response(URI.parse(fixedurl))
status = res.code # If there was an error, pass that code back to our caller
#page = res.body.encode('UTF-8')
content_type = res['content-type']
end
}
rescue Timeout::Error
#page = "TIMEOUT"
status = 504 # 504 Gateway Timeout We're the gateway, we timed out. Seems logical.
end
render :layout => false, :status => status, :content_type => content_type
end
end
The corresponding view is quite simple:
<%= raw #page %>
When I use this proxy to fetch XML containing an è (for example), I get the following error:
Encoding::UndefinedConversionError in HomeController#geoproxy
"\xE8" from ASCII-8BIT to UTF-8
This error occurs at the following line:
#page = res.body.encode('UTF-8')
If I remove the .encode(), the error is resolved, but my XML contains a placeholder instead of the è.
How can I get my project to display the XML properly?
Could you check if the following code works for you? I was able to fix similar problem of mine with it.
#page = res.body.force_encoding('Windows-1254').encode('UTF-8')

Resources