I am parsing JSON output by using the curl mentioned below ,
However I am getting no method error while parsing the output due to DESCRIPTION #{h["incident_updates"][1]["status"] . because [["incident_updates"][1][status]] is not present in some cases, only the values of [["incident_updates"][0][status]] is available .
( But others iteration contains values for both [0][status]
[1][status] )
So to avoid the error used rescue method ,however error is prevented but script is not executing after the first error(iteration stopped on first error itself, script is not continuing ) . Need to run the script till the iteration completes (i.e., though no values present for [1] [status] script should run to bring the value for next iteration element because next element may contain [1] [status])
Please help on this .
Thanks in advance
def inc
begin
page3 = `curl https://api.statuspage.io/v1/pages/incidents.json?page=3 -H "Authorization: OAuth a8ef42"`
JSON.parse(page3).each do |h|
puts "ID : #{h["id"]} , CREATED AT : #{h["created_at"]} , LINK : #{h["shortlink"]} , ISSUE NAME : #{h["name"]} , DESCRIPTION #{h["incident_updates"][0]["status"]} , DESCRIPTION #{h["incident_updates"][1]["status"]}"
end
rescue NoMethodError => e
end
end
Try this:
def inc
page3 = `curl https://api.statuspage.io/v1/pages/incidents.json?page=3 -H "Authorization: OAuth a8ef42"`
JSON.parse(page3).each do |h|
begin
puts "ID : #{h["id"]} , CREATED AT : #{h["created_at"]} , LINK : #{h["shortlink"]} , ISSUE NAME : #{h["name"]} , DESCRIPTION #{h["incident_updates"][0]["status"]} , DESCRIPTION #{h["incident_updates"][1]["status"]}"
rescue NoMethodError => e
puts e
end
end
end
Explanation:
Whenever the exception is caught it tries to exit out of the block in which the exception has occurred.
In your previous code, you're handling it in the scope of the function. So, when the exception was occurring in the iteration it was exiting out of the loop because because it wasn't handled inside the scope in which it was occurring (loop) and was caught right outside the loop because you wrote it there (outside the loop).
To continue the iteration process you must handle it where it was occurring so that the system must know that it's been handled perfectly and it can perform the next iteration.
Related
I am writing a twitter tool that harvests some data. Below is a snippet of the code
replies_without_root_tweet.each do |r|
begin
t = client.status(r.in_reply_to_status_id)
RootTweet.find_or_create(t)
rescue Twitter::Error::NotFound,Twitter::Error::Forbidden => e
puts e
end
Now the thing is, the Twitter search API has rate limit which i hit a lot. The issue here is how can i resume the process in 15 minutes if i hit this exception
Twitter::Error::TooManyRequests
As you can see i rescue from another two exceptions, if i add the too many requests exception as well, it will be a problem as i will probably hit that exception all the time unless a specified amount of time passes.
Is there a way to know when a specific exception fires up so i can sleep the process?
You can have as many rescue statements as you want, so you can do this to handle TooManyRequests differently than the others:
begin
t = client.status(r.in_reply_to_status_id)
RootTweet.find_or_create(t)
rescue Twitter::Error::NotFound, Twitter::Error::Forbidden => e
puts e
rescue Twitter::Error::TooManyRequests => e
puts e
sleep 10
end
You can also ask the error object what it is, e.g. e.is_a?(Twitter::Error::TooManyRequests).
uniqUsers = User.find(params[:userid]).events.where("comingfrom != ''").uniq_by {|obj| obj.comingfrom}
uniqUsers.map do |elem|
begin
#tag = nil
open('http://localhost:3000/search_dbs/show?userid='+ params[:userid] + '&fromnumber=' + elem.comingfrom + '&format=json', 'r', :read_timeout=>1) do |http|
#tag = http.read
end
rescue Exception => e
puts "failes"
puts e
end
end
hi , this is driving me crazy , for some reason the open url command is running out of time with no error. when i try the same url in chrome everything works like a charm, when im doing this from the code i get Timeout::Error
One second is optimistic.
When I was writing spiders, I'd create a retry queue, that contained sub-arrays or objects that contain the number of retries previously attempted, the URL, and maybe the last timeout value. Using an incrementing timeout value, the first time I'd try one second, the second try two seconds, four, eight, sixteen, etc. until I determined the site wasn't going to respond.
def checkdomains
#domains = Domain.all
##domains.where(:confirmed => "yes").each do |f|
#domains.each do |f|
r = Whois.whois(f.domain)
if r.available? == true
EmailNotify.notify_email(f).deliver
end
end
end
This method crashes when it comes upon an invalid url (the whois gem gives an error), and doesn't keep on checking the rest of the domains. Is there any way I can have it continue to check the rest of the domains even if it crashes on one? At least until I can sort out phising out each domain.
#domains.each do |f|
begin
r = Whois.whois(f.domain)
if r.available? == true
EmailNotify.notify_email(f).deliver
end
rescue Exception => e
puts "Error #{e}"
next # <= This is what you were looking for
end
end
When you say
crashing out
I assume you mean that you are getting an exception raised. If this is the case then just trap the exception, do what you want with it (Store the address in a bad_email table or whatever) then carry on doing what you are doing. Your log file will tell what exception is being raised so you know what your rescue statement should be
so
begin
r = Whois.whois(f.domain)
if r.available? == true
EmailNotify.notify_email(f).deliver
rescue WhateverException
#do something here like re raise the error or store the email address in a bad_emails table or do both just simply do nothing at all
end
If you are referring to something else like the whole app dying then I haven'ty got a clue and there is not enough info to advise further. Sorry
As jamesw suggests, you can wrap the statements in an exception handler, dealing with them as they occur. Let me suggest further that, wherever your program gets these (possibly invalid) domain names, you validate them as soon as you get them, and throw out the invalid ones. That way, by the time you reach this loop, you already know you're iterating over a list of good domains.
EDIT: For domain name validation, check here.
Hi I've this piece of code
class Place < ActiveRecord::Base
def self.find_or_create_by_latlon(lat, lon)
place_id = call_external_webapi
result = Place.where(:place_id => place_id).limit(1)
result = Place.create(:place_id => place_id, ... ) if result.empty? #!
result
end
end
Then I'd like to do in another model or controller
p = Post.new
p.place = Place.find_or_create_by_latlon(XXXXX, YYYYY) # race-condition
p.save
But Place.find_or_create_by_latlon takes too much time to get the data if the action executed is create and sometimes in production p.place is nil.
How can I force to wait for the response before execute p.save ?
thanks for your advices
You're right that this is a race condition and it can often be triggered by people who double click submit buttons on forms. What you might do is loop back if you encounter an error.
result = Place.find_by_place_id(...) ||
Place.create(...) ||
Place.find_by_place_id(...)
There are more elegant ways of doing this, but the basic method is here.
I had to deal with a similar problem. In our backend a user is is created from a token if the user doesn't exist. AFTER a user record is already created, a slow API call gets sent to update the users information.
def self.find_or_create_by_facebook_id(facebook_id)
User.find_by_facebook_id(facebook_id) || User.create(facebook_id: facebook_id)
rescue ActiveRecord::RecordNotUnique => e
User.find_by_facebook_id(facebook_id)
end
def self.find_by_token(token)
facebook_id = get_facebook_id_from_token(token)
user = User.find_or_create_by_facebook_id(facebook_id)
if user.unregistered?
user.update_profile_from_facebook
user.mark_as_registered
user.save
end
return user
end
The step of the strategy is to first remove the slow API call (in my case update_profile_from_facebook) from the create method. Because the operation takes so long, you are significantly increasing the chance of duplicate insert operations when you include the operation as part of the call to create.
The second step is to add a unique constraint to your database column to ensure duplicates aren't created.
The final step is to create a function that will catch the RecordNotUnique exception in the rare case where duplicate insert operations are sent to the database.
This may not be the most elegant solution but it worked for us.
I hit this inside a sidekick job that retries and gets the error repeatedly and eventually clears itself. The best explanation I've found is on a blog post here. The gist is that postgres keeps an internally stored value for incrementing the primary key that gets messed up somehow. This rings true for me because I'm setting the primary key and not just using an incremented value so that's likely how this cropped up. The solution from the comments in the link above appears to be to call ActiveRecord::Base.connection.reset_pk_sequence!(table_name) This cleared up the issue for me.
begin
result = Place.where(:place_id => place_id).limit(1)
result = Place.create(:place_id => place_id, ... ) if result.empty? #!
rescue ActiveRecord::StatementInvalid => error
#save_retry_count = (#save_retry_count || 1)
ActiveRecord::Base.connection.reset_pk_sequence!(:place)
retry if( (#save_retry_count -= 1) >= 0 )
raise error
end
I have this piece of code:
begin
complete_results = Timeout.timeout(4) do
results = platform.search(artist, album_name)
end
rescue Timeout::Error
puts 'Print me something please'
end
I then launch the method containing this code, and well, here is the beginning of a stack trace:
Exception message : execution expired
Exception backtrace : /***/****/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/timeout.rb:64:i
So I naively thinks that my call timed out, but 'Print me something please' is never printed and complete_results which is suppose to be the timeout status return value (either true or false, as mentioned in the documentation), is definitively not a boolean.
Am I doing something wrong?
Your code is correct
require 'timeout'
begin
complete_results = Timeout.timeout(1) do
sleep(2)
end
rescue Timeout::Error
puts 'Print me something please'
end
does print out "print me something please".
Try the basic code as above. If that works, you have an issue in platform.search.
The problem is that platform.search is catching the exception that Timeout#timeout throws.
You can get around this by wrapping your inner code in another thread -- YMMV.
begin
complete_results = Timeout.timeout(4) do
Thread.new{ results = platform.search(artist, album_name) }.value
end
rescue Timeout::Error
puts 'Print me something please'
end
According to the documentation:
If the block execution terminates
before sec seconds has passed, it
returns true. If not, it terminates
the execution and raises exception
(which defaults to Timeout::Error)
This means it only returns true if it's successful, otherwise the variable will not be set (ie it's nil NOT false).
As far as your example goes, it's definitely timing out for me and getting to the rescue part...