I am writing a twitter tool that harvests some data. Below is a snippet of the code
replies_without_root_tweet.each do |r|
begin
t = client.status(r.in_reply_to_status_id)
RootTweet.find_or_create(t)
rescue Twitter::Error::NotFound,Twitter::Error::Forbidden => e
puts e
end
Now the thing is, the Twitter search API has rate limit which i hit a lot. The issue here is how can i resume the process in 15 minutes if i hit this exception
Twitter::Error::TooManyRequests
As you can see i rescue from another two exceptions, if i add the too many requests exception as well, it will be a problem as i will probably hit that exception all the time unless a specified amount of time passes.
Is there a way to know when a specific exception fires up so i can sleep the process?
You can have as many rescue statements as you want, so you can do this to handle TooManyRequests differently than the others:
begin
t = client.status(r.in_reply_to_status_id)
RootTweet.find_or_create(t)
rescue Twitter::Error::NotFound, Twitter::Error::Forbidden => e
puts e
rescue Twitter::Error::TooManyRequests => e
puts e
sleep 10
end
You can also ask the error object what it is, e.g. e.is_a?(Twitter::Error::TooManyRequests).
Related
I am parsing JSON output by using the curl mentioned below ,
However I am getting no method error while parsing the output due to DESCRIPTION #{h["incident_updates"][1]["status"] . because [["incident_updates"][1][status]] is not present in some cases, only the values of [["incident_updates"][0][status]] is available .
( But others iteration contains values for both [0][status]
[1][status] )
So to avoid the error used rescue method ,however error is prevented but script is not executing after the first error(iteration stopped on first error itself, script is not continuing ) . Need to run the script till the iteration completes (i.e., though no values present for [1] [status] script should run to bring the value for next iteration element because next element may contain [1] [status])
Please help on this .
Thanks in advance
def inc
begin
page3 = `curl https://api.statuspage.io/v1/pages/incidents.json?page=3 -H "Authorization: OAuth a8ef42"`
JSON.parse(page3).each do |h|
puts "ID : #{h["id"]} , CREATED AT : #{h["created_at"]} , LINK : #{h["shortlink"]} , ISSUE NAME : #{h["name"]} , DESCRIPTION #{h["incident_updates"][0]["status"]} , DESCRIPTION #{h["incident_updates"][1]["status"]}"
end
rescue NoMethodError => e
end
end
Try this:
def inc
page3 = `curl https://api.statuspage.io/v1/pages/incidents.json?page=3 -H "Authorization: OAuth a8ef42"`
JSON.parse(page3).each do |h|
begin
puts "ID : #{h["id"]} , CREATED AT : #{h["created_at"]} , LINK : #{h["shortlink"]} , ISSUE NAME : #{h["name"]} , DESCRIPTION #{h["incident_updates"][0]["status"]} , DESCRIPTION #{h["incident_updates"][1]["status"]}"
rescue NoMethodError => e
puts e
end
end
end
Explanation:
Whenever the exception is caught it tries to exit out of the block in which the exception has occurred.
In your previous code, you're handling it in the scope of the function. So, when the exception was occurring in the iteration it was exiting out of the loop because because it wasn't handled inside the scope in which it was occurring (loop) and was caught right outside the loop because you wrote it there (outside the loop).
To continue the iteration process you must handle it where it was occurring so that the system must know that it's been handled perfectly and it can perform the next iteration.
uniqUsers = User.find(params[:userid]).events.where("comingfrom != ''").uniq_by {|obj| obj.comingfrom}
uniqUsers.map do |elem|
begin
#tag = nil
open('http://localhost:3000/search_dbs/show?userid='+ params[:userid] + '&fromnumber=' + elem.comingfrom + '&format=json', 'r', :read_timeout=>1) do |http|
#tag = http.read
end
rescue Exception => e
puts "failes"
puts e
end
end
hi , this is driving me crazy , for some reason the open url command is running out of time with no error. when i try the same url in chrome everything works like a charm, when im doing this from the code i get Timeout::Error
One second is optimistic.
When I was writing spiders, I'd create a retry queue, that contained sub-arrays or objects that contain the number of retries previously attempted, the URL, and maybe the last timeout value. Using an incrementing timeout value, the first time I'd try one second, the second try two seconds, four, eight, sixteen, etc. until I determined the site wasn't going to respond.
Imagine this scenario (just a sample)
file = open("/file1")
file2 = open("/file2")
file3 = open("/file3")
How can i handle this situation, what i want to do is allow statements that don't rise exception run while catching any exception that they might rise, In other words i have 3 lines that can rise the same exception, how should this be handled. i can check the msg, but still if the first line throws an exception the next two lines won't run.
a relating question is can i have a nesting rescue blocks (meaning throw and rescue an exception inside a rescue block?)
You might want a loop for this specific case:
open_files = ['file1', 'file2', 'file3'].map do |file|
begin
File.open file
rescue => e
...
end
end
Well, figured it out.
not the smartest question!
just have each one in different method and call them.
Loadfile1()
Loadfile2()
Loadfile3()
loadfile1()
file1 = open("/file1")
rescue......
...
end
def checkdomains
#domains = Domain.all
##domains.where(:confirmed => "yes").each do |f|
#domains.each do |f|
r = Whois.whois(f.domain)
if r.available? == true
EmailNotify.notify_email(f).deliver
end
end
end
This method crashes when it comes upon an invalid url (the whois gem gives an error), and doesn't keep on checking the rest of the domains. Is there any way I can have it continue to check the rest of the domains even if it crashes on one? At least until I can sort out phising out each domain.
#domains.each do |f|
begin
r = Whois.whois(f.domain)
if r.available? == true
EmailNotify.notify_email(f).deliver
end
rescue Exception => e
puts "Error #{e}"
next # <= This is what you were looking for
end
end
When you say
crashing out
I assume you mean that you are getting an exception raised. If this is the case then just trap the exception, do what you want with it (Store the address in a bad_email table or whatever) then carry on doing what you are doing. Your log file will tell what exception is being raised so you know what your rescue statement should be
so
begin
r = Whois.whois(f.domain)
if r.available? == true
EmailNotify.notify_email(f).deliver
rescue WhateverException
#do something here like re raise the error or store the email address in a bad_emails table or do both just simply do nothing at all
end
If you are referring to something else like the whole app dying then I haven'ty got a clue and there is not enough info to advise further. Sorry
As jamesw suggests, you can wrap the statements in an exception handler, dealing with them as they occur. Let me suggest further that, wherever your program gets these (possibly invalid) domain names, you validate them as soon as you get them, and throw out the invalid ones. That way, by the time you reach this loop, you already know you're iterating over a list of good domains.
EDIT: For domain name validation, check here.
I have this piece of code:
begin
complete_results = Timeout.timeout(4) do
results = platform.search(artist, album_name)
end
rescue Timeout::Error
puts 'Print me something please'
end
I then launch the method containing this code, and well, here is the beginning of a stack trace:
Exception message : execution expired
Exception backtrace : /***/****/.rvm/rubies/ruby-1.8.7-p302/lib/ruby/1.8/timeout.rb:64:i
So I naively thinks that my call timed out, but 'Print me something please' is never printed and complete_results which is suppose to be the timeout status return value (either true or false, as mentioned in the documentation), is definitively not a boolean.
Am I doing something wrong?
Your code is correct
require 'timeout'
begin
complete_results = Timeout.timeout(1) do
sleep(2)
end
rescue Timeout::Error
puts 'Print me something please'
end
does print out "print me something please".
Try the basic code as above. If that works, you have an issue in platform.search.
The problem is that platform.search is catching the exception that Timeout#timeout throws.
You can get around this by wrapping your inner code in another thread -- YMMV.
begin
complete_results = Timeout.timeout(4) do
Thread.new{ results = platform.search(artist, album_name) }.value
end
rescue Timeout::Error
puts 'Print me something please'
end
According to the documentation:
If the block execution terminates
before sec seconds has passed, it
returns true. If not, it terminates
the execution and raises exception
(which defaults to Timeout::Error)
This means it only returns true if it's successful, otherwise the variable will not be set (ie it's nil NOT false).
As far as your example goes, it's definitely timing out for me and getting to the rescue part...