I use this code to get info of an youtube video(youtube_itgem)
video_info = Video.yt_session.video_by(self.source_id)
hits = video_info.view_count
if video_info.rating
self.rating = video_info.rating.average
self.likes = video_info.rating.likes.to_i
end
The problem is that sometimes, the original youtube video is deleted, like this: https://www.youtube.com/watch?v=lsZFkiimpKI.
So video_info = Video.yt_session.video_by(self.source_id) will reports OpenURI::HTTPError: 404, and the code cannot continue to get info of next videos.
I want to delete this non-exist video its source is not available anymore, how can I do that? Maybe by rescue?
I tried if Video.yt_session.video_by(self.source_id), but it doesn't work.
Did you try the following:
begin
video_info = Video.yt_session.video_by(self.source_id)
hits = video_info.view_count
if video_info.rating
self.rating = video_info.rating.average
self.likes = video_info.rating.likes.to_i
end
rescue OpenURI::HTTPError
# Delete the entry
end
Example with multiple cases of catching errors:
begin
# Do the thing
rescue OpenURI::HTTPError
# Delete the entry
rescue OpenURI::DifferentErrorType
# Do something different
resuce OpenURI::AnotherType, OpenURI::ASpecialError
# Do something radically different
end
Related
I am working on an app where I am required to fetch and save the open graph data of a website.
So far I have been able to grab properties such as title, description, url by using this code
before_save :get_meta_from_link
def check_link
begin
#page_link = MetaInspector.new(sanitized_url)
rescue Faraday::ConnectionFailed => e
errors.add(:link, "Oops, can't be processed ATM")
end
end
def get_meta_from_link
page = #page_link
return unless page.to_hash.present?
if page.title.present?
self.title = page.title
end
if page.description.present?
self.description = page.description
end
if page.url.present?
self.url = page.url
end
end
I am using the metainspector gem and trying to grab values such as og:locale, og:type. How can I fetch those values?
This is the link I am using to cross reference values: https://metainspectordemo.herokuapp.com
Ok, so I managed to solve it using
def check_link
begin
#page_link = MetaInspector.new(sanitized_url)
rescue MetaInspector::RequestError => e
errors.add(:link, "you provided is not being read by our system. Please check the link.")
end
end
in my link model
followed by
def get_meta_from_link
page = #page_link
paje = #page_link.meta_tags
return unless page.to_hash.present?
if page.title.present?
self.btitle = page.title
end
end
My ruby environment is: Ruby 2.3.1 and Rails 5.0.0.1.
I'm trying to import a text file for import a lot of purchases items.
Example of purchase file:
data.txt
Customer\tDescription\tUnit Price\tQuantity\tAddress\tSupply company\n
Athos Matthew\tChocolate\t10\t5\tSome address.\tChocolate company\n
The columns are divided by a tab (\t) and it has an enter at the final (\n).
I have the purchase class where all attributes cannot be null. The attributes are:
custumer_name:string
product_id:integer # It has relationship with the Product Resource
product_quantity:integer
supply_company_id:integer # It has relationship with the SupplyCompany Resource
To import the file, I decided to create a PurchaseImporter class to do this job and keep the code cleaner.
My problem is that transaction part:
begin
ActiveRecord::Base.transaction do
purchase = Purchase.new
data = line.force_encoding('UTF-8').split(/\t/)
purchase.customer_name = data[0]
product = Product.find_or_create_by!(description: data[1], price: data[2])
purchase.product_quantity = data[3]
purchase.product = product
supply_company = SupplyCompany.find_or_create_by!(name: data[5], address: data[4])
purchase.supply_company = supply_company
purchase.save!
end
rescue Exception => e
#errors[:import][index] = e.message
end
My problem is that I want to catch all the raised errors from the Product, SupplyCompany and Purchase that could happen inside this transaction.
This is the order of the happenings without the unnecessary code to explain it.
product = Product.find_or_create_by!(description: data[1], price: data[2])
supply_company = SupplyCompany.find_or_create_by!(name: data[5], address: data[4])
purchase.save!
I need to print this errors information to this 3 classes in the screen, but with my code, I can only catch the first exception error, generated by the product. If some error happened in the SupplyCompany or in the Purchase, I lost these errors messages.
Are there other ways to import and log the errors message when importing files?
You can have more specific exception handling... do a rescue for each section you want to trap, at the end raise an error if any previous error was encountered (to get you out of the transaction block) and test in that final error that you're rescuing your own raise otherwise it's some other problem and you need to halt.
begin
ActiveRecord::Base.transaction do
error_encountered = false
purchase = Purchase.new
data = line.force_encoding('UTF-8').split(/\t/)
purchase.customer_name = data[0]
begin
product = Product.find_or_create_by!(description: data[1], price: data[2])
purchase.product_quantity = data[3]
purchase.product = product
rescue Exception => e
#errors[:import][index] = e.message
error_encountered = true
end
begin
supply_company = SupplyCompany.find_or_create_by!(name: data[5], address: data[4])
purchase.supply_company = supply_company
rescue Exception => e
#errors[:import][index] = e.message
error_encountered = true
end
begin
purchase.save!
rescue Exception => e
#errors[:import][index] = e.message
error_encountered = true
end
raise 'break out of transaction' if error_encountered
end
rescue Exception => e
raise unless e.message == 'break out of transaction'
end
Since you're rescuing Exception it's hard to know what error is actually emerging. When rescuing, you should try to use a more specific class when possible.
You also might not need to use rescue at all. The active methods you're using: find_or_create_by! and save! can be written without the exclamation point so that they don't raise errors.
In active record, if you try and save something with validation errors then the <record>.errors.full_messages array is populated. It won't necessarily raise an error if you don't use the exclamation point (although errors can be raised from all sorts of things regardless).
So, for example you can try and save a record and check for errors like this:
product = Product.find_or_initialize_by(description: data[1], price: data[2])
product.save
errors[:import][index] ||= []
errors[:import][index].concat product.errors_full_messages
Actually, in this case I think your approach makes some sense. You're saving a few records in sequence. If the first fails, then probably the others will fail - so is it worth even attempting to save those subsequent records? I'll let you decide.
I'm fetching videos' data from youtube, including general video info, and likes number of them.
The problem is that sometimes, there is no corresponding data. Maybe the video is private, or the likes number is 0.
In either case, there will report NoMethodError: undefined method 'likes' for nil:NilClass
How can I handle this? I just want to ignore them and continue to the next youtube video.
I think I can use something like rescue, but I don't find many tutorial of it.
Here is my code:
client = YouTubeIt::Client.new(:dev_key => my_key)
Video.all.each do |video|
video_info = client.video_by(video.url)
like_number = video_info.rating.likes.to_i
video.likes = like_number
if video.save
puts "#{video.title} likes: #{video.likes}"
end
end
Rather that using the exception handling, which is slower, in this case just check for the presence of the value before calling the method.
Assuming the error is generated on the line
like_number = video_info.rating.likes.to_i
simply use
client = YouTubeIt::Client.new(:dev_key => my_key)
Video.all.each do |video|
video_info = client.video_by(video.url)
# here check for the presence.
if video_info.rating
like_number = video_info.rating.likes.to_i
video.likes = like_number
if video.save
puts "#{video.title} likes: #{video.likes}"
end
end
end
You should check for all possible nil values with, eg, video.nil? and corresponding conditionals. Rescuing should always be your last resort (it's slow and dirty), but, if there is no way to predict where the script will fail, you can always rescue some piece of code:
begin
#your error-prone code
rescue
#action you want to take in case your error-prone code raises error
end
Here is a good article on exceptions: http://www.tutorialspoint.com/ruby/ruby_exceptions.htm.
In this example, the method body acts as default begin block so need of begin block
def any_method
client = YouTubeIt::Client.new(:dev_key => my_key)
Video.all.each do |video|
video_info = client.video_by(video.url)
# try tries to find the method, if not found returns nil
like_number = video_info.try(:rating).try(:likes).to_i
video.likes = like_number
if video.save
puts "#{video.title} likes: #{video.likes}"
end
end
# For any type of standard errors, use rescue block
rescue => error_object
p 'Sorry, some error occured'
p error_object
end
Assuming the message argument is a string, I have the following snippet:
users.each do |user|
posted = Facebook.post_to_facebook(message,user.token)
end
Then, post_to_facebook is a method like this:
def post_to_facebook
facebook_graph = Koala::Facebook::GraphAPI.new(token)
object_from_koala = facebook_graph.put_wall_post(message)
end
For some users, when calling the put_wall_post I get an exception of this type: Koala::Facebook::APIError. I just want to skip posting to that user's wall, and go to the next user, but I fail to see how to do that managing the exception.
you could use rescue like this, and maybe handle your exceptions
def post_to_facebook
begin
facebook_graph = Koala::Facebook::GraphAPI.new(token)
object_from_koala = facebook_graph.put_wall_post(message)
rescue
do something else
end
I want to get data from this page:
http://www.canadapost.ca/cpotools/apps/track/personal/findByTrackNumber?trackingNumber=0656887000494793
But that page forwards to:
http://www.canadapost.ca/cpotools/apps/track/personal/findByTrackNumber?execution=eXs1
So, when I use open, from OpenUri, to try and fetch the data, it throws a RuntimeError error saying HTTP redirection loop:
I'm not really sure how to get that data after it redirects and throws that error.
You need a tool like Mechanize. From it's description:
The Mechanize library is used for
automating interaction with websites.
Mechanize automatically stores and
sends cookies, follows redirects, can
follow links, and submit forms. Form
fields can be populated and submitted.
Mechanize also keeps track of the
sites that you have visited as a
history.
which is exactly what you need. So,
sudo gem install mechanize
then
require 'mechanize'
agent = WWW::Mechanize.new
page = agent.get "http://www.canadapost.ca/cpotools/apps/track/personal/findByTrackNumber trackingNumber=0656887000494793"
page.content # Get the resulting page as a string
page.body # Get the body content of the resulting page as a string
page.search(".somecss") # Search for specific elements by XPath/CSS using nokogiri
and you're ready to rock 'n' roll.
The site seems to be doing some of the redirection logic with sessions. If you don't send back the session cookies they are sending on the first request you will end up in a redirect loop. IMHO it's a crappy implementation on their part.
However, I tried to pass the cookies back to them, but I didn't get it to work, so I can't be completely sure that that is all that's going on here.
While mechanize is a wonderful tool I prefer to "cook" my own thing.
If you are serious about parsing you can take a look at this code. It serves to crawl thousands of site on an international level everyday and as far as I have researched and tweaked there isn't a more stable approach to this that also allows you to highly customize later on your needs.
require "open-uri"
require "zlib"
require "nokogiri"
require "sanitize"
require "htmlentities"
require "readability"
def crawl(url_address)
self.errors = Array.new
begin
begin
url_address = URI.parse(url_address)
rescue URI::InvalidURIError
url_address = URI.decode(url_address)
url_address = URI.encode(url_address)
url_address = URI.parse(url_address)
end
url_address.normalize!
stream = ""
timeout(8) { stream = url_address.open(SHINSO_HEADERS) }
if stream.size > 0
url_crawled = URI.parse(stream.base_uri.to_s)
else
self.errors << "Server said status 200 OK but document file is zero bytes."
return
end
rescue Exception => exception
self.errors << exception
return
end
# extract information before html parsing
self.url_posted = url_address.to_s
self.url_parsed = url_crawled.to_s
self.url_host = url_crawled.host
self.status = stream.status
self.content_type = stream.content_type
self.content_encoding = stream.content_encoding
self.charset = stream.charset
if stream.content_encoding.include?('gzip')
document = Zlib::GzipReader.new(stream).read
elsif stream.content_encoding.include?('deflate')
document = Zlib::Deflate.new().deflate(stream).read
#elsif stream.content_encoding.include?('x-gzip') or
#elsif stream.content_encoding.include?('compress')
else
document = stream.read
end
self.charset_guess = CharGuess.guess(document)
if not self.charset_guess.blank? and (not self.charset_guess.downcase == 'utf-8' or not self.charset_guess.downcase == 'utf8')
document = Iconv.iconv("UTF-8", self.charset_guess, document).to_s
end
document = Nokogiri::HTML.parse(document,nil,"utf8")
document.xpath('//script').remove
document.xpath('//SCRIPT').remove
for item in document.xpath('//*[translate(#src, "ABCDEFGHIJKLMNOPQRSTUVWXYZ", "abcdefghijklmnopqrstuvwxyz")]')
item.set_attribute('src',make_absolute_address(item['src']))
end
document = document.to_s.gsub(/<!--(.|\s)*?-->/,'')
self.content = Nokogiri::HTML.parse(document,nil,"utf8")
end