Scraping images from user input URL using MetaInspector (Rails) - ruby-on-rails

Question

I'm trying to create an app where a user can submit a URL link, a title and description, and it'll create a post with the title, description and an image. I want to be able to scrape the best or main image from directly from the URL path that the user submitted and display it on the show page using MetaInspector. (The reason I didn't use Nokogiri or Mechanize is because I didn't understand it all that well and MetaInspector seems alot less daunting)
The problem is I'm very new to rails and I'm having a hard time following most tutorials.
Is anyone able to explain to me step by step how to do this or show me a source that's very detailed and noob friendly?
I have a Post model that contains the link, and should also save the scraped image as a Paperclip attachment:
class Post < ActiveRecord::Base
belongs_to :user
has_attached_file :image
end
# == Schema Information
#
# Table name: posts
#
# id :integer not null, primary key
# title :string
# link :string
# description :text
# created_at :datetime
# updated_at :datetime
# user_id :integer
# image_file_name :string
# image_content_type :string
# image_file_size :integer
# image_updated_at :datetime
The full code of my app is available at github.com/johnnyji/wanderful.
I really appreciate any help at all! Thank you

Answer 1

Let's walk through this step by step.
First, add the MetaInspector gem to your Gemfile
gem 'metainspector'
and run the bundle command.
We need another bit of code: open-uri. With it, we can read remote files from URLs as if they were local files. It is part of Rubys standard library, so it's already built in, but we still need to require it at the top of your post.rb:
require 'open-uri'
class Post < ActiveRecord::Base
belongs_to :user
has_attached_file :image
end
We want to grab an image whenever a Posts link changes, so we make a before_save callback that triggers whenever that happens:
class Post < ActiveRecord::Base
belongs_to :user
has_attached_file :image
before_save :get_image_from_link,
if: ->(post) { post.link_changed? }
end
you can find more about before_save and other callbacks in the ActiveRecord::Callbacks guide.
the link_changed? method is part of the "dirty tracking" functionality ActiveModel::Dirty provides
that if: ->(post) thing is called a "stabby lambda" - it's basically just a Ruby function that is called with the current post as an argument. If it returns true, the before_action is run. It could also be written as if: Proc.new { |post| post.link_changed? }
Now we need our get_image_from_link method. Since it's only supposed to be called from within the Post model itself and not from the outside (say, Post.find(5).get_image_from_link), we make it a private method:
class Post < ActiveRecord::Base
belongs_to :user
has_attached_file :image
before_save :get_image_from_link,
if: ->(post) { post.link_changed? }
private
def get_image_from_link
end
end
Reading MetaInspectors README, it has a cool method called page.images.best that does the hard work for us selecting the right image from that page. So we are going to
parse the link with MetaInspector
open the image it selected as best with open-uri as a File-like object
give that File-like object to Paperclip to save as an attachment
So:
def get_image_from_link
# `link` here is `self.link` = the current post.
# At least when reading attributes, `self` is implicit
# in Ruby
page = MetaInspector.new(link)
# maybe the page didn't have images?
return unless page.images.best.present?
# when you use IO resources such as files, you need
# to take care that you `.close` everything you open.
# Using the block form takes care of that automatically.
open(page.images.best) do |file|
# when writing/assigning a value, `self` is not
# implicit, because when you write `something = 5`,
# Ruby cannot know whether you want to assign to
# `self.something` or create a new local variable
# called `something`
self.image = file
end
end
This is far from perfect, because it lacks some error handling (what if MetaInspector fails to open the page? Or open-uri cannot read the image URL?). Also, this has the drawback that all that parsing, downloading and so on takes place right when the user submits or updates her post, so when she clicks on the save button, she'll have to wait for all this to complete.
For the next iteration, look into doing things like these asynchronously, for example with a job queue. Rails' new Active Job system might be a good starting point.