I've just started using Ruby on Rails and so far it's working nicely. I'm now trying to implement a gem but it's not working, and I am hoping it's just a beginner mistake - something which I've not yet grasped!!
I've followed tutorials and got my hello world example - also managed to git push this to my Heroku account. I started to follow the tutorial here: http://railscasts.com/episodes/190-screen-scraping-with-nokogiri and got the following code working in Terminal (on mac)
require 'rubygems'
require 'nokogiri'
require 'open-uri'
url = "http://www.walmart.com/search/search-ng.do?search_constraint=0&ic=48_0&search_query=batman&Find.x=0&Find.y=0&Find=Find"
doc = Nokogiri::HTML(open(url))
puts doc.at_css("title").text
So that worked nicely. I can see the title inside of terminal. However, when I try to put this code in my view controller, it cannot find the nokogiri gem. The code in my controller is
class HomeController < ApplicationController
def index
require 'rubygems'
require 'nokogiri'
require 'open-uri'
url = "http://www.walmart.com/search/search-ng.do?search_constraint=0&ic=48_0&search_query=batman&Find.x=0&Find.y=0&Find=Find"
doc = Nokogiri::HTML(open(url))
#mattVar = doc.at_css("title").text
end
end
So then in my HTML, I have
<h1>Hello!</h1>
<p><%= #mattVar %></p>
Which should output the title as it does in the terminal window. However, I get an error saying no such file to load -- nokogiri
Any ideas where I'm going wrong?
If I do gem list I can see the nokogiri gem there.
UPDATE
I closed the local rails server and restarted. It's now working. Not sure if it was because of the work adding the gem to my root or doing the 'bundle install' (as the posts below suggested). Thanks for the help.
Firstly try changing your controller to be somthing like
class HomeController < ApplicationController
require 'nokogiri'
require 'open-uri'
def index
url = "http://www.walmart.com/search/search-ng.do?search_constraint=0&ic=48_0&search_query=batman&Find.x=0&Find.y=0&Find=Find"
doc = Nokogiri::HTML(open(url))
#mattVar = doc.at_css("title").text
end
end
I'm guessing you are also using bundler, check that you have include this gem and run bundle install.
You can find the documentation for this at http://gembundler.com/
You should put your various require statements outside of your index method. It won't slow you down much to have them in there, but you're asking Ruby to ensure that they are loaded every time the method is called. Better to just require them once at the top of your code. This won't cause your problem, however.
I suspect that you are testing your server under a different environment than what IRB is running in. Try removing the offending require statements for the moment and changing your HTML template to:
<h1>Environment</h1>
<ul>
<li>Ruby: <%=RUBY_VERSION%></li>
<li>Gems: <%=`gem list`%></li>
</ul>
Related
This is my Scraper Controller
class ScraperController < ApplicationController
def getinformation
require 'open-uri'
require 'nokogiri'
#information = Nokogiri::HTML(open('https://ibotta.com/rebates'))
end
end
And this is a webpage of the information that I'm getting from nokogiri
https://rails-tutorial2-chriscma.c9users.io/scraper/getinformation
I'm not getting any of the product names, and I'm not sure why?
The page you're trying to scrape is dynamically generated by executing javascript, so you won't be able to use nokogiri to download the content. It looks like the offers on the page are loaded from https://ibotta.com/web_v1/offers.json, but this isn't accessible directly. Therefore, I think you'll need to use something which can execute javascript like selenium / phantomjs / chrome headless / watir etc in order to load the page.
I am using Nokogiri with open_uri to crawl web data in a Rails app. I have a NokogiriCrawler class:
class NokogiriCrawler
require 'open_uri'
def initialize
end
end
but always have this error: cannot load such file -- open_uri
Try this instead:
require 'open-uri'
P.S. This is unrelated to your actual problem, but requires should almost always go at the top of your file, not inside your class declaration.
I've recently run into a problem using OpenURI. Every open method results in the following error:
"No such file or directory # rb_sysopen".
My code looks simply like the following:
data = open("http://google.ca/")
I noticed the error shortly after adding gem 'nokogiri' to my Gemfile and running bundle install, though I have no indication of whether or not this caused the problem and have since removed the entry with no positive impact on the problem. Any help would be appreciated.
Try to write require 'open-uri' before your code.
I am using Ruby 3.0.1 and a part from the:
require "open-uri"
I have to explicitly call URI.open instead of just open:
data = URI.open("http://google.ca/")
Maybe it is something on new Ruby versions
I am attempting to use the songkickr gem in a rails app. I have installed the gem using the
gem install songkickr
The next step of the instruction is to:
require 'songkickr'
remote = Songkickr::Remote.new API_KEY
Where do I do this? In what file?
The github page for the gem is https://github.com/jrmehle/songkickr
and my github for this is on https://github.com/jeremybelcher/travel
Sorry for the beginner question, trying to lean al this. Any help is very much appreciated.
In your gemfile at your application root you need to add the line
gem 'songkickr'
Then run
bundle
There are several places where you can put ruby code in a Ruby on Rails app and you should just choose one of them, depending which makes the most sense for you application. For a start, how about you just try putting that code in one of your controllers?
In your Gemfile, add:
gem 'songkickr'
Then run the following command: bundle
At the top of the controller put:
require 'songkickr'
API_KEY = "" # edit this line
Then in some action you can do this to test it out:
remote = Songkickr::Remote.new API_KEY
Later you might decide to make the "remote" object persist between requests and then you would need to do something more complicated.
Add require 'songkickr' to your application controller
Example:
class ApplicationController < ActionController::Base
protect_from_forgery
require 'songkickr'
end
I'm trying to read in the html of a certain website.
Trying #something = open("http://www.google.com/") fails with the following error:
Errno::ENOENT in testController#show
No such file or directory - http://www.google.com/
Going to http://www.google.com/, I obviously see the site. What am I doing wrong?
Thanks!
You need to require 'open-uri' first to be able to open() remote paths.
See the docs for more info.
You should use a utility like Nokogiri to parse the returned content like so:
(From the Nokogiri site front page # http://nokogiri.org/)
require 'nokogiri'
require 'open-uri'
# Get a Nokogiri::HTML:Document for the page we’re interested in...
doc = Nokogiri::HTML(open('http://www.google.com/search?q=tenderlove'))
# Do funky things with it using Nokogiri::XML::Node methods...
# Search for nodes by css
doc.css('h3.r a.l').each do |link|
puts link.content
end
will print to the screen:
Some Link