I'm new to Rails and I'm trying to make a simple weather API to get weather by zipcode
is there a way to get the zipcode from user input from a simple form, this will be just for learning so I'm not trying to make users devise, or users model
require 'net/http'
require 'json'
#url = 'http://api.openweathermap.org/data/2.5/weather?zip=#{zipcode}&appid=APIKEY'
#uri = URI(#url)
#response = Net::HTTP.get(#uri)
#output = JSON.parse(#response)
actually I figured it out, i needed to add
def zipcode
#zip_query = params[:zipcode]
if params[:zipcode] == ""
#zip_query = "Hey you forgot to enter a zipcode!"
elsif params[:zipcode]
# Do Api stuff
require 'net/http'
require 'json'
#url = 'http://api.openweathermap.org/data/2.5/weather?zip='+ #zip_query +'&appid=APIKEY'
#uri = URI(#url)
#response = Net::HTTP.get(#uri)
#output = JSON.parse(#response)
#name = #output['name']
# Check for empty return result
if #output.empty?
#final_output = "Error"
elsif !#output
#final_output = "Error"
else
#final_output = ((#output['main']['temp'] - 273.15) * 9/5 +32).round(2)
end
end
end
in the controller.rb file
and add
post "zipcode" => 'home#zipcode'
get "home/zipcode"
in the routes file
but I'm sure this is not the best practice
Related
I have this code
def index
require 'net/http'
require 'json'
#url = 'https://pro-api.coinmarketcap.com/v1/cryptocurrency/listings/latest?start=1&limit=100&CMC_PRO_API_KEY=mykey'
#uri = URI(#url)
#response = Net::HTTP.get(#uri)
#coins = JSON.parse(#response)
#my_coins = ["BTC", "XRP", "ADA", "ETH", "USDT"]
end
The url brings
{"status"=>{"timestamp"=>"2021-02-16T03:55:40.727Z", "error_code"=>0, "error_message"=>nil, "elapsed"=>21, "credit_count"=>1, "notice"=>nil, "total_count"=>4078}, "data"=>[{"id"=>1, "name"=>"
Using that variable (#coins) how could I give the instruction to delete everythin until ' "data"=>'?
def index
require 'net/http'
require 'json'
#url = 'https://pro-api.coinmarketcap.com/v1/cryptocurrency/listings/latest?start=1&limit=100&CMC_PRO_API_KEY=mykey'
#uri = URI(#url)
#response = Net::HTTP.get(#uri)
#coins = get_coins(#response)
#my_coins = ["BTC", "XRP", "ADA", "ETH", "USDT"]
end
def get_coins(response)
coins = JSON.parse(response)
coins.slice('data')
end
it will give you only 'data' part. because 'data' is a key of hash same as 'status'
the #coins variable will remain the same, but the output is a new variable which is result from slice operation
you can also delete using delete operation then #coins will change to remaining key
#coins.delete('status')
puts #coins #{"data"=>[{"id"=>1, "name"=>"somename"}]
def index
require 'net/http'
require 'json'
#url = 'https://pro-api.coinmarketcap.com/v1/cryptocurrency/listings/latest?start=1&limit=100&CMC_PRO_API_KEY=mykey'
#uri = URI(#url)
#response = Net::HTTP.get(#uri)
#coins = JSON.parse(response)
#coins.delete('status')
#my_coins = ["BTC", "XRP", "ADA", "ETH", "USDT"]
end
Im trying to use Kimurai to scrape a website. Im running into this error when I want to do /scrape.
def scrape
url = "https://www.tripadvisor.com/Restaurants-g31892-Rogers_Arkansas.html"
response = RestaurantsScraper.parse!(response, url, data: {})
if response[status] == :completed && response[error].nil?
flash.now[notice] = "Successfully scraped url"
else
flash.now[alert] = response[error]
end
end
Here is my scraper class
class RestaurantsScraper < Kimurai::Base
#name = "restaurants_scraper"
#driver = :selenium_chrome
#start_urls = ["https://www.tripadvisor.com/Restaurants-g31892-Rogers_Arkansas.html"]
def parse(response, url:, data: {})
response.xpath("//div[#class=_1llCuDZj]").each do |a|
request_to :parse_repo_page, url: absolute_url(a[:href], base: url)
end
end
def parse_repo_page(response, url:, data: {})
item = {}
item["title"] = t.css('a._15_ydu6b')&.text&.squish&.gsub('[^0-9].', '')
item["type"] = t.css('span._1p0FLy4t')&.text&.squish
item["reviews"] = t.css('span.w726Ki5B').text&.squish
item["top_reviews"] = t.css('a._2uEVo25r _3mPt7dFq').text&.squish
Restaurant.where(item).first_or_create
end
end
Here is the error im getting
It's because response from RestaurantsScraper.parse!(response, url, data: {}) isn't defined.
From the kimurai docs it says you need to pass a Nokogiri::HTML::Document object.
I haven't used Kimurai and it feels like there is definitely a better way to do this, but something like the following may be enough to get you to the next step:
def scrape
require 'open-uri'
url = "https://www.tripadvisor.com/Restaurants-g31892-Rogers_Arkansas.html"
html = Nokogiri.parse open(url)
response = RestaurantsScraper.parse!(html, url, data: {})
if response[status] == :completed && response[error].nil?
flash.now[notice] = "Successfully scraped url"
else
flash.now[alert] = response[error]
end
end
I have a model with an initializer in it, which basically creates a user from a user hash.
After it gets the user information, it checks whether the "privileges" key in the hash is an array. If it's not, it turns it into an array.
Now the obvious way of doing this would be crafting an entire user_hash so that it would skip those "create user" lines and then check if it turns the input into an array if necessary. However, I was wondering if there is a more DRY way of doing this?
Here is the user model I'm talking about:
def initialize(opts={})
#first_name = opts[:user_hash][:first]
#last_name = opts[:user_hash][:last]
#user_name = opts[:user_hash][:user_name]
#email = opts[:user_hash][:email]
#user_id = opts[:user_hash][:id]
#privileges = {}
if opts[:privs].present?
if !opts[:privs].kind_of?(Array)
opts[:privs] = [opts[:privs]]
end
end
end
You can pass a double which returns the needed value when the proper key is requested, and itself (or something else) otherwise:
it 'turns privs into an array' do
opts = double(:opts)
allow(opts)to receive(:[]).and_return(opts)
allow(opts)to receive(:[]).with(:privs).and_return('not array')
expect(MyClass.new(opts).privileges).to eq(['not array'])
end
Btw, your code could be simplified using the splat operator:
privs = [*opts[:privs]]
sample behavior:
privs = nil
[*privs]
# => []
privs = ['my', 'array']
[*privs]
# => ["my", "array"]
privs = 'my array'
[*privs]
# => ["my array"]
You can even use the idempotent Kernel#Array
def initialize(opts = {})
#first_name = opts[:user_hash][:first]
#last_name = opts[:user_hash][:last]
#user_name = opts[:user_hash][:user_name]
#email = opts[:user_hash][:email]
#user_id = opts[:user_hash][:id]
#privileges = {}
Array(opts[:privs])
end
I hope that helps
Rather than testing the implementation (value is turned into an array), I would test the desired behavior (takes single privilege or multiple privileges):
describe User do
describe '#initialize' do
it "takes single privilege" do
user = User.new(user_hash: {}, privs: 'foo')
expect(user.privileges).to eq(['foo'])
end
it "takes multiple privileges" do
user = User.new(user_hash: {}, privs: ['foo', 'bar'])
expect(user.privileges).to eq(['foo', 'bar'])
end
end
end
First of all Thanks for you all for helping programmers like me with your valuable inputs in solving day to day issues.
This is my first question in stack overflow as I am experiencing this problems from almost one week.
WE are building a crawler which crawls the specific websites and extract the contents from it, we are using mechanize to acheive this , as it was taking loads of time we decided to run the crawling process as a background task using resque with redis gem , but while sending the process to background I am experiencing the error as the title saying,
my code in lib/parsers/home.rb
require 'resque'
require File.dirname(__FILE__)+"/../index"
class Home < Index
Resque.enqueue(Index , :page )
def self.perform(page)
super (page)
search_form = page.form_with :name=>"frmAgent"
resuts_page = search_form.submit
total_entries = resuts_page.parser.xpath('//*[#id="PagingTable"]/tr[2]/td[2]').text
if total_entries =~ /(\d+)\s*$/
total_entries = $1
else
total_entries = "unknown"
end
start_res_idx = 1
while true
puts "Found #{total_entries} entries"
detail_links = resuts_page.parser.xpath('//*[#id="MainTable"]/tr/td/a')
detail_links.each do |d_link|
if d_link.attribute("class")
next
else
data_page = #agent.get d_link.attribute("href")
fields = get_fields_from_page data_page
save_result_page page.uri.to_s, fields
#break
end
end
site_done
rescue Exception => e
puts "error: #{e}"
end
end
and the superclass in lib/index.rb is
require 'resque'
require 'mechanize'
require 'mechanize/form'
class Index
#queue = :Index_queue
def initialize(site)
#site = site
#agent = Mechanize.new
#agent.user_agent = Mechanize::AGENT_ALIASES['Windows Mozilla']
#agent.follow_meta_refresh = true
#rows_parsed = 0
#rows_total = 0
rescue Exception => e
log "Unable to login: #{e.message}"
end
def run
log "Parsing..."
url = "unknown"
if #site.url
url = #site.url
log "Opening #{url} as a data page"
#page = #agent.get(url)
#perform method should be override in subclasses
#data = self.perform(#page)
else
#some sites do not have "datapage" URL
#for example after login you're already on your very own datapage
#this is to be addressed in 'perform' method of subclass
#data = self.perform(nil)
end
rescue Exception=>e
puts "Failed to parse URL '#{url}', exception=>"+e.message
set_site_status("error "+e.message)
end
#overriding method
def self.perform(page)
end
def save_result_page(url, result_params)
result = Result.find_by_sql(["select * from results where site_id = ? AND ref_code = ?", #site.id, utf8(result_params[:ref_code])]).first
if result.nil?
result_params[:site_id] = #site.id
result_params[:time_crawled] = DateTime.now().strftime "%Y-%m-%d %H:%M:%S"
result_params[:link] = url
result = Result.create result_params
else
result.result_fields.each do |f|
f.delete
end
result.link = url
result.time_crawled = DateTime.now().strftime "%Y-%m-%d %H:%M:%S"
result.html = result_params[:html]
fields = []
result_params[:result_fields_attributes].each do |f|
fields.push ResultField.new(f)
end
result.result_fields = fields
result.save
end
#rows_parsed +=1
msg = "Saved #{#rows_parsed}"
msg +=" of #{#rows_total}" if #rows_total.to_i > 0
log msg
return result
end
end
What's Wrong with this code?
Thanks
I'm trying to make a request to a web service (fwix), and in my rails app I've created the following initializer, which works... sorta, I have two problems however:
For some reason the values of the parameters need to have +'s as the spaces, is this a standard thing that I can accomplish with ruby? Additionally is this a standard way to form a url? I thought that spaces were %20.
In my code how can I take any of the options sent in and just use them instead of having to state each one like query_items << "api_key=#{options[:api_key]}" if options[:api_key]
The following is my code, the trouble area I'm having are the lines starting with query_items for each parameter in the last method, any ideas would be awesome!
require 'httparty'
module Fwix
class API
include HTTParty
class JSONParser < HTTParty::Parser
def json
JSON.parse(body)
end
end
parser JSONParser
base_uri "http://geoapi.fwix.com"
def self.query(options = {})
begin
query_url = query_url(options)
puts "querying: #{base_uri}#{query_url}"
response = get( query_url )
rescue
raise "Connection to Fwix API failed" if response.nil?
end
end
def self.query_url(input_options = {})
#defaults ||= {
:api_key => "my_api_key",
}
options = #defaults.merge(input_options)
query_url = "/content.json?"
query_items = []
query_items << "api_key=#{options[:api_key]}" if options[:api_key]
query_items << "province=#{options[:province]}" if options[:province]
query_items << "city=#{options[:city]}" if options[:city]
query_items << "address=#{options[:address]}" if options[:address]
query_url += query_items.join('&')
query_url
end
end
end
For 1)
You API provider is expecting '+' because the API is expecting in a CGI formatted string instead of URL formatted string.
require 'cgi'
my_query = "hel lo"
CGI.escape(my_query)
this should give you
"hel+lo"
as you expect
for Question 2) I would do something like
query_items = options.keys.collect { |key| "#{key.to_s}=#{options[key]}" }
def self.query_url(input_options = {})
options = {
:api_key => "my_api_key",
}.merge(input_options)
query_url = "/content.json?"
query_items = []
options.each { |k, v| query_items << "#{k}=#{v.gsub(/\s/, '+')}" }
query_url += query_items.join('&')
end
I'm a developer at Fwix and wanted to help you with your url escaping issue. However, escaping with %20 works for me:
wget 'http://geoapi.fwix.com/content.xml?api_key=mark&province=ca&city=san%20francisco&query=gavin%20newsom'
I was hoping you could provide me with the specific request you're making that you're unable to escape with %20.