view by newspaper category with rails 4 - ruby-on-rails

My problem is that I need to fill an array with rails 4 to display top 3 results categorized by newspaper
Category of newspaper is: 0,1,2,3 or other, is random.
parsed_json: Is one array with values de newspapers, after news_source fill with news from newspapers.
SQL: http://i.stack.imgur.com/dP8BM.png
With this code only the last value of the array of a newspaper.
def index
if get_api_key
#parsed_json = Array.new
#news_source = Array.new
#Trae todos los datos asociados a la llave
#emission = Emission.where(key: get_api_key)
#Envío de datos a json
#emission.each do |p|
#weather_now = WeatherNowUy.where(city: p.state)
#weather_next_days = WeatherNextDaysUy.where(city: p.state)
#parsed_json << JSON.parse(p.news_source)
end
#parsed_json.each do |u|
#news_source = NewsUy.where(newspaper: u).limit(3)
end
end
end
Example of return, no show by newspaper 1 is also found in the array parsed_json.
Not
"News": [
{
"title": "Con un gol de Tevez, Boca venció a Godoy Cruz 2-0 y sigue en lo más alto",
"description": "El delantero marcó de penal en la victoria en La Bombonera. Meli había abierto el marcador. Los \"Xeneizes\" comparten con San Lorenzo la primera posición.",
"newspaper": "0"
},
{
"title": "Barcelona picó en punta",
"description": "La primera fecha del fútbol español se terminará el lunes con el partido entre Granada y Eibar.",
"newspaper": "0"
},
{
"title": "Sampdoria, Chievo Verona y Fiorentina los líderes",
"description": "Este fin de semana se disputó la primera fecha de la serie A italiana.",
"newspaper": "0"
}
]
}

Change #news_source = NewsUy.where(newspaper: u).limit(3) to #news_source << NewsUy.where(newspaper: u).limit(3) in your last #parsed_json.each do loop. Because, if you use = operator, it will replace it every time with the next value and that's why you are getting only the last value. To get all the values, you need to accumulate those into the #news_source array using << operator.
So, you need this:
#parsed_json.each do |u|
#news_source << NewsUy.where(newspaper: u).limit(3)
end

Related

Rake task to update records

I am doing a rake task that is scrapping a website in order to find concerts:
lib/tasks/my_task.rake
task :find_concerts => :environment do
doc = Nokogiri::HTML(open(url))
data = doc.search('#dateconcert table')
data = data.css('.jaunec' ).map { |tr| tr.css('td').map(&:text) } + doc.css('.jaunef' ).map { |tr| tr.css('td').map(&:text) }
data.each do |concert|
c = Concert.new
c.date = concert[0]
c.city = concert[1]
c.save
end
end
What I want
I want to get an alert when a new concert is added ( on the website I am scrapping), so the tasks will be run everyday.
My problem
I want my Concerts list to be updated if there is a new record...
With the task I wrote it finds again the records that are already stored and duplicated them...
I only want the new records that could be found...
In the end I would like to compare what is new between the two last tasks in order to send an alert if something new was found.
EDIT
This is what data returns
[
["03 Décembre 2017", "PONT L\u0092ABBE (29) | Centre Culturel Le Triskell "],
["26 Janvier 2018", "MONTPELLIER (34) | Le Jam "],
["17 Février 2018", "BLOIS (41) | All That Jazz / Les Lobis "],
["22 Mars 2018", "MOISSAC (82) | Hall de Paris "],
["24 Mars 2018", "LAX (Baraqueville) (12) | Festival Lax'N Blues LAX\u0092N "],
["08 Décembre 2017", "ECHANGE CULTUREL CAMEROUN (0) | au 18 décembre 2017 - Organisation tournée MFR "],
["27 Janvier 2018", "LE THOR (84) | Le Sonograf "],
["16 Mars 2018", "CHAUMONT (52) | Le Nouveau Relax "],
["23 Mars 2018", "AUCH (32) | Le Cri'Art "]
]
I found a solution that may need some refactoring, though.
So I created two tasks,
find_concerts that will be run manually for the first scrap
update_concerts that will be run every day
task
require "nokogiri"
require "open-uri"
require "date"
require "time"
namespace :scrap do
desc "This get MM concerts"
url = "http://mountain-men.fr/concerts/"
doc = Nokogiri::HTML(open(url))
data = doc.search('#dateconcert table')
data = data.css('.jaunec' ).map { |tr| tr.css('td').map(&:text) } + doc.css('.jaunef' ).map { |tr| tr.css('td').map(&:text) }
task :find_concerts => :environment do
data.each do |concert|
c = Concert.create
c.date = concert[0]
c.city = concert[1]
c.save
end
end
task :update_concerts => :environment do
existing_date = Concert.all.map { |c| [c.date, c.city] }
data.each do |concert|
c = Concert.create
c.date = concert[0]
c.city = concert[1]
c.save unless existing_date.include?([concert[0], concert[1]])
end
Concert.where(city: nil, date: nil).destroy_all
end
end
I am not sure if I understand you correctly, but what you need to do is to check if it exists and other wise create.
I would do some thing like this.
task :find_concerts => :environment do
doc = Nokogiri::HTML(open(url))
data = doc.search('#dateconcert table')
data = data.css('.jaunec' ).map { |tr| tr.css('td').map(&:text) } + doc.css('.jaunef' ).map { |tr| tr.css('td').map(&:text) }
data.each do |concert|
Concert.where(data: concert[0], city: concert[1]).first_or_create
end
end
Now you only create if it doesn't exist. Then when you create one you can set a hook.
class Concern
after_create :send_notification
def send_notification
# send email here
end
end

How to use Ruby's send method without receiver?

I am trying to convert user input à la "6 months" into a Ruby method à la 6.months.
How can this be done?
I keep getting an undefined method 6.months error in line 10.
Thanks for any help.
class Coupon < ActiveRecord::Base
def self.duration_options
["3 years", "2 years", "1 year", "6 months", "3 months", "1 month"]
end
def end_at
if Coupon.duration_options.include?(duration)
method = duration.sub!(' ', '.')
time = Time.zone.now + send(method)
end
time
end
end
Your send is on the object itself, in this case it's self.send(method)
You could do like this
if Coupon.duration_options.include?(duration)
off_set = duration.split(" ")
time = Time.zone.now + off_set.first.to_i.send(off_set.last.to_sym)
end
time

Checking if hash value has a text

I have a hash:
universityname = e.university
topuniversities = CSV.read('lib/assets/topuniversities.csv',{encoding: "UTF-8", headers:true, header_converters: :symbol, converters: :all})
hashed_topuniversities = topuniversities.map {|d| d.to_hash}
hashed_topuniversities.any? {|rank, name| name.split(' ').include?(universityname) }.each do |s|
if s[:universityrank] <= 10
new_score += 10
elsif s[:universityrank] >= 11 && s[:universityrank] <= 25
new_score += 5
elsif s[:universityrank] >= 26 && s[:universityrank] <= 50
new_score += 3
elsif s[:universityrank] >= 51 && s[:universityrank] <= 100
new_score += 2
end
Basically what this is doing is looking at a hash and checking if the hash value contains a university name is an input.
For example the user input can be "Oxford University" and in the hash its stored as "Oxford". The User needs to type in as it stored in the hash to be able to be assigned a score, But I want it that if the user types in "oxford university" then the hash value "Oxford" should be selected and then go through.
Everything else in this works fine but the .include? does not work correctly, I still need to type the exact word.
hashed_topuniversities = topuniversities.map &:to_hash
univ = hashed_topuniversities.detect do |rank, name|
name.downcase.split(' ').include?(universityname.downcase)
end
new_score += case univ[:universityrank]
when -Float::INFINITY..10 then 10
when 11..25 then 5
when 26..50 then 3
when 50..100 then 2
else 0
end
Besides some code improvements in terms of being more idiomatic ruby, the main change is downcase called on both university name and user input. Now they are compared case insensitive.
I don't think your approach will work (in real-life, anyway). "University of Oxford" is an easy one--just look for the presence of the word, "Oxford". What about "University of Kansas"? Would you merely try to match "Kansas"? What about "Kansas State University"?
Also, some universities are are customarily referred to by well-know acronyms or shortened names, such as "LSE", "UCLA", "USC", "SUNY", "LSU", "RPI", "Penn State", "Georgia Tech", "Berkeley" and "Cal Tech". You also need to think about punctuation and "little words" (e.g., "at", "the", "of") in university names (e.g., "University of California, Los Angeles").
For any serious application, I think you need to construct a list of all commonly-used names for each university and then require an exact match between those names and the given university name (after punctuation and little words have been removed). You can do that by modifying the hash hashed_top_universities, perhaps like this:
hashed_top_universities
#=> { "University of California at Berkeley" =>
# { rank: 1, names: ["university california", "berkeley", "cal"] },
# "University of California at Los Angeles" =>
# { rank: 2, names: ["ucla"] },
# "University of Oxford" =>
# { rank: 3, names: ["oxford", "oxford university"] }
# }
Names of some universities contain non-ASCII characters, which is a further complication (that I will not address).
Here's how you might code it.
Given a university name, the first step is to construct a hash (reverse_hash) that maps university names to ranks. The names consist of the elements of the value of the key :names in the inner hashes in hashed_top_universities, together with the complete university names that comprise the keys in that hash, after they have been downcased and punctuation and "little words" have been removed.
PUNCTUATION = ",."
EXCLUSIONS = %w| of for the at u |
SCORE = { 1=>10, 3=>7, 25=>5, 50=>3, 100=>2, Float::INFINITY=>0 }
reverse_hash = hashed_top_universities.each_with_object({}) { |(k,v),h|
(v[:names] + [simplify(k)]).each { |name| h[name] = v[:rank] } }
#=> {"university california"=>1, "berkeley"=>1, "cal"=>1,
# "university california berkeley"=>1,
# "ucla"=>2, "university california los angeles"=>2,
# "oxford"=>3, "oxford university"=>3, "university oxford"=>3}
def simplify(str)
str.downcase.delete(PUNCTUATION).
gsub(/\b#{Regexp.union(EXCLUSIONS)}\b/,'').
squeeze(' ')
end
def score(name, reverse_hash)
rank = reverse_hash[simplify(name)]
SCORE.find { |k,_| rank <= k }.last
end
Let's try it.
score("University of California at Berkeley", reverse_hash)
#=> 10
score("Cal", reverse_hash)
#=> 10
score("UCLA", reverse_hash)
#=> 7
score("Oxford", reverse_hash)
#=> 7

No implicit conversion of String into Integer, open date filter

I'm actually trying to use Json, from open data paris, and to filter some stuff.
I can create what I want with those lines:
def import
#result = ''
url = 'http://opendata.paris.fr/api/records/1.0/search/?rows=1000&dataset=arc_innovation&facet=code_postal&facet=commune&facet=etat_1&facet=typologie_carto&facet=type_innovation&facet=image'
response = HTTParty.get(url)
hash = response.parsed_response
hash['records'].each do |record|
lime = Lime.where(datasetid: record['datasetid'], recordid: record['recordid']).first_or_initialize
lime.title = record['fields']['nom']
lime.text = record['fields']['txt_descriptif']
lime.url = record['fields']['site_internet']
lime.xy = record['fields']['xy']
category = category_from_typeinnovation(record['fields']['type_innovation'])
lime.categories << category unless category.nil?
if record['fields'].has_key? 'image'
image_id = record['fields']['image']['id']
image_filename = record['fields']['image']['filename']
image_url = "http://opendata.paris.fr/explore/dataset/arc_innovation/files/#{image_id}/download"
lime.image = image_url
end
lime.save
#result += "imported #{lime.title}\n"
end
render text: #result
end
Here is how looks my Json like
"records":[
{
"fields":{
"commune":"Paris 19e",
"code_postal":75019,
"id_formularie":1,
"etat_1":"existant",
"contact_tel":"01 53 35 50 00",
"xy":[
48.889979,
2.371504
],
"nom":"104",
"txt_descriptif":"Plateforme collaborative qui s’intéresse à la création artistique contemporaine dans toute sa diversité, elle accueille des artistes en résidence et accompagne l’émergence de nouvelles formes d’art avec le Cinq destiné aux pratiques amateurs, la Maison desPetits et un incubateur qui héberge et soutient le développement de start-ups.",
"type_innovation":"Lieux innovants de la culture et du sport",
},
I try to filter by "type-innovation". I don't want to create a post with the data that own "type_innovation":"Lieux innovants de la culture et du sport" for example.
I tried to filter like this:
def import
#result = ''
url = 'http://opendata.paris.fr/api/records/1.0/search/? rows=1000&dataset=arc_innovation&facet=code_postal&facet=commune&facet=etat_1&facet=typologie_carto&facet=type_innovation&facet=image'
response = HTTParty.get(url)
hash = response.parsed_response
hash['records'].each do |record|
lime = Lime.where(datasetid: record['datasetid'], recordid: record['recordid']).first_or_initialize
lime.title = record['fields']['nom']
lime.text = record['fields']['txt_descriptif']
lime.url = record['fields']['site_internet']
lime.xy = record['fields']['xy']
category = category_from_typeinnovation(record['fields']['type_innovation'])
lime.categories << category unless category.nil?
if record['fields'].has_key? 'image'
image_id = record['fields']['image']['id']
image_filename = record['fields']['image']['filename']
image_url = "http://opendata.paris.fr/explore/dataset/arc_innovation/files/#{image_id}/download"
lime.image = image_url
end
if record['fields']['type_innovation'] != ['fields']['type_innovation']['Lieux innovants de la culture et du sport']
lime.save
#result += "imported #{lime.title}\n"
end
end
render text: #result
end
When I try to import it returns me the error. How Should I do the filtering?

Rails 4.1.2 - Reject nested attributes result being ignored

I have a Rails form that has two types of nested attributes: Pendencies and Address.
However, those two are not obligatory fields and should be ignored when blank:
accepts_nested_attributes_for :pendencies, allow_destroy: true,
reject_if: proc { |attributes|
attributes['status'].blank? &&
attributes['description'].blank? &&
(attributes['pendency_type'].blank? || attributes['pendency_type'] == 0) }
accepts_nested_attributes_for :address, allow_destroy: true,
reject_if: proc { |attributes|
attributes['zipcode'].blank? &&
attributes['street'].blank? &&
(attributes['number'].blank? || attributes['number'] <= 0) &&
attributes['neighbour'].blank? &&
attributes['city'].blank? &&
attributes['state'].blank? }
I have used binding.pry on both procs to ensure they are running and they are. Both are returning true or false as expected.
Yet, once the model which hold those associations gets validated when calling model.update, it returns errors about the exactly same fields it should be ignoring:
#messages=
{:"pendencies.status"=>["não pode ficar em branco"],
:"pendencies.description"=>["não pode ficar em branco"],
:"pendencies.pendency_type"=>["não pode ficar em branco"],
:"address.zipcode"=>["não pode ficar em branco", "não é válido", "não possui o tamanho esperado (8 caracteres)"],
:"address.street"=>["não pode ficar em branco", "é muito curto (mínimo: 3 caracteres)"],
:"address.number"=>["não pode ficar em branco", "não é um número", "é muito curto (mínimo: 1 caracteres)"],
:"address.city"=>["não pode ficar em branco", "é muito curto (mínimo: 2 caracteres)"],
:"address.state"=>["não pode ficar em branco", "é muito curto (mínimo: 2 caracteres)"]}>
Sorry for the PT-BR in the messages, all it says is that the length is short and the field is invalid.
I would like to know how can I ignore those nested attributes so Rails would not try to save them.
Thank you.

Resources