Sum Identical Keys in a Ruby Hash - ruby-on-rails

I have simple app that is meant to pull Adwords account metrics (via the adwordsapi) and present it to the user in a table inside a Rails view. It is working properly pulling down all info for multiple campaigns except for one issue..
I am unable to get the totals of each field (total cost, total impressions, etc.). I am able to serve impressions for each campaign within the account, but am unable to get the totals for the account.
I apologize ahead of time for the noob code to follow :)
Here is the adwordscampaign_controller.rb
class AdwordscampaignController < ApplicationController
PAGE_SIZE = 50
def index()
#selected_account = selected_account
if #selected_account
response = request_campaigns_list()
if response
#campaigns = Adwordscampaign.get_campaigns_list(response)
#campaign_count = response[:total_num_entries]
#start = params[:start]
#end = params[:end]
#myhash = Adwordscampaign.get_campaigns_list(response)
end
end
end
private
def request_campaigns_list()
# Prepare start and end date for the last week.
if params[:start].nil?
start_date = DateTime.parse((Date.today - 7).to_s).strftime("%Y%m%d")
end_date = DateTime.parse((Date.today - 1).to_s).strftime("%Y%m%d")
else
start_date = params[:start]
end_date = params[:end]
end
api = get_adwords_api()
service = api.service(:CampaignService, get_api_version())
selector = {
:fields => ['Id', 'Name', 'Status', 'Cost', 'Impressions', 'Clicks', 'Ctr', 'Conversions', 'Amount'],
:ordering => [{:field => 'Id', :sort_order => 'ASCENDING'}],
:date_range => {:min => start_date, :max => end_date},
:paging => {:start_index => 0, :number_results => PAGE_SIZE}
}
result = nil
begin
result = service.get(selector)
rescue AdwordsApi::Errors::ApiException => e
logger.fatal("Exception occurred: %s\n%s" % [e.to_s, e.message])
flash.now[:alert] =
'API request failed with an error, see logs for details'
end
return result
end
end
the relevant model: adwordscampaign.rb
class Adwordscampaign
attr_reader :id
attr_reader :name
attr_reader :status
attr_reader :cost
attr_reader :impressions
attr_reader :clicks
attr_reader :ctr
attr_reader :costdecimal
attr_reader :costperconversiondecimal
def initialize(api_campaign)
#id = api_campaign[:id]
#name = api_campaign[:name]
#status = api_campaign[:status]
budget = api_campaign[:budget]
stats = api_campaign[:campaign_stats]
#cost = (stats[:cost][:micro_amount] / 10000)
#costdecimal = (#cost * 10000).round.to_f / 1000000
#impressions = stats[:impressions]
#clicks = stats[:clicks]
#ctr = (stats[:ctr] * 100)
end
def self.get_campaigns_list(response)
result = {}
if response[:entries]
response[:entries].each do |api_campaign|
campaign = Adwordscampaign.new(api_campaign)
result[campaign.id] = campaign
end
end
return result
end
end
The table from the views\adwordscampaign\index.html.erb
<table class="table table-striped table-bordered">
<tr>
<th>ID
<th>Name
<th>Status
<th>Impressions
<th>Clicks
<th>CTR
<th>Cost
<% #campaigns.each do |id, campaign| %>
<tr>
<td><%= campaign.id %></td>
<td><%= campaign.name %></td>
<td><%= campaign.status %></td>
<td><%= number_with_delimiter(campaign.impressions) %></td>
<td><%= number_with_delimiter(campaign.clicks) %></td>
<td><%= number_with_precision(campaign.ctr, precision: 2) %>%</td>
<td>$<%= number_with_delimiter(campaign.costdecimal) %></td>
<% end %>
<td><%= #campaigns %></td>
</table>
I threw #campaigns in the view (at the bottom) so I could see what the output of that was. The syntax is a little unfamiliar to me but it appears to be hashes nested in a hash (correct?)
output of #campaigns in the view
{109886905=>#<Adwordscampaign:0x4528ba0 #id=109879905, #name="Upholstery Cleaning", #status="ACTIVE", #cost=2702, #costdecimal=27.02, #impressions=824, #clicks=7, #ctr=0.8495145631067961>, 103480025=>#<Adwordscampaign:0x7028b28 #id=109880025, #name="Carpet Cleaning", #status="ACTIVE", #cost=16739, #costdecimal=167.39, #impressions=4457, #clicks=29, #ctr=0.6506618801884676>, 104560145=>#<Adwordscampaign:0x3e9ibac8 #id=109880145, #name="Competitors", #status="ACTIVE", #cost=1596, #costdecimal=15.96, #impressions=515, #clicks=5, #ctr=0.9708737864077669>
Finally to the question - How would I get the total #clicks (or #impressions, #cost, etc.) for each of the 3 campaigns found here?
I've been searching for things like "how to sum identical hash keys/values" or "how to merge nested hashes" to no avail.
Thanks in advance!

#campaigns looks like a simple hash of { id -> Adwordscampaign }. The #<ObjectName:data> notation is Ruby doing its best to give you something readable for that object.
To sum clicks, for example, is a simple map-reduce:
total_clicks = #campaigns.map { |id, campaign| campaign.clicks } .reduce(&:+)
This creates an array of all campaign.clicks values, then combines them all using the + operator.
This assumes there is a #clicks on those objects. If not, tweak accordingly.
If all you're doing is summing various attributes, you can simplify using it multiple times like so:
camp_list = #campaigns.map { |id, campaign| campaign }
total_clicks = camp_list.map(&:clicks).reduce(&:+)
total_cost = camp_list.map(&:cost).reduce(&:+)
total_impressions = camp_list.map(&:impressions).reduce(&:+)
Reducing it further is an exercise to the reader. :)

Related

How would I refactor this code, using Presenters || Decorators || Poros

My code works perfectly, however, I know my approach isn't best practice practice. I would appreciate the effort of anyone who tries to explain in details on how I could refactor this code using either(presenters, decorators or poros) - I have no deep understanding of either yet.
Here is the code below:
In my events controller, I have a method called tickets_report defined as such
def tickets_report
#all_bookings = #event.bookings
#ticket_types = #event.ticket_types
#attendees = #event.attendees
end
then, I use the #ticket_types instance variable in my views to plot a table as follows:
<tbody>
<% grand_quantity = ticket_left = quantity_sold = total_price = total_amount = 0 %>
<% #tickets.each_with_index do |ticket, index| %>
<% remains = ticket.quantity.to_i - ticket.user_tickets.count.to_i
amount = ticket.price.to_f * ticket.user_tickets.count.to_i
grand_quantity += ticket.quantity
ticket_left += remains
quantity_sold += ticket.user_tickets.count
total_price += ticket.price.to_f
total_amount += amount
%>
<tr>
<td><%= index + 1%></td>
<td><%= ticket.name %></td>
<td><%= ticket.quantity %></td>
<td><%= remains %></td>
<td><%= ticket.user_tickets.count %></td>
<td><%= number_to_currency(ticket.price)%></td>
<td><%= number_to_currency(amount)%></td>
</tr>
<% end %>
<tr class="gtotal">
<td colspan="2">Grand Total</td>
<td><%= grand_quantity %></td>
<td><%= ticket_left %></td>
<td><%= quantity_sold %></td>
<td><%= number_to_currency(total_price)%></td>
<td><%= number_to_currency(total_amount)%></td>
</tr>
</tbody>
My problem is that calculations done in the view is really not a good thing, and this is how far I have gone trying to fix this myself
module Events
class Ticket
def initialize(ticket)
#ticket = ticket
end
def name
#ticket.name
end
def quantity
#ticket.quantity
end
def price
#ticket.price.to_f
end
def user_count
#ticket.user_tickets.count.to_i
end
def remains
#ticket.quantity.to_i - user_count
end
def amount
price * user_count
end
end
class TicketReportPresenter < Ticket
##grand_quantity = ##ticket_left = ##quantity_sold = 0
##total_price = ##total_amount = 0
def initialize(ticket)
#ticket = Ticket.new(ticket)
##grand_quantity += #ticket.quantity.to_i
##ticket_left += #ticket.remains
##quantity_sold += #ticket.user_count
##total_price += #ticket.price
##total_amount ++ #ticket.amount
end
class << self
def grand_quantity
##grand_quantity
end
def ticket_left
##ticket_left
end
def quantity_sold
##quantity_sold
end
def total_price
##total_price
end
def total_amount
##total_amount
end
end
end
end
The new tickets_report method in my controller
def tickets_report
#all_bookings = #event.bookings
#ticket_types = Events::TicketReportPresenter.new(#event.ticket_types.first)
#attendees = #event.attendees
end
However, things don't seem to be going right at this point. For instance I cannot call ActiveRecord relationships on the tickets object I defined. What is the best approach to refactor the logic present in the view, a detailed description and source code will suffice plus links to other resources. Thanks.

Ruby: How should I remove duplicate lines from an Array?

I may not be using the correct terminology but here goes..
I'm displaying IPS alerts on a dashboard app and there are many duplicate lines. For example, if one script kiddie is trying to brute force an RDP server, I could get 150 Alerts but could be slimmed down to about 5 because that's how many hosts they are going after. So I'm trying to remove the duplicate alerts, and I'm looking to use the sid, src_addr, and dst_addr as my metrics to determine if they are duplicates.
Currently I display #filtered_snort_detail_query using this code:
This is my view
<% if #filtered_snort_detail_query.count > 0 %>
<table>
<tr>
<th>Timestamp</th>
<th>Tag Info</th>
<th>Message</th>
</tr>
<% #filtered_snort_detail_query.each do |d|
text_msg = d['_source']['message']
if d['_source']['message'].nil?
end
%>
<tr>
<td class='timestamp'><%= d['_source']['#timestamp'].to_time %></td>
<td class='tags'><%= d['_source']['tags'] %></td>
<td class='message'><%= text_msg %></td>
</tr>
<% end %>
</table>
<% else %>
<div> No Results Returned. </div>
<% end %>
Here is my controller
if #es_snort_detail_query.count > 0
#filtered_snort_detail_query = Array.new
#es_snort_detail_query.each do |ips_detail|
next if ips_detail['_source']['type'] != 'snort-ips'
next if ips_detail['_source']['#timestamp'] < #ts
#filtered_snort_detail_query.push(ips_detail)
end
end
Here is what I think I need to do to get the metrics I need to compare lines in my controller.
I'm just not sure the best way to look at each line of #filtered_snort_detail_query and build a new array to display in my view using these parameters:
show me all lines, but not if sid_data, src_ip_data, and dst_ip_data happen two or more times.
if #es_snort_detail_query.count > 0
#filtered_snort_detail_query = Array.new
#es_snort_detail_query.each do |ips_detail|
next if ips_detail['_source']['type'] != 'snort-ips'
next if ips_detail['_source']['#timestamp'] < #ts
#filtered_snort_detail_query.push(ips_detail)
end
if #filtered_snort_detail_query.count > 0
ip_src = Array.new
ip_dst = Array.new
sid = Array.new
#filtered_snort_detail_query.each do |find_ip, find_sid|
unless find_ip.nil?
sid_data = find_sid.scan(/\[\d+\:\d+\:\d+\]/)
src_ip_data = find_ip.scan(/(?:[0-9]{1,3}\.){3}[0-9]{1,3}/)
dst_ip_data = find_ip.scan(/(?:[0-9]{1,3}\.){3}[0-9]{1,3}/)
sid.push(sid_data[0]) unless sid_data[0].nil?
ip_src.push(src_ip_data[0]) unless src_ip_data[0].nil?
ip_dst.push(dst_ip_data[1]) unless dst_ip_data[1].nil?
end
end
end
end
Sorry if I misunderstood the question.
If you have a bunch of objects in an array, and you want to remove duplicates based on a subset of their properties, you can use the uniq method with a block:
queries.uniq do |query|
[query.sid_data, query.src_ip_data, query.dst_ip_data]
end
It will compare the queries based on the array created in the block, and remove duplicates.
Go to http://www.compileonline.com/execute_ruby_online.php, copy and paste the code below, and click execute script in the top left.
queries = [
{ :a => "ab", :b => "ba" },
{ :a => "ab", :b => "xy" },
{ :a => "xy", :b => "xy" }
]
unique_queries = queries.uniq do |q|
q[:a]
end
puts unique_queries
See? The comparaison was done only based on the value of the :a key. That's the same principle.

How to determine the order from a controller

I have a restaurant with many employees and each employee has many customer ratings.
I want to create a stats page that shows the employees ordered by their monthly ratings average.
In the employee model:
def avg_rating
date = Date.today
ratings_total = self.ratings.sum(:score, :conditions => {:created_at => (date.beginning_of_month..date.end_of_month)}).to_f
ratings_count = self.ratings.count(:conditions => {:created_at => (date.beginning_of_month..date.end_of_month)}).to_f
return (ratings_total/ ratings_count)
end
In the restaurant controller I have:
def restaurant_stats
#restaurant = Restaurant.find(params[:restaurant_id])
#employees = #restaurant.employees.all
end
In the restaurant stats view:
<table>
<% #employees.each do |employee| %>
<tr>
<td><%= employee.name %></td>
<td><%= employee.avg_rating %></td>
</tr>
<% end %>
</table>
I'm not sure how to get the employees in the correct order? I assume I would have to retrieve the values in the correct order in the restaurant_stats action instead of just #restaurant.employees.all but I'm not sure how to because of the functions used in the employees model
You could do, from the controller:
#employees = #restaurant.employees.all.sort_by {|employee| employee.avg_rating}
or more concisely
#employees = #restaurant.employees.all.sort_by(&:avg_rating)
Note that this will load all employees in memory for sorting.
Try in the restaurant controller:
#employees = #restaurant.employees.all.sort {|x,y| y.avg_rating <=> x.avg_rating }
or
#employees = #restaurant.employees.all.sort_by(:avg_rating)
you could create an array and sort it
I think below works but haven't checked it
#employees = #restaurant.employees.collect {|p| [ p.name, p.avg_rating ]}
#employees.sort!{ |a,b| (a[1] <=> b[1]) }

Ruby on Rails - Compare Active Record Result to Array Containing New Instansiated Objects

In my Rails app I am doing two things - comparing an Active Record result to an array built from a JSON response from external API.
First step is checking which results are in the database and not in the API and I do this as follows:
def self.orphaned_in_db
db_numbers = self.find(:all).map{|x| x.number}
listed_numbers = self.all_telco.map{|x| x.number} # gets JSON from API
orphaned_numbers = db_numbers - listed_numbers
orphaned_results = self.find_all_by_number(orphaned_numbers)
return orphaned_results
end
This is the new version as the old version was taking far too long after the result sets of each increased dramatically in the last few weeks.
# def self.orphaned_in_db
# old version
# db_numbers = self.find(:all)
# listed_numbers = self.all_telco
# orphaned_numbers = []
# db_numbers.each do |db|
# scan = listed_numbers.select{ |l| l.number == db.number}
# orphaned_numbers.push(db) if scan.empty?
# end
# return orphaned_numbers
# end
I am now finding it tricky to do the opposite of this - find numbers in the API array that are not in my database table.
def self.orphaned_in_telco
db_numbers = self.find(:all).map{|x| x.number}
all_numbers = self.all_telco
listed_numbers = all_numbers.map{|x| x.number}
orphaned_numbers = listed_numbers - db_numbers
orphaned_results = # how to filter all_numbers by orphaned_numbers?
return oprhaned_results
end
Again, the old way that now fails to work because it's so slow:
# def self.orphaned_in_telco
# original, inefficient way
# db_numbers = self.find(:all)
# listed_numbers = self.all_telco
# orphaned_numbers = []
# listed_numbers.each do |l|
# scan = db_numbers.select{ |db| l.number == db.number}
# orphaned_numbers.push(l) if scan.empty?
# end
# return orphaned_numbers
# end
I am finding this difficult because it's the same view and partial that was used before to display these orphaned numbers (both iterations) but it's legacy code so I've never seen it working in action but I'm just confused how it worked before with Active Record results and a normal array.
The view is just:
<%= render :partial => 'list_item', :collection => #telco_numbers ) %>
Where #telco_numbers is set to the return value of the above methods. (#telco_numbers = TelcoNumber.orphaned_in_telco params[:page])
The partial is as follows:
<tr>
<td>
<%= (link_to list_item.organisation.name.truncate(30), :controller => 'organisation', :action => 'update', :id => list_item.organisation.id) if list_item.organisation %>
</td>
<td class="centre"> <%= link_to ((list_item.countrycode == "44" ? Telco.format_uk_number(list_item.number) : "+#{list_item.countrycode} #{list_item.number}")), {:action => 'update', :id => list_item.id} %></td>
<td class="centre"><%= list_item.route_technology %></td>
<td><%= list_item.route_destination if list_item.route_destination %></td>
<td class="centre">
<% if !list_item.new_record? %>
[ <%= link_to 'Edit', {:action => 'update', :id => list_item.id} %> ]
<% else %>
[ <%= link_to 'Add New', {:action => 'update', :telco_number => list_item.attributes } %> ]
<% end %>
</td>
So as I understand it, for the the version I am trying to fix, instead of having an Edit action, it would have an Add New link since it's not in my database table yet so I am just trying to figure out how to refactor the inefficient version so that it will still work with the shared view.
If it helps, the format of the JSON API response is :
[{"country_code":"44","number":"1133508889","block":null,"type":"Legacy","SMS":"0"},
So only country_code and number correspond to columns in my database table, the rest isn't necessary, hence the if statements in the partial to only show certain parameters if they are available.
UPDATE
After changing the method to the following as suggested by Chris Vo, it finally works after taking ages to complete but it's still not quite right.
def self.orphaned_in_telco
# original, inefficient way
db_numbers = self.find(:all)
listed_numbers = self.all_telco
orphaned_numbers = listed_numbers - db_numbers
orphaned_results = []
orphaned_numbers.each do |n|
item = self.new()
item.id = n
orphaned_results.push(item)
end
return orphaned_results
end
The number column in my view html table just contains the + character and the Add New link doesn't have any value for the countrycode and number url parameters (link url is correct and all parameters are in the query string but they are all empty).
Some of the methods in my model:
def self.max_search_results
return ##max_search_results
end
#for pagination
def self.per_page
20
end
def self.some_telco(per_page, page = 1)
page = 1 if page.nil?
api_call = TelcoApiv3.new("post", "/numbers/#{TelcoApiv3.account_id}/allocated/all")
listed_numbers = TelcoApiv3.poll(api_call.response["link"])
return listed_numbers.collect do |ln|
ln.store("countrycode", ln["country_code"])
TelcoNumber.new ln
end
end
def self.all_telco(page = 1)
listed_numbers = some_telco(##max_nlist_results, page)
if listed_numbers.length == ##max_nlist_results
return listed_numbers.concat(all_telco(page + 1))
else
return listed_numbers
end
end
What if, for the orphaned_in_telco method, after you find the orphaned_numbers you create an instance of your model for every number in that set and then push it in a table to return them... or at least something in that direction. e.g.
orphaned_results = []
orphaned_numbers.each do |n|
item = self.new()
item.id = n
orphaned_results.push(item)
end
return orphaned_results
And then from the partial when you call Add new you would only need to call save on that instance.
This way you won't have the problem of Active Record and array for the partial, since you will be returning an array of Active record instances.
Also, my suggestion for speeding up things, would be to use a Hash to store the keys/numbers.
Hope it helps!
UPDATE
In order to have the countrycode and speed up things a little, I will continue with my Hash suggestion:
So, let's start from your initial implementation:
#this returns everything from the API
all_numbers = self.all_telco
#this returns a Hash in the form {:number => :country_code}
listed_numbers = Hash[all_numbers.map{|x| [x.number,x.country_code]}]
#so now you can do
orphaned_numbers = listed_numbers.keys - db_numbers
orphaned_results = []
orphaned_numbers.each do |n|
item = self.new()
item.number = n
item.countrycode = listed_numbers[n]
orphaned_results.push(item)
end
return orphaned_results
This should give it a boost and also send the country_code to the front side.
But first you're going to want to cut down on your db calls.
self.find(:all) #stupid slow
I'll point you in a right direction with
self.where('number NOT IN (?)', array_of_phone_numbers)
This will find all records that are not in the json data.

How do you get the rows and the columns in the result of a query with ActiveRecord?

Is there a way with ActiveRecord to execute a custom SQL query and have it return an array of arrays where the first row is the column names and each following row is the row data? I want to execute something like:
connection.select_rows_with_headers "SELECT id, concat(first_name, ' ', last_name) as name, email FROM users"
And have it return:
[["id","name","email"],["1","Bob Johnson","bob#example.com"],["2","Joe Smith","joe#example.com"]]
This would allow me to print the results of the custom query in an HTML table like this:
<table>
<% result.each_with_index do |r,i| %>
<tr>
<% r.each do |c| %>
<% if i == 0 %>
<th><%=h c %></th>
<% else %>
<td><%=h c %></td>
<% end %>
<% end %>
</tr>
<% end %>
</table>
Note that select_all doesn't work because the keys in each hash are unordered, so you've lost the ordering of the results as specified in the query.
Not EXACTLY what you're looking for, but maybe:
connection.execute('select * from users').all_hashes
and you'll get back
[{:id => 1, :name => 'Bob', :email => 'bob#example.com'},{:id => 1, :name => 'Joe', :email => 'joe#example.com'}]
and you could do:
results = connection.execute('select * from users').all_hashes
munged_results = []
columns = results.first.keys.map(&:to_s)
munged_results << results.first.keys.map(&:to_s)
munged_results += results.map{|r| columns.map{|c| r[c]} }
something like that
edit:
results = connection.execute('select * from users').all_hashes
munged_results = []
columns = User.column_names
munged_results << columns
munged_results += results.map{|r| columns.map{|c| r[c]} }
That should be ordered properly.
Beyond that there is the result object that is returned from #execute that can be interrogated for bits of information. Methods like #fetch_fields get you the fields in order and #fetch_row will get you each row of the result set as an array (works like an iterator).
edit again:
OK, here's a good solution, modify for whatever DB you're using:
class Mysql::Result
def all_arrays
results = []
results << fetch_fields.map{|f| f.name}
while r = fetch_row
results << r
end
results
end
end
That will get them without a ton of overhead.
Use it like this:
connection.execute('select salt, id from users').all_arrays

Resources