Rails 3.0.3 application (stuck on a Dreamhost shared server).
I have a page that displays averages calculated from subsets of data from one model.
Right now, each average is calculated individually, like this:
From the view, I'm using the current_user helper provided by Devise authentication to call the average methods that are located in the user model, like so:
<%= current_user.seven_day_weight_average %>
<%= current_user.fourteen_day_weight_average %>
<%= current_user.thirty_day_weight_average %>
Here's the public methods and the averaging method in the user model:
def seven_day_weight_average
calculate_average_weight(7)
end
def fourteen_day_weight_average
calculate_average_weight(14)
end
def thirty_day_weight_average
calculate_average_weight(30)
end
. . .
private
def calculate_average_weight(number_days)
temp_weight = 0
weights_array = self.weights.find_all_by_entry_date(number_days.days.ago..Date.today)
unless weights_array.count.zero?
weights_array.each do |weight|
temp_weight += weight.converted_weight
end
return (temp_weight/weights_array.count).round(1).to_s
else
return '0.0'
end
end
This doesn't seem very efficient - the database is queried for every average calculated.
How can I calculate and make these averages available to the page with one database query?
You could cache an array of converted weights for the last 30 days (presuming 30 is the maximum days back), something like this:
def calculate_average_weight(number_days)
#converted_weights ||= weights.where("entry_date > ?", 30.days.ago).group_by(&:entry_date).sort_by do |date,weights|
date
end.collect do |date,weights|
weights.collect(&:converted_weight)
end
weights_during_period = #converted_weights[0..number_days-1].flatten
weights_during_period.sum / weights_during_period.length
end
Explanation:
Firstly, ||= gets or sets #converted_weights (ie don't bother setting it unless it's nil or false). This ensures only one db hit. Next, we find all weights from 30 days ago and group by date. This returns an array of [date, weights], which we sort by date. Then we collect the converted weights for each date, so we end up with: [weights on day 1], [weights on day 2], ....
Now, the calculation: we store values spanning the number of days from the array in weights_during_period. We flatten the values and calculate the average value.
Related
Here's the situation:
I have an Event model and I want to add prev / next buttons to a view to get the next event, but sorted by the event start datetime, not the ID/created_at.
So the events are created in the order that start, so I can compare IDs or get the next highest ID or anything like that. E.g. Event ID 2 starts before Event ID 3. So Event.next(3) should return Event ID 2.
At first I was passing the start datetime as a param and getting the next one, but this failed when there were 2 events with the same start. The param start datetime doesn't include microseconds, so what would happen is something like this:
order("start > ?",current_start).first
would keep returning the same event over and over because current_start wouldn't include microseconds, so the current event would technically be > than current_start by 0.000000124 seconds or something like that.
The way I got to work for everything was with a concern like this:
module PrevNext
extend ActiveSupport::Concern
module ClassMethods
def next(id)
find_by(id: chron_ids[current_index(id)+1])
end
def prev(id)
find_by(id: chron_ids[current_index(id)-1])
end
def chron_ids
#chron_ids ||= order("#{order_by_attr} ASC").ids
end
def current_index(id)
chron_ids.find_index(id)
end
def order_by_attr
#order_by_attr ||= 'created_at'
end
end
end
Model:
class Event < ActiveRecord::Base
...
include PrevNext
def self.order_by_attr
#order_by_attr ||= "start_datetime"
end
...
end
I know pulling all the IDs into an array is bad and dumb* but i don't know how to
Get a list of the records in the order I want
Jump to a specific record in that list (current event)
and then get the next record
...all in one ActiveRecord query. (Using Rails 4 w/ PostgreSQL)
*This table will likely never have more than 10k records, so it's not catastrophically bad and dumb.
The best I could manage was to pull out only the IDs in order and then memoize them.
Ideally, i'd like to do this by just passing the Event ID, rather than a start date params, since it's passed via GET param, so the less URL encoding and decoding the better.
There has to be a better way to do this. I posted it on Reddit as well, but the only suggested response didn't actually work.
Reddit Link
Any help or insight is appreciated. Thanks!
You can get the next n records by using the SQL OFFSET keyword:
china = Country.order(:population).first
india = City.order(:population).offset(1).take
# SELECT * FROM countries ORDER BY population LIMIT 1 OFFSET 1
Which is how pagination for example often is done:
#countries = Country.order(:population).limit(50)
#countries = scope.offset( params[:page].to_i * 50 ) if params[:page]
Another way to do this is by using would be query cursors. However ActiveRecord does not support this and it building a generally reusable solution would be quite a task and may not be very useful in the end.
Context:
Trying to generating an array with 1 element for each created_at day in db table. Each element is the average of the points (integer) column from records with that created_at day.
This will later be graphed to display the avg number of points on each day.
Result:
I've been successful in doing this, but it feels like an unnecessary amount of code to generate the desired result.
Code:
def daily_avg
# get all data for current user
records = current_user.rounds
# make array of long dates
long_date_array = records.pluck(:created_at)
# create array to store short dates
short_date_array = []
# remove time of day
long_date_array.each do |date|
short_date_array << date.strftime('%Y%m%d')
end
# remove duplicate dates
short_date_array.uniq!
# array of avg by date
array_of_avg_values = []
# iterate through each day
short_date_array.each do |date|
temp_array = []
# make array of records with this day
records.each do |record|
if date === record.created_at.strftime('%Y%m%d')
temp_array << record.audio_points
end
end
# calc avg by day and append to array_of_avg_values
array_of_avg_values << temp_array.inject(0.0) { |sum, el| sum + el } / temp_array.size
end
render json: array_of_avg_values
end
Question:
I think this is a common extraction problem needing to be solved by lots of applications, so I'm wondering if there's a known repeatable pattern for solving something like this?
Or a more optimal way to solve this?
(I'm barely a junior developer so any advice you can share would be appreciated!)
Yes, that's a lot of unnecessary stuff when you can just go down to SQL to do it (I'm assuming you have a class called Round in your app):
class Round
DAILY_AVERAGE_SELECT = "SELECT
DATE(rounds.created_at) AS day_date,
AVG(rounds.audio_points) AS audio_points
FROM rounds
WHERE rounds.user_id = ?
GROUP BY DATE(rounds.created_at)
"
def self.daily_average(user_id)
connection.select_all(sanitize_sql_array([DAILY_AVERAGE_SELECT, user_id]), "daily-average")
end
end
Doing this straight into the database will be faster (and also include less code) than doing it in ruby as you're doing now.
I advice you to do something like this:
grouped =
records.order(:created_at).group_by do |r|
r.created_at.strftime('%Y%m%d')
end
At first here you generate proper SQL near to that you wish to get in first approximation, then group result records by created_at field converted to just a date.
points =
grouped.map do |(date, values)|
[ date, values.reduce(0.0, :audio_points) / values.size ]
end.to_h
# => { "1-1-1970" => 155.0, ... }
Then you remap your grouped hash via array, to calculate average values with audio_points.
You can use group and calculations methods built in AR: http://guides.rubyonrails.org/active_record_querying.html#group
http://guides.rubyonrails.org/active_record_querying.html#calculations
Ok so I have an app that allows users to pull App Store data, specifically top free top paid etc. The various attributes are quite limited, but users can filter by category and country. So obviously this leads to a lot of repeated queries, now normally this wouldn't be a problem, but I also use this data with google api which has a credits system. So What I want to do is save these results in my database if the results are unique. I have this all set up and fine but my only hang up is how I determine if a query has been made before, so my solution is to make a hashtable that stores all queries that have been made before and if not NULL(nil) then I call the api to fetch the data then create a new record.
Issue is the App Store refreshes every day or so(not exactly sure the schedule but will look it up later). I would like to have this Hashtable reference function refresh or reset itself to all NULL at this interval.
What would be the most efficient or simple way to start a refresh for this? Additionally I am kinda new to rails, so where should I place this function? In the helper modules? Controller?
Thanks!
Edit:
ok so here is my HashTable helper module
module MapsHelper
queryHistoryLookUp = {}
i = 0
31.times do |i|
queryTableLookup.merge!(i =>[] )
end
def queryTableLookup(asciiNum, queryString)
if queryTableLookup[asciiNum % 31].size == 0
queryTableLookup[asciiNum % 31].push(queryString)
else
a = queryTableLookup[asciiNum % 31].size
arrayOfQueries = queryTableLookup[asciiNum % 31]
a.times do |i|
if arrayOfQueries[i] == queryString
return true
else
return false
end
end
end
end
end
def queryHash(query)
asciSum = 0
query.each_char do |i|
asciSum += i.sum
end
queryTableLookup(asciSum, query)
end
end
additionally I am kinda new to rails, can I interact with these functions using Javascript, since on the client side I create the string query.
In my opinion, your best bet would be to use the Rails cache system. It provides a method of caching data, with an optional expires_in time.
From the docs:
http://guides.rubyonrails.org/caching_with_rails.html#low-level-caching
class MyModel < ActiveRecord::Base
def self.get_api_data(key)
Rails.cache.fetch("my_model/api_data:#{key}", expires_in: 12.hours) do
SomeService::API.get_data(key)
end
end
end
In your hash (which I think it could exist in a class variable) you can store both the query and the last access datetime:
Suppose you have a hash as class variable to the Foo class with name cache and that the query variable is your current query that you want to check.
if Foo.cache[query].nil? || (DateTime.now - Foo.cache[query].last_fetch).to_i > 0
results = your_method_to_fetch_data_for(query)
Foo.cache[query] = {:results => results, :last_fetch => Datetime.now}
else
results = Foo.cache[query][:results]
end
The following query runs fairly quickly, but the series processing that needs to take place afterwards is really slowing this method down. I could use some help in refactoring.
def self.sum_amount_chart_series(start_time)
orders_by_day = Widget.archived.not_void.
where(:print_datetime => start_time.beginning_of_day..Time.zone.now.end_of_day).
group(pg_print_date_group).
select("#{pg_print_date_group} as print_date, sum(amount) as total_amount")
# THIS IS WHAT IS SLOWING THE METHOD DOWN!
(start_time.to_date..Date.today).map do |date|
order = orders_by_day.detect { |order| order.print_date.to_date == date }
order && order.total_amount.to_f.round(2) || 0.0
end
end
def self.pg_print_date_group
"CAST((print_datetime + interval '#{tz_offset_hours} hours') AS date)"
end
I have benchmarked this method and the offending code is the series loop where it generates a series of dates and then maps out a new array with an amount for each date. This way I get a series back with amounts for every date, regardless if it has an amount or not.
When the query only returns a few dates, it runs fairly quickly. But set the start date back a year or two and it becomes impossibly slow. The real offender is the .detect method. It's very slow at scanning the array of activerecord objects.
Is there a faster method to generates this series?
orders_by_day is grouped by "pg_print_date_group" so it should be a hash of "date" to objects. so why don't you just do
(start_time.to_date..Date.today).map do |date|
order = orders_by_day[date.to_s(:db)]
order && order.total_amount.to_f.round(2) || 0.0
end
That should seriously reduce the Big O of your run. And if I'm misunderstanding and your orders_by_day isn't a hash, preprocess it into a hash and then run the map, you definitely don't want to detect for every date.
Since the primary offender in your code is the detect method that has to scan the array again and again, I suggest that you invert the order in which you create the series, so that you only scan the array once, and your code runs in O(n) time.
Try something along the lines of:
series = []
next_date = start_time.to_date
orders_by_day.each do |order|
while order.print_date.to_date < next_date
series << 0.0
next_date = next_date.next
end
series << order.total_amount.to_f.round(2)
next_date += 1
end
while next_date < Date.today
series << 0.0
next_date = next_date.next
end
Please note that my code is untested ;)
Is there an easy way to obtain the average of an attribute in a collection?
For instance, each user has a score.
Given a collection of user(s) (#users), how can you get the average score for the group?
Is there anything like #users.average(:score)? I think I came across something like this for database fields, but I need it to work for a collection...
For your question, one could actually do:
#users.collect(&:score).sum.to_f/#users.length if #users.length > 0
Earlier I thought, #users.collect(&:score).average would have worked. For database fields, User.average(:score) will work. You can also add :conditions like other activerecord queries.
I use to extend our friend Array with this method:
class Array
# Calculates average of anything that responds to :"+" and :to_f
def avg
blank? and 0.0 or sum.to_f/size
end
end
Here's a little snippet to not only get the average but also the standard deviation.
class User
attr_accessor :score
def initialize(score)
#score = score
end
end
#users=[User.new(10), User.new(20), User.new(30), User.new(40)]
mean=#users.inject(0){|acc, user| acc + user.score} / #users.length.to_f
stddev = Math.sqrt(#users.inject(0) { |sum, u| sum + (u.score - mean) ** 2 } / #users.length.to_f )
u can use this here
http://api.rubyonrails.org/classes/ActiveRecord/Calculations.html#method-i-average