Ruby on Rails method to calculate percentiles - can it be refactored? - ruby-on-rails

I have written a method to calculate a given percentile for a set of numbers for use in an application I am building. Typically the user needs to know the 25th percentile of a given set of numbers and the 75th percentile.
My method is as follows:
def calculate_percentile(array,percentile)
#get number of items in array
return nil if array.empty?
#sort the array
array.sort!
#get the array length
arr_length = array.length
#multiply items in the array by the required percentile (e.g. 0.75 for 75th percentile)
#round the result up to the next whole number
#then subtract one to get the array item we need to return
arr_item = ((array.length * percentile).ceil)-1
#return the matching number from the array
return array[arr_item]
end
This looks to provide the results I was expecting but can anybody refactor this or offer an improved method to return specific percentiles for a set of numbers?

Some remarks:
If a particular index of an Array does not exist, [] will return nil, so your initial check for an empty Array is unnecessary.
You should not sort! the Array argument, because you are affecting the order of the items in the Array in the code that called your method. Use sort (without !) instead.
You don't actually use arr_length after assignment.
A return statement on the last line is unnecessary in Ruby.
There is no standard definition for the percentile function (there can be a lot of subtleties with rounding), so I'll just assume that how you implemented it is how you want it to behave. Therefore I can't really comment on the logic.
That said, the function that you wrote can be written much more tersely while still being readable.
def calculate_percentile(array, percentile)
array.sort[(percentile * array.length).ceil - 1]
end

Here's the same refactored into a one liner. You don't need an explicit return as the last line in Ruby. The return value of the last statement of the method is what's returned.
def calculate_percentile(array=[],percentile=0.0)
# multiply items in the array by the required percentile
# (e.g. 0.75 for 75th percentile)
# round the result up to the next whole number
# then subtract one to get the array item we need to return
array ? array.sort[((array.length * percentile).ceil)-1] : nil
end

Not sure if it's worth it, but here is how I did it for the quartiles:
def median(list)
(list[(list.size - 1) / 2] + list[list.size / 2]) / 2
end
numbers = [1, 2, 3, 4, 5, 6]
if numbers.size % 2 == 0
puts median(numbers[0...(numbers.size / 2)])
puts median(numbers)
puts median(numbers[(numbers.size / 2)..-1])
else
median_index = numbers.index(median(numbers))
puts median(numbers[0..(median_index - 1)])
puts median(numbers)
puts median(numbers[(median_index + 1)..-1])
end

If you're calculating both quartiles, you might want to move the "sort" outside the function, so that it only needs to be done once. This also means you aren't modifying your caller's data (sort!), nor making a copy every time the function is called (sort).
I know, premature optimisation and all that. And it's a bit awkward for the function to say, "the array must be sorted before calling this function". So it's reasonable to leave it as it is.
But sorting already-sorted data is going to take considerably longer than the whole rest of the function put together(*). It also has higher algorithmic complexity: O(N) at best, when the function could be O(1) for the second quartile (although O(N log N) for the first one if the data is not already sorted, of course). So it's worth avoiding if performance might ever be an issue for this function.
There are slightly faster ways of finding the two quartiles than a full sort (look up "selection algorithms"). For instance if you're familiar with the way qsort uses pivots, observe that if you need to know the 25th and 75th items out of 100, and your pivot at some stage ends up in position 80, then there's absolutely no point recursing into the block above the pivot. You really don't care what order those elements are in, just that they're in the top quartile. But this will considerably increase the complexity of the code compared with just calling a library to sort for you. Unless you really need a minor performance boost, I think you're good as you are.
(*) Unless ruby arrays have a flag to remember they're already sorted and haven't been modified since. I don't know whether they do, but if so then using sort! a second time is of course free.

Related

Lua: Sort table of numbers with multiple dots

I have a table of strings like this:
{
"1",
"1.5",
"3.13",
"1.2.5.7",
"2.5",
"1.3.5",
"2.2.5.7.10",
"1.17",
"1.10.5",
"2.3.14.9",
"3.5.21.9.3",
"4"
}
And would like to sort that like this:
{
"1",
"1.2.5.7",
"1.3.5",
"1.5",
"1.10.5",
"1.17",
"2.2.5.7.10",
"2.3.14.9",
"2.5",
"3.5.21.9.3",
"3.13",
"4"
}
How do I sort this in Lua? I know that table.sort() will be used, I just don't know the function (second parameter) to use for comparison.
Given your requirements, you probably want something like natural sort order. I described several possible solution as well as their impact on the results in a blog post.
The simplest solution may look like this (below), but there are 5 different solutions listed with different complexity and the results:
function alphanumsort(o)
local function padnum(d) return ("%03d%s"):format(#d, d) end
table.sort(o, function(a,b)
return tostring(a):gsub("%d+",padnum) < tostring(b):gsub("%d+",padnum) end)
return o
end
table.sort sorts ascending by default. You don't have to provide a second parameter then. As you're sorting strings Lua will compare the strings character by character. Hence you must implement a sorting function that tells Lua which comes first.
I just don't know the function (second parameter) to use for
comparison.
That's why people wrote the Lua Reference Manual
table.sort (list [, comp])
Sorts the list elements in a given order, in-place, from list1 to
list[#list]. If comp is given, then it must be a function that
receives two list elements and returns true when the first element
must come before the second in the final order, so that, after the
sort, i <= j implies not comp(list[j],list[i]). If comp is not given,
then the standard Lua operator < is used instead.
The comp function must define a consistent order; more formally, the
function must define a strict weak order. (A weak order is similar to
a total order, but it can equate different elements for comparison
purposes.)
The sort algorithm is not stable: Different elements considered equal
by the given order may have their relative positions changed by the
sort.
Think about how you would do it with pen an paper. You would compare each number segment. As soon as a segment is smaller than the other you know this number comes first.
So a solution would probably require you to get those segments for the strings, convert them to numbers so you can compare their values...

How to iterate over an ActiveRecord resultset in one line with nil check in Ruby

I have an array of Active Record result and I want to iterate over each record to get a specific attribute and add all of them in one line with a nil check. Here is what I got so far
def total_cost(cost_rec)
total= 0.0
unless cost_rec.nil?
cost_rec.each { |c| total += c.cost }
end
total
end
Is there an elegant way to do the same thing in one line?
You could combine safe-navigation (to "hide" the nil check), summation inside the database (to avoid pulling a bunch of data out of the database that you don't need), and a #to_f call to hide the final nil check:
cost_rec&.sum(:cost).to_f
If the cost is an integer, then:
cost_rec&.sum(:cost).to_i
and if cost is a numeric inside the database and you don't want to worry about precision issues:
cost_rec&.sum(:cost).to_d
If cost_rec is an array rather than a relation (i.e. you've already pulled all the data out of the database), then one of:
cost_rec&.sum(&:cost).to_f
cost_rec&.sum(&:cost).to_i
cost_rec&.sum(&:cost).to_d
depending on what type cost is.
You could also use Kernel#Array to ignore nils (since Array(nil) is []) and ignore the difference between arrays and ActiveRecord relations (since #Array calls #to_ary and relations respond to that) and say:
Array(cost_rec).sum(&:cost)
that'll even allow cost_rec to be a single model instance. This also bypasses the need for the final #to_X call since [].sum is 0. The downside of this approach is that you can't push the summation into the database when cost_rec is a relation.
anything like these?
def total_cost(cost_rec)
(cost_rec || []).inject(0) { |memo, c| memo + c.cost }
end
or
def total_cost(cost_rec)
(cost_rec || []).sum(&:cost)
end
Either one of these should work
total = cost_rec.map(&:cost).compact.sum
total = cost_rec.map{|c| c.cost }.compact.sum
total = cost_rec.pluck(:cost).compact.sum
Edit: if cost_rec is nil
total = (cost_rec || []).map{|c| c.cost }.compact.sum
When cost_rec is an ActiveRecord::Relatation then this should work out of the box:
cost_rec.sum(:cost)
See ActiveRecord::Calculations#sum.

table size difference. are both examples identical?

tNum={[2]=true , [3]=true,[4]=true, [5]=true ,[6]=true }
#tNum-->0
tNum={}
tNum[2]=true
tNum[3]=true
tNum[4]=true
tNum[5]=true
tNum[6]=true
#tNum-->6
why such a difference in size?
are both examples identical?
Your two tables are semantically identical, but using # on them is ambiguous. Both 0 and 6 are correct lengths. Here's an abridged version of the docs:
The length operator applied on a table returns a border in that table. A border in a table t is any natural number that satisfies the following condition:
(border == 0 or t[border] ~= nil) and t[border + 1] == nil
A table with exactly one border is called a sequence.
When t is not a sequence, #t can return any of its borders. (The exact one depends on details of the internal representation of the table, which in turn can depend on how the table was populated and the memory addresses of its non-numeric keys.)
This is an example of undefined behavior (UB). (That may not be the right word, because the behavior is partially defined. UB in Lua can't launch nuclear weapons, as it can in C.) Undefined behavior is important, because it gives the devs the freedom to choose the fastest possible algorithm without worrying about what happens when a user violates their assumptions.
To find a length, Lua makes, at most, log n guesses instead of looking at every element to find an unambiguous length. For large arrays, this speeds things up a lot.
The issue is that when you define a table as starting at index [2], the length operator breaks because it assumes that tables start at index [1].
The following code works as intended:
tNum = {[1]=false, [2]=true, [3]=true, [4]=true, [5]=true, [6]=true}
#tNum => 6
The odd behaviour is caused because when you initialize an array with tNum={} it initializes by assigning every index to nil, and the first index is [1] (It doesn't actually initialize every value to nil, but it's easier to explain that way).
Conversely, when you initialize an array with tNum={[2]=true} you are explicitly telling the array that tNum[1] does not exist and the array begins at index 2. The length calculation breaks when you do this.
For a more thorough explanation, see this section of the lua wiki near the bottom where it explains:
For those that really want their arrays starting at 0, it is not difficult to write the following:
days = {[0]="Sunday", "Monday", "Tuesday", "Wednesday",
"Thursday", "Friday", "Saturday"}
Now, the first value, "Sunday", is at index 0. That zero does not affect the other fields, but "Monday" naturally goes to index 1, because it is the first list value in the constructor; the other values follow it. Despite this facility, I do not recommend the use of arrays starting at 0 in Lua. Remember that most functions assume that arrays start at index 1, and therefore will not handle such arrays correctly.
The Length operator assumes your array will begin at index [1], and since it does not, it doesn't work correctly.
I hope this was helpful, good luck with your code!

remove all entries with specified value in table

I want to delete all entries from a table, wich equals a given value.
Now, I got a pretty little problem one might to know, how to handle.
This is the Code:
function(list_to_search_in, compared_value, invert)
for k,v in pairs(list_to_search_in) do
if invert and v ~= compared_value then
table.remove(list_to_search_in, v)
if not invert and v == compared_value then
table.remove(list_to_search_in, v)
end
end
end
The Problem:
Let's say the table is { 1, 2, 3, 2 }. So when I'm iterating through that loop and come to the first match, it's removed from the table. This means the value and the key is deleted.
Now the key of the deleten value is assigned to the next value in line. But due the skript will check the value of the next key, this value (whichs kay has been just altered) will never be checked.
I thought, a simple
k = k - 1
after a remove would do the job, but it doesn't.
v = nil
would do great I think, but only if garbage-collector does not do his job in this very moment the pairs iterates to the next value.
Anyone has an idea? I would prefer an text-based hint to a finished syntax which solves the problem.
Don't use table.remove for this. It squeezes the "hole" out of array-like tables. That's not allowed during an iteration using pairs/next. Just set the value to nil.
If you need to squeeze holes out of the table then you can either create a new table and populate it with only the values you want to keep or do the removals during the first pass and then squeeze out holes in a second pass.
Also the order of item traversal when using pairs is not guaranteed in any way.

How do I collect and combine multiple arrays for calculation?

I am collecting the values for a specific column from a named_scope as follows:
a = survey_job.survey_responses.collect(&:base_pay)
This gives me a numeric array for example (1,2,3,4,5). I can then pass this array into various functions I have created to retrieve the mean, median, standard deviation of the number set. This all works fine however I now need to start combining multiple columns of data to carry out the same types of calculation.
I need to collect the details of perhaps three fields as follows:
survey_job.survey_responses.collect(&:base_pay)
survey_job.survey_responses.collect(&:bonus_pay)
survey_job.survey_responses.collect(&:overtime_pay)
This will give me 3 arrays. I then need to combine these into a single array by adding each of the matching values together - i.e. add the first result from each array, the second result from each array and so on so I have an array of the totals.
How do I create a method which will collect all of this data together and how do I call it from the view template?
Really appreciate any help on this one...
Thanks
Simon
s = survey_job.survey_responses
pay = s.collect(&:base_pay).zip(s.collect(&:bonus_pay), s.collect(&:overtime_pay))
pay.map{|i| i.compact.inject(&:+) }
Do that, but with meaningful variable names and I think it will work.
Define a normal method in app/helpers/_helper.rb and it will work in the view
Edit: now it works if they contain nil or are of different sizes (as long as the longest array is the one on which zip is called.
Here's a method that will combine an arbitrary number of arrays by taking the sum at each index. It'll allow each array to be of different length, too.
def combine(*arrays)
# Get the length of the largest array, that'll be the number of iterations needed
maxlen = arrays.map(&:length).max
out = []
maxlen.times do |i|
# Push the sum of all array elements at a given index to the result array
out.push( arrays.map{|a| a[i]}.inject(0) { |memo, value| memo += value.to_i } )
end
out
end
Then, in the controller, you could do
base_pay = survey_job.survey_responses.collect(&:base_pay)
bonus_pay = survey_job.survey_responses.collect(&:bonus_pay)
overtime_pay = survey_job.survey_responses.collect(&:overtime_pay)
#total_pay = combine(base_pay, bonus_pay, overtime_pay)
And then refer to #total_pay as needed in your view.

Resources