This is the code in my reports controller, it just looks so bad, can anyone give me some suggestions on how to tidy it up?
# app\controller\reports_controller.rb
#report_lines = []
#sum_wp, #sum_projcted_wp, #sum_il, #sum_projcted_il, #sum_li,#sum_gross_profit ,#sum_opportunities = [0,0,0,0,0,0,0]
date = #start_date
num_of_months.times do
wp,projected_wp, invoice_line,projected_il,line_item, opp = Report.data_of_invoicing_and_delivery_report(#part_or_service,date)
#sum_wp += wp
#sum_projcted_wp +=projected_wp
#sum_il=invoice_line
#sum_projcted_il +=projected_il
#sum_li += line_item
gross_profit = invoice_line - line_item
#sum_gross_profit += gross_profit
#sum_opportunities += opp
#report_lines << [date.strftime("%m/%Y"),wp,projected_wp ,invoice_line,projected_il,line_item,gross_profit,opp]
date = date.next_month
end
I'm looking to use some method like
#sum_a,#sum_b,#sum_c += [1,2,3]
My instant thought is: move the code to a model.
The objective should be "Thin Controllers", so they should not contain business logic.
Second, I like to present my report lines to my Views as OpenStruct() objects, which seems cleaner to me.
So I'd consider moving this accumulation logic into (most likely) a class method on Report and returning an array of "report line" OpenStructs and a single totals OpenStruct to pass to my View.
My controller code would become something like this:
#report_lines, #report_totals = Report.summarised_data_of_inv_and_dlvry_rpt(#part_or_service, #start_date, num_of_months)
EDIT: (A day later)
Looking at that adding accumulating-into-an-array thing, I came up with this:
require 'test/unit'
class Array
def add_corresponding(other)
each_index { |i| self[i] += other[i] }
end
end
class TestProblem < Test::Unit::TestCase
def test_add_corresponding
a = [1,2,3,4,5]
assert_equal [3,5,8,11,16], a.add_corresponding([2,3,5,7,11])
assert_equal [2,3,6,8,10], a.add_corresponding([-1,-2,-2,-3,-6])
end
end
Look: a test! It seems to work OK. There are no checks for differences in size between the two arrays, so there's lots of ways it could go wrong, but the concept seems sound enough. I'm considering trying something similar that would let me take an ActiveRecord resultset and accumulate it into an OpenStruct, which is what I tend to use in my reports...
Our new Array method might reduce the original code to something like this:
totals = [0,0,0,0,0,0,0]
date = #start_date
num_of_months.times do
wp, projected_wp, invoice_line, projected_il, line_item, opp = Report.data_of_invoicing_and_delivery_report(#part_or_service,date)
totals.add_corresponding [wp, projected_wp, invoice_line, projected_il, line_item, opp, invoice_line - line_item]
#report_lines << [date.strftime("%m/%Y"),wp,projected_wp ,invoice_line,projected_il,line_item,gross_profit,opp]
date = date.next_month
end
#sum_wp, #sum_projcted_wp, #sum_il, #sum_projcted_il, #sum_li, #sum_opportunities, #sum_gross_profit = totals
...which if Report#data_of_invoicing_and_delivery_report could also calculate gross_profit would reduce even further to:
num_of_months.times do
totals.add_corresponding(Report.data_of_invoicing_and_delivery_report(#part_or_service,date))
end
Completely un-tested, but that's a hell of a reduction for the addition of a one-line method to Array and performing a single extra subtraction in a model.
Create a summation object that contains all those fields, pass the entire array to #sum.increment_sums(Report.data_of...)
Related
Given this model:
class User < ActiveRecord::Base
has_many :things
end
Then we can do this::
#user = User.find(123)
#user.things.find_each{ |t| print t.name }
#user.thing_ids.each{ |id| print id }
There are a large number of #user.things and I want to iterate through only their ids in batches, like with find_each. Is there a handy way to do this?
The goal is to:
not load the entire thing_ids array into memory at once
still only load arrays of thing_ids, and not instantiate a Thing for each id
Rails 5 introduced in_batches method, which yields a relation and uses pluck(primary_key) internally. And we can make use of the where_values_hash method of the relation in order to retrieve already-plucked ids:
#user.things.in_batches { |batch_rel| p batch_rel.where_values_hash['id'] }
Note that in_batches has order and limit restrictions similar to find_each.
This approach is a bit hacky since it depends on the internal implementation of in_batches and will fail if in_batches stops plucking ids in the future. A non-hacky method would be batch_rel.pluck(:id), but this runs the same pluck query twice.
You can try something like below, the each slice will take 4 elements at a time and them you can loop around the 4
#user.thing_ids.each_slice(4) do |batch|
batch.each do |id|
puts id
end
end
It is, unfortunately, not a one-liner or helper that will allow you to do this, so instead:
limit = 1000
offset = 0
loop do
batch = #user.things.limit(limit).offset(offset).pluck(:id)
batch.each { |id| puts id }
break if batch.count < limit
offset += limit
end
UPDATE Final EDIT:
I have updated my answer after reviewing your updated question (not sure why you would downvote after I backed up my answer with source code to prove it...but I don't hold grudges :)
Here is my solution, tested and working, so you can accept this as the answer if it pleases you.
Below, I have extended ActiveRecord::Relation, overriding the find_in_batches method to accept one additional option, :relation. When set to true, it will return the activerecord relation to your block, so you can then use your desired method 'pluck' to get only the ids of the target query.
#put this file in your lib directory:
#active_record_extension.rb
module ARAExtension
extend ActiveSupport::Concern
def find_in_batches(options = {})
options.assert_valid_keys(:start, :batch_size, :relation)
relation = self
start = options[:start]
batch_size = options[:batch_size] || 1000
unless block_given?
return to_enum(:find_in_batches, options) do
total = start ? where(table[primary_key].gteq(start)).size : size
(total - 1).div(batch_size) + 1
end
end
if logger && (arel.orders.present? || arel.taken.present?)
logger.warn("Scoped order and limit are ignored, it's forced to be batch order and batch size")
end
relation = relation.reorder(batch_order).limit(batch_size)
records = start ? relation.where(table[primary_key].gteq(start)) : relation
records = records.to_a unless options[:relation]
while records.any?
records_size = records.size
primary_key_offset = records.last.id
raise "Primary key not included in the custom select clause" unless primary_key_offset
yield records
break if records_size < batch_size
records = relation.where(table[primary_key].gt(primary_key_offset))
records = records.to_a unless options[:relation]
end
end
end
ActiveRecord::Relation.send(:include, ARAExtension)
here is the initializer
#put this file in config/initializers directory:
#extensions.rb
require "active_record_extension"
Originally, this method forced a conversion of the relation to an array of activrecord objects and returned it to you. Now, I optionally allow you to return the query before the conversion to the array happens. Here is an example of how to use it:
#user.things.find_in_batches(:batch_size=>10, :relation=>true).each do |batch_query|
# do any kind of further querying/filtering/mapping that you want
# show that this is actually an activerecord relation, not an array of AR objects
puts batch_query.to_sql
# add more conditions to this query, this is just an example
batch_query = batch_query.where(:color=>"blue")
# pluck just the ids
puts batch_query.pluck(:id)
end
Ultimately, if you don't like any of the answers given on an SO post, you can roll-your-own solution. Consider only downvoting when an answer is either way off topic or not helpful in any way. We are all just trying to help. Downvoting an answer that has source code to prove it will only deter others from trying to help you.
Previous EDIT
In response to your comment (because my comment would not fit):
calling
thing_ids
internally uses
pluck
pluck internally uses
select_all
...which instantiates an activerecord Result
Previous 2nd EDIT:
This line of code within pluck returns an activerecord Result:
....
result = klass.connection.select_all(relation.arel, nil, bound_attributes)
...
I just stepped through the source code for you. Using select_all will save you some memory, but in the end, an activerecord Result was still created and mapped over even when you are using the pluck method.
I would use something like this:
User.things.find_each(batch_size: 1000).map(&:id)
This will give you an array of the ids.
Objective: Parse data to display all the id's in the erb file
Problem: NoMethodError in DemoController#index due to this piece of code
#x = obj[i]["id"]
When I replace the "i" in the above piece of code with a number, one id number displays which leads me to believe that the while loop is correct. It just doesn't understand what "i" is.
What am I doing wrong?
Here is my code for my Controller and View
demo_controller.rb
require 'rubygems'
require 'json'
require 'net/http'
require 'httparty'
class DemoController < ApplicationController
respond_to :json
$angelURI = "https://api.angel.co/1/jobs"
def index
response = HTTParty.get('https://api.angel.co/1/jobs/')
obj = JSON.parse(response.body)["jobs"]
arraylength = obj.length
i = 0
while i <= arraylength do
#x = obj[i]["id"]
i += 1
end
end
end
index.html.erb
<%=#x%>
You are assigning a value to the same #x variable at each level of your loop - this will end with #x having the value of the last id - is that the intended behavior ?
I don't see something weird with your array right now, but Ruby tend to favor using each over for:
obj.each do |elem|
#x = elem["id"]
end
Upate: Following zishe good catch about the loop, using each also avoid that kind of question ("do I need to go to the ith element or stop at the ith-1").
By combining best of answers we get:
#x = []
obj.each do |job|
#x << job["id"]
end
i is a counter in while loop, it's basics. I think you looping to more, change <= on < in this:
i = 0
while i < arraylength do
#x = obj[i]["id"]
i += 1
end
Or better do like Martin suggests.
So, you have a off-by-one error: your while loop runs too far (because of the <=). Simple solution: use each (so you do not have to maintain a counter yourself --why make it hard). But on top, I would propose to add a file in lib that will do the parsing of the page.
So, e.g. add a file called lib/jobs_parser.rb that contains something like
require 'httparty'
module JobsParser
ANGEL_JOBS_URI = "https://api.angel.co/1/jobs"
def all_job_ids
all_jobs.map{|j| j["id"]}
end
def all_jobs
response = HTTParty.get(ANGEL_JOBS_URI)
jobs = JSON.parse(response.body)["jobs"]
end
end
What do I do here: the map generates an array containing just the "id" field.
I think it makes more sense, on this level to keep the complete array of jobs or ids.
Note: I drastically shortened the list require statements, most should be auto-required via your Gemfile.
And then in your controller you can write:
class DemoController < ApplicationController
def index
all_job_ids = JobsParser.all_job_ids
#x = all_job_ids.last
end
end
and your view remains the same :)
This has the advantage that you can simply test the JobsParser, through tests, or manually in the rails console, and that your code is a bit more readable.
You have a off-by-one error in your code. Basically, you are looping over the array and are then trying to access one more element than is in the array, which is then returned as nil and naturally doesn't act as a Hash.
Say your obj is an array with 3 elements, thus arraylength is three. You are now fetching 4 elements from the array, the elements with the indexes of 0, 1, 2, and 3. As you only have the 3 elements 0..2, the last one obj[3] doesn't exist.
To keep your existing code, you could change your loop to read as follows:
while i < arraylength do
#...
end
However, to just get the id of the last element in your array, it is much clearer (and much faster) to just use idiomatic ruby and write your whole algorithm as
def index
response = HTTParty.get('https://api.angel.co/1/jobs/')
jobs = JSON.parse(response.body)["jobs"]
#x = jobs.last["id"]
end
I have created a small Ruby class here:
class Star
#Star initialization
def initialize(star, number)
#star = star
#number = number
end
end
and I am looking to initialize a class called Solar System with 100 stars. This is what I have done and it doesn't seem to be working. Any help would be greatly appreciated.
require_relative 'star.rb'
class SolarSystem
#Initialize Game
def initialize(partOfSolarSystem)
#partOfSolarSystem = partOfSolarSystem
#stars_array = []
for i in 0..99
stars_array = Star.new('unknown_star',i)
end
end
def show_solar_system
#code here to show all the initialized stars in solar system
end
end
I can't seem to get it to initialize the array in the constructor. I would then like to be able to print out all of the elements in the stars array. Any help with this would be greatly appreciated.
Also in an effort to eventually move this to a database with rails or something of that nature, should I be looking to hash this or will this be easily converted to mySQL or another DB with some helper functions? I would eventually like to write this into rails and have a dynamic website for it.
Once again, thanks very much.
Your problem is assigning a new value to #stars_array variable on each iteration. There are multiple ways to deal with it:
#stars_array = (0..99).map { |i| Star.new('unknown_star',i) }
By the way, there is a couple of design issues (just for your attention):
Why variable is called stars_array, not just stars?
Why would ever instance of Star class have some object named #star inside? Recursion? :) Seems like #name would be proper and more clear attribute's name.
Don't miss indentation.
EDIT: About DB-mapping. Most common way - inherit both classes from ActiveRecord::Base, and create one-to-many relation from solar system to stars. Each class will have it's own table. Takes absolutely no efforts.
You are assigning the new object every time round the loop. The fix is to append the new object:
#stars_array << Star.new('unknown_star',i)
Or, if you prefer words rather than symbols:
#stars_array.push(Star.new('unknown_star',i))
Or, to be more terse:
100.times {|i| #stars_array << Star.new('unknown_star',i) }
A few things to fix to make it work. In your loop you're assigning a new value to the array rather than appending to it. Secondly, in your loop you're using a local variable stars_array instead of the instance variable #stars_array.
Your initialize method should look like this:
def initialize(part_of_solar_system)
#part_of_solar_system = part_of_solar_system
#stars_array = []
for i in 0..99
#stars_array << Star.new('unknown_star', i)
end
end
Also, you might want to revisit your Ruby idioms, like preferring snake_case to camelCase for variable names and avoiding for loops in favor of each, e.g.
def initialize(part_of_solar_system)
#part_of_solar_system = part_of_solar_system
#stars_array = []
(0..99).each { |i| #stars_array << Star.new('unknown_star', i) }
end
It's a vague question I know....but the performance on this block of code is horrible. It takes about 15secs from the original post to the action to rendering the page...
The purpose of this action is to retrieve all Occupations from a CV, all the skills from that CV and the occupations. They need to be organized in 2 arrays:
the first array contains all the Occupations (no duplicates) and has them ordered according their score. Fo each double entry found the score is increased by 1
the second array contains ALL the skills from both the occupation array and the cv. Again no doubles are allowed, but for every double encountered the score of the existing is increased by one.
Below is the code block that performs this operation. It's relatively big compared to my other code snippets, but i hope it's understandable. I know working with the arrays like i do is confusing, but here is what each array location means:
position 0 : the actuall skill/occupation object
position 1 : the score of the entry
position 2 : the location found in the db
position 3 : the location found in the cv
def categorize
#cv = Cv.find(params[:cv_id], :include => [:desired_occupations, :past_occupations, :educational_skills])
#menu = :second
#language = Language.resolve(:code => :en, :name => :en)
#occupation_hashes = []
#skill_hashes = []
(#cv.desired_occupations + #cv.past_occupations).each do |occupation|
section = []
section << 'Desired occupation' if #cv.desired_occupations.include? occupation
section << 'Work experience' if #cv.past_occupations.include? occupation
unless (array = #occupation_hashes.assoc(occupation)).blank?
array[1] += 1
array[2] = (array[2] & section).uniq
else
#occupation_hashes << [occupation, 1, section]
end
occupation.skills.each do |skill|
unless (array = #skill_hashes.assoc skill).blank?
label = occupation.concept.label(#language).value
array[1]+= 1
array[3] << label unless array[3].include? label
else
#skill_hashes << [skill, 1, [], [occupation.concept.label(#language).value]]
end
end
end
#cv.educational_skills.each do |skill|
unless (array = #skill_hashes.assoc skill).blank?
array[1]+= 1
array[3] << 'Education skills' unless array[3].include? 'Education skills'
else
#skill_hashes << [skill, 1, ['Education skills'], []]
end
end
# Sort the hashes
#occupation_hashes.sort! { |x,y| y[1] <=> x[1]}
#skill_hashes.sort! { |x,y| y[1] <=> x[1]}
#max = #skill_hashes.first[1]
#min = #skill_hashes.last[1] end
I can post the additional models and migrations to make it clear what each class does, but I think the first few lines of the above script should be clear on the associations. I'm looking for a way to optimize the each-loops...
That's quite the block of code there. Generally if you're writing methods that serious you're going to have trouble maintaining it in the future. A technique that would help is breaking up that monolithic chunk of code and turning it into a helper class that does the processing in more logical stages, making it easier to fine-tune aspects of it.
For instance, an interface might be:
#categorizer = CvCategorizer.new(params[:cv_id])
This would encapsulate all of the above and save it into instance variables made accessible by being declared with attr_reader.
Using a utility class means you can break up the initialization into steps that are made more clear:
def initialize(cv_id)
# Call a wrapper method that loads the CV
#cv = self.load_cv(cv_id)
# Perform discrete steps to re-order the imported data
self.organize_occupations
self.organize_skills
end
It's really hard to say why this is slow by just looking at it, though I would pay very close attention to log/development.log to see what's going on in there. It could be the initial load is painfully slow but the rest of the method is fine.
You should do a but of profiling in your code to see what is taking a large chunk of time. You can figure out how to work on of the profilers, or just sprinkle some simple puts or logger.info statements throughout your code with a timestamp. Probably easiest to do this by using Benchmark. Note: you may need to require 'benchmark'... not sure if it is auto required in Rails or not.
For a single line, you can do something like this:
logger.info Benchmark.measure { #cv = Cv.find(params[:cv_id], :include => [:desired_occupations, :past_occupations, :educational_skills]) }
And for timing larger blocks of code:
logger.info Benchmark.measure do
(#cv.desired_occupations + #cv.past_occupations).each do |occupation|
section = []
section << 'Desired occupation' if #cv.desired_occupations.include? occupation
section << 'Work experience' if #cv.past_occupations.include? occupation
unless (array = #occupation_hashes.assoc(occupation)).blank?
array[1] += 1
array[2] = (array[2] & section).uniq
else
#occupation_hashes << [occupation, 1, section]
end
end
end
I'd just start with large blocks and then narrow it down. Not knowing how large of a dataset you are dealing with, it is hard to say what the problem zone is.
I'll also concur with others that you will be way better off to break this thing into smaller methods. This will also make it easier to test for performance, as you can do things like:
Benchmark.measure { 10000.times { foo.do_that_thing_that_might_be_slow }}
I am trying to use a time_select to input a time into a model that will then perform some calculations.
the time_select helper prepares the params that is return so that it can be used in a multi-parameter assignment to an Active Record object.
Something like the following
Parameters: {"commit"=>"Calculate", "authenticity_token"=>"eQ/wixLHfrboPd/Ol5IkhQ4lENpt9vc4j0PcIw0Iy/M=", "calculator"=>{"time(2i)"=>"6", "time(3i)"=>"10", "time(4i)"=>"17", "time(5i)"=>"15", "time(1i)"=>"2009"}}
My question is, what is the best way to use this format in a non-active record model. Also on a side note. What is the meaning of the (5i), (4i) etc.? (Other than the obvious reason to distinguish the different time values, basically why it was named this way)
Thank you
You can create a method in the non active record model as follows
# This will return a Time object from provided hash
def parse_calculator_time(hash)
Time.parse("#{hash['time1i']}-#{hash['time2i']}-#{hash['time3i']} #{hash['time4i']}:#{hash['time5i']}")
end
You can then call the method from the controller action as follows
time_object = YourModel.parse_calculator_time(params[:calculator])
It may not be the best solution, but it is simple to use.
Cheers :)
The letter after the number stands for the type to which you wish it to be cast. In this case, integer. It could also be f for float or s for string.
I just did this myself and the easiest way that I could find was to basically copy/paste the Rails code into my base module (or abstract object).
I copied the following functions verbatim from ActiveRecord::Base
assign_multiparameter_attributes(pairs)
extract_callstack_for_multiparameter_attributes(pairs)
type_cast_attribute_value(multiparameter_name, value)
find_parameter_position(multiparameter_name)
I also have the following methods which call/use them:
def setup_parameters(params = {})
new_params = {}
multi_parameter_attributes = []
params.each do |k,v|
if k.to_s.include?("(")
multi_parameter_attributes << [ k.to_s, v ]
else
new_params[k.to_s] = v
end
end
new_params.merge(assign_multiparameter_attributes(multi_parameter_attributes))
end
# Very simplified version of the ActiveRecord::Base method that handles only dates/times
def execute_callstack_for_multiparameter_attributes(callstack)
attributes = {}
callstack.each do |name, values|
if values.empty?
send(name + '=', nil)
else
value = case values.size
when 2 then t = Time.new; Time.local(t.year, t.month, t.day, values[0], values[min], 0, 0)
when 5 then t = Time.time_with_datetime_fallback(:local, *values)
when 3 then Date.new(*values)
else nil
end
attributes[name.to_s] = value
end
end
attributes
end
If you find a better solution, please let me know :-)