I'm just wondering is there any chance to get fresh eye on code below and make some code refactor?
def call
inq_proc_ids = InquiryProcess.all.includes(inquiry_field_responses: :inquiry_field).select do |process|
process.inquiry_field_responses.select do |inquiry_field_responses|
inquiry_field_responses.inquiry_field.name == 'company_name'
end.last&.value&.start_with?(company_filter)
end.map(&:id)
InquiryProcess.where(id: inq_proc_ids)
end
I think I should leave only InquiryProcess.where(id: inq_proc_ids) in my call method but I don't know how to handle with all these .last&.value&.start_with?(company_filter) and .map(&:id) stuff.
EDIT:
I was trying to split it to the new methods
def call
InquiryProcess.where(id: inquiry_process_id)
end
private
attr_reader :company_filter, :inquiry_field_response
def inquiry_process_id
InquiryProcess.all.includes(inquiry_field_responses: :inquiry_field).select do |process|
process.inquiry_field_responses.select_company_name
end.map(&:id)
end
def select_company_name
select do |inquiry_field_responses|
inquiry_field_responses.inquiry_field.name == 'company_name'
end.last&.value&.start_with?(company_filter)
end
but I got an error:
NoMethodError (undefined method `select_company_name' for ActiveRecord::Associations::CollectionProxy []>):
The code you posted is not only hard to follow, but I remember we had a massive memory leak connected to ActiveReocrd caching when using precalculated ids in a query.
That said, I'd try to utilise the above within a single sql query:
def call
id_select = InquiryProcess
.joins(inquiry_field_responses: :inquiry_field)
.where(inquire_fields: { name: 'company_name' })
.where(InquiryField.arel_table[:value].matches("#{company_filter}%"))
.select(:id)
InquiryProcess.where(id: id_select)
end
Note that id_select is not an array of ids but ActiveRecord scope, the above will translate to following SQL:
SELECT "inquiry_processes".*
FROM "inquiry_processes"
WHERE "inquiry_processes"."id" IN (
SELECT "inquiry_processes"."id"
FROM "inquiry_processes"
INNER JOIN ...
WHERE ...
)
And to answer another question - why do we query table by matching id to a result of another subquery on the same table? This is to avoid all sort of painful issues when you deal with an active record relation that has a join in it - e.g. it would affect all further includes statements, as the preloaded association would only include records matching the relation join conditions.
I really hope for you that this bit is quite well tested or you have someone who can verify validity of the behaviour.
Related
I have a database query where I want to get an array of Users that are distinct for the set:
#range is a predefinded date range
#shift_list is a list of filtered shifts
def listing
Shift
.where(date: #range, shiftname: #shift_list)
.select(:user_id)
.distinct
.map { |id| User.find( id.user_id ) }
.sort
end
and I read somewhere that for readability, or isolating for testing, or code reuse, you could split this into seperate methods:
def listing
shiftlist
.select(:user_id)
.distinct
.map { |id| User.find( id.user_id ) }
.sort
end
def shift_list
Shift
.where(date: #range, shiftname: #shift_list)
end
So I rewrote this and some other code, and now the page takes 4 times as long to load.
My question is, does this type of method splitting cause the database to be hit twice? Or is it something that I did elsewhere?
And I'd love a suggestion to improve the efficiency of this code.
Further to the need to remove mapping from the code, this shift list is being created with the following code:
def _month_shift_list
Shift
.select(:shiftname)
.distinct
.where(date: #range)
.map {|x| x.shiftname }
end
My intention is to create an array of shiftnames as strings.
I am obviously missing some key understanding in database access, as this method is clearly creating part of the problem.
And I think I have found the solution to this with the following:
def month_shift_list
Shift.
.where(date: #range)
.pluck(:shiftname)
.uniq
end
Nope, the database will not be hit twice. The queries in both methods are lazy loaded. The issue you have with the slow page load times is because the map function now has to do multiple finds which translates to multiple SELECT from the DB. You can re-write your query to this:
def listing
User.
joins(:shift).
merge(Shift.where(date: #range, shiftname: #shift_list).
uniq.
sort
end
This has just one hit to the DB and will be much faster and should produce the same result as above.
The assumption here is that there is a has_one/has_many relationship on the User model for Shifts
class User < ActiveRecord::Base
has_one :shift
end
If you don't want to establish the has_one/has_many relationship on User, you can re-write it to:
def listing
User.
joins("INNER JOIN shifts on shifts.user_id = users.id").
merge(Shift.where(date: #range, shiftname: #shift_list).
uniq.
sort
end
ALTERNATIVE:
You can use 2 queries if you experience issues with using ActiveRecord#merge.
def listing
user_ids = Shift.where(date: #range, shiftname: #shift_list).uniq.pluck(:user_id).sort
User.find(user_ids)
end
Using model concerns which include scopes, what is the best way to write these knowing that nested and/or self-referencing queries are likely?
In one of my concerns, I have scopes similar to these:
scope :current, ->(as_at = Time.now) { current_and_expired(as_at).current_and_future(as_at) }
scope :current_and_future, ->(as_at = Time.now) { where("#{upper_bound_column} IS NULL OR #{upper_bound_column} >= ?", as_at) }
scope :current_and_expired, ->(as_at = Time.now) { where("#{lower_bound_column} IS NULL OR #{lower_bound_column} <= ?", as_at) }
def self.lower_bound_column
lower_bound_field
end
def self.upper_bound_column
upper_bound_field
end
And is referred to via has_many's, example: has_many :company_users, -> { current }
If an ActiveRecord query is made which refers to a few models that include the concern, this results in an 'ambiguous column name' exception which makes sense.
To help overcome this, I change the column name helper methods to now be
def self.lower_bound_column
"#{self.table_name}.#{lower_bound_field}"
end
def self.upper_bound_column
"#{self.table_name}.#{upper_bound_field}"
end
Which works great, until you require self-referencing queries. Arel helps mitigate these issues by aliasing the table name in the resulting SQL, for example:
LEFT OUTER JOIN "company_users" "company_users_companies" ON "company_users_companies"."company_id" = "companies"."id"
and
INNER JOIN "company_users" ON "users"."id" = "company_users"."user_id" WHERE "company_users"."company_id" = $2
The issue here is that self.table_name no longer refers to the table name in the query. And this results in the tongue in cheek hint: HINT: Perhaps you meant to reference the table alias "company_users_companies"
In an attempt to migrate these queries over to Arel, I changed the column name helper methods to:
def self.lower_bound_column
self.class.arel_table[lower_bound_field.to_sym]
end
def self.upper_bound_column
self.class.arel_table[upper_bound_field.to_sym]
end
and updated the scopes to reflect:
lower_bound_column.eq(nil).or(lower_bound_column.lteq(as_at))
but this just ported the issue across since self.class.arel_table will always be the same regardless of the query.
I guess my question is, is how do I create scopes that can be used in self-referencing queries, which require operators such as <= and >=?
Edits
I have created a basic application to help showcase this issue.
git clone git#github.com:fattymiller/expirable_test.git
cd expirable_test
createdb expirable_test-development
bundle install
rake db:migrate
rake db:seed
rails s
Findings and assumptions
Works in sqlite3, not Postgres. Most likely because Postgres enforces the order of queries in the SQL?
Well, well, well. After quite a big time looking through the sources of Arel, ActiveRecord and Rails issues (it seems this is not new), I was able to find the way to access the current arel_table object, with its table_aliases if they are being used, inside the current scope at the moment of its execution.
That made possible to know if the scope is going to be used within a JOIN that has the table name aliased, or if on the other hand the scope can be used on the real table name.
I just added this method to your Expirable concern:
def self.current_table_name
current_table = current_scope.arel.source.left
case current_table
when Arel::Table
current_table.name
when Arel::Nodes::TableAlias
current_table.right
else
fail
end
end
As you can see, I'm using current_scope as the base object to look for the arel table, instead of the prior attempts of using self.class.arel_table or even relation.arel_table, which as you said remained the same regardless of where the scope was used. I'm just calling source on that object to obtain an Arel::SelectManager that in turn will give you the current table on the #left. At this moment there are two options: that you have there an Arel::Table (no alias, table name is on #name) or that you have an Arel::Nodes::TableAlias with the alias on its #right.
With that table_name you can revert to your first attempt of #{current_table_name}.#{lower_bound_field} and #{current_table_name}.#{upper_bound_field} in your scopes:
def self.lower_bound_column
"#{current_table_name}.#{lower_bound_field}"
end
def self.upper_bound_column
"#{current_table_name}.#{upper_bound_field}"
end
scope :current_and_future, ->(as_at = Time.now) { where("#{upper_bound_column} IS NULL OR #{upper_bound_column} >= ?", as_at) }
scope :current_and_expired, ->(as_at = Time.now) { where("#{lower_bound_column} IS NULL OR #{lower_bound_column} <= ?", as_at) }
This current_table_name method seems to me to be something that would be useful to have on the AR / Arel public API, so it can be maintained across version upgrades. What do you think?
If you are interested, here are some references I used down the road:
A similar question on SO, answered with a ton of code, that you could use instead of your beautiful and concise Ability.
This Rails issue and this other one.
And the commit on your test app on github that made tests green!
I have a slightly modified approach from #dgilperez, which uses the full power of Arel
def self.current_table_name
current_table = current_scope.arel.source.left
end
now you could modify your methods with arel_table syntax
def self.lower_bound_column
current_table[:lower_bound_field]
end
def self.upper_bound_column
current_table[:upper_bound_field]
end
and use it query like this
lower_bound_column.eq(nil).or(lower_bound_column.lteq(as_at))
I have a RoR app (in development). I have models based on sql views.
However, I would like to create & update thoses models. They are stored in database through 2 tables (generic design pattern, that's why I use sql views).
I've heard about hooks such as before_save, but as mentioned here there is still issues saving or updating other objects in before_save callback.
So, I am wonderring how to safely override save method in rails ?
Any suggestions are welcomed.
Thanks
EDIT :
sql of my view
CREATE VIEW my_objects AS
SELECT o.* ,
at.value as "column1",
FROM
Generic_object o
LEFT JOIN (SELECT at.* FROM Another_table at ON at.genreic_object_id = o.id AND at.name = "param1" )
This lead to a "reconstructed" object from my generic table and some column from another table :
o.col1, o.col2, ..., at.param1
If my object have several parameters I have to JOIN the another table again, like this
LEFT JOIN ( SELECT at2.* FROM Another_table at2 ON at2.generic_object_id = o.id AND at2.name = "param2")
And then, i get this object :
o.col1, o.col2, ..., at.param1, at2.param2
It seems rare (weird? :p), but this pattern is my constraint.... :(
From my experience I rarely find such need to I override common methods. And I don't recommend so because there will be other code depends on the original method. Your problem must have another better solution.
However, if you really have to, you can safely override it by Ruby's alias_method, or alias_method_chain provided by ActivitySupport.
class Foo < ActiveRecord::Base
alias_method_chain :save, :my_feature
def save
blah_blah
save_without_my_feature # The original save method
end
end
def save
# new object ?
# yes
q = Array.new
qualifiers.each do |qk|
q.push(Another_table.create(value: self.read_attribute(qk),name: qk))
end
e = Generic_object.create(name: name,type: self.class,qualifiers: q)
# existing object
#bla bla
end
I am using Ruby on Rails 3.2.2 and I would like to know if in scope methods it is possible to "dynamically" join a table only if that table is not joined yet. That it, I have:
def self.scope_method_name(user)
joins(:joining_association_name).where("joining_table_name.user_id = ?", user.id)
end
I would like to make something like the following:
# Note: the following code is just a sample in order to understand what I mean.
def self.scope_method_name(user)
if table_is_joined?(joining_table_name)
where("joining_table_name.user_id = ?", user.id)
else
joins(:joining_association_name).where("joining_table_name.user_id = ?", user.id)
end
end
Is it possible / advised to make that? If so, how could / should I proceed?
I would like to use this approach in order to avoid multiple database table statements in INNER JOIN of SQL queries (in some cases it seems to make my SQL querying not working as expected since multiple table statements) and so to use the scope_method_name without caring related SQL query concerns (in my case, without caring to join database tables).
Note: It could raise SQL errors (for example, errors as-like "ActiveRecord::StatementInvalid: Mysql2::Error: Unknown column 'joining_table_name.user_id' in 'where clause'") when you have not joined yet the database table (for example, this could happen when you run code like ClassName.scope_method_name(#user) without to previously join the joining_association_name and so without to join the related joining_table_name table).
Where is the method loaded? to check if an association has been loaded. You could try to use that.
if association_name.loaded?
where("joining_table_name.user_id = ?", user.id)
else
joins(:joining_association_name).where("joining_table_name.user_id = ?", user.id)
end
So I have an ActiveRecord class with a couple different named scopes that include join parameters. While running a report, I happen to have a situation where one gets called inside of the other:
1 Model.scope_with_some_joins.find_in_batches do |models|
2 models.each do |mdl|
3 other_comparisons = Model.scope_with_other_joins
4 end
5 end
My problem is on line 3 -- I get a runtime error showing me that for some reason when running the second query it's maintaining the join scope from the outer query. Really I need it to be run separately on it's own, without sharing context with the outer query. Any thoughts or ideas?
(I should mention that the problem is an "ambigious column" error because there is one table that is joined in from both queries)
You're looking for
Model.with_exclusive_scope { ...do your find in here... }
This will remove any scopes that are currently in use for the block.
An example usage:
# in model.rb
def self.find_stuff
self.scope_with_some_joins.find_in_batches do |models|
models.each do |mdl|
self.with_exclusive_scope do
other_comparisons = self.scope_with_other_joins
end
end
end
end
Then you query with Model.find_stuff. This way the logic is wrapped up in the model, not in the controller.