I am working on a plugin for Discourse, which means that I can modify classes with class_eval, but I cannot change the DB schema. To store extra data about the Topic model, I can perform joins with TopicCustomField, which is provided for this purpose.
I am able to store and retrieve all the data I need, but when many Topics are loaded at once, the DB performance is inefficient because my indirect data is loaded once for each Topic by itself. It would be much better if this data were loaded all at once for each Topic, like can happen when using preload or includes.
For example, each Topic has a topic_guid, and a set of parent_guids (stored in a single string with dashes because order is important). These parent_guids point to both other Topic's topic_guids as well as the name of other Groups.
I would love to be able write something like:
has_many :topic_custom_fields
has_many :parent_guids, -> { where(name: 'parent_guids').pluck(:value).first }, :through => :topic_custom_fields
has_many :parent_groups, class_name: 'Group', primary_key: :parent_guids, foreign_key: :name
But this :through complains about not being able to find an association ":parent_guids" in TopicCustomField, and primary_key won't actually take an association instead of a DB column.
I've also tried the following, but the :through clauses are not able to use the functions as associations.
has_many :topic_custom_fields do
def parent_guids
parent_guids_str = where(name: PARENT_GUIDS_FIELD_NAME).pluck(:value).first
return [] unless parent_guids_str
parent_guids_str.split('-').delete_if { |s| s.length == 0 }
end
def parent_groups
Group.where(name: parent_guids)
end
end
has_many :parent_guids, :through => :topic_custom_fields
has_many :parent_groups, :through => :topic_custom_fields
Using Rails 4.2.7.1
Actually, through parameter of rails associations is to set a many-to-many association with a model, passing "through" other model:
http://guides.rubyonrails.org/association_basics.html#the-has-many-through-association
So you can't do
has_many :parent_guids, :through => :topic_custom_fields
since ParentGuid is not a model related to TopicCustomFields. Also, pass a block to has_many is only to extend the association with new methods as the ones rails already provide you, like topic_custom_fields.create, topic_custom_fields.build, etc.
Why don't you define the methods inside the block at your second example in the Topic class to retrieve the groups? Is there something you want that would not be possible only with the methods?
Update
Well, I don't think it's possible achieve the same improved performance in this case, since the group ids still have to be handled from topic_custom_fields, and the improved performance is reached through JOINs. Maybe a complex combination of preload, where and references could do the trick, but I don't know if it's possible.
You could try to minimize the db calls instead, maybe gathering all the parent_guids before querying the groups.
I hope there is a more elegant solution, but this is what I have done in order to preload my data efficiently. This should be fairly easy to extend to other applications.
I modify Relation's exec_queries, which calls other preloading functions.
ActiveRecord::Relation.class_eval do
attr_accessor :preload_funcs
old_exec_queries = self.instance_method(:exec_queries)
define_method(:exec_queries) do |&block|
records = old_exec_queries.bind(self).call(&block)
if preload_funcs
preload_funcs.each do |func|
func.call(self, records)
end
end
records
end
end
To Topic, I added:
has_many :topic_custom_fields
attr_accessor :parent_groups
def parent_guids
parent_guids_str = topic_custom_fields.select { |a| a.name == PARENT_GUIDS_FIELD_NAME }.first
return [] unless parent_guids_str
parent_guids_str.value.split('-').delete_if { |s| s.length == 0 }
end
And then in order to preload the parent_groups, I do:
def preload_parent_groups(topics)
topics.preload_funcs ||= []
topics.preload_funcs <<= Proc.new do |association, records|
parent_guidss = association.map {|t| t.parent_guids}.flatten
parent_groupss = Group.where(name: parent_guidss).to_a
records.each do |t|
t.parent_groups = t.parent_guids.map {|guid| parent_groupss.select {|group| group.name == guid }.first}
end
end
topics
end
And finally, I add the preloaders to my Relation query:
result = result.preload(:topic_custom_fields)
result = preload_parent_groups(result)
Related
I have a product model setup like the following:
class Product < ActiveRecord::Base
has_many :product_atts, :dependent => :destroy
has_many :atts, :through => :product_atts
has_many :variants, :class_name => "Product", :foreign_key => "parent_id", :dependent => :destroy
end
And I want to search for products that have associations with multiple attributes.
I thought maybe this would work:
Product.joins(:product_atts).where(parent_id: params[:product_id]).where(product_atts: {att_id: [5,7]})
But this does not seem to do what I am looking for. This does where ID or ID.
So I tried the following:
Product.joins(:product_atts).where(parent_id: 3).where(product_atts: {att_id: 5}).where(product_atts: {att_id: 7})
But this doesn't work either, it returns 0 results.
So my question is how do I look for a model by passing in attributes of multiple join models of the same model type?
SOLUTION:
att_ids = params[:att_ids] #This is an array of attribute ids
product = Product.find(params[:product_id]) #This is the parent product
scope = att_ids.reduce(product.variants) do |relation, att_id|
relation.where('EXISTS (SELECT 1 FROM product_atts WHERE product_id=products.id AND att_id=?)', att_id)
end
product_variant = scope.first
This is a seemingly-simple request made actually pretty tricky by how SQL works. Joins are always just joining rows together, and your WHERE clauses are only going to be looking at one row at a time (hence why your expectations are not working like you expect -- it's not possible for one row to have two values for the same column.
There are a bunch of ways to solve this when dealing with raw SQL, but in Rails, I've found the simplest (not most efficient) way is to embed subqueries using the EXISTS keyword. Wrapping that up in a solution which handles arbitrary number of desired att_ids, you get:
scope = att_ids_to_find.reduce(Product) do |relation, att_id|
relation.where('EXISTS (SELECT 1 FROM product_atts WHERE parent_id=products.id AND att_id=?)', att_id)
end
products = scope.all
If you're not familiar with reduce, what's going on is it's taking Product, then adding one additional where clause for each att_id. The end result is something like Product.where(...).where(...).where(...), but you don't need to worry about that too much. This solution also works well when mixed with scopes and other joins.
There's a way to condition something to an associative table of ActiveRecord?
I retrieve segments this way:
#segments = Segment.all
But, a Segment has_many products. See:
models/product.rb:
class Product < ActiveRecord::Base
belongs_to :segment, dependent: :destroy
end
models/segment.rb:
class Segment < ActiveRecord::Base
has_many :products
end
The problem is: I just want to retrieve products whose its status is equals to 1. I can condition something like this using where on Segment model, but how can I achieve this for products?
What I already tried
I found a solution. Take a look:
#segments = Segment.find(:all, include: :products, conditions: {products: {status: 1}})
It worked, but I think the code can be better.
Why I think the code can be better
Well, why should I use include: :products if the association is already live within the models? We're associating things through the model and I'm sure that is something near to enough.
Ideas?
Segment.joins(:products).where("products.status = 1")
You can also use includes instead of joins. But rails will convert it into a join internally since you are using the products table attribute in the query
A few tips, that might help you.
For easy naming purposes, I am considering the status==1 as being active. Of course I have no idea what it means in your specific case.
class Product
ACTIVE=1
def self.active
where(status: ACTIVE)
end
end
Now you write something like:
segment.products.active
and this will return only the active products for the given segment.
The solution you found, which will retrieve all segments with (active) products, could be written differently as follows:
Segment.includes(:products).where(products: {status: 1})
Now, why so elaborate: this actually translates to a sql query, so you have to be a little more explicit about it.
If you only ever want those with a status of 1
class Segment < ActiveRecord::Base
has_many :products, :conditions => { :status => 1 }
end
In rails 3 or
class Segment < ActiveRecord::Base
has_many :products, -> { where status: 1 }
end
In rails 4
Obviously can use status: true if it's a boolean
Then
#segments = Segment.includes(:products)
The association has_many :products makes it possible to use include: :products in your scope. Therefore you shouldn't doubt in your solution. It is right, and it is just the same as solutions presented in the other answers but by other syntacsis.
This should do the job - and it's compatibile with AREL syntax:
#segments = Segment.joins(:products).where(products: {status: 1})
It's quite different that solution with include (or includes, as it would be Rails 3/4), because it generates query with INNER JOIN, while includes generates LEFT OUTER JOIN. Also, includes is usually used for eager loading associated records, not for queries with JOIN.
I'm trying to list all the user's products with a probable association where a flag 'notification' is set to zero.
user.probable_associations.where(:notified => 0).collect{|a| Product.where(:id => a.product_id).collect{|p| p.name}}.to_sentence
It seems like using a where and collect method twice within the statement isn't very good. Is there a better way to go about this?
Also, the result is something like
"[\"Product A\"] and [\"Product B\"]"
which is pretty ugly...and I still need to remove the extra punctuation "[\" \"]
instead of something clean like
"Product A and Product B"
EDIT based on Rich's Answer, still have issues because notified is a field in associations NOT product:
has_many :probable_associations, -> { where "associations.category = 3"}, class_name: 'Association', before_add: :set_probable_category
has_many :probable_products, class_name: 'Product', through: :probable_associations, source: :product do
def not_notified
select(:name).where(notified: 0)
end
end
I'd use an ActiveRecord Association extension:
#app/models/user.rb
Class User < ActiveRecord::Base
has_many :products do
def not_notified
select(:name).where(notified: 0)
end
end
end
#-> #user.products.not_notified
That's my contribution, but you could then use #spickermann & #tompave's controbutions and use .flatten.to_sentence
Without knowing what probable_associations does would I rewrite the code to something like this:
product_ids = user.probable_associations.where(:notified => 0).map(&:product_id)
Product.where(:id => product_ids).map(&:name).to_sentence
Assuming that probable_associations is just an ActiveRecord has_many association, and that you want to end up with a list of titles for Product records, you can use this:
ids = user.probable_associations
.where(notified: 0)
.pluck(:product_id)
result = Product.where(id: ids).pluck(:name).to_sentence
It's similar to #spikermann's answer, but pluck(:column_name) is faster than using a block and only extracts the required column from the DB.
Also, the reason your code produces that string is that, by the time you call to_sentence, you have an Array of sub-arrays. Each sub-array contains a single element: a product name.
That's because the second collect is sent to an ActiveRecord::Relation containing just one record.
You could have solved that problem with flatten, but the whole operation could just be refactored.
Update: This may be something that just isn't doable. See this
TLDR: How do you conditionally load an association (say, only load the association for the current user) while also including records that don't have that association at all?
Rails 3.1, here's roughly the model I'm working with.
class User
has_many :subscriptions
has_many :collections, :through => :subscriptions
end
class Collection
has_many :things
end
class Thing
has_many :user_thing_states, :dependent => :destroy
belongs_to :collection
end
class Subscription
belongs_to :user
belongs_to :collection
end
class UserThingState
belongs_to :user
belongs_to :thing
end
There exist many collections which have many things. Users subscribe to many collections and thereby they subscribe to many things. Users have a state with respect to things, but not necessarily, and are still subscribed to things even if they don't happen to have a state for them. When a user subscribes to a collection and its associated things, a state is not generated for every single thing (which could be in the hundreds). Instead, states are generated when a user first interacts with a given thing. Now, the problem: I want to select all of the user's subscribed things while loading the user's state for each thing where the state exists.
Conceptually this isn't that hard. For reference, the SQL that would get me the data needed for this is:
SELECT things.*, user_thing_states.* FROM things
# Next line gets me all things subscribed to
INNER JOIN subscriptions as subs ON things.collection_id = subs.collection_id AND subs.user_id = :user_id
# Next line pulls in the state data for the user
LEFT JOIN user_thing_states as uts ON things.id = uts.thing_id AND uqs.user_id = :user_id
I just don't know how to piece it together in rails. What happens in the Thing class? Thing.includes(:user_thing_states) would load all states for all users and that looks like the only tool. I need something like this but am not sure how (or if it's possible):
class Thing
has_many :user_thing_states
delegates :some_state_property, :to => :state, :allow_nil => true
def state
# There should be only one user_thing_state if the include is correct, state method to access it.
self.user_thing_states.first
end
end
I need something like:
Thing.includes(:user_question_states, **where 'user_question_state.user_id => :user_id**).by_collections(user.collections)
Then I can do
things = User.things_subscribed_to
things.first.some_state_property # the property of the state loaded for the current user.
You don't need to do anything.
class User
has_many :user_thing_states
has_many :things, :through => :user_thing_states
end
# All Users w/ Things eager loaded through States association
User.all.includes(:things)
# Lookup specific user, Load all States w/ Things (if they exist for that user)
user = User.find_by_login 'bob'
user.user_thing_states.all(:include => :things)
Using includes() for this already loads up the associated object if they exist.
There's no need to do any filtering or add extra behavior for the Users who don't have an associated object.
Just ran into this issue ourselves, and my coworker pointed out that Rails 6 seems to include support for this now: https://github.com/rails/rails/pull/32655
*Nope, didn't solve it :( Here's a treatment of the specific issue I seem to have hit.
Think I've got it, easier than expected:
class Thing
has_many :user_thing_states
delegates :some_state_property, :to => :state, :allow_nil => true
scope :with_user_state, lambda { |user|
includes(:user_thing_states).where('user_thing_states.user_id = :user_id
OR user_thing_states.user_id IS NULL',
{:user_id => user.id}) }
def state
self.user_thing_states.first
end
end
So:
Thing.with_user_state(current_user).all
Will load all Things and each thing will have only one user_question_state accessible via state, and won't exclude Things with no state.
Answering my own question twice... bit awkward but anyway.
Rails doesn't seem to let you specify additional conditions for an includes() statement. If it did, my previous answer would work - you could put an additional condition on the includes() statement that would let the where conditions work correctly. To solve this we'd need to get includes() to use something like the following SQL (Getting the 'AND' condition is the problem):
LEFT JOIN user_thing_states as uts ON things.id = uts.thing_id AND uqs.user_id = :user_id
I'm resorting to this for now which is a bit awful.
class User
...
def subscribed_things
self.subscribed_things_with_state + self.subscribed_things_with_no_state
end
def subscribed_things_with_state
self.things.includes(:user_thing_states).by_subscribed_collections(self).all
end
def subscribed_things_with_no_state
Thing.with_no_state().by_subscribed_collections(self).all
end
end
I have a polymorphic association like this -
class Image < ActiveRecord::Base
has_one :approval, :as => :approvable
end
class Page < ActiveRecord::Base
has_one :approval, :as => :approvable
end
class Site < ActiveRecord::Base
has_one :approval, :as => :approvable
end
class Approval < ActiveRecord::Base
belongs_to :approvable, :polymorphic => true
end
I need to find approvals where approval.apporvable.deleted = false
I have tried something like this -
#approvals = Approval.find(:all,
:include => [:approvable],
:conditions => [":approvable.deleted = ?", false ])
This gives "Can not eagerly load the polymorphic association :approvable" error
How can the condition be given correctly so that I get a result set with approvals who's approvable item is not deleted ?
Thanks for any help in advance
This is not possible, since all "approvables" reside in different tables. Instead you will have to fetch all approvals, and then use the normal array methods.
#approvals = Approval.all.select { |approval| !approval.approvable.deleted? }
What your asking, in terms of SQL, is projecting data from different tables for different rows in the resultset. It is not possible to my knowledge.
So you'll have to be content with:
#approvals = Approval.all.reject{|a| a.approvable.deleted? }
# I assume you have a deleted? method in all the approvables
I would recommend either of the answers already presented here (they are the same thing) but I would also recommend putting that deleted flag into the Approval model if you really care to do it all in a single query.
With a polymorphic relationship rails can use eager fetching on the polys, but you can't join to them because yet again, the relationships are not known so the query is actually multiple queried intersected.
So in the end if you REALLY need to, drop into sql and intersect all the possible joins you can do to all the types of approvables in a single query, but you will have to do lots of joining manually. (manually meaning not using rails' built-in mechanisms...)
thanks for your answers
I was pretty sure that this couldn't be done. I wanted some more confirmation
besides that I was hoping for some other soln than looping thru the result set
to avoid performance related issues later
Although for the time being both reject/select are fine but in the long run I
will have to do those sql joins manually.
Thanks again for your help!!
M