Update: This may be something that just isn't doable. See this
TLDR: How do you conditionally load an association (say, only load the association for the current user) while also including records that don't have that association at all?
Rails 3.1, here's roughly the model I'm working with.
class User
has_many :subscriptions
has_many :collections, :through => :subscriptions
end
class Collection
has_many :things
end
class Thing
has_many :user_thing_states, :dependent => :destroy
belongs_to :collection
end
class Subscription
belongs_to :user
belongs_to :collection
end
class UserThingState
belongs_to :user
belongs_to :thing
end
There exist many collections which have many things. Users subscribe to many collections and thereby they subscribe to many things. Users have a state with respect to things, but not necessarily, and are still subscribed to things even if they don't happen to have a state for them. When a user subscribes to a collection and its associated things, a state is not generated for every single thing (which could be in the hundreds). Instead, states are generated when a user first interacts with a given thing. Now, the problem: I want to select all of the user's subscribed things while loading the user's state for each thing where the state exists.
Conceptually this isn't that hard. For reference, the SQL that would get me the data needed for this is:
SELECT things.*, user_thing_states.* FROM things
# Next line gets me all things subscribed to
INNER JOIN subscriptions as subs ON things.collection_id = subs.collection_id AND subs.user_id = :user_id
# Next line pulls in the state data for the user
LEFT JOIN user_thing_states as uts ON things.id = uts.thing_id AND uqs.user_id = :user_id
I just don't know how to piece it together in rails. What happens in the Thing class? Thing.includes(:user_thing_states) would load all states for all users and that looks like the only tool. I need something like this but am not sure how (or if it's possible):
class Thing
has_many :user_thing_states
delegates :some_state_property, :to => :state, :allow_nil => true
def state
# There should be only one user_thing_state if the include is correct, state method to access it.
self.user_thing_states.first
end
end
I need something like:
Thing.includes(:user_question_states, **where 'user_question_state.user_id => :user_id**).by_collections(user.collections)
Then I can do
things = User.things_subscribed_to
things.first.some_state_property # the property of the state loaded for the current user.
You don't need to do anything.
class User
has_many :user_thing_states
has_many :things, :through => :user_thing_states
end
# All Users w/ Things eager loaded through States association
User.all.includes(:things)
# Lookup specific user, Load all States w/ Things (if they exist for that user)
user = User.find_by_login 'bob'
user.user_thing_states.all(:include => :things)
Using includes() for this already loads up the associated object if they exist.
There's no need to do any filtering or add extra behavior for the Users who don't have an associated object.
Just ran into this issue ourselves, and my coworker pointed out that Rails 6 seems to include support for this now: https://github.com/rails/rails/pull/32655
*Nope, didn't solve it :( Here's a treatment of the specific issue I seem to have hit.
Think I've got it, easier than expected:
class Thing
has_many :user_thing_states
delegates :some_state_property, :to => :state, :allow_nil => true
scope :with_user_state, lambda { |user|
includes(:user_thing_states).where('user_thing_states.user_id = :user_id
OR user_thing_states.user_id IS NULL',
{:user_id => user.id}) }
def state
self.user_thing_states.first
end
end
So:
Thing.with_user_state(current_user).all
Will load all Things and each thing will have only one user_question_state accessible via state, and won't exclude Things with no state.
Answering my own question twice... bit awkward but anyway.
Rails doesn't seem to let you specify additional conditions for an includes() statement. If it did, my previous answer would work - you could put an additional condition on the includes() statement that would let the where conditions work correctly. To solve this we'd need to get includes() to use something like the following SQL (Getting the 'AND' condition is the problem):
LEFT JOIN user_thing_states as uts ON things.id = uts.thing_id AND uqs.user_id = :user_id
I'm resorting to this for now which is a bit awful.
class User
...
def subscribed_things
self.subscribed_things_with_state + self.subscribed_things_with_no_state
end
def subscribed_things_with_state
self.things.includes(:user_thing_states).by_subscribed_collections(self).all
end
def subscribed_things_with_no_state
Thing.with_no_state().by_subscribed_collections(self).all
end
end
Related
So I've got a User model, a Building model, and a MaintenanceRequest model.
A user has_many :maintenance_requests, but belongs_to :building.
A maintenance requests belongs_to :building, and belongs_to: user
I'm trying to figure out how to send a new, then create a maintenance request.
What I'd like to do is:
#maintenance_request = current_user.building.maintenance_requests.build(permitted_mr_params)
=> #<MaintenanceRequest id: nil, user_id: 1, building_id: 1>
And have a new maintenance request with the user and building set to it's parent associations.
What I have to do:
#maintenance_request = current_user.maintenance_requests.build(permitted_mr_params)
#maintenance_request.building = current_user.building
It would be nice if I could get the maintenance request to set its building based of the user's building.
Obviously, I can work around this, but I'd really appreciate the syntactic sugar.
From the has_many doc
You can pass a second argument scope as a callable (i.e. proc or lambda) to retrieve a specific set of records or customize the generated query when you access the associated collection.
I.e
class User < ActiveRecord::Base
has_many :maintenance_requests, ->(user){building: user.building}, through: :users
end
Then your desired one line should "just work" current_user.building.maintenance_requests.build(permitted_mr_params)
Alternatively, if you are using cancancan you can add hash conditions in your ability file
can :create, MaintenanceRequest, user: #user.id, building: #user.building_id
In my opinion, I think the approach you propose is fine. It's one extra line of code, but doesn't really increase the complexity of your controller.
Another option is to merge the user_id and building_id, in your request params:
permitted_mr_params.merge(user_id: current_user.id, building_id: current_user.building_id)
#maintenance_request = MaintenanceRequest.create(permitted_mr_params)
Or, if you're not concerned about mass-assignment, set user_id and building_id as a hidden field in your form. I don't see a tremendous benefit, however, as you'll have to whitelist the params.
My approach would be to skip
maintenance_request belongs_to :building
since it already belongs to it through the user. Instead, you can define a method
class MaintenanceRequest
belongs_to :user
def building
user.building
end
#more class stuff
end
Also, in building class
class Building
has_many :users
has_many :maintenance_requests, through: :users
#more stuff
end
So you can completely omit explicit building association with maintenance_request
UPDATE
Since users can move across buildings, you can set automatic behavior with a callback. The job will be done like you do it, but in a more Railsey way
class MaintenanceRequest
#stuff
before_create {
building=user.building
}
end
So, when you create the maintenance_request for the user, the building will be set accordingly
I've just changed an action in competitors_controller.rb from...
def audit
#review = Review.find(params[:review_id])
#competitor = #review.competitors.find(params[:id])
respond_with(#review, #competitor)
end
...to...
def audit
#review = Review.find(params[:review_id])
#competitor = Competitor.find(params[:id], :include => {:surveys => {:question => [:condition, :precondition]}})
respond_with(#review, #competitor)
end
...as the page was timing out on loading.
The underlying associations are:
class Competitor < ActiveRecord::Base
has_many :surveys
end
class Survey < ActiveRecord::Base
belongs_to :competitor
belongs_to :question
delegate :dependencies, :precondition, :condition, :to => :question
end
class Question < ActiveRecord::Base
has_many :dependancies, :class_name => "Question", :foreign_key => "precondition_id"
belongs_to :precondition, :class_name => "Question"
has_many :surveys, :dependent => :delete_all
end
Basically, the audit.html.haml page loads:
#competitor.surveys.{sorting, etc}.each do |s|
s.foo, s.bar
s.{sorting, etc}.dependant_surveys.each do |s2|
s2.foo, s2.bar
s2.{sorting, etc}.dependant_surveys.each do |s3|
s3.foo, s3.bar, etc etc
If I nest it too far, the page doesn't load before it times out.
What I need to know is whether the eager loading I have inserted into competitor_controllers.rb theoretically speeds up each of the following two methods, which are called so often when building audit.html.haml?
class Survey < ActiveRecord::Base
def dependant_surveys
self.review.surveys.select{|survey| self.dependencies.include?(survey.question)}
end
end
class Question < ActiveRecord::Base
def dependencies
Question.all.select{|question| question.precondition == self}
end
end
(I say "theoretically" as I know the question could also be answered with benchmarking. But before I get that far I want to check I have the theory right.)
You do a lot of processing in ruby, and there's no need for that. You should move all operations like
Question.all.select{|question| question.precondition == self}
#competitor.surveys.{sorting, etc}
to database.
If I understand properly, first line is meant to select all records that have a precondition set to given question. Remember that Question.all returns an array so you perform select in array, and you could do it in db with simple scope scope :has_precondition, -> {|q| where(precondition_id: q.id} or so.
Given you always sort models in same way, you may consider creating a default scope with order clause. Doing it in ruby is very counter-efficient.
The eager loading looks like it should work, but much more importantly these are things that you could and should be doing using SQL. Loading and iterating through all the ActiveRecord models in your DB has the potential to be incredibly time consuming (as you've found), whereas working out how to do it in SQL will allow you to load all the models directly from a single query. It can take a while to get a handle on this, but it's well worth it!
As far as I know you can only include associations for eager loading. I don't think what you have will work at all. You haven't shown us where Question.condition is defined, or Survey.review. Also, the associations that you are eager loading won't be used by your methods dependent_surveys and dependencies as they're performing their own queries.
AFAICT your eager loading won't make a jot of difference, or it will probably slow it down. I think you'll have to refactor your dependent_surveys as an association and eager load that. I haven't got the slightest clue what that method is trying to do, on a high level, so I'm not even going to attempt to refactor it.
I believe this is a bug in Rails 3. I am hoping someone here can steer me in the correct direction. The code posted below, is purely for illustration of this problem. Hopefully this does not confuse the issue.
Given I have a Post model, and a Comment model. Post has_many Comments, and Comment belongs_to Post.
With a default_scope set on the Post model, defining joins() and where() relations. In this case where() is dependent on joins().
Normally Posts wouldn't be dependent on Comments. Again, I just want to give a simple example. This could be any case when where() is dependent on joins().
class Post < ActiveRecord::Base
has_many :comments, :dependent => :destroy
default_scope joins(:comments).where("comments.id < 999")
end
class Comment < ActiveRecord::Base
belongs_to :post, :counter_cache => true
end
Running the following command:
Post.update_all(:title => Time.now)
Produces the following query, and ultimately throws ActiveRecord::StatementInvalid:
UPDATE `posts` SET `title` = '2010-10-15 15:59:27' WHERE (comments.id < 999)
Again, update_all, delete_all, destroy_all behave the same way. I discovered this behaviour when my application complained when trying to update the counter_cache. Which eventually drills down into update_all.
I had this problem also, but we really needed to be able to use update_all with complex conditions in the default_scope (for example, without the default scope eager-loading is impossible, and pasting a named scope literally everywhere is no fun at all). I have opened a pull request here with my fix:
https://github.com/rails/rails/pull/8449
For delete_all I've raised an error if there's a join condition to make it more obvious what you have to do (instead of just tossing the join condition and running the delete_all on everything, you get an error).
Not sure what the rails guys are going to do with my pull request, but thought it was relevant to this discussion. (Also, if you need this bug fixed, you could try out my branch and post a comment on the pull request.)
I ran into this as well.
If you have
class Topic < ActiveRecord::Base
default_scope :conditions => "forums.preferences > 1", :include => [:forum]
end
and you do a
Topic.update_all(...)
it’ll fail with
Mysql::Error: Unknown column 'forums.preferences' in 'where clause'
The work around for this is:
Topic.send(:with_exclusive_scope) { Topic.update_all(...) }
You can monkey patch this using this code (and requiring it in environment.rb or else where)
module ActiveRecordMixins
class ActiveRecord::Base
def self.update_all!(*args)
self.send(:with_exclusive_scope) { self.update_all(*args) }
end
def self.delete_all!(*args)
self.send(:with_exclusive_scope) { self.delete_all(*args) }
end
end
end
end
Then just you update_all! or delete_all! when it has a default scope.
You can also do this on the class level, without creating new methods, like so:
def self.update_all(*args)
self.send(:with_exclusive_scope) { super(*args) }
end
def self.delete_all(*args)
self.send(:with_exclusive_scope) { super(*args) }
end
I don't think I'd call it a bug. The behavior seems logical enough to me, although not immediately obvious. But I worked out a SQL solution that seems to be working well. Using your example, it would be:
class Post < ActiveRecord::Base
has_many :comments, :dependent => :destroy
default_scope do
with_scope :find => {:readonly => false} do
joins("INNER JOIN comments ON comments.post_id = posts.id AND comments.id < 999")
end
end
end
In reality I'm using reflection to make it more robust, but the above gets the idea cross. Moving the WHERE logic into the JOIN ensures that it won't be applied in inappropriate places. The :readonly option is to counteract Rails's default behavior of making joins'd objects readonly.
Also, I know that some people deride the use of default_scope. But for multi-tenant apps, it's a perfect fit.
I have a polymorphic association like this -
class Image < ActiveRecord::Base
has_one :approval, :as => :approvable
end
class Page < ActiveRecord::Base
has_one :approval, :as => :approvable
end
class Site < ActiveRecord::Base
has_one :approval, :as => :approvable
end
class Approval < ActiveRecord::Base
belongs_to :approvable, :polymorphic => true
end
I need to find approvals where approval.apporvable.deleted = false
I have tried something like this -
#approvals = Approval.find(:all,
:include => [:approvable],
:conditions => [":approvable.deleted = ?", false ])
This gives "Can not eagerly load the polymorphic association :approvable" error
How can the condition be given correctly so that I get a result set with approvals who's approvable item is not deleted ?
Thanks for any help in advance
This is not possible, since all "approvables" reside in different tables. Instead you will have to fetch all approvals, and then use the normal array methods.
#approvals = Approval.all.select { |approval| !approval.approvable.deleted? }
What your asking, in terms of SQL, is projecting data from different tables for different rows in the resultset. It is not possible to my knowledge.
So you'll have to be content with:
#approvals = Approval.all.reject{|a| a.approvable.deleted? }
# I assume you have a deleted? method in all the approvables
I would recommend either of the answers already presented here (they are the same thing) but I would also recommend putting that deleted flag into the Approval model if you really care to do it all in a single query.
With a polymorphic relationship rails can use eager fetching on the polys, but you can't join to them because yet again, the relationships are not known so the query is actually multiple queried intersected.
So in the end if you REALLY need to, drop into sql and intersect all the possible joins you can do to all the types of approvables in a single query, but you will have to do lots of joining manually. (manually meaning not using rails' built-in mechanisms...)
thanks for your answers
I was pretty sure that this couldn't be done. I wanted some more confirmation
besides that I was hoping for some other soln than looping thru the result set
to avoid performance related issues later
Although for the time being both reject/select are fine but in the long run I
will have to do those sql joins manually.
Thanks again for your help!!
M
The business logic is this: Users are in a Boat through a join table, I guess let's call that model a Ticket. But when a User instance wants to check who else is on the boat, there's a condition that asks if that user has permission see everyone on the Boat, or just certain people on the Boat. If a User can see everyone, the normal deal is fine: some_user.boats.first.users returns all users with a ticket for that boat. But for some users, the only people that are on the boat (as far as they're concerned) are people in, let's say the dining room. So if User's ticket is "tagged" (using an acts_as_taggable style system) with "Dining Room", the only Users returned from some_user.boats.first.users should be Users with tickets tagged "Dining Room".
Just for the record, I'm not trying to design something to be insane from the getgo - I'm trying to wedge this arbitrary grouping into a (mostly) existent system.
So we've got:
class User
has_many :tickets
has_many :boats, :through => :tickets
end
class Ticket
belongs_to :user
belongs_to :boat
end
class Boat
has_many :tickets
has_many :users, :through => :tickets
end
Initially, I thought that I could conditionally modify the virtual class like:
singleton = class << a_user_instance ; self ; end
singleton.class_eval(<<-code
has_many :tickets, :include => :tags, :conditions => ['tags.id in (?)', [#{tag_ids.to_s(:db)}]]
code
)
That gets all the way down to generating the SQL, but when generated, it generates SQL ending in:
LEFT OUTER JOIN "tags" ON ("tags"."id" = "taggings"."tag_id") WHERE ("tickets"._id = 1069416589 AND (tags.id in (5001,4502)))
I've tried digging around the ActiveRecord code, but I can't find anywhere that would prefix that 'id' in the SQL above with an underscore. I know that associations are loaded when an ActiveRecord class is loaded, and I'd assume the same with a singleton class. shrug.
I also used an alias_method_chain like:
singleton = class << a_user_instance ; self ; end
singleton.class_eval(<<-code
def tickets_with_tag_filtering
tags = Tag.find(etc, etc)
tickets_without_tag_filtering.scoped(:include => :tags, :conditions => {:'tags.id' => tags})
end
alias_method_chain :tickets, :tag_filtering
code
)
But while that approach produces the desired Tickets, any joins on those tickets use the conditions in the class, not the virtual class. some_user.boats.first.users returns all users.
Any type of comment will be appreciated, especially if I'm barking up the wrong tree with this approach. Thanks!
So a wild guess about your underscore issue is that Rails is generating the assocation code based on the context at the time of evaluation. Being in a singleton class could mess this up, like so:
"#{owner.table_name}.#{association.class.name}_id = #{association.id}"
You could get in there and define a class name property on your singleton class and see if that fixes the issue.
On the whole I don't recommend this. It creates behavior that is agonizing to track down and impossible to extend effectively. It creates a landmine in the codebase that will wound you or someone you love at a later time.
Instead, consider using a named_scope declaration:
class User
has_many :taggings, :through => :tickets
named_scope :visible_to, lambda { |looking_user|
{ :include => [ :tickets, :taggings ],
:conditions => [ "tickets.boat_id in (?) and taggings.ticket_id = tickets.id and taggings.tag_id in (?)", looking_user.boat_ids, looking_user.tag_ids ]
}
}
end
While you may have to go back and edit some code, this is much more flexible in the ways it can be used:
Boat.last.users.visible_to( current_user )
It's clear that a restriction is being placed on the find, and what the purpose of that restriction is. Because the conditions are dynamically calculated at runtime, you can deal with the next weird modification your client hits you with. Say some of their users have xray vision and clairvoyance:
class User
named_scope :visible_to, lambda { |looking_user|
if looking_user.superhuman?
{}
else
{ :include => [ :tickets, :taggings ],
:conditions => [ "tickets.boat_id in (?) and taggings.ticket_id = tickets.id and taggings.tag_id in (?)", looking_user.boat_ids, looking_user.tag_ids ]
}
end
}
end
By returning an empty hash, you can effectively nullify the effect of the scope.
Why not just grab all users on the boat and include their tags.
Then run a quick filter to include & return only the users with the same tag as the inquiring user.
What version of Rails are you using? Have you tried upgrading to see if the underscore issue is fixed? It's like it can't find the foreign key to put in as "tag_id" or somethin'.
My ruby-fu is limited, so I'm not sure how to dynamically include the correct method options at run-time.
Just to help you clarify, you have to worry about this two places. You want to filter a user's viewable users so they only see users with the same tags. Your structure is:
user <--> tickets <--> boats <--> tickets <--> users
... right?
So, you need to filter both sets of tickets down to the ones with the current_user's tags.
Maybe you just need a current_user.viewable_users() method and then filter everything through that? I'm not sure what existing functionality you've got to preserve.
Blech, I don't feel like I'm helping you at all. Sorry.
Your approach is the problem. I know it seems expedient at the moment to hack something in where you don't have to refactor the existing call sites, but I believe given time this will come back to haunt you as the source of bugs and complexity.
Sleeping dogs that lie come back to bite you hard, in my experience. Typically in the form of a future developer who doesn't know your association is "magic" and uses it assuming it's just pail ole rails. He/she likely won't even have a reason to write a test case that would expose the behavior either, which raises the odds you'll only find out about the bug when it's live in production and the client is unhappy. Is it really worth the time you're saving now?
Austinfrombostin is pointing the way. Different semantics? Different names. Rule number one is always to write code that says what it does as clearly as possible. Anything else is the path of madness.