Two models (Rails 2.3.8):
User; username & disabled properties; User has_one :profile
Profile; full_name & hidden properties
I am trying to create a named_scope that eliminate the disabled=1 and hidden=1 User-Profiles. The User model is usually used in conjunction with the Profile model, so I attempt to eager-load the Profile model (:include => :profile).
I created a named_scope on the User model called 'visible':
named_scope :visible, {
:joins => "INNER JOIN profiles ON users.id=profiles.user_id",
:conditions => ["users.disabled = ? AND profiles.hidden = ?", false, false]
}
I've noticed that when I use the named_scope in a query, the eager-loading instruction is ignored.
Variation 1 - User model only:
# UserController
#users = User.find(:all)
# User's Index view
<% for user in #users %>
<p><%= user.username %></p>
<% end %>
# generates a single query:
SELECT * FROM `users`
Variation 2 - use Profile model in view; lazy load Profile model
# UserController
#users = User.find(:all)
# User's Index view
<% for user in #users %>
<p><%= user.username %></p>
<p><%= user.profile.full_name %></p>
<% end %>
# generates multiple queries:
SELECT * FROM `profiles` WHERE (`profiles`.user_id = 1) ORDER BY full_name ASC LIMIT 1
SHOW FIELDS FROM `profiles`
SELECT * FROM `profiles` WHERE (`profiles`.user_id = 2) ORDER BY full_name ASC LIMIT 1
SELECT * FROM `profiles` WHERE (`profiles`.user_id = 3) ORDER BY full_name ASC LIMIT 1
SELECT * FROM `profiles` WHERE (`profiles`.user_id = 4) ORDER BY full_name ASC LIMIT 1
SELECT * FROM `profiles` WHERE (`profiles`.user_id = 5) ORDER BY full_name ASC LIMIT 1
SELECT * FROM `profiles` WHERE (`profiles`.user_id = 6) ORDER BY full_name ASC LIMIT 1
Variation 3 - eager load Profile model
# UserController
#users = User.find(:all, :include => :profile)
#view; no changes
# two queries
SELECT * FROM `users`
SELECT `profiles`.* FROM `profiles` WHERE (`profiles`.user_id IN (1,2,3,4,5,6))
Variation 4 - use name_scope, including eager-loading instruction
#UserConroller
#users = User.visible(:include => :profile)
#view; no changes
# generates multiple queries
SELECT `users`.* FROM `users` INNER JOIN profiles ON users.id=profiles.user_id WHERE (users.disabled = 0 AND profiles.hidden = 0)
SELECT * FROM `profiles` WHERE (`profiles`.user_id = 1) ORDER BY full_name ASC LIMIT 1
SELECT * FROM `profiles` WHERE (`profiles`.user_id = 2) ORDER BY full_name ASC LIMIT 1
SELECT * FROM `profiles` WHERE (`profiles`.user_id = 3) ORDER BY full_name ASC LIMIT 1
SELECT * FROM `profiles` WHERE (`profiles`.user_id = 4) ORDER BY full_name ASC LIMIT 1
Variation 4 does return the correct number of records, but also appears to be ignoring the eager-loading instruction.
Is this an issue with cross-model named scopes? Perhaps I'm not using it correctly.
Is this sort of situation handled better by Rails 3?
From railsapi.com:
Eager loading of associations
[...] Since only one table is loaded
at a time, conditions or orders
cannot reference tables other than the
main one. If this is the case Active
Record falls back to the previously
used LEFT OUTER JOIN based strategy.
For example
Post.find(:all, :include => [ :author, :comments ],
:conditions => ['comments.approved = ?', true])
will result in a single SQL query with
joins along the lines of: LEFT OUTER
JOIN comments ON comments.post_id =
posts.id and LEFT OUTER JOIN authors
ON authors.id = posts.author_id.
I believe this answers your question... there's no eager loading in "variation #4" because you references profiles table on your named_scope.
I believe the following may give you what you are looking for:
#users = User.visible.scoped(:include => :profile)
This did the trick for me, but I'm not joining with other tables in the definition of my named scope.
Jim Benton provides an elegant way of adding this to ActiveRecord on his blog: http://autonomousmachine.com/posts/2009/10/28/add-a-scope-for-easier-eager-loading
Related
I have an Order model, it has many items, it looks like this
class Order < ActiveRecord::Base
has_many :items
def total
items.sum('price * quantity')
end
end
And I have an order index view, querying order table like this
def index
#orders = Order.includes(:items)
end
Then, in the view, I access total of order, as a result, you will see tons of SUM query like this
SELECT SUM(price * quantity) FROM "items" WHERE "items"."order_id" = $1 [["order_id", 1]]
SELECT SUM(price * quantity) FROM "items" WHERE "items"."order_id" = $1 [["order_id", 2]]
SELECT SUM(price * quantity) FROM "items" WHERE "items"."order_id" = $1 [["order_id", 3]]
...
It's pretty slow to load order.total one by one, I wonder how can I load the sum in a eager manner via single query, but still I can access order.total just like before.
Try this:
subquery = Order.joins(:items).select('orders.id, sum(items.price * items.quantity) AS total').group('orders.id')
#orders = Order.includes(:items).joins("INNER JOIN (#{subquery.to_sql}) totals ON totals.id = orders.id")
This will create a subquery that sums the total of the orders, and then you join that subquery to your other query.
I wrote up two options for this in this blog post on using find_by_sql or joins to solve this.
For your example above, using find_by_sql you could write something like this:
Order.find_by_sql("select
orders.id,
SUM(items.price * items.quantity) as total
from orders
join items
on orders.id = items.order_id
group by
order.id")
Using joins, you could rewrite as:
Order.all.select("order.id, SUM(items.price * items.quantity) as total").joins(:items).group("order.id")
Include all the fields you want in your select list in both the select clause and the group by clause. Hope that helps!
I have 2 models, User and UserProfile. A user has_one user_profile and a user_profile belongs_to user.
1) Find without select
This query in console works fine, and take only 2 SQL queries.
>> User.find(:all, :limit => 10, :include => [ :user_profile ])
User Load (0.3ms) SELECT * FROM `users` LIMIT 10
UserProfile Load (0.3ms) SELECT `user_profiles`.* FROM `user_profiles`
WHERE (`user_profiles`.user_id IN (1,2,3,...))
2) Find with select on user model
I can select columns from User model, with
>> User.find(:all, :select => '`users`.id, `users`.last_name',
:limit => 10, :include => [ :user_profile ])
User Load (0.3ms) SELECT `users`.id, `users`.last_name FROM `users` LIMIT 10
UserProfile Load (0.2ms) SELECT `user_profiles`.* FROM `user_profiles`
WHERE (`user_profiles`.user_id IN (17510,18087,17508,17288...))
Everything works fine. Note that I must set users.id in the user selected columns, because the second query doesn't work (return NULL).
3) Find with select on user_profile model
But when I try to select columns from UserProfile model, I got only 1 query, which doesn't take care of my :select
>> User.find(:all,
:select => '`users`.id, `users`.last_name, `user_profiles`.permalink',
:limit => 10, :include => [ :user_profile ])
User Load Including Associations (0.6ms) SELECT `users`.`id` AS t0_r0,
`users`.`login` AS t0_r1, ....
`user_profiles`.`id` AS t1_r0,
`user_profiles`.`birth_date` AS t1_r1,
LEFT OUTER JOIN `user_profiles` ON user_profiles.user_id = users.id LIMIT 10
As you can see, the Rails query contains fiels from users and fields from user_profiles that I didn't select.
4) Join method
Codeit purpose a method with join function :
user_details = User.find(:all,
:select => '`users`.id, `users`.last_name, `user_profiles`.permalink',
:limit => 10, :joins => [ :user_profile ]
)
User Load (0.2ms) SELECT `users`.id, `users`.last_name, `user_profiles`.permalink
FROM `users`
INNER JOIN `user_profiles` ON user_profiles.user_id = users.id
LIMIT 10
This solution works fine with SQL queries, but doesn't make 'link' between User and User Profile. 10 new queries are needed, while the method 1 and 2 make only 2 SQL queries.
user_details.map(&:user_profile).map(&:permalink)
UserProfile Load (0.3ms) SELECT * FROM `user_profiles` WHERE (`user_profiles`.user_id = 1) LIMIT 1
UserProfile Load (0.2ms) SELECT * FROM `user_profiles` WHERE (`user_profiles`.user_id = 2) LIMIT 1
... (10 times) ...
UserProfile Load (0.3ms) SELECT * FROM `user_profiles` WHERE (`user_profiles`.user_id = 10) LIMIT 1
Is there a right syntax to have same results than the 2 first queries, but with a :select witch select only a few columns of my models ?
Use join:
User.find(:all,
:select => '`users`.id, `users`.last_name, `user_profiles`.permalink',
:limit => 10, :joins => [ :user_profile ])
include is used for eager loading. It is used to solve (N+1) queries problem for accessing user_profile when you have large users with user_profile. If you want to select columns of included table you need to use join. If you use columns of included table it will just ignored from select clause.
EDIT:
user_details = User.find(:all,
:select => '`users`.id, `users`.last_name, `user_profiles`.permalink',
:limit => 10, :joins => [ :user_profile ]
)
user_details.map(&:permalink)
I have 2 models in my rails app, one with an UUID primary key :
class User < ActiveRecord::Base
belongs_to :country, :foreign_key => 'country_uuid'
end
class Country < ActiveRecord::Base
set_primary_key :uuid
has_many :users
end
When I try something like that:
<% #user = User.find :first, :include => [:country] %>
<%= #user.country.name %>
I have the good result, but I see 2 requests in the log file. Why eager loading is not working when we change the ID key for UUID key ?
User Load (0.4ms) SELECT `users`.* FROM `users` LIMIT 1
Country Load (0.4ms) SELECT `countries`.* FROM `countries` WHERE (`countries`.`uuid` = '1')
And I would have something like:
User Load (0.4ms) SELECT `users`.* FROM `users` INNER JOIN countries ON countries.uuid = users.country_uuid LIMIT 1
Is there a workaround ?
If I change uuid key for id key, but keep the string format to store an uuid, will it be ok ?
Thanks,
Use joins instead of include to get the inner join
includes always issues a 2nd query but not n+1 queries (lazy)
for the direction you are going in user -> 1 country it is not so important
but if you were going the other direction country -> many users
country = Country.first
# => select countries.* from countries where id = xxxx limit 1;
country.users.each do
# select users.* from users where user_id = xxxx;
# this could be bad because of lazy loading, one query per iteration
end
# vs...
country = Country.first.includes(:users)
# => select countries.* from countries where id = xxxx limit 1;
# => select users.* from users where country_uuid IN (xxxx);
country.users.each do
# users are all in memory
end
see http://guides.rubyonrails.org/active_record_querying.html for more info
I don't think the fact you are using UUID should make any difference
I'm attempting to eager load in my Rails 3 app. I've narrowed it down to a very basic sample, and instead of generating the one query I'm expecting, it's generating 4.
First, here's a simple breakdown of my models.
class Profile < ActiveRecord::Base
belongs_to :gender
def to_param
self.name
end
end
class Gender < ActiveRecord::Base
has_many :profiles, :dependent => :nullify
end
I then has a ProfilesController::show action, where's I'm querying for the model.
def ProfilesController < ApplicationController
before_filter :find_profile, :only => [:show]
def show
end
private
def find_profile
#profile = Profile.find_by_username(params[:id], :include => :gender)
raise ActiveRecord::RecordNotFound, "Page not found" unless #profile
end
end
When I look at the queries this generates, it shows the following:
SELECT `profiles`.* FROM `profiles` WHERE `profiles`.`username` = 'matt' LIMIT 1
SELECT `genders`.* FROM `genders` WHERE (`genders`.`id` = 1)
What I expected to see is a single query:
SELECT `profiles`.*, `genders`.* FROM `profiles` LEFT JOIN `genders` ON `profiles`.gender_id = `genders`.id WHERE `profiles`.`username` = 'matt' LIMIT 1
Anyone know what I'm doing wrong here? Everything I've found on eager loading makes it sound like this should work.
Edit: After trying joins, as recommended by sled, I'm still seeing the same results.
The code:
#profile = Profile.joins(:gender).where(:username => params[:id]).limit(1).first
The query:
SELECT `profiles`.* FROM `profiles` INNER JOIN `genders` ON `genders`.`id` = `profiles`.`gender_id` WHERE `profiles`.`username` = 'matt' LIMIT 1
Again, you can see no genders data is being retrieved, and so a second query to genders is being made.
I even tried adding a select, to no avail:
#profile = Profile.joins(:gender).select('profiles.*, genders.*').where(:username => params[:id]).limit(1).first
which correctly resulted in:
SELECT profiles.*, genders.* FROM `profiles` INNER JOIN `genders` ON `genders`.`id` = `profiles`.`gender_id` WHERE `profiles`.`username` = 'matt' LIMIT 1
...but it still performed a second query on genders later when accessing #profile.gender's attributes.
Edit 2: I also tried creating a scope that includes both select and joins in order to get all the fields I require, (similar to the custom left join method sled demonstrated). It looks like this:
class Profile < ActiveRecord::Base
# ...
ALL_ATTRIBUTES = [:photo, :city, :gender, :relationship_status, :physique, :children,
:diet, :drink, :smoke, :drug, :education, :income, :job, :politic, :religion, :zodiac]
scope :with_attributes,
select((ALL_ATTRIBUTES.collect { |a| "`#{reflect_on_association(a).table_name}`.*" } + ["`#{table_name}`.*"]).join(', ')).
joins(ALL_ATTRIBUTES.collect { |a|
assoc = reflect_on_association(a)
"LEFT JOIN `#{assoc.table_name}` ON `#{table_name}`.#{assoc.primary_key_name} = `#{assoc.table_name}`.#{assoc.active_record_primary_key}"
}.join(' '))
# ...
end
This generates the following query, which appears correct:
SELECT `photos`.*, `cities`.*, `profile_genders`.*, `profile_relationship_statuses`.*, `profile_physiques`.*, `profile_children`.*, `profile_diets`.*, `profile_drinks`.*, `profile_smokes`.*, `profile_drugs`.*, `profile_educations`.*, `profile_incomes`.*, `profile_jobs`.*, `profile_politics`.*, `profile_religions`.*, `profile_zodiacs`.*, `profiles`.* FROM `profiles` LEFT JOIN `photos` ON `profiles`.photo_id = `photos`.id LEFT JOIN `cities` ON `profiles`.city_id = `cities`.id LEFT JOIN `profile_genders` ON `profiles`.gender_id = `profile_genders`.id LEFT JOIN `profile_relationship_statuses` ON `profiles`.relationship_status_id = `profile_relationship_statuses`.id LEFT JOIN `profile_physiques` ON `profiles`.physique_id = `profile_physiques`.id LEFT JOIN `profile_children` ON `profiles`.children_id = `profile_children`.id LEFT JOIN `profile_diets` ON `profiles`.diet_id = `profile_diets`.id LEFT JOIN `profile_drinks` ON `profiles`.drink_id = `profile_drinks`.id LEFT JOIN `profile_smokes` ON `profiles`.smoke_id = `profile_smokes`.id LEFT JOIN `profile_drugs` ON `profiles`.drug_id = `profile_drugs`.id LEFT JOIN `profile_educations` ON `profiles`.education_id = `profile_educations`.id LEFT JOIN `profile_incomes` ON `profiles`.income_id = `profile_incomes`.id LEFT JOIN `profile_jobs` ON `profiles`.job_id = `profile_jobs`.id LEFT JOIN `profile_politics` ON `profiles`.politic_id = `profile_politics`.id LEFT JOIN `profile_religions` ON `profiles`.religion_id = `profile_religions`.id LEFT JOIN `profile_zodiacs` ON `profiles`.zodiac_id = `profile_zodiacs`.id WHERE `profiles`.`username` = 'matt' LIMIT 1
Unfortunately, it doesn't seem that calls to relationship attributes (e.g.: #profile.gender.name) are using the data that was returned in the original SELECT. Instead, I see a flood of queries following this first one:
Profile::Gender Load (0.2ms) SELECT `profile_genders`.* FROM `profile_genders` WHERE `profile_genders`.`id` = 1 LIMIT 1
Profile::Gender Load (0.4ms) SELECT `profile_genders`.* FROM `profile_genders` INNER JOIN `profile_attractions` ON `profile_genders`.id = `profile_attractions`.gender_id WHERE ((`profile_attractions`.profile_id = 2))
City Load (0.4ms) SELECT `cities`.* FROM `cities` WHERE `cities`.`id` = 1 LIMIT 1
Country Load (0.3ms) SELECT `countries`.* FROM `countries` WHERE `countries`.`id` = 228 ORDER BY FIELD(code, 'US') DESC, name ASC LIMIT 1
Profile Load (0.4ms) SELECT `profiles`.* FROM `profiles` WHERE `profiles`.`id` = 2 LIMIT 1
Profile::Language Load (0.4ms) SELECT `profile_languages`.* FROM `profile_languages` INNER JOIN `profile_profiles_languages` ON `profile_languages`.id = `profile_profiles_languages`.language_id WHERE ((`profile_profiles_languages`.profile_id = 2))
SQL (0.3ms) SELECT COUNT(*) FROM `profile_ethnicities` INNER JOIN `profile_profiles_ethnicities` ON `profile_ethnicities`.id = `profile_profiles_ethnicities`.ethnicity_id WHERE ((`profile_profiles_ethnicities`.profile_id = 2))
Profile::Religion Load (0.5ms) SELECT `profile_religions`.* FROM `profile_religions` WHERE `profile_religions`.`id` = 2 LIMIT 1
Profile::Politic Load (0.2ms) SELECT `profile_politics`.* FROM `profile_politics` WHERE `profile_politics`.`id` = 3 LIMIT 1
your example is fine and it will end up in two queries because that's how eager loading is implemented in rails. It becomes handy if you have many associated records. You can read more about it here
What you probably want is a simple join:
#profile = Profile.joins(:gender).where(:username => params[:id])
Edit
If the profile consists of many pieces there are multiple approaches here:
Custom left joins - maybe there is a plugin out there which does the job otherwise I'd suggest to do something like:
class Profile < ActiveRecord::Base
# .... code .....
def self.with_dependencies
attr_joins = []
attr_selects = []
attr_selects << "`profiles`.*"
attr_selects << "`genders`.*"
attr_selects << "`colors`.*"
attr_joins << "LEFT JOIN `genders` ON `gender`.`id` = `profiles`.gender_id"
attr_joins << "LEFT JOIN `colors` ON `colors`.`id` = `profiles`.color_id"
prep_model = select(attr_selects.join(','))
attr_joins.each do |c_join|
prep_model = prep_model.joins(c_join)
end
return prep_model
end
end
Now you could do something like:
#profile = Profile.with_dependencies.where(:username => params[:id])
Another solution is to use the :include => [:gender, :color] it may be some queries more but it's the cleaner "rails way". If you run into performance issues you may want to rethink your DB Schema but do you have really such a heavy load?
A friend of mine wrote a nice little solution for this simple 1:n relations (like genders) it's called simple_enum
After working with sled's suggestions, I finally came up with this solution. I'm sure it could be made cleaner with a plugin, but here's what I've got for now:
class Profile < ActiveRecord::Base
ALL_ATTRIBUTES = [:photo, :city, :gender, :relationship_status, :physique, :children,
:diet, :drink, :smoke, :drug, :education, :income, :job, :politic, :religion, :zodiac]
scope :with_attributes,
includes(ALL_ATTRIBUTES).
select((ALL_ATTRIBUTES.collect { |a| "`#{reflect_on_association(a).table_name}`.*" } + ["`#{table_name}`.*"]).join(', '))
end
The two main points are:
A call to includes, which passes the symbols of the relationships I want
A call to select that makes sure to retrieve all columns for the related tables. Note that I call reflect_on_association so that I don't have to hard-code the related tables' names, letting the Rails models do the work for me.
I can now call:
Profile.with_attributes.where(:username => params[:id]).limit(1).first
Going to mark sled's answer as correct since it's his help (answers + comments combined) that led me here, even though this is the code I'm ultimately using.
Rails version 3.0.3, I am new to rails, but been in webdev for a long time.
I am using awesome nested set.
I have the tables "posts", "labels", and "labels_posts"
posts has_and_belongs_to_many labels
labels has_and_belongs_to_many posts
labels acts_as_nested_set
I have a label_id and I want to get all posts that are associated to that label and its children, all as a single ordered result set.
Let us say that I have Labels: "L1, L1.1, L1.1.1, L1.2, L2"
Given L1, and knowing that therefore I have L1, L1.1, L1.1.1, and L1.2, I would normally run the query:
select id, title
from posts
where exists (select * from labels_posts where labels_posts.post_id = posts.id and labels_posts.label_id IN ('L1', 'L1.1', 'L1.1.1', 'L1.2'))
order by created_at desc
This query would return all the posts associated with each of those labels.
So, what is the rails way to do this?
EDIT:
So, here is my controller
#label = Label.find(params[:label])
#posts = Post.all.select do |post|
post.label_ids.include?(#label.self_and_descendants.map(&:id))
end
And here is the rails server output
Label Load (0.5ms) SELECT "labels".* FROM "labels" WHERE ("labels"."cached_slug" = 'caribbean') LIMIT 1
Post Load (0.6ms) SELECT "posts".* FROM "posts"
Label Load (0.2ms) SELECT "labels".id FROM "labels" INNER JOIN "labels_posts" ON "labels".id = "labels_posts".label_id WHERE ("labels_posts".post_id = 1 )
Label Load (0.8ms) SELECT "labels".* FROM "labels" WHERE ("labels"."lft" >= 1 AND "labels"."rgt" <= 8) ORDER BY "lft"
Label Load (0.1ms) SELECT "labels".id FROM "labels" INNER JOIN "labels_posts" ON "labels".id = "labels_posts".label_id WHERE ("labels_posts".post_id = 2 )
CACHE (0.0ms) SELECT "labels".* FROM "labels" WHERE ("labels"."lft" >= 1 AND "labels"."rgt" <= 8) ORDER BY "lft"
I am not sure the select method is the one that is needed.
EDIT ANSWER:
So here is the answer I arrived at
#label = Label.find(params[:label])
#posts = Post.order('posts.created_at desc').where('labels_posts.label_id IN (?)', #label.self_and_descendants.map(&:id)).includes(:labels)
try
Post.all(:conditions => "id = labels_posts.post_id AND label_posts.label_id in('L1', 'L1.1', 'L1.1.1', 'L1.2')", :include => [:label => labels_posts], :order => :created_at)
Lets first collect the ids of the Label (label_id) and its children.
Then use the Ruby Array Select method to select those posts which are associated with the appropriate labels.
label_ids = Label.find(label_id).children.map(&:id).push(label_id)
posts = Post.all.select{|post| post.label_ids.include?(label_ids)}