Say I'm modeling Students, Lessons, and Teachers. Given a single student enrolled in many lessons, how would I find all of their teachers of classes that are level 102? For that matter, how would I find all of their lessons' teachers? Right now, I have this:
s = Mongoid::Student.find_by(name: 'Billy')
l = s.lessons.where(level: 102)
t = l.map { |lesson| lesson.teachers }.flatten
Is there a way to do turn the second two lines into one query?
Each collection requires at least one query, there's no way to access more than one collection in one query (i.e. no JOINs) so that's the best you can do. However, this:
t = l.map { |lesson| lesson.teachers }.flatten
is doing l.length queries to get the teachers per lesson. You can clean that up by collecting all the teacher IDs from the lessons:
teacher_ids = l.map(&:teacher_ids).flatten.uniq # Or use `inject` or ...
and then grab the teachers based on those IDs:
t = Teacher.find(teacher_ids)
# or
t = Teacher.where(:id.in => teacher_ids).to_a
If all those queries don't work for you then you're stuck with denormalizing something so that you have everything you need embedded in a single collection; this would of course mean that you'd have to maintain the copies as things change and periodically sanity check the copies for consistency problems.
Related
Assuming this simplified schema:
users has_many discount_codes
discount_codes has_many orders
I want to grab all users, and if they happen to have any orders, only include the orders that were created between two dates. But if they don't have orders, or have orders only outside of those two dates, still return the users and do not exclude any users ever.
What I'm doing now:
users = User.all.includes(discount_codes: :orders)
users = users.where("orders.created_at BETWEEN ? AND ?", date1, date2).
or(users.where(orders: { id: nil })
I believe my OR clause allows me to retain users who do not have any orders whatsoever, but what happens is if I have a user who only has orders outside of date1 and date2, then my query will exclude that user.
For what it's worth, I want to use this orders where clause here specifically so I can avoid n + 1 issues later in determining orders per user.
Thanks in advance!
It doesn't make sense to try and control the orders that are loaded as part of the where clause for users. If you were to control that it'd have to be part of the includes (which I think means it'd have to be a part of the association).
Although technically it can combine them into a single query in some cases, activerecord is going to do this as two queries.
The first query will be executed when you go to iterate over the users and will use that where clause to limit the users found.
It will then run a second query behind the scenes based on that includes statement. This will simply be a query to get all orders which are associated with the users that were found by the previous query. As such the only way to control the orders that are found through the user's where clause is to omit users from the result set.
If I were you I would create an instance method in User model for what you are looking for but instead of using where use a select block:
def orders_in_timespan(start, end)
orders.select{ |o| o.between?(start, end) }
end
Because of the way ActiveRecord will cache the found orders from the includes against the instance then if you start off with an includes in your users query then I believe this will not result in n queries.
Something like:
render json: User.includes(:orders), methods: :orders_in_timespan
Of course, the easiest way to confirm the number of queries is to look at the logs. I believe this approach should have two queries regardless of the number of users being rendered (as likely does your code in the question).
Also, I'm not sure how familiar you are with sql but you can call .to_sql on the end of things such as your users variable in order to see the sql that would be generated which might help shed some light on the discrepancies between what you're getting and what you're looking for.
Option 1: Write a custom query in SQL (ugly).
Option 2: Create 2 separate queries like below...
#users = User.limit(10)
#orders = Order.joins(:discount_code)
.where(created_at: [10.days.ago..1.day.ago], discount_codes: {user_id: users.select(:id)})
.group_by{|order| order.discount_code.user_id}
Now you can use it like this ...
#users.each do |user|
orders = #orders[user.id]
puts user.name
puts user.id
puts orders.count
end
I hope this will solve your problem.
You need to use joins instead of includes. Rails joins use inner joins and will reject all the records which don't have associations.
User.joins(discount_codes: :orders).where(orders: {created_at: [10.days.ago..1.day.ago]}).distinct
This will give you all distinct users who placed orders in a given period of time.
user = User.joins(:discount_codes).joins(:orders).where("orders.created_at BETWEEN ? AND ?", date1, date2) +
User.left_joins(:discount_codes).left_joins(:orders).group("users.id").having("count(orders.id) = 0")
There is an association query I seem to be unable to do without triggering a N+1 query.
Suppose I host Parties. I have many Friends, and each time a friend comes to a party, they create a Presence.
And so:
Presence.belongs_to :party
Presence.belongs_to :friend
Friend.has_many :presences
Party.has_many :presences
So far so good.
I want to obtain a list of every one of my Friends, knowing whether or not they are present at this Party, without triggering a N+1 query.
My dataset would look like this:
friends: [
{name: "Dave Vlopment", presences: [{created_at: "8pm", party_id: 2012}]},
{name: "Brett E. Hardproblem", presences: [nil]},
{name: "Ann Plosswan-Quarry", presences: [{created_at: "10pm", party_id: 2012}]},
...
]
and so on.
I have a lot of friends and do a lot of parties, of course. (This is of course a fictional example.)
I would do:
Friend.all.includes(:presence).map{ |them| them.parties }
# But then, `them.parties` is not filtered to tonight's party.
Friend.all.includes(:presence).map{ |them| them.parties.where(party_id: pid) }
# And there I have an N+1.
I could always filter at the Ruby layer:
Friend.all.includes(:presence).map{ |them| them.parties.select{ |it| it.party_id = party.id } }
But this works pretty badly with as_json(includes: {}) and so on. I'm discovering this is very error-prone as I'll be making calculations on the results.
And I make a lot of parties, you know? (still fictional)
If I where on the first query, I lose the left join:
Friend.all.includes(:presence).where(party: party)
I have no idea that tonight, Brett and a bunch of friends, who are always there, are absent. (this one is not guaranteed to be a fictional experience)
I will only see friends who are present.
And if I go through party, well of course I will not see who is absent either.
Now I know there are ways I can do this in SQL, and other ways we can wrangle around some Ruby to pull it together.
However, I'm looking for a "first-class" way to do this in Activerecord, without getting N+1s.
Is there a way to do this using only the Activerecord tools? I haven't found anything yet.
I'm not sure whether this meets your expectation about "first-class" way or not.
But you can use this approach to avoids N+1
# fetch all friends
friends = Friend.all
# fetch all presences. grouped by friend_id
grouped_presences = Presence.all.group_by(&:friend_id)
# arrange data
data = []
friends.each do |friend|
json = friend.as_json
json["presences"] = grouped_presences[friend.id].as_json
data << json
end
puts data
It only executes 2 queries
SELECT `friends`.* FROM `friends`
SELECT `presences`.* FROM `presences`
I have a model Profile that has a onetoone with the model Interest. If I query an array of Profiles using where, how would I then get another separate array composed of the Interests associated with those originally queried Profiles?
Something like
#foundProfiles = Profile.where(field: data)
into
#foundInterests = #foundProfiles.Interest.all
this doesn't work, but that's the idea I'm trying to get at.
Make use of association:
#foundProfiles = Profile.includes(:interest).where(field: data)
# Eager load interest to avoid n + 1 queries
#foundInterests = #foundProfiles.map(&:interest)
EDIT
If you need to query further on the Interest records you can do something like:
#foundInterests = Interest.where(profile_id: #foundProfiles.map(&:id))
This will return you Interest records associated with #foundProfiles and you can chain where on it
I have a habtm relationship between my Product and Category model.
I'm trying to write a query that searches for products with minimum of 2 categories.
I got it working with the following code:
p = Product.joins(:categories).group("product_id").having("count(product_id) > 1")
p.length # 178
When iterating on it though, for each time I call product.categories, it will do a new call to the database - not good. I want to prevent these calls and have the same result. Doing more research I've seen that I could include (includes) my categories table and it would load all the table in memory so it's not necessary to call the database again when iterating. So I got it working with the following code:
p2 = Product.includes(:categories).joins(:categories).group("product_id").having("count(product_id) > 1")
p2.length # 178 - I compared and the objects are the same as last query
Here come's what I am confused about:
p.first.eql? p2.first # true
p.first.categories.eql? p2.first.categories # false
p.first.categories.length # 2
p2.first.categories.length # 1
Why with the includes query I get the right objects but I don't get the categories relationship right?
It has something to do with the group method. Your p2 only contains the first category for each product.
You could break this up into two queries:
product_ids = Product.joins(:categories).group("product_id").having("count(product_id) > 1").pluck(:product_id)
result = Product.includes(:categories).find(product_ids)
Yeah, you hit the database twice, but at least you don't go to the database when you're iterating.
You must know that includes doesn't play well with joins (joins will just suppress the former).
Also When you include an association ActiveRecord figures out if it'll use eager_load (with a left join) or preload (with a separate query). Includes is just a wrapper for one of those 2.
The thing is preload plays well with joins ! So you can do this :
products = Product.preload(:categories). # this will trigger a separate query
joins(:categories). # this will build the relevant query
group("products.id").
having("count(product_id) > 1").
select("products.*")
Note that this will also hit the database twice, but you will not have any O(n) query.
Lets say I have 2 tables in my database, both have 'updated_at' field.
I want to retrieve all the records in both tables in one active record query.
I should return one large activerecord relation ordered by updated_at field.
Let assume these two tables are really similar.
If it is possible, it would be better to have the ability to only return the first 50, or second 50, or so on.
here's an example
cats = Cat.all
dog = Dog.all
animals = (cats | dogs).sort!{|a,b| a.updated_at <=> b.updated_at}.limit(50)
It is pretty costly to get all dogs then cats then merge them into one, and then get the first 50 animals.
I need a way to make one query to db and the db returns the first 50 animals.
or the second 50 animals.
Given that these models are so similar, wouldn't be better to use ActiveRecord's Single table inheritance? For the example you provided, you would have base class Animal, and you would get the results you want with just Animal.order("updated_at").limit(50)
It's not a single query, but an obvious optimization would be to only grab the first 50 of each instead of the whole tables:
cats = Cat.order_by(:updated_at).limit(50)
dogs = Dog.order_by(:updated_at).limit(50)
animals = (cats + dogs).sort_by(&:updated_at).first(50)
You could then get the next 50 by querying on the last updated_at
last_animal_date = animals.last.updated_a
next_cats = Cat.where("updated_at > ?", last_date).order_by(:updated_at).limit(50)
next_dogs = Dog.where("updated_at > ?", last_date).order_by(:updated_at).limit(50)
# etc ...
Clearly with the amount of repetition this needs to be factored into a method but I leave that as an exercise to the reader :)