Some of my classes :
class User
embeds_many :notifications
field :first_name
field :last_name
def name{ "#{first_name} #{last_name}" }
class Notification
embedded_in :user
belongs_to :sender, class_name: "User", inverse_of: nil
Now in my views, I implemented a small mailbox system for notifications. However, it's currently hitting N+1 times the database :
<% current_user.notifications.sort{...}.each do |notif|%>
...
<%= notif.sender.name if notif.sender %>
The problem here is the notif.sender.name which causes N hits on the database. Can I somehow preload/eager load this ? Something like current_user.notifications.includes(:sender) (but which would work :D)
I currently only need the sender name.
I think you're half out of luck here. Mongoid has an error message like:
Eager loading in Mongoid only supports providing arguments to M.includes that are the names of relations on the M model, and only supports one level of eager loading. (ie, eager loading associations not on the M but one step away via another relation is not allowed).
Note the last parenthesized sentence in particular:
eager loading associations not on the M but one step away via another relation is not allowed
Embedding is a relation but you want to apply includes to the embedded relation and that's one step too far for Mongoid.
The fine manual does say that:
This will work for embedded relations that reference another collection via belongs_to as well.
but that means that you'd call includes on the embedded relation rather than what the models are embedded in. In your case, that means that you could eager load the senders for each set of embedded Notifications:
current_user.notifications.includes(:sender).sort { ... }
That still leaves you with the N+1 problem that eager loading is supposed to get around but your N will be smaller.
If that's still too heavy then you could denormalize the name into each embedded document (i.e. copy it rather than referencing it through the sender). Of course, you'd need to maintain the copies if people were allowed to change their names.
It's not perfect, but this article presents a possible solution.
You can load all the senders and use set_relation to avoid them to be loaded every time.
def notifications_with_senders
sender_ids = notifications.map(:sender_id)
senders = User.in(id: sender_ids).index_by(&:id)
notifications.each do |notification|
notification.set_relation(:sender, senders[notification.sender_id])
end
end
Would be great to have that as a Relation method (like includes of Rails Active Record)
Related
Using Ruby-on-Rails 5.2.3 with ruby 2.5
Problem:
I've written in Rails a network-router configuration frontend, and this has some circular definitions, like this one:
A Forward has many Addresses (that are forwarded).
An Address has many Forwards (that do forward to it).
An Address has many Forwards (for which the responses come in from it).
Addendum:
As this may appear a bit weird at first glance, here is a little picture:
One can see the three distinct associations between the two objects, where the "Address" in each association plays quite a different role. (But they can change roles or even play multilple ones - and thats exactly the point of the application: always generate a syntactically and logically correct configuration for the end device - and that part works already).
Plus, obviousely is also needed:
An Address belongs to an interface.
I built this in the usual way:
class Forward < ApplicationRecord
has_many :addresses, dependent: :nullify
belongs_to :remoteip, class_name: 'Address'
belongs_to :responding, class_name: 'Address', optional: true
end
class Address < ApplicationRecord
belongs_to :interface
belongs_to :forward, optional: true
has_many :remoteip_forwards, class_name: 'Forward',
foreign_key: :remoteip_id, dependent: :destroy
has_many :responding_forwards, class_name: 'Forward',
foreign_key: :responding_id, dependent: :nullify
end
Then there are CONSTRAINTS on the Database, as objects must not be orphans, in thie case:
address.interface_id NOT NULL
Then I have a top-level Design model, which chains together all the other models via proper associations, so I can grab one design and get all the stuff as one tree.
This all works just fine, so far. But when I try to copy an entire Design (with the deep_clone gem), when saving, some of the rather complex Designs give me NotNullViolation.
Analysis:
Logs show that Rails does save some of the Address records twice: it does an INSERT with valid forward_id but empty interface_id, and only later does an UPDATE for the valid interface_id, all within the transaction.
And obviousely this kicks the NOT NULL constraint, and the action fails (when removing the constraint, the copy succeeds properly).
I would understand this if the actual design would indeed contain a circular reference (because then there is no other way to save it), but there is none! It is only logically possible to create circular references with these models.
I could not find proper documentation on how Rails handles such (potentially) circular things. inverse_of doesn't help anything (but that's to be expected, as my namings are well-defined).
When I use autosafe: false on the offending associations, the issue is resolved and Rails gets the tree saved - but then records which are not referenced twice, are missing in the copy (that's no surprize).
So it seems that the Rails code that walks the association tree to find and save the records is not fully extensive, and too early resorts on writing a record twice (probably to enhance performance?).
How to approach this in the best way?
Sadly, NOT NULL constraints are not DEFERRABLE in postgresql. Too bad, as that would seem the most charming solution.
use autosave=false, walk the association tree by foot to find the proper sequence and save each record individually? No thanks.
Layout the associations somehow differently? How? (I already have the working code that converts all the data for the targeted device, and that works well with the current layout.)
Do away with the NOT NULL constraints? Hmm...
Or, is there somewhere a knob in Rails that could be adjusted?
Update
I have, for the time being, workarounded the issue as explained here: https://discuss.rubyonrails.org/t/request-failures-lots-of-activerecord-not-nullviolation/79156
(The literal number 3 there might be made configurable, as it is a tradeoff between getting the associations resolved and running out of stack space.)
For this setup with default (unspecified) search_data:
class Item < ActiveRecord::Base
searchkick
has_many :quantities, dependent: :destroy
scope :search_import, -> { includes(:quantities) }
end
When importing all database records by running Item.reindex, each "batch" eager loads the quantities for all of the item ids as expected.
However, if I want to specify the index model document differently than the default attributes using the search_data method including the associated model data with something like this:
class Item < ActiveRecord::Base
searchkick
has_many :quantities, dependent: :destroy
def search_data
{
part_number: standard_part_number,
category_id: category_id.to_i,
content_source_name: content_source_name,
standard_price: standard_price.to_i,
locations: quantities.map {|q| [q.location_code,
quantity: q.value.to_i,
bins: q.location_bins]
}.to_h
}
end
scope :search_import, -> { includes(:quantities) }
end
where I am operating on the quantities using map to define a locations attribute, returning to import using Item.reindex I see that it not only eager loads all of the associated quantities each batch, but it also then loads all quantities per item with a hit to the database again for each item record it is indexing.
Am I missing something here to get Searchkick to eager load the quantities models and be able to customize the search data record using that already loaded associated model without it doing another pull from the database again per item?
Update
I've determined there is something interfering with the way the ActiveRecord normally responds to the method name used for the association with the eager loaded models in our app and may not be exclusively related to us using Searchkick, although it did reveal the issue. I am not sure what it is that is interfering at this time but it has something to do with the target method on the association. Perhaps a gem that is installed is causing this problem. I did manage to find a way to work around this (for now) using item.association(:quantities).target as a replacement for item.quantities and now when running the reindex it makes use of the already eager loaded quantities and doesn't hit the db again for each item
I see that it not only eager loads all of the associated quantities each batch
This is expected behaviour (and very likely performant), since each batch you will get different quantities to load, since they are connecting to different items, so you don't need to keep all quantities in memory.
each batch, but it also then loads all quantities per item with a hit to the database again for each item record it is indexing.
This is actually unexpected, but my guess here is, that in one of the methods in Quantity (#location_code #value or #location_bins) or even one of the methods on Item that you call (#standard_part_number, #category_id, #content_source_name, #standard_price), there is some implementation, that requires reloading of records.
Without knowing the code of that methods it is purely speculative, but the presented part of the code looks fine.
So I've read a lot about the rails includes method but I'm still a bit confused about what's the best situation to use it.
I have a situation where I have a user record and then this user is related to multiple models like client, player, game, team_player, team, server and server_center.
I need to display specific attributes from the related models in a view. I only need around 1-2 attributes from a specific model and I don't use the others.
I already added delegates for example to get the server.name from player I can use server_name but in this situation do I include all of the tables from which I need the attributes or is there something else I do because I only need a couple of attributes from the model.
My query is as follows at the moment:
#user_profile = User
.includes({:client => [:player, :team_player => [:team]]},
:game,
{:server_center => :server})
.where(game_id: #master.admin.games)
Includes ensures that all of the specified associations are loaded using the minimum possible number of queries.
Let say we have 2 models named User and Profile :
class User < ActiveRecord::Base
has_one :profile
end
class Profile < ActiveRecord::Base
belongs_to :user
end
If we are iterating through each of the users and display the name of each user were name field resides in Profile model which has a association with User model, we would normally have to retrieve the name with a separate database query each time. However, when using the includes method, it has already eagerly loaded the associated person table, so this block only required a single query.
without includes:
users = User.all
users.each do |user|
puts user.profile.name # need extra database query for each time we call name
end
with includes
# 1st query to get all users 2nd to get all profiles and loads to the memory
users = User.includes(:profile).all
users.each do |user|
puts user.profile.name # no extra query needed instead it loads from memory.
end
Eager Loading is used to prevent N+1 query problems. basically it does left outer join and this plays an important role in speeding up request response or optimizing the queries. eg: if we are having huge amount users and if we want to iterate through those users and their corresponding profile. no of time which we will be hitting database will be equals to number of users. but if we are using includes it will keep all profile into memory later when we iterate through the users it will fetch from this memory instead of querying.
Eager loading may not always be the best the cure for our N+1 queries for eg: if you are dealing with some complex queries preferably looks for some caching solutions like Russian Doll caching etc.. still both method has his own pros & cons end of the day it's up to you to determine the best approach.
one useful gem which helps to detect N+1 query is bullet
I'm using subclasses in mongoid and I would like to eager load relations of both the parent and its subclasses.
class Event
include Mongoid::Document
belongs_to :modifier
end
class Fixture < Event
belongs_to :club
end
When I run Event.includes(:modifier, :club) I get:
Mongoid::Errors::InvalidIncludes:
Problem:
Invalid includes directive: Event.includes(:modifier, :club)
Summary:
Eager loading in Mongoid only supports providing arguments to
Event.includes that are the names of relations on the Event model,
and only supports one level of eager loading. (ie, eager
loading associations not on the Event but one step away via
another relation is not allowed.
Resolution:
Ensure that each parameter passed to Event.includes is a valid name
of a relation on the Event model. These are: "modifier".
The error message is reasonable but I'd like to know if there is a workaround, short of running a separate query on each class?
I'd like to be able to also chain further criteria, i.e. Event.includes(:modifier, :club).desc(:updated_at) rather than sorting the result array in rails.
versions: Mongoid v3.16 & Rails 3.2.15
Edit: I'd better make it clearer what it is I want.
I want all events and all fixtures documents.
I also want all their relations: the modifiers on Events and Fixtures, and the clubs on Fixtures.
And I want those related documents to be retrieved from mongo all at once, i.e. typically done through 'eager loading' using the '.includes()' method .
There will be further subclasses, i.e. class Election < Events, class Seminar < Events. So I'd like to avoid querying each subclass separately.
I'd also like to chain further criteria, such as desc(:updated_at).
I am not exactly sure what you try to do, but I guess this could solve your problem.
Event.where(:modifier_id.ne => nil, :club_id.ne => nil).desc(:updated_at)
Let me know if that works for you.
so far MongoID doesn't support nested document eager loading. you better check out this gem
I'm porting some functionality to Rails, and I'm working with an existing table which is for comments.
Basically, there are two types of comments - profile comments (photo_id column is null) and photo comments (photo_id column is set to photo's ID)
I got single table inheritance working just fine by adding a type field to the table, but I'm wondering if there's a way to get my single table inheritance working without the type field. According to the Rails API documentation, "If you don‘t have a type column defined in your table, single-table inheritance won‘t be triggered. In that case, it‘ll work just like normal subclasses with no special magic for differentiating between them or reloading the right type with find."
I'm wondering if there's a way that I can customize my models to determine type based on photo_id being nil or having an integer value, rather than using the database column (which I'd rather not add if I don't have to.) Any ideas?
If comments models doesn't differ much, I wouldn't bother with single table inheritance at all. Just add:
# to Comment model
belongs_to :photo
belongs_to :profile
# to Profile model
has_many :comments
# to Photo model
has_many :comments
Then:
#photo.comments # will return comments associated with photos
#profile.comments # will return comments associated with profiles
There may be problem if you had both photo_id and profile_id set (I suppose it may happen when you comment a photo that is associated with profile), so you can change in Profile model:
has_many :comments, :conditions => "photo_id is not null"
Another approach (I think better) it to you polymorphic associations but you will need to modify you sql tables.
I suspect you cannot do this trivially. However, one possibility is to trick active record into using a view rather than a table, and write some database functions to set this magic attribute based on which id is set.
However, in the end, I suspect it would be far, far easier to just add the column.