Imagine a simple database where students have multiple courses and multiple exams, given a list of students already loaded from the db, I want to populate the lists of courses and exams with a single database call for each.
I know I can use Include up front which results in a single call to retrieve everything:
var students = context.Students.Include("Courses").Include("Exams").ToList();
but I also need to be able to load the courses and exams at a later time.
I tried:
var courses = students.SelectMany(x => x.Courses).ToList();
var exams = students.SelectMany(x => x.Exams).ToList();
but this resulted in two db calls for each student. How can I achieve this more efficiently?
From my experience, you cannot load multiple EntityCollections simultaneously. You provided the two options in your question. You can either eagerly load the related entities in a single query or you can load them later via a query for each EntityCollection to load.
Make sure you turn lazy loading off for your entity framework model, IIRC it'll cause those collection properties to fire queries whenever you iterate over them. Otherwise it looks like it should work with just one query.
Related
I have three models, let's call them Product, TemplateProduct and ReadyProduct. There is only one table for these, the products table, and both TemplateProduct and ReadyProduct inherit from the Product model. There is a has_many/belongs_to association between TemplateProduct and ReadyProduct. Templates are used to lay out general characteristics for products, Readys are used to customize the products and what are actually made available for view by the customer. Each TemplateProduct has an id and each ReadyProduct has a template_product_id which ties to it's template.
The project is built using Rails 5.
What I want to be able to do is to gather a list of TemplateProducts, then get a count of each templates associated ReadyProducts and do so in such a way that won't hammer the database. I understand ActiveRecord associations but my SQL is weak and I have only a limited understanding of joins. I can gather a list of TemplateProducts with a simple Product.where(conditions) but I don't know what to once I have this. For the sake of flexability sake I want to be able to base my ReadyProduct count off of this initial collection as sometimes I'll need the additional count and sometimes I won't. I'm sure there must be a simple way to do this but I haven't found a solution.
If you just need a mapping of TemplateProduct ids to ReadyProduct counts, then all you need is:
TemplateProduct.joins(:ready_products).group(:id).count
If you want TemplateProduct instances with a baked in ReadyProduct count, then you'll need this instead:
tps = TemplateProduct.joins(:ready_products).select('products.*, COUNT(ready_products_products.id) ready_product_count').group(:id)
tps.first.ready_product_count
#=> 6
ready_products_products gets defined by Rails, it prefixes the actual table name (products) with the model name's "table form" (pluralized, snake case, lower case, ready_products), joined with an underscore.
I received some really good help in solving an issue in which I needed to get objects from objects in one query. That worked like a charm.
This is a follow up to that question. I chose to create a new question to this since the last one was answered according to my previous specification. Please do refer to that previous question for details.
Basically, I wanted to get all the objects of multiple objects in one single query. E.g. if a Product has several Categories which in turn has several Products, I want to get all the Products in that relation, easier (and erronously) put:
all_products = #my_product.categories.products
This was solved with this query. It is this query I would (preferably) like to alter:
Product.includes(:categories).where(categories: { id: #my_product.categories.pluck(:id) } )
Now, I realized something I missed using this solution was that I only get a list of unique Products (which one would expect as well). I would however like to get a list with possible duplicates as well.
Basically, if a "Blue, Electric Car" is included in categories ("Blue", "Electric" and "Car") I would like to get three instances of that object returned, instead of one unique.
I guess this does not make Rails-sense but is there a way to alter the query above so that it does not serve me a list of unique objects in the returned list but rather the "complete" list?
The includes method of AREL will choose between two strategies to make the query, one of which simply does two distinct query and the other one does an INNER JOIN.
In both cases the products will be distinct.
You have to do manually a right outer join:
Product.joins('RIGHT JOIN categories ON categories.product_id = products.id').where(categories: { id: #my_product.categories.pluck(:id) } )
adds also .preload(:categories) if you want to keep the eager loading of the categories.
Since you want duplicates, just change includes to joins, (I tested this just now). joins will essentially combine (inner-join) the two tables giving you a list of records that are all unique (per Product and Category). includes does eager loading which just loads the associated tables already but does an outer-join, and therefore, the retrieved records are also unique (but only per Product).
Product.joins(:categories).where(categories: { id: #my_product.categories.pluck(:id) } )
I have a scaffolded Grails application with two domains, Person and Course. Person belongs to Course, and Course hasMany Persons. I have modified show.gsp for Course to list all of the Persons associated with the selected Course.
To achieve this, Course.groovy contains the following line:
List persons = new ArrayList()
And, as a result, the "person" database table contains a persons_idx field. I frequently will be adding new data to the "person" table outside of my Grails application, from an external website.
When INSERTing new data, how to I figure out what to set persons_idx as?
I had originally used a SortedSet instead of an ArrayList for persons, since I care about sorting. But since I am sorting on Person.lastName, and there will always be multiple people with the same last name, then the list will exclude those persons who have the same last names as others. I wish there was another way...
Thanks.
Having two applications manipulate the same Database is a thing to avoid, when possible. Can your 2nd application instead call an action on the controlling app to add a Person to the Course with parameters passed to specify each? That way, only one app is writing to the DB, reducing caching, index, and sequence headaches.
You also state that Person belongsTo Course... so you create a new Person for "Bob Jenkins" for each course that he's in? This seems excessive. You should probably look into a ManyToMany for this.
Without moving to a service, unfortunately, you'd want to change the indices on some if not many of the rows for the children of the Course you're trying to add a Person to, as that index is the sorted index for all the Persons in the Course.
I would suggest going back to a "Set", and do your sorting in the app. Your other question about sorting already told you not to override compareTo to just check the last name. If I were you, I'd forget about overriding compareTo at all (except to check IDs, if you want), and just use the sort() method, passing in a closure that correctly sorts the objects.
I watched this rails cast http://railscasts.com/episodes/22-eager-loading but still I have some confusions about what is the best way of writing an efficient GET REST service for a scenario like this:
Let's say we have an Organization table and there are like twenty other tables that there is a belongs_to and has_many relations between them. (so all those tables have a organization_id field).
Now I want to write a GET and INDEX request in form of a Rails REST service that based on the organization id being passed to the request in URL, it can go and read those tables and fill the JSON BUT NOT for ALL of those table, only for a few of them, for example let's say for a Patients, Orders and Visits table, not all of those twenty tables.
So still I have trouble with getting my head around how to write such a
.find( :all )
sort of query ?
Can someone show some example so I can understand how to do this sort of queries?
You can include all of those tables in one SQL query:
#organization = Organization.includes(:patients, :orders, :visits).find(1)
Now when you do something like:
#organization.patients
It will load the patients in-memory, since it already fetched them in the original query. Without includes, #organization.patients would trigger another database query. This is why it's called "eager loading", because you are loading the patients of the organization before you actually reference them (eagerly), because you know you will need that data later.
You can use includes anytime, whether using all or not. Personally I find it to be more explicit and clear when I chain the includes method onto the model, instead of including it as some sort of hash option (as in the Railscast episode).
Is there a way to do a second query and have that replicate the functionality of eager loading? Lets say a real basic example. A Person has many Cars and a Car has many Parts..
Typically to eager load you would say:
person.cars.includes(:parts)
I don't really understand what this does.. does it just load those other objects in memory? Are they associated with each car at this point in a more substantial way then just sharing keys? If its just sharing keys.. could I load these 'Parts' in memory and have them accessed without calling the 'includes' explicitly? For example.. something like:
cars = person.cars
cars_id_array << insert cars ids here
parts = Parts.where(:user_id => person_id, "car_id IN cars_id_array") (just an example)
In this case (assuming i did the write code which I'm sure I didn't.. could I go car1.parts, car2.parts and would those parts be already loaded in memory?
If not, what is eager loading doing?
Eager loading is doing something very similar to what you are doing, but in addition to simply querying the database, it's also establishing relationships. When you retrieve a model from the database that has related models, the related models are generally loaded on demand. The example from the guides is:
clients = Client.limit(10)
clients.each do |client|
puts client.address.postcode
end
When you do the first query, there's a call to the database to grab ten clients. As you iterate through them to get the postcode that is associated with a client address (which presumably lives in a different table), you'll need to do 10 more queries to grab the proper addresses. By using eager loading, you can do the whole thing in just two queries (which are handled under the covers).
clients = Client.includes(:address).limit(10)
This grabs the ten clients, then does another query (similar to yours) to grab the addresses that are in the array of client_id's. These are then resident in memory, with relationships in tact, so you don't need to make another database call to get them.
If you tried to replicate this by just loading them seperately:
clients = Client.limit(10)
addresses = Address.where("client_id in [my ID array]")
clients.each do |client|
puts client.address.postcode
end
You're still going to be hitting the database for each of the addresses, since those objects aren't associated with each other except through the database id's. The objects are loaded, but the relationships aren't created between them.