Let's say we have a Post model.
In the beginning of a view I need to display the total number of posts, typically by
#posts.size
Later in the view I need to display all posts, typically by
#posts.each do |post|
end
This results in two queries.
If I had done the queries in the opposite order, it had resulted in one single query (presumably utilizing CACHE).
Is there any "trick" where I can achieve the (first) mentioned order with only one db query?
You can use for example #fragment-caching
<% cache 'posts_size' do %>
<%= #posts.size %>
<% end %>
Then when a Post is created you can expire the fragment
expire_fragment 'posts_size'
Related
I have a table called Person with roughly a million records. I would like to display each of them as a card on the index page, but I am having performance issues. The view is rendered very slowly and is causing significant lag.
I simply had #persons = Person.all but it didn't work since it just crashed my app. Now, I am using the Person.find_in_batches method, which now allows me to display all the records within the page. Not only it takes a long time to load up the records but when I try to click a few buttons on the page, it either freezes or lags heavily. I'm lost on how I can improve performance.
For the record, here is my following code:
Person controller:
def index
batch_size = 5000
#people_in_batches = Person.find_in_batches(batch_size: batch_size)
end
Index view:
<% #people_in_batches.each do |batch|%>
<% batch.each do |person| %>
<div>
<span><%= person.name %></span>
<span><%= person.date_if_birth.strftime("%m/%d/%Y") %></span>
</div>
<% end %>
<% end %>
Thank you!
It doesn't matter how you query the million records, it still a million records.
What you should do is consider adding pagination on your page, so you would only needs to fetch records for the corresponding page. Just like Gmail.
So on the backend, you can use gem like https://github.com/kaminari/kaminari to help you with pagination.
Other than that, you should also consider adding filters on the page too, e.g. Active / Not Active, to reduce the number of records on page load.
From what i understand of Russian doll caching in Rails it would be detrimental to eager load related objects or object lists when we are doing RDC (Russian Doll Caching) because in RDC we just load the top level object from the database, and look up its cached rendered template and serve. If we were to eager load related object lists that will be useless if the cache is not stale.
Is my understanding correct? If yes, how do we make sure that we eager load all the related objects on the very first call so as not to pay the cost of N+1 queries during the very first load (when the cache is not warm).
Correct - when loading a collection or a complicated object with many associations, a costly call to eager load all objects and associations can be avoided by doing a fast, simple call.
The rails guide for caching does have a good example, however, it's split up a bit. Looking at the common use case of caching a collection (ie the index action in Rails):
<% cache("products/all-#{Product.maximum(:updated_at).try(:to_i)}") do %>
All available products:
<% Product.all.each do |p| %>
<% cache(p) do %>
<%= link_to p.name, product_url(p) %>
<% end %>
<% end %>
<% end %>
This (condensed) example does 1 simple DB call Product.maximum(:updated_at) to avoid doing a much more expensive call Product.all.
For a cold cache (the second question), it is important to avoid N+1's by eager-loading associated objects. However, we know we need to do this expensive call because the first cache read for the collection missed. In Rails, this is typically done using includes. If a Product belongs to many Orders, then something like:
<% cache("products/all-#{Product.maximum(:updated_at).try(:to_i)}") do %>
All available products:
<% Product.includes(:orders).all.each do |p| %>
<% cache(p) do %>
<%= link_to p.name, product_url(p) %>
Bought at:
<ul>
<% p.orders.each do |o| %>
<li><%= o.created_at.to_s %></li>
<% end %>
</ul>
<% end %>
<% end %>
<% end %>
In the cold cache case we still do a cache read for collection and each member, however, in the partially warm cache case, we will skip rendering for a portion of the members. Note that this strategy relies on a Products associations being correctly set up to touch when associated objects are updated.
Update: This blog post describes a more complex pattern to further optimize building responses for partially cached collections. Instead of rebuilding the entire collection, it bulk fetches all available cached values then does a bulk query for the remaining values (and updates the cache). This is helpful in a couple of ways: the bulk cache read is faster than N+1 cache reads and the bulk query to the DB to build the cache is also smaller.
My way
controller pattern 1 (note: Here, it's calling all users!!)
#users = User.confirmed.joins(:profile)
view pattern 1 (note: Here, it only shows first 10 users but it show the number of all users!!)
<%= "ALL ("+ #users.count.to_s + " users)" %>
<% #users.limit(10).each do |users| %>
<%= render 'users/user', :user => users %>
<% end %>
Should it be just like this below if I'm considering page loading speed?
Or it won't be changed?
controller pattern 2 (note: I added limit(10), and #users_count to count all users)
#users = User.confirmed.joins(:profile).limit(10)
#users_count = User.confirmed.joins(:profile).count
view pattern 2 (note: I took it off limit(10) and use #users_count for count)
<%= "ALL ("+ #users_count.to_s + " users)" %>
<% #users.each do |users| %>
<%= render 'users/user', :user => users %>
<% end %>
If you have lazy loading disabled, then the second approach would be faster because Rails doesn't need to fetch all records from the database. You should really fetch only the records you need when performing queries.
If you have lazy loading enabled (by default), then it is the same, because the data is fetched when it is needed, so the effect will be the same. You can also put two variables in controller and write the same query as you did in the view and the data will be fetched only if and when it is needed.
#users = User.confirmed.joins(:profile)
#users_count = #users.count
#users = #users.limit(10)
You can check sql generated by the app in your rails console and then decide.
Also, if you are using profile in user.html.erb, consider using includes instead of join. Join can cause n+1 problem if you need associated records. If you don't, you do not want to fetch records you don't need. You can read more about it here, in 12 Eager Loading Associations.
The two options are exactly the same. Neither of them loads all the Users because you're just chaining scopes. The query is only run when you call .each in the view, at which point you've applied the .limit(10) anyway. I'd go with the first option because the code is cleaner.
#users.count does one query to get the count, it doesn't instantiate any User objects.
#users.limit(10).each ... does one query (actually two because you've used includes) with a limit, so it will instantiate 10 objects plus your includes.
you can try #users.find_in_batches
Please take a look
Find in batches
Please let me know
If you want speed loading
I can suggest you memcache Memcache
Using Rails 3.1.1 and Heroku.
I believe this should be a fairly easy fix but I cannot find (and easily verify) how to do this. I have a very slow controller (6 sec) Product#show, with lots of N+1 and other things I will have to solve.
The website is a two-column website (main-column and right-column) where the main content from Product#show is shown in one column and daily product are shown in the other, including a "Random Product from the Database".
What I want to do is to let the content in main-column that is created by Product#show be cached (and thus bypass the controller and win 6 seconds). I do, however, want the right column to be dynamic (and loaded for each page request).
If I use caches_page :show it will cache the entire website, including the right-column, which makes me have to expire the cache every day in order to be able to load a new Daily Product. Not a good solution.
If I use cache('product-show' + #product.slug) do it only caches the view (right?) and still have to go through the controller.
So, how can I solve this?
You can achieve this with fragment caching like below:
def show
if !fragment_exist?("main_content")
#products = Product.all
#users_count = User.count
end
#random_products = Product.order("RANDOM()").limit(10)
end
show.html.erb
<!--MAIN CONTENT-->
<% cache("main_content") do %>
<%= #users_count %>
<% #products.each do |product| %>
<%= product.name %>
<% end %>
<% end %>
<!--SIDE CONTENT-->
<% #random_products.each do %>
<%= product.name %>
<% end %>
Use fragment caching, and don't load things in the controller.
If you have a very complex query, let it live in the controller as a scope, and only evaluate it in the view.
If you have a complex process to do so the query must be executed, use a helper method.
If you manage to just load lazy queries in the controller, if the cache is hit none of them will be executed.
I have an object called #events containing about 50 records being pulled from a find condition of my model.
I'm currently displaying the results of the #object in my view like this....
<% for event in #events %>
<p><%= #event.name %></p>
<% end %>
Instead of displaying the entire 50 I would like shrink the set to about 10 records so it displays better on the page.
I cannot use :limit in the find condition since the object is being composed from a variety of loops where after each iteration it adds a few specific records.
So the issue is I have this object #events with 50 records, how can I change the object after its been composed so only the first 10 records remain?
First of all, if you'd like to have pagination, I strongly suggest taking a look at will_paginate
Alternatively, you can do the following to read the first 10 records only.
<% #events.first(10).each do |event| %>
<p><%= event.name %></p>
<% end %>
Or the last 10 records
<% #events.last(10).each do |event| %>
<p><%= event.name %></p>
<% end %>
I didn't test it but you get the point.
are you looking to completely do away with the other 40 or are you just wanting to pull off 10 per page for display purposes. if you are just doing this for display purposes i would look into the will_paginate gem. through its options you could set it so only 10 results per page are shown.
Take a look at will_paginate and kaminari. They both are designed to limit the records retrieved from the database, plus offer helpers for your views to provide the usual number of pages and current page lists.
Will_paginate has been around a while, and is pretty flexible. Kaminari is newer and looks like it has a cleaner interface.