Background:
I'm creating a dashboard as a project and I have a query that I think is going to be a big performance issue:
<% for outlet in #outlets %>
<% if Monitoring.where(:outlet_id => outlet.id).where('date(created_at) = ?', Date.today).exists? %>
<li>
<a class="done" href="<%= outlet_url(outlet) %>" rel="tooltip" title="<%= outlet.name %>"></a>
</li>
<% else %>
<li>
</li>
<% end %>
<% end %>
What I'm trying to achieve is a series of dots on a page. If the anchor tag has a class of done, it will display as green, if not it will be red (done through CSS).
Aside from the obvious DRY issues here, this query is very heavy, so I'm looking at ways to improve it.
An Outlet is Monitored at least once a day (An Outlet has_many :monitorings). For each outlet I need to check if it has been monitored on that particular day, and output the HTML accordingly.
If anyone could help me with this it would be fantastic.
(Also, any advice on caching this would be appreciated).
Cheers in advance :).
You might make a conditioned association for current monitors, then use includes to fetch the associated current monitorings on the original query.
class Outlet
has many :current_monitorings, :class_name => "Monitoring",
:conditions => proc { [ 'monitorings.created_at > ?', Time.now.midnight ] }
end
#outlets = Outlet.includes(:current_monitorings)
#outlets.each do |outlet|
if outlet.current_monitorings.empty?
# the no monitor today case
else
# it's been monitored today
end
end
At the Postgres level, you'll likely benefit from an index on monitorings(outlet_id,created_at) to support the outer join implied by #includes.
BTW, it's bad style to be executing database queries in your view. Put the domain logic in your models, and have your controller execute the query and supply the results to the presentation layer.
Maybe try:
<% #outlets.includes(:monitorings).each do |outlet| %>
<% css_class = outlet.monitorings.any? { |m| m.created_at == Date.today } ? 'done' : '' %>
<li><%= link_to '', outlet_url(outlet), :class => css_class, :rel => "tooltip", :title => outlet.name %></li>
<% end %>
It'll perform 1 big query.
A good way to cache this would be denormalizing your database with the use of callbacks. In your Outlet model, you could add a field called last_monitored_on; anytime a Monitor is saved, update the appropriate Outlet model with the date. Then, you wouldn't have to query Monitors at all.
You could also consider caching that page fragment, and let it expire daily.
Related
My portfolio_controller.rb has an index method like this:
def index
#portfolio = PortfolioItem.all
end
How can I specify in the condition that the code in this block should be executed 6 times? In other words, how can I access exactly 6 values from the #portfolio object in my view, using a loop? This is what I have so far:
<% #portfolio.shuffle.each do |portfo| %>
Using all, followed by shuffle, is a bad solution for two reasons.
A slight improvement would be to use sample(6) instead of shuffle.first(6), as this removes a step from the process.
However, the bigger issue here is that Portfolio.all.<something> (where the <something> method requires converting the data into a ruby Array) will fetch all of the data into memory - which is a bad idea. As the table grows, this will become a bigger performance issue.
A better idea is to perform the "random selection" in SQL (with the order and limit methods), rather than in ruby. This avoids the need to fetch other data into memory.
The exact solution is database-specific, unfortunately. For PostgreSQL and SQLite, use:
Portfolio.order('RANDOM()').limit(6).each do |portfolio|
Or for MySQL, use:
Portfolio.order('RAND()').limit(6).each do |portfolio|
You could define this as a helper in the Portfolio model - for example:
class Portfolio < ApplicationRecord
# ...
scope :random_sample, ->(n) { order('RANDOM()').limit(n) }
# ...
end
And then in your view:
#portfolio.random_sample(6).each do |portfolio|
You can something like this :
<%(1..6).each do |i| %>
<% #your statements %>
<%end%>
<% #portfolio.shuffle.each_with_index do |portfo, index| %>
<%if index <= 6%>
<p><%= portfo.title %></p>
<%end%>
<% end %>
Or You can do it as
<% #portfolio.shuffle.take(6).each do |portfo| %>
<p><%= portfo.title %></p>
<% end %>
I have four different Rails-apps (ruby 2.3.3/rails 4.1.13) on Heroku running on Unicorn (1 worker per app).
Two of them have a few thousand visitors per day, the other two around hundred a day. They all have a major issue in common: They are always running out on memory on Heroku! They are almost always above 100% of the 500 Mb limit on Heroku, thus using the slow Swap-memory. As all 4 share the same issue, I believe there is something in my programming habits that cause this - and possibly the way I render sub-items in partial lists (specifically after reading this question). I would like to hear whether this code is likely to bloat memory in my apps:
I have three files (the code obviously is quite simplified):
#show
render 'partials/product_list', :vars => { :products => Product.where(:foo => "bar"), :display_format => :grid }
#partials/product_list
<% if vars[:products].empty? %>
No products exist
<% else %>
<% if vars[:display_format].eql?(:grid) %>
<div class="abc">
<% vars[:products].each do |product|
<%= render 'partials/product_item', :product => product %>
<% end %>
</div>
<% elsif vars[:display_format].eql?(:wide) %>
<ul>
<% vars[:products].each do |product|
<li><%= render 'partials/product_item', :product => product %></li>
<% end %>
</ul>
<% elsif vars[:display_format].eql?(:rectangular) %>
<div class="ghi">
<ul class="rectangular_container">
<% vars[:products].each do |product|
<li class="rectangle"><%= render 'partials/product_item', :product => product %></li>
<% end %>
</ul>
</div>
<% else %>
<div class="jkl">
<% vars[:products].each do |product|
<%= render 'partials/product_item', :product => product %>
<% end %>
</div>
<% end >
#partials/product_item
<% if vars[:display_format].eql?(:wide) %>
<h1><%= product.name %></h1>
<p><%= product.description %></p>
<% elsif vars[:display_format].eql?(:rectangular)%>
// Similar but with lots of divs and other html
<% end %>
This may seem very weird but I reference partials/product_list from here and there all over the website (with different layouts and different product sets) and if I want to change the setup of e.g. grid-layout, I want to do it in one place only.
I have started using references in #show with Product.all.pluck(:id) and in partials/product_item start with product = Product.find(vars[:product]) but I can't really tell if this is making any difference. Edit: As max says in the comments, this is probably less efficient due to the amount of .find-calls I need to do.
Before I dig really deep into this, I have a few questions:
Is this something that strikes you as: "YES, this type of rendering will bloat your memory!"
If yes: How should I solve this type of rendering without memory problems?
Would there be a difference if I would use :collection, :layout etc instead of my own variables?
Any light on this issue would be highly appreciated!
One issue that stands out to me is it looks like you're doing some lookups in your views and breaking the MVC pattern. In my opinion, you should put that code into the controller for that action
so rewrite this line:
render 'partials/product_list', :vars => { :products => Product.where(:foo => "bar"), :display_format => :grid }
to this:
# in show controller action
#products = Product.all # <- This will get cached by rails
# in view
render 'partials/product_list', :vars => { :products => #products, :display_format => :grid }
Removing the Product call in the view and replacing it an instance variable that rails will cache. Keep in mind default rails cache is memory based. Which may be causing some of the bloat as your data grows. If you have memcache available perhaps changing your cache store to memcache would help too.
You can just chain a where clauses to your instance variable like you do now and the AREL will handle it.
# partial rendering with chained scope
render 'partials/product_list', :vars => { :products => #products.where(foo: "bar"), :display_format => :grid }
If you are finding that further filtering of items is becoming a common pattern, then you can use a scope in your Products model that will be chainable on the #products instance variable. This will keep it in AREL (SQL) and maintain the lazy loading/query caching while simplifying your code a bit.
# in Products model
scope :filter_to_bar, -> (term) {where(foo: term)}
# partial rendering with chained scope
render 'partials/product_list', :vars => { :products => #products.filter_to_bar("bar"), :display_format => :grid }
Personally I say keep your data fetches in SQL when you can, as this puts the strain on SQL and is lazy loaded, i.e. doesn't affect ruby memory. SQL done properly is fast, ruby is much slower and more memory intensive. SQL is only slow when your queries are O^n or you're making repeated small queries and putting unneccessary hits on the database. In that case, fetch the data and work with it on the rails application as suggested. While this will add to the memory consumption of your application is consuming, it will be more performant.
One thing you can do to speed up SQL fetches is to implement pagination tables or some sort of infinite scroll mechanism. This way you only load the data you need when you need it.
Also if you want a faster version of rails' partial pattern, check out the cells gem as suggested. It will require some refactoring but should be faster than rails' pure partials. However, caching will probably take you much further with less refactoring.
I have the following view to show the categories with the count
<% #categories.each do |category| %>
<% category.sub_categories.sort.each do |sub_category| %>
<li><%= link_to sub_category.name, "category/#{sub_category.slug}", title: sub_category.name.capitalize %> <%= sub_category.posts.where(status: 1).count %></li>
<% end %>
<% end %>
But I dont think using where in view is not good idea. Is there any other way to perform such operation.
I am getting correct count but I need better way to do this. can anyone help
Your Post model should have a scope on it that encapsulates this status logic:
class Post < ActiveRecord::Base
def self.active
where(status: 1)
end
end
Then call it like so from the view:
sub_category.posts.active.count
use scope to do the same thing, your solution is ok otherwise. you don't need to do this controller.
scope :active_posts, lambda{ where(status: 1)}
The only problem I see with this is it causes N+1 queries, because you do 1 query for the categories THEN another query for EACH category. This is ok with small quantities, however can cause serious performance problems later on.
1) I recommend you look into "counter_cache"ing:
- http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html#method-i-belongs_to
2) OR upon activation/deactivation of Posts, create a method that will increment/decrement a attribute on the Post's Category (i.e. "active_posts")
I've just started programming in rails 3 days ago, learned ruby by the same time, and I'm having a hard time since yesterday figuring how to with one form, delete/update some of my instances, I only use one model "Task". Here's the code:
<%= form_for #task do |f| %>
<ul>
<% #tasks.each do |task| %>
<li id="task"><%= f.check_box :done %> <%= f.label :name %> </li>
<% end %>
<button onclick="doUpdate()">Mark Selected as done </button>
<%= button_to "Delete selected", :method => :delete %>
</ul>
<% end %>
Here's the controller:
def delete
#tasks = Task.find(:all, :conditions => ["task.done = ?", true])
#tasks.each do |task|
task.delete
end
#tasks = Task.all
end
My model have only 2 parameters. name:String and done:Boolean, I wan't to delete all the selected checkboxes. But this don't work for me
Thanks in advance.
The problem is, you are doing it wrong(and I'll tell you why). I could paste the code that would make it work but I'd rather explain, as you are probably doing it to learn.
Task.find(:all, :conditions => ["done = ?", true]) will return EVERYTHING in your database where done = true. You will be erasing everything that is marked as done in the DATABASE, not what were marked on the form. Task is your model, you can access the database by using find, where and other methods from activerecord(if activerecord doesn't sound natural to you, activerecord lets you get stuff from the database without the need of writing SQL queries).
What you really need to do in your controller is:
- You have to get what was sent from the form (check the documentation/web resources for the usage of param[] ).
- For every checkbox marked true, you erase a record. (you got the each part right, this is good!)
I don't think your view is right, I advise you to first be sure that the data that you receive is right(the params[]), then proceed to try to erase the record, or do whatever you want to do with it.
To "test" if your variables and code that is inside your controllers and models, use print #variable or something else(check rails docs how to debug).
I advise you use destroy instead of delete as other fellow stackoverflowers have said. Read the docs of destroy and delete.
Keep going :)
On a first glance, try using task.destroy instead of task.delete, and done instead of task.done See delete vs. destroy.
def destroy
#tasks = Task.find(:all, :conditions => ["done = ?", true])
#tasks.each do |task|
task.destroy
end
#tasks = Task.all
end
I'm having a tough time deciding how to refactor this method in my controller. The idea is that (in this case) it graphs the users that joined (or were created) in the past two weeks.
You might be wondering why I did the #graph_limit thing, and that is because I always want the day that has the most results to be the tallest bar on my bar chart (which in the view are just created with css by making the height of the <div> using css).
Basically I want to dry it up and... ya know just about improve this method as much as possible:
# Controller
def index
two_weeks_ago = Date.today - 13.days
#users_graphed = User.count(:conditions=>["created_at >= ?", two_weeks_ago], :order => 'DATE(created_at) DESC', :group => ["DATE(created_at)"])
two_weeks_ago.upto(Date.today) do |day|
#graph_limit = 100/#users_graphed.values.max.to_f
#users_graphed[day.to_s] ||= 0
end
end
Also I should mention, that you guys are probably going to rip my code to shreds... so I'm bracing for the outcome.
# View
<% #users_graphed.sort.reverse.each do |user| %>
<li>
<% content_tag :div, :style => "height: #{number_with_precision(user[1] * #graph_limit, :precision => 2)}px; ", :class => "stat_bar" do %>
<%= content_tag(:span, user[1]) unless user[1] == 0 %>
<% end %>
</li>
<% end %>
Ultimately and what my real goal here is to put this into my application controller and be able to chart any models by it's create_at times. maybe something like tasks.chart_by(2.weeks). How would you guys get this separated out into something I can use throughout the whole app?
I agree with Joseph that your controller here is doing a lot of work that should be done in the model. Any time you're specifying multiple find parameters in your controller, ask yourself whether or not that should be in your model instead.
You're doing a lot of iterating here that seems needless. Firstly, You shouldn't be calculating #graph_limit inside the loop. You're recalculating it 14 times, but the value is going to be the same every time. Do that outside the loop.
Secondly, that sort.reverse in your view sticks out. You're already sorting in your finder (:order => 'DATE(created_at) DESC'), and then you're sorting again in your view and then reversing it? You should instead be asking the database for the values in the final order you want them. Then to make your zero-filling code work you can just reverse it, doing Date.today.downto(two_weeks_ago) instead of upto.
I would say that you should really be doing this all in SQL, but unfortunately (as perhaps you've discovered) MySQL makes it difficult to fill in missing days without creating a calendar table to join against.
Thanks Jordan, per your ideas (which were really great by the way) I've created a helper that is like such:
def graph_by_time(klass, time_ago)
time_range_start = Date.today - time_ago
#elements_graphed = klass.count(:conditions=>["created_at >= ?", time_range_start], :order => 'DATE(created_at) DESC', :group => ["DATE(created_at)"])
#graph_limit = 100/#elements_graphed.values.max.to_f
time_range_start.upto(Date.today) do |element|
#elements_graphed[element.to_s] ||= 0
end
return #elements_graphed.sort.reverse
end
The biggest issue here is zero filling the days which have no records associated with them, your method of switching to from upto to downto didnt work and only returned the records which did result in a integer other than zero.