What is the best way to profile a controller action in Ruby on Rails. Currently I am using the brute-force method of throwing in puts Time.now calls between what I think will be a bottleneck. But that feels really, really dirty. There has got to be a better way.
I picked up this technique a while back and have found it quite handy.
When it's in place, you can add ?profile=true to any URL that hits a controller. Your action will run as usual, but instead of delivering the rendered page to the browser, it'll send a detailed, nicely formatted ruby-prof page that shows where your action spent its time.
First, add ruby-prof to your Gemfile, probably in the development group:
group :development do
gem "ruby-prof"
end
Then add an around filter to your ApplicationController:
around_action :performance_profile if Rails.env == 'development'
def performance_profile
if params[:profile] && result = RubyProf.profile { yield }
out = StringIO.new
RubyProf::GraphHtmlPrinter.new(result).print out, :min_percent => 0
self.response_body = out.string
else
yield
end
end
Reading the ruby-prof output is a bit of an art, but I'll leave that as an exercise.
Additional note by ScottJShea:
If you want to change the measurement type place this:
RubyProf.measure_mode = RubyProf::GC_TIME #example
Before the if in the profile method of the application controller. You can find a list of the available measurements at the ruby-prof page. As of this writing the memory and allocations data streams seem to be corrupted (see defect).
Use the Benchmark standard library and the various tests available in Rails (unit, functional, integration). Here's an example:
def test_do_something
elapsed_time = Benchmark.realtime do
100.downto(1) do |index|
# do something here
end
end
assert elapsed_time < SOME_LIMIT
end
So here we just do something 100 times, time it via the Benchmark library, and ensure that it took less than SOME_LIMIT amount of time.
You also may find these links useful: The Benchmark.realtime reference and the Test::Unit reference. Also, if you're into the 'book reading' thing, I picked up the idea for the example from Agile Web Development with Rails, which talks all about the different testing types and a little on performance testing.
There's a Railscast on profiling that's well worth watching
http://railscasts.com/episodes/98-request-profiling
You might want to give the FiveRuns TuneUp service a try, as it's really rather impressive. Disclaimer: I'm not associated with FiveRuns in any way, I've just tried this service out.
TuneUp is a free service whereby you download a plugin and when you run your application it injects a panel at the top of the screen that can be expanded to display detailed performance metrics.
It gives you some nice graphs, including one that shows what proportion of time is spent in the Model, View and Controller. You can even drill right down to see the individual SQL queries that ActiveRecord is executing if you need to and it can show you the underlying database schema with another click.
Finally, you can optionally upload your profiling data to the FiveRuns site for community performance analysis and advice.
This works in Rails 4.2.6:
o=OpenStruct.new(logger: Rails.logger)
o.extend ActiveSupport::Benchmarkable
o.benchmark 'name' do
# ... your code ...
end
Related
I want to know how much memory is consumed if I create a ruby object. Does Ruby have any method to tell?
Is there any difference for memory consumption in the following?
users = User.where("created_at > ?", 2.months.ago) # select all fields
users = User.select(:user_name).where("created_at > ?", 2.months.ago) # just select one field
You could use ruby-prof, a wonderful ruby profiler that will tell you everything your code is doing, including memory allocation. The usage is really simple:
require 'ruby-prof'
# Profile the code
result = RubyProf.profile do
...
[code to profile]
...
end
# Print a flat profile to text
printer = RubyProf::FlatPrinter.new(result)
printer.print(STDOUT)
It can output results as text, text graph, html graph, call stack and more. In the readme there is also a section about profiling rails applications. The installation is immediate, so give it a try:
gem install ruby-prof
Firstly there's no easy way to measure how much memory an object consumes. If it's a Rails App you can use this Unix Script to check. Also there's a great blog post that may help you about this issue.
In your second question. The 2nd query is probably gonna consume less memory since ActiveRecord isnt processing all the fields to build an AR object. Ultimately it's better to use .pluck for you second query.
users = User.where("created_at > ?", 2.months.ago).pluck(:user_name)
I've got a Document model basically, which has_many Pages, and I have a view where I need to enumerate a bunch of documents (eg. 300) and make a button for each page. (I want to do pagination client side using the DataTables jQuery plugin so that the table can be sortable and searchable). The problem is that if I try to enumerate all the buttons for each Page in each Document, it takes over 10 seconds to render which is just not useful.
Is there any 'trick' to doing this kind of nested collection rendering fast? Should I just cache the fragments for each document (they don't change much once they're created)? Or is this just a bad situation for Rails partials and would my best bet be to do some client side rendering as part of the pagination in DataTables?
Edit: I'm already including associations so that I don't have an N+1 query problem, that's not the issue. And I tried caching, and it seems like that might be my solution for now because this index page gets reloaded often between every few documents being added, so it never has to rebuild the full cache of all the partials.
The immediate need for caching seems like a code smell. In lieu of guessing, have you tried profiling? (e.g. http://hiltmon.com/blog/2012/02/27/quick-and-dirty-rails-performance-profiling/)
gem 'ruby-prof'
Define a profiler:
# /lib/development_profiler.rb
class DevelopmentProfiler
def self.prof(file_name)
RubyProf.start
yield
results = RubyProf.stop
# Print a flat profile to text
File.open "#{Rails.root}/tmp/performance/#{file_name}-graph.html", 'w' do |file|
RubyProf::GraphHtmlPrinter.new(results).print(file)
end
File.open "#{Rails.root}/tmp/performance/#{file_name}-flat.txt", 'w' do |file|
# RubyProf::FlatPrinter.new(results).print(file)
RubyProf::FlatPrinterWithLineNumbers.new(results).print(file)
end
File.open "#{Rails.root}/tmp/performance/#{file_name}-stack.html", 'w' do |file|
RubyProf::CallStackPrinter.new(results).print(file)
end
end
end
Wrap your view-logic with...
DevelopmentProfiler::prof("render-profiling") do
# Your slow code here
end
Edit - An additional thought: Because your model's data is not likely to change, it may be better to eat the rendering cost, once. You could statically-generate the whole rendered page in an after_save callback. Then just serve that single file for subsequent requests.
Though if charging a more traditional cache isn't a huge inconvenience, this may be more trouble than it's worth.
What I'm doing
I'm using the twitter gem (a Ruby wrapper for the Twitter API) in my app, which is run on Heroku. I use Heroku's Scheduler to periodically run caching tasks that use the twitter gem to, for example, update the list of retweets for a particular user. I'm also using delayed_job so scheduler calls a rake task, which calls a method that is 'delayed' (see scheduler.rake below). The method loops through "authentications" (for users who have authenticated twitter through my app) to update each authorized user's retweet cache in the app.
My question
What am I doing wrong? For example, since I'm using Heroku's Scheduler, is delayed_job redundant? Also, you can see I'm not catching (rescuing) any errors. So, if Twitter is unreachable, or if a user's auth token has expired, everything chokes. This is obviously dumb and terrible because if there's an error, the entire thing chokes and ends up creating a failed delayed_job, which causes ripple effects for my app. I can see this is bad, but I'm not sure what the best solution is. How/where should I be catching errors?
I'll put all my code (from the scheduler down to the method being called) for one of my cache methods. I'm really just hoping for a bulleted list (and maybe some code or pseudo-code) berating me for poor coding practice and telling me where I can improve things.
I have seen this SO question, which helps me a little with the begin/rescue block, but I could use more guidance on catching errors, and one the higher-level "is this a good way to do this?" plane.
Code
Heroku Scheduler job:
rake update_retweet_cache
scheduler.rake (in my app)
task :update_retweet_cache => :environment do
Tweet.delay.cache_retweets_for_all_auths
end
Tweet.rb, update_retweet_cache method:
def self.cache_retweets_for_all_auths
#authentications = Authentication.find_all_by_provider("twitter")
#authentications.each do |authentication|
authentication.user.twitter.retweeted_to_me(include_entities: true, count: 200).each do |tweet|
# Actually build the cache - this is good - removing to keep this short
end
end
end
User.rb, twitter method:
def twitter
authentication = Authentication.find_by_user_id_and_provider(self.id, "twitter")
if authentication
#twitter ||= Twitter::Client.new(:oauth_token => authentication.oauth_token, :oauth_token_secret => authentication.oauth_secret)
end
end
Note: As I was posting this, I noticed that I'm finding all "twitter" authentications in the "cache_retweets_for_all_auths" method, then calling the "User.twitter" method, which specifically limits to "twitter" authentications. This is obviously redundant, and I'll fix it.
First what is the exact error you are getting, and what do you want to happen when there is an error?
Edit:
If you just want to catch the errors and log them then the following should work.
def self.cache_retweets_for_all_auths
#authentications = Authentication.find_all_by_provider("twitter")
#authentications.each do |authentication|
being
authentication.user.twitter.retweeted_to_me(include_entities: true, count: 200).each do |tweet|
# Actually build the cache - this is good - removing to keep this short
end
rescue => e
#Either create an object where the error is log, or output it to what ever log you wish.
end
end
end
This way when it fails it will keep moving on to the next user but will still making a note of the error. Most of the time with twitter its just better to do something like this then try to do with each error on its own. I have seen so many weird things out of the twitter API, and random errors, that trying to track down every error almost always turns into a wild goose chase, though it is still good to keep track just in case.
Next for when you should use what.
You should use a scheduler when you need something to happen based on time only, delayed jobs for when its based on an user action, but the 'action' you are going to delay would take to long for a normal response. Sometimes you can just put the thing plainly in the controller also.
So in other words
The scheduler will be fine as long as the time between updates X is less then the time it will take for the update to happen, time Y.
If X < Y then you might want to look at calling the logic from the controller when each indvidual entry is accessed, isntead of trying to do them all at once. The idea being you would only update it after a certain time as passed so. You could store the last time update either on the model itself in a field like twitter_udpate_time or in a redis or memecache instance at a unquie key for the user/auth.
But if the individual update itself is still too long, then thats when you should do the above, but instead of doing the actually update, call a delayed job.
You could even set it up that it only updates or calls the delayed job after a certain number of views, to further limit stuff.
Possible Fancy Pants
Or if you want to get really fancy you could still do it as a cron job, but have a point system based on views that weights which entries should be updated. The idea being certain actions would add points to certain users, and if their points are over a certain amount you update them, and then remove their points. That way you could target the ones you think are the most important, or have the most traffic or show up in the most search results etc etc.
Next off a nick picky thing.
http://api.rubyonrails.org/classes/ActiveRecord/Batches.html
You should be using
#authentications.find_each do |authentication|
instead of
#authentications.each do |authentication|
find_each pulls in only 1000 entries at a time so if you end up with a lof of Authentications you don't end up pulling a crazy amount of entries into memory.
We have a Rails application that we test with RSpec. We want to spec operations that rely on Memcached. What is the best practice to do so?
I thought of doing this by stubbing all calls to Rails.cache. Is this a good idea?
As per #Pan Thomakos suggestion, I'm adding some additional details about one of the scenarios I'm trying to test:
We have the concept of accounts in our system, therefore on every request we retrieve the current user and the current account. Because there are not many accounts in the system, we keep them all in cache and retrieve them from there.
def self.find_by_slug(slug)
Rails.cache.fetch(Account.cache_key_for_slug(slug), :expires_in => 1.day) { super }
end
For this reason, caching in this case isn't just a nice to have behavior, but the expected behavior and the thing I want to test. Therefore turning off caching won't do.
Test without stubbing IMHO!
The sequence would look like this:
Cache.flush # or equivalent
Cache.get(slug).shouldbe null # test cache is empty
Method.find_by_slug(slug).should == 'some value' # test that method words
Cache.get(slug).should == 'some value' # test that cache has value.
Personally, I believe if you have the resources on hand then stubbing SHOULD NOT be used. If you do not have the resources on hand (IE a 3rd party service) then stubbing SHOULD BE used.
The problem with stubbing, is that if you changed the code that you are stubbing, then you won't know if it breaks.
An example in this case would be if you switched from the standard memcache gem, to Dahli?, or some other memcache gem which handed cache misses by returning false, null or some other value differently. I mean really! Cache.set("my_key", false)! :)
A example for switching, would be to leave the ASCII protocol and move to the faster binary protocol.
Memcache is a cheap resource, you can set it up with 1 meg of ram to do this testing. I would even go as far as to say you can do the same for mysql. Anything bigger than mysql, then I would start leaning towards stubbing as the cost to "setup" those resources becomes significant. YMMV.
-daniel
It seems like if you're using Rails.cache.fetch directly your best option is to stub. But if you use the controller helpers (which are now in seperate gems in rails 4), I came across this gem which is helpful https://github.com/avit/rspec-rails-caching
I'm sure "global variable" will get the hair on the back of everyone's neck standing up. What I'm trying to do is store a hierarchical menu in an acts_as_tree data table (done). In application_helper.rb, I create an html menu by querying the database and walking the tree (done). I don't want to do this for every page load.
Here's what I tried:
application.rb
config.menu = nil
application_helper.rb
def my_menu_builder
return MyApp::Application.config.menu if MyApp::Application.config.menu
# All the menu building code that should only run once
MyApp::Application.config.menu = menu_html
end
menu_controller.rb
def create
# whatever create code
expire_menu_cache
end
protected
def expire_menu_cache
MyApp::Application.config.menu = nil
end
Where I stand right now is that on first page load, the database is, indeed, queried and the menu built. The results are stored in the config variable and the database is never again hit for this.
It's the cache expiration part that's not working. When I reset the config.menu variable to nil, presumably the next time through my_menu_builder, it will detect that change and rebuild the menu, caching the new results. Doesn't seem to happen.
Questions:
Is Application.config a good place to store stuff like this?
Does anyone see an obvious flaw in this caching strategy?
Don't say premature optimization -- that's the phase I'm in. The premature-optimization iteration :)
Thanks!
I would avoid global variables, and use Rails' caching facilities.
http://guides.rubyonrails.org/caching_with_rails.html
One way to achieve this is to set an empty hash in your application.rb file:
MY_VARS = {}
Then you can add whatever you want in this hash which is accessible everywhere.
MY_VARS[:foo] = "bar"
and elsewhere:
MY_VARS[:foo]
As you felt, this is not the Rails way to behave, even if it works. There are different ways to use caching in Rails:
simple cache in memory explained here:
Rails.cache.read("city") # => nil
Rails.cache.write("city", "Duckburgh")
Rails.cache.read("city") # => "Duckburgh"
use of a real engine like memcached
I encourage you to have a look at http://railslab.newrelic.com/scaling-rails
This is THE place to learn caching in all it's shapes.