We have a Rails application that we test with RSpec. We want to spec operations that rely on Memcached. What is the best practice to do so?
I thought of doing this by stubbing all calls to Rails.cache. Is this a good idea?
As per #Pan Thomakos suggestion, I'm adding some additional details about one of the scenarios I'm trying to test:
We have the concept of accounts in our system, therefore on every request we retrieve the current user and the current account. Because there are not many accounts in the system, we keep them all in cache and retrieve them from there.
def self.find_by_slug(slug)
Rails.cache.fetch(Account.cache_key_for_slug(slug), :expires_in => 1.day) { super }
end
For this reason, caching in this case isn't just a nice to have behavior, but the expected behavior and the thing I want to test. Therefore turning off caching won't do.
Test without stubbing IMHO!
The sequence would look like this:
Cache.flush # or equivalent
Cache.get(slug).shouldbe null # test cache is empty
Method.find_by_slug(slug).should == 'some value' # test that method words
Cache.get(slug).should == 'some value' # test that cache has value.
Personally, I believe if you have the resources on hand then stubbing SHOULD NOT be used. If you do not have the resources on hand (IE a 3rd party service) then stubbing SHOULD BE used.
The problem with stubbing, is that if you changed the code that you are stubbing, then you won't know if it breaks.
An example in this case would be if you switched from the standard memcache gem, to Dahli?, or some other memcache gem which handed cache misses by returning false, null or some other value differently. I mean really! Cache.set("my_key", false)! :)
A example for switching, would be to leave the ASCII protocol and move to the faster binary protocol.
Memcache is a cheap resource, you can set it up with 1 meg of ram to do this testing. I would even go as far as to say you can do the same for mysql. Anything bigger than mysql, then I would start leaning towards stubbing as the cost to "setup" those resources becomes significant. YMMV.
-daniel
It seems like if you're using Rails.cache.fetch directly your best option is to stub. But if you use the controller helpers (which are now in seperate gems in rails 4), I came across this gem which is helpful https://github.com/avit/rspec-rails-caching
Related
In one of my old apps, I'm using several API connectors - like AWS or Mandill as example.
For some reason (may be I saw it somewhere, don't remember), I using class constant to initialize this objects on init stage of application.
As example:
/initializers/mandrill.rb:
require 'mandrill'
MANDRILL = Mandrill::API.new ENV['MANDRILL_APIKEY']
Now I can access MANDRILL class constant of my application in any method and use it. (full path MyApplication::Application::MANDRILL, or just MANDRILL). All working fine, example:
def update_mandrill
result = MANDRILL.inbound.update_route id, pattern, url
end
The question is: it is good practice to use such class constants? Or better create new class instance in every method that using this instance, like in example:
def update_mandrill
require 'mandrill'
mandrill = Mandrill::API.new ENV['MANDRILL_APIKEY']
result = mandrill.inbound.update_route id, pattern, url
end
Interesting question.
It's very handy approach but it may have cons in some scenarios.
Imagine you have a constant that either takes a lot of time to initialize or it loads a lot of data into memory. When its initialization takes long you essentially degrade app boot time (which may or may not be a problem, usually it will in development).
If it loads a lot of data into memory it may turn out it's gonna be a problem when running rake tasks for example which load entire environment. You may hit memory boundaries in use cases in which you essentially do not need this data at all.
I know one application which load a lot of data during boot - and it's done very deliberately. Sure, use case is a bit uncommon, but still.
Another thing to consider is - imagine, you're trying to establish connection to external service like Mongo or anything else. If this service is unavailable (what happens) your application won't be able to boot. Maybe this service is essential for app to work, and without it it would be "useless" anyway, but it's also possible that you essentially stop everything because storage in which you keeps log does not work.
I'm not saying you shouldn't use it as you suggested - I do it also in my apps, but you should be aware of potential drawbacks.
Yes, pre-creating a pseudo-constant object (like that api client) is usually a good idea. However, there is, approximately, a thousand ways go about it and the constant is not on top of my personal list.
These days I usually go with setting it in the env files.
# config/environments/production.rb
config.email_client = Mandrill::API.new ENV['MANDRILL_APIKEY'] # the real thing
# config/environments/test.rb
config.email_client = a_null_object # something that conforms to the same api, but does absolutely nothing
# config/environments/development.rb
config.email_client = a_dev_object # post to local smtp, or something
Then you refer to the client like this:
Rails.application.configuration.email_client
And the correct behaviour will be picked up in each env.
If I don't need this per-env variation, then I either use some kind of singleton object (EmailClient.get) or a global variable in the initializer ($email_client). It can be argued that a constant is better than global variable, semantically and because it raises a warning when you try to re-assign it. But I like that global variable stands out more. You see right away that it's something special. (And then again, it's only #3 in the list, so I don't do it very often.).
I seem to have encountered literature alluding to it being bad practice to use RSpec's any_instance_of methods (e.g. expect_any_instance_of). The relish docs even list these methods under the "Working with legacy code" section (http://www.relishapp.com/rspec/rspec-mocks/v/3-4/docs/working-with-legacy-code/any-instance) which implies I shouldn't be writing new code leveraging this.
I feel that I am routinely writing new specs that rely on this capability. A great example is any method that creates a new instance and then calls a method on it. (In Rails where MyModel is an ActiveRecord) I routinely write methods that do something like the following:
def my_method
my_active_record_model = MyModel.create(my_param: my_val)
my_active_record_model.do_something_productive
end
I generally write my specs looking for the do_something_productive being called with use of the expect_any_instance_of. e.g.:
expect_any_instance_of(MyModel).to receive(:do_something_productive)
subject.my_method
The only other way I can see to spec this would be with a bunch of stubs like this:
my_double = double('my_model')
expect(MyModel).to receive(:create).and_return(my_double)
expect(my_double).to receive(:do_something_productive)
subject.my_method
However, I consider this worse because a) it's longer and slower to write, and b) it's much more brittle and white box than the first way. To illustrate the second point, if I change my_method to the following:
def my_method
my_active_record_model = MyModel.new(my_param: my_val)
my_active_record_model.save
my_active_record_model.do_something_productive
end
then the double version of the spec breaks, but the any_instance_of version works just fine.
So my questions are, how are other developers doing this? Is my approach of using any_instance_of frowned upon? And if so, why?
This is kind of a rant, but here are my thoughts:
The relish docs even list these methods under the "Working with legacy code" section (http://www.relishapp.com/rspec/rspec-mocks/v/3-4/docs/working-with-legacy-code/any-instance) which implies I shouldn't be writing new code leveraging this.
I don't agree with this. Mocking/stubbing is a valuable tool when used effectively and should be used in tandem with assertion style testing. The reason for this is that mocking/stubbing enables an "outside-in" testing approach where you can minimize coupling and test high level functionality without needing to call every little db transaction, API call, or helper method in your stack.
The question really is do you want to test state or behavior? Obviously, your app involves both so it doesn't make sense to tether yourself to a single paradigm of testing. Traditional testing via assertions/expectations is effective for testing state and is seldom concerned with how state is changed. On the other hand, mocking forces you to think about interfaces and interactions between objects, with less burden on the mutation of state itself since you can stub and shim return values, etc.
I would, however, urge caution when using *_any_instance_of and avoid it if possible. It's a very blunt instrument and can be easy to abuse especially when a project is small only to become a liability when the project is larger. I usually take *_any_instance_of as a smell that either my code or tests, can be improved, but there are times wen it's necessary to use.
That being said, between the two approaches you propose, I prefer this one:
my_double = double('my_model')
expect(MyModel).to receive(:create).and_return(my_double)
expect(my_double).to receive(:do_something_productive)
subject.my_method
It's explicit, well-isolated, and doesn't incur overhead with database calls. It will likely require a rewrite if the implementation of my_method changes, but that's OK. Since it's well-isolated it probably won't need to be rewritten if any code outside of my_method changes. Contrast this with assertions where dropping a column in a database can break almost the entire test suite.
I don't have a better solution to testing code like that than either of the two you gave. In the stubbing/mocking solution I'd use allow rather than expect for the create call, since the create call isn't the point of the spec, but that's a side issue. I agree that the stubbing and mocking is painful, but that's usually what I do.
However, that code has just a bit of Feature Envy. Extracting a method onto MyModel clears up the smell and eliminates the testing issue:
class MyModel < ActiveRecord::Base
def self.create_productively(attrs)
create(attrs).do_something_productive
end
end
def my_method
MyModel.create_productively(attrs)
end
# in the spec
expect(MyModel).to receive(:create_productively)
subject.my_method
create_productively is a model method, so it can and should be tested with real instances and there's no need to stub or mock.
I often notice that the need to use less-commonly-used features of RSpec means that my code could use a little refactoring.
def self.my_method(attrs)
create(attrs).tap {|m| m.do_something_productive}
end
# Spec
let(:attrs) { # valid hash }
describe "when calling my_method with valid attributes" do
it "does something productive" do
expect(MyModel.my_method(attrs)).to have_done_something_productive
end
end
Naturally, you will have other tests for #do_something_productive itself.
The trade-off is always the same: mocks and stubs are fast, but brittle. Real objects are slower but less brittle, and generally require less test maintenance.
I tend to reserve mocks/stubs for external dependencies (e.g. API calls), or when using interfaces that have been defined but not implemented.
I'm working through Michael Hartel's rails tutorial, on 6.3 and need alternative code for the user_spec model. The code that he has is:
let(:found_user) { User.find_by(email: #user.email) }
It looks like I can use where, but unsure of the correct syntax. I tried several variations of the following:
let(:found_user) { User.where(:email => "#user.email")}
I'm sure this is a pretty easy answer, but cant quite get it.
let(:found_user){User.where(email: #user.email).first}
or
let(:found_user){User.find_by_email(#user.email)}
That first one that uses where returns a collection of users that match the where clauses, which is why you would need that .first (It doesn't execute the sql until you grab the records with something like .all, .first, or .each).
I would say it's not the best practice to execute database commands in a unit test though. What are you testing specifically? Is there a reason you need the user to be saved in the database and can't just do something like:
let(:user){User.new(email: 'some email')}
ActiveRecord::Base#find_by is effectively where(options).first, but that's a whole extra call that you needn't make.
Rails also provides mildly deprecated "magic" find_by_<attribute>[and_<attribute>] methods which used method_missing to parse out what was meant based on the name of the method. While the framework does provide these, I caution against using them as they are necessarily slower than "native" methods, and are more resistant to refactoring.
I would recommend sticking with find_by for the general case, and would try to avoid hitting the database in specs and tests.
The factory_girl gem provides a method to create a stubbed version of the class which quacks like a record returned from the database by answering true for persisted? and providing an id.
Alternatively, you can just build a new record without saving it: User.new(attribute: value, ...) and run your tests on that:
it "does some things" do
user = User.new(attributes)
# make user do some things
expect(things).to have_happened
end
I have a rails 2.3.4 app and a line that looks like:
temp = Rails.cache.fetch(:temp_id) { User.find_by_name('Temp').id }
and everything worked fine, until I decided to switch the caching layer to memcached by adding the following to my environment.rb:
config.cache_store = :mem_cache_store
Now the line which used to work fine gives me the following error:
undefined method 'length' for :temp_id:Symbol
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.4/lib/active_support/vendor/memcache-client-1.7.4/memcache.rb:645:in 'get_server_for_key'
I understand the error, but I would imagine this common case would have been quickly discovered by a rails test case, so I am wondering if I am doing something wrong. Otherwise, I'm sure I can monkeypatch this issue to convert the symbol to a string.
Thanks
Just use string keys if you can. All the documentation examples use string keys. Although it's not explicitly mentioned as far as I can see, other keys are not supported.
The key arguments are passed directly to the cache implementation, so the different caching flavours may disagree on whether or not they accept anything other than strings.
Because the caches are external with the exception of in-memory cache, I'm not sure that supporting symbols would be useful apart from preventing cases like yours. The key will actually be written to some output somewhere (it's not just internal to your Ruby app), so conceptually the key should be a string.
Edit in reaction to comment: yes, it is of course possible and perfectly reasonable in this case to create a monkey patch to circumvent having to change all calls. What you're suggesting is this (copied into the answer for readability):
class MemCache
def get_server_for_key_with_symbols(key, options = {})
key = key.to_s if key.is_a? Symbol
get_server_for_key_without_symbols(key, options)
end
alias_method_chain :get_server_for_key, :symbols
end
I would also consider just doing a project wide search-and-replace for \.fetch(:\w+) and replace it with \.fetch("$1") (repeat for read and write if necessary). This should probably cover 95% of all cases and a subsequent run of your test suite should catch the rest of the errors.
In general: While the documentation of Rails is pretty good these days, a lot of assumptions are unfortunately still implicit. It's generally a good idea to take a good look at the examples that are given in the documentation, and use the same style. The documented examples are always how the framework was intended to be used.
FWIW, it's canonically Rails.cache.read and Rails.cache.write.
What is the best way to profile a controller action in Ruby on Rails. Currently I am using the brute-force method of throwing in puts Time.now calls between what I think will be a bottleneck. But that feels really, really dirty. There has got to be a better way.
I picked up this technique a while back and have found it quite handy.
When it's in place, you can add ?profile=true to any URL that hits a controller. Your action will run as usual, but instead of delivering the rendered page to the browser, it'll send a detailed, nicely formatted ruby-prof page that shows where your action spent its time.
First, add ruby-prof to your Gemfile, probably in the development group:
group :development do
gem "ruby-prof"
end
Then add an around filter to your ApplicationController:
around_action :performance_profile if Rails.env == 'development'
def performance_profile
if params[:profile] && result = RubyProf.profile { yield }
out = StringIO.new
RubyProf::GraphHtmlPrinter.new(result).print out, :min_percent => 0
self.response_body = out.string
else
yield
end
end
Reading the ruby-prof output is a bit of an art, but I'll leave that as an exercise.
Additional note by ScottJShea:
If you want to change the measurement type place this:
RubyProf.measure_mode = RubyProf::GC_TIME #example
Before the if in the profile method of the application controller. You can find a list of the available measurements at the ruby-prof page. As of this writing the memory and allocations data streams seem to be corrupted (see defect).
Use the Benchmark standard library and the various tests available in Rails (unit, functional, integration). Here's an example:
def test_do_something
elapsed_time = Benchmark.realtime do
100.downto(1) do |index|
# do something here
end
end
assert elapsed_time < SOME_LIMIT
end
So here we just do something 100 times, time it via the Benchmark library, and ensure that it took less than SOME_LIMIT amount of time.
You also may find these links useful: The Benchmark.realtime reference and the Test::Unit reference. Also, if you're into the 'book reading' thing, I picked up the idea for the example from Agile Web Development with Rails, which talks all about the different testing types and a little on performance testing.
There's a Railscast on profiling that's well worth watching
http://railscasts.com/episodes/98-request-profiling
You might want to give the FiveRuns TuneUp service a try, as it's really rather impressive. Disclaimer: I'm not associated with FiveRuns in any way, I've just tried this service out.
TuneUp is a free service whereby you download a plugin and when you run your application it injects a panel at the top of the screen that can be expanded to display detailed performance metrics.
It gives you some nice graphs, including one that shows what proportion of time is spent in the Model, View and Controller. You can even drill right down to see the individual SQL queries that ActiveRecord is executing if you need to and it can show you the underlying database schema with another click.
Finally, you can optionally upload your profiling data to the FiveRuns site for community performance analysis and advice.
This works in Rails 4.2.6:
o=OpenStruct.new(logger: Rails.logger)
o.extend ActiveSupport::Benchmarkable
o.benchmark 'name' do
# ... your code ...
end