Curious about best practices for testing a particular situation.
I have a model that requires some time consuming operations to set up- reaching out to external services, parsing, kernel stuff, etc. One particular part of the set up is basically optional- I'd like to check that it's been run, but the result won't matter for almost all tests.
This model is used as input to many other classes, so I want to avoid a lengthy test suite and overbearing setup for a relatively unimportant step.
I'd like to know if this covers my bases, or if I'm going about this all wrong.
Currently, I am:
Stubbing out the operation globally
config.before(:each) do
LongOperation.any_instance.stub(:the_operation)
end
Testing that it gets called in my background job
code:
class BackgroundSetupWorker
def perform
LongOperation.the_operation
end
end
and test:
LongOperation.should_receive(:the_operation)
Unit testing the long-running operation
before(:each) do
LongOperation.unstub(:the_operation)
end
it "works preoperly" do
expect(LongOperation.the_operation).to ...
end
I think the ideal thing would be to take the LongOperation class as a param so you can switch it out in the tests however you like.
class BackgroundSetupWorker
def initialize(op_provider = LongOperation)
#op_provider = op_provider
end
def perform
#op_provider.the_operation
end
end
#in spec
describe BackgroundSetupWorker do
let(:op_provider){ double(the_operation: nil) }
subject(:worker){ BackgroundSetupWorker.new(op_provider) }
it 'should call op_provider' do
worker.perform
expect(op_provider).to have_received(:the_operation)
end
end
Related
Today I'm tryng to speed up my tests suite. My application is basically a big integrator between systems so most of my tests are using Savon mocks like this
RSpec.describe MyClass do
describe 'a function which sends a SOAP request'do
before do
savon.mock!
savon.expects(action).returns(File.read("spec/fixtures/somefile.xml"))
end
after { savon.unmock! }
it 'checks something'
it 'checks something else'
it 'checks something more'
it 'checks something different'
end
end
Obviously most of those tests are quite slow as they are loading a file. Moreover sometimes these mocks are inside nested contexts in order to combine multiple shared examples which increases the amount of loads.
Thinking to speed up some of these tests I tried to reduce the number of file loads moving them outside the before block. like this
RSpec.describe MyClass do
describe 'a function which sends a SOAP request'do
the_file = File.read("spec/fixtures/somefile.xml")
before do
savon.mock!
savon.expects(action).returns(the_file)
end
after { savon.unmock! }
it 'checks something'
it 'checks something else'
it 'checks something more'
it 'checks something different'
end
end
Indeed, the speed does not change; I have blocks of 96 tests with multiples, nested contexts and checks and I haven't gained not even 0.01 seconds. So my questions are:
I supposed the before block loads the file for each it, am I
right?
Does Rspec or Savon have some kind of cache?
How can I track the number of times I'm really loading my example file?
Thank you!
Perhaps you should look into the hooks order and specify something that suits you better like before (:suite) or before (:context). Depending the one you use it will be executed
https://relishapp.com/rspec/rspec-core/docs/hooks/before-and-after-hooks
Using let let(:the_file) { File.read("spec/fixtures/somefile.xml") } will solve your problem as let is lazy evaluated
I have several spec files that look like the following:
describe "My DSL" do
before :each do
#object = prepare_my_object
end
describe "foo" do
before :each do
#result = #object.read_my_dsl_and_store_stuff_in_database__this_is_expensive
end
it "should do this" do
#result.should be_this
end
it "should not do that" do
#result.should_not be_that
end
# ... several more tests about the state of #result
end
# ...
end
These tests take a long time, essentially because the second before :each block runs every time. Using before :all instead does not really help, because it gets called before the outer before :each. Putting all expectations in one single it block would help, but this is considered bad style.
What is best practice to have my expensive method being executed only once?
The fastest way to speed up rspec is to completely decouple the database. The DSL problem is a different problem from the get stuff in to and out of a db problem. If you have one method doing both, is it is possible to break the method into pieces?
Ideally, your DSL would be cached locally, so it wouldn't have to be pulled from the db on every request anyway. It could get loaded once in memory and held there before refreshing.
If you run against a local, in-memory cache, and decouple the db, does that speed things up? If yes, then it's the db call that's slow. If your DSL is completely loaded up in memory and the tests are still slow, then the problem is your DSL itself.
I'm very new on TDD and unit-testing, and I'm having quite a lot of doubts about the correct approach I should take during the tests of the custom model validations.
Suppose I have a custom validation:
User < ActiveRecord::Base
validate :weird_validation
def weird_validation
# Validate the weird attribute
end
end
Should I take this approach:
context "validation"
it "pass the validation with weird stuff" do
user = User.new weird: "something weird"
user.should be_valid
end
it "should't pass the validation with normal stuff" do
user = User.new weird: "something"
user.should_not be_valid
user.errors[:weird].size.should eq 1
end
end
Or this one:
context "#weird_validation" do
it "should not add an error if weird is weird" do
user = User.new
user.stub(:weird){"something weird"}
user.errors.should_not_receive :add
user.weird_validation.should eq true
end
it "should add an error if weird is not weird" do
user = User.new
user.stub(:weird){"something"}
user.errors.should_receive(:add).with(:weird, anything())
user.weird_validation.should eq false
end
end
So IMHO
The first approach
Pros
It test behaviour
Easy refactoring
Cons
Dependable of other methods
Something unrelated could make the test fail
The second approach
Pros
It doesn't relay on anything else, since everything else is stubbed
It's very specific of all the things the code should do
Cons
It's very specific of all the things the code should do
Refactoring the validations could potentially break the test
I personally think the correct approach should be the first one, but I can't avoid to think that I'm relying too much in other methods rather than the one I want to test, and if the test fails it may be because of any method withing the model. For example, it would not validate if the validation of other attribute failed.
Using the second approach I'm practically writing the code twice, which seems like a waste of time and effort. But I'm unit-testing the isolated method about what it should do. (and I'm personally doing this for every single method, which is very bad and time consuming I think)
Is there any midway when it comes to using stubs and mocks? Because I've taken the second approach and I think I'm abusing it.
IMO the second approach is the best one because you test your model properties and validations one at a time (the "unit" part).
To avoid overhead, you may consider using shoulda. It is really efficient for models unit testing. We're usually using a factory_girl/mocha/shoulda combination for functional testing (factory_girl and mocha are also very helpful to test queries and named scopes). Tests are easy to write, read and maintain :
# class UserTest < ActiveSupport::TestCase
# or
# describe 'User' do
should have_db_column(:weird).of_type(:string).with_options(:limit=>255)
should allow_value("something weird").for(:weird)
should_not allow_value("something").for(:weird)
should ensure_length_of(:weird).is_at_least(1).is_at_most(255)
# end
Shoulda generates positive/negative matchers therefore avoids a lot of code duplication.
My basic logic is to have an infinite loop running somewhere and test it as best as possible. The reason for having an infinite loop is not important (main loop for games, daemon-like logic...) and I'm more asking about best practices regarding a situation like that.
Let's take this code for example:
module Blah
extend self
def run
some_initializer_method
loop do
some_other_method
yet_another_method
end
end
end
I want to test the method Blah.run using Rspec (also I use RR, but plain rspec would be an acceptable answer).
I figure the best way to do it would be to decompose a bit more, like separating the loop into another method or something:
module Blah
extend self
def run
some_initializer_method
do_some_looping
end
def do_some_looping
loop do
some_other_method
yet_another_method
end
end
end
... this allows us to test run and mock the loop... but at some point the code inside the loop needs to be tested.
So what would you do in such a situation?
Simply not testing this logic, meaning test some_other_method & yet_another_method but not do_some_looping ?
Have the loop break at some point via a mock?
... something else?
What might be more practical is to execute the loop in a separate thread, assert that everything is working correctly, and then terminate the thread when it is no longer required.
thread = Thread.new do
Blah.run
end
assert_equal 0, Blah.foo
thread.kill
in rspec 3.3, add this line
allow(subject).to receive(:loop).and_yield
to your before hook will simple yield to the block without any looping
How about having the body of the loop in a separate method, like calculateOneWorldIteration? That way you can spin the loop in the test as needed. And it doesn’t hurt the API, it’s quite a natural method to have in the public interface.
You can not test that something that runs forever.
When faced with a section of code that is difficult (or impossible) to test you should:-
Refactor to isolate the difficult to test part of the code. Make the untestable parts tiny and trivial. Comment to ensure they are not later expanded to become non-trivial
Unit test the other parts, which are now separated from the difficult to test section
The difficult to test part would be covered by an integration or acceptance test
If the main loop in your game only goes around once, this will be immediately obvious when you run it.
What about mocking the loop so that it gets executed only the number of times you specify ?
Module Object
private
def loop
3.times { yield }
end
end
Of course, you mock this only in your specs.
I know this is a little old, but you can also use the yields method to fake a block and pass a single iteration to a loop method. This should allow you to test the methods you're calling within your loop without actually putting it into an infinite loop.
require 'test/unit'
require 'mocha'
class Something
def test_method
puts "test_method"
loop do
puts String.new("frederick")
end
end
end
class LoopTest < Test::Unit::TestCase
def test_loop_yields
something = Something.new
something.expects(:loop).yields.with() do
String.expects(:new).returns("samantha")
end
something.test_method
end
end
# Started
# test_method
# samantha
# .
# Finished in 0.005 seconds.
#
# 1 tests, 2 assertions, 0 failures, 0 errors
I almost always use a catch/throw construct to test infinite loops.
Raising an error may also work, but that's not ideal especially if your loop's block rescue all errors, including Exceptions. If your block doesn't rescue Exception (or some other error class), then you can subclass Exception (or another non-rescued class) and rescue your subclass:
Exception example
Setup
class RspecLoopStop < Exception; end
Test
blah.stub!(:some_initializer_method)
blah.should_receive(:some_other_method)
blah.should_receive(:yet_another_method)
# make sure it repeats
blah.should_receive(:some_other_method).and_raise RspecLoopStop
begin
blah.run
rescue RspecLoopStop
# all done
end
Catch/throw example:
blah.stub!(:some_initializer_method)
blah.should_receive(:some_other_method)
blah.should_receive(:yet_another_method)
blah.should_receive(:some_other_method).and_throw :rspec_loop_stop
catch :rspec_loop_stop
blah.run
end
When I first tried this, I was concerned that using should_receive a second time on :some_other_method would "overwrite" the first one, but this is not the case. If you want to see for yourself, add blocks to should_receive to see if it's called the expected number of times:
blah.should_receive(:some_other_method) { puts 'received some_other_method' }
Our solution to testing a loop that only exits on signals was to stub the exit condition method to return false the first time but true the second time, ensuring the loop is only executed once.
Class with infinite loop:
class Scheduling::Daemon
def run
loop do
if daemon_received_stop_signal?
break
end
# do stuff
end
end
end
spec testing the behaviour of the loop:
describe Scheduling::Daemon do
describe "#run" do
before do
Scheduling::Daemon.should_receive(:daemon_received_stop_signal?).
and_return(false, true) # execute loop once then exit
end
it "does stuff" do
Scheduling::Daemon.run
# assert stuff was done
end
end
end
:) I had this query a few months ago.
The short answer is there is no easy way to test that. You test drive the internals of the loop. Then you plonk it into a loop method & do a manual test that the loop works till the terminating condition occurs.
The easiest solution I found is to yield the loop one time and than return. I've used mocha here.
require 'spec_helper'
require 'blah'
describe Blah do
it 'loops' do
Blah.stubs(:some_initializer_method)
Blah.stubs(:some_other_method)
Blah.stubs(:yet_another_method)
Blah.expects(:loop).yields().then().returns()
Blah.run
end
end
We're expecting that the loop is actually executed and it's ensured it will exit after one iteration.
Nevertheless as stated above it's good practice to keep the looping method as small and stupid as possible.
Hope this helps!
I am making a concerted effort to wrap my head around Rspec in order to move towards more of a TDD/BDD development pattern. However, I'm a long way off and struggling with some of the fundamentals:
Like, when exactly should I be using mocks/stubs and when shouldn't I?
Take for example this scenario: I have a Site model that has_many :blogs and the Blog model has_many :articles. In my Site model I have a callback filter that creates a default set of blogs and articles for every new site. I want to test that code, so here goes:
describe Site, "when created" do
include SiteSpecHelper
before(:each) do
#site = Site.create valid_site_attributes
end
it "should have 2 blogs" do
#site.should have(2).blogs
end
it "should have 1 main blog article" do
#site.blogs.find_by_slug("main").should have(1).articles
end
it "should have 2 secondary blog articles" do
#site.blogs.find_by_slug("secondary").should have(2).articles
end
end
Now, if I run that test, everything passes. However, it's also pretty slow as it's creating a new Site, two new Blogs and three new Articles - for every single test! So I wonder, is this a good candidate for using stubs? Let's give it a go:
describe Site, "when created" do
include SiteSpecHelper
before(:each) do
site = Site.new
#blog = Blog.new
#article = Article.new
Site.stub!(:create).and_return(site)
Blog.stub!(:create).and_return(#blog)
Article.stub!(:create).and_return(#article)
#site = Site.create valid_site_attributes
end
it "should have 2 blogs" do
#site.stub!(:blogs).and_return([#blog, #blog])
#site.should have(2).blogs
end
it "should have 1 main blog article" do
#blog.stub!(:articles).and_return([#article])
#site.stub_chain(:blogs, :find_by_slug).with("main").and_return(#blog)
#site.blogs.find_by_slug("main").should have(1).articles
end
it "should have 2 secondary blog articles" do
#blog.stub!(:articles).and_return([#article, #article])
#site.stub_chain(:blogs, :find_by_slug).with("secondary").and_return(#blog)
#site.blogs.find_by_slug("secondary").should have(2).articles
end
end
Now all the tests still pass, and things are a bit speedier too. But, I've doubled the length of my tests and the whole exercise just strikes me as utterly pointless, because I'm no longer testing my code, I'm just testing my tests.
Now, either I've completely missed the point of mocks/stubs, or I'm approaching it fundamentally wrong, but I'm hoping someone might be able to either:
Improve me tests above so it uses stubs or mocks in a way that actually tests my code, rather than my tests.
Or, tell me if I should even be using stubs here - or whether in fact this is completely unnecessary and I should be writing these models to the test database.
But, I've doubled the length of my tests and the whole exercise just strikes me as utterly pointless, because I'm no longer testing my code, I'm just testing my tests.
This is the key right here. Tests that don't test your code aren't useful. If you can negatively change the code that your tests are supposed to be testing, and the tests don't fail, they're not worth having.
As a rule of thumb, I don't like to mock/stub anything unless I have to. For example, when I'm writing a controller test, and I want to make sure that the appropriate action happens when a record fails to save, I find it easier to stub the object's save method to return false, rather than carefully crafting parameters just so in order to make sure a model fails to save.
Another example is for a helper called admin? that just returns true or false based on whether or not the currently logged-in user is an admin or not. I didn't want to go through faking a user login, so I did this:
# helper
def admin?
unless current_user.nil?
return current_user.is_admin?
else
return false
end
end
# spec
describe "#admin?" do
it "should return false if no user is logged in" do
stubs(:current_user).returns(nil)
admin?.should be_false
end
it "should return false if the current user is not an admin" do
stubs(:current_user).returns(mock(:is_admin? => false))
admin?.should be_false
end
it "should return true if the current user is an admin" do
stubs(:current_user).returns(mock(:is_admin? => true))
admin?.should be_true
end
end
As a middle ground, you might want to look into Shoulda. This way you can just make sure your models have an association defined, and trust that Rails is well-tested enough that the association will "just work" without you having to create an associated model and then counting it.
I've got a model called Member that basically everything in my app is related to. It has 10 associations defined. I could test each of those associations, or I could just do this:
it { should have_many(:achievements).through(:completed_achievements) }
it { should have_many(:attendees).dependent(:destroy) }
it { should have_many(:completed_achievements).dependent(:destroy) }
it { should have_many(:loots).dependent(:nullify) }
it { should have_one(:last_loot) }
it { should have_many(:punishments).dependent(:destroy) }
it { should have_many(:raids).through(:attendees) }
it { should belong_to(:rank) }
it { should belong_to(:user) }
it { should have_many(:wishlists).dependent(:destroy) }
This is exactly why I use stubs/mocks very rarely (really only when I'm going to be hitting an external webservice). The time saved just isn't worth the added complexity.
There are better ways to speed up your testing time, and Nick Gauthier gives a good talk covering a bunch of them - see the video and the slides.
Also, I think a good option is to try out an in-memory sqlite database for your test runs. This should cut down on your database time by quite a bit by not having to hit the disk for everything. I haven't tried this myself, though (I primarily use MongoDB, which has the same benefit), so tread carefully. Here's a fairly recent blog post on it.
I'm not so sure with agreeing on the others. The real problem (as I see it) here, is that you're testing multiple pieces of interesting behavior with the same tests (the finding behavior, and the creation). For reasons on why this is bad, see this talk: http://www.infoq.com/presentations/integration-tests-scam. I'm assuming for the rest of this answer that you want to test that creation is what you want to test.
Isolationist tests often seem unwieldy, but that's often because they have design lessons to teach you. Below are some basic things I can see out of this (though without seeing the production code, I can't do too much good).
For starters, to query the design, does having the Site add articles to a blog make sense? What about a class method on Blog called something like Blog.with_one_article. This then means all you have to test is that that class method has been called twice (if [as I understand it for now], you have a "primary" and "secondary" Blog for each Site, and that the associations are set up (I haven't found a great way to do this in rails yet, I usually don't test it).
Furthermore, are you overriding ActiveRecord's create method when you call Site.create? If so, I'd suggest making a new class method on Site named something else (Site.with_default_blogs possibly?). This is just a general habit of mine, overriding stuff generally causes problems later on in projects.