Multiple assertions for single setup in RSpec - ruby-on-rails

I have a few slower specs that I would like to optimise.
The example of such spec looks like:
require 'rspec'
class HeavyComputation
def compute_result
sleep 1 # something compute heavy here
"very big string"
end
end
describe HeavyComputation, 'preferred style, but slow' do
subject { described_class.new.compute_result }
it { should include 'big' }
it { should match 'string' }
it { should match /very/ }
# +50 others
end
This is very readable and I'm happy with it generally, except that every additional spec will add at least 1 second to the total run-time. That is not very acceptable.
(Please let's not discuss the optimisation on the HeavyComputation class as it is outside of the scope of this question).
So what I have to resort to is spec like this:
describe HeavyComputation, 'faster, but ugly' do
subject { described_class.new.compute_result }
it 'should have expected result overall' do
should include 'big'
should match 'string'
should match /very/
# +50 others
end
end
This is obviously much better performance wise because the time to run it will always be nearly constant.
The problem is that failures are very hard to track down and it is not very intuitive to read.
So ideally, I would like to have a mix of both. Something along these lines:
describe HeavyComputation, 'what I want ideally' do
with_shared_setup_or_subject_or_something_similar_with do
shared(:result) { described_class.new.compute_result }
subject { result }
it { should include 'big' }
it { should match 'string' }
it { should match /very/ }
# +50 others
end
end
But unfortunately I cannot see where to even start implementing it. There are multiple potential issues with it (should the hooks be called on shared result is among those).
What I want to know if there is an existing solution to this problem.
If no, what would be the best way to tackle it?

You can use a before(:context) hook to achieve this:
describe HeavyComputation, 'what I want ideally' do
before(:context) { #result = described_class.new.compute_result }
subject { #result }
it { should include 'big' }
it { should match 'string' }
it { should match /very/ }
# +50 others
end
Be aware that before(:context) comes with a number of caveats, however:
Warning: before(:context)
It is very tempting to use before(:context) to speed things up, but we
recommend that you avoid this as there are a number of gotchas, as well
as things that simply don't work.
context
before(:context) is run in an example that is generated to provide group
context for the block.
instance variables
Instance variables declared in before(:context) are shared across all the
examples in the group. This means that each example can change the
state of a shared object, resulting in an ordering dependency that can
make it difficult to reason about failures.
unsupported rspec constructs
RSpec has several constructs that reset state between each example
automatically. These are not intended for use from within before(:context):
let declarations
subject declarations
Any mocking, stubbing or test double declaration
other frameworks
Mock object frameworks and database transaction managers (like
ActiveRecord) are typically designed around the idea of setting up
before an example, running that one example, and then tearing down.
This means that mocks and stubs can (sometimes) be declared in
before(:context), but get torn down before the first real example is ever
run.
You can create database-backed model objects in a before(:context) in
rspec-rails, but it will not be wrapped in a transaction for you, so
you are on your own to clean up in an after(:context) block.
(from http://rubydoc.info/gems/rspec-core/RSpec/Core/Hooks:before)
As long as you understand that your before(:context) hook is outside the normal per-example lifecycle of things like test doubles and DB transactions, and manage the necessary setup and teardown yourself explicitly, you'll be fine -- but others who work on your code base may not be aware of these gotchas.

#Myron Marston gave some inspiration, so my first attempt to implement it in a more or less reusable way ended up with the following usage (note the shared_subject):
describe HeavyComputation do
shared_subject { described_class.new.compute_result }
it { should include 'big' }
it { should match 'string' }
it { should match /very/ }
# +50 others
end
The idea is to only render subject once, on the very first spec instead of in the shared blocks.
It makes it pretty much unnecessary to change anything (since all the hooks will be executed).
Of course shared_subject assumes the shared state with all its quirks.
But every new nested context will create a new shared subject and to some extent eliminates a possibility of a state leak.
More importantly, all we need to do in order to deal the state leaks s(should those sneak in) is to replace shared_subject back to subject. Then you're running normal RSpec examples.
I'm sure the implementation has some quirks but should be a pretty good start.

aggregate_failures, added in version 3.3, will do some of what you're asking about. It allows you to have multiple expectations inside of a spec, and RSpec will run each and report all failures instead of stopping at the first one.
The catch is that since you have to put it inside of a single spec, you don't get to name each expectation.
There's a block form:
it 'succeeds' do
aggregate_failures "testing response" do
expect(response.status).to eq(200)
expect(response.body).to eq('{"msg":"success"}')
end
end
And a metadata form, which applies to the whole spec:
it 'succeeds', :aggregate_failures do
expect(response.status).to eq(200)
expect(response.body).to eq('{"msg":"success"}')
end
See: https://www.relishapp.com/rspec/rspec-core/docs/expectation-framework-integration/aggregating-failures

Related

Class has leaked into another example and can no longer be used in spec

I'm unable to replicate this locally, but for some reason I am getting the following error when running tests in CircleCi:
<Double Mylogger> was originally created in one example but has leaked into another example and can no longer be used. rspec-mocks' doubles are designed to only last for one example, and you need to create a new one in each example you wish to use it for.
Here is a simplified version of my code:
# frozen_string_literal: true
describe 'my_rake_task' do
let(:my_log) { Mylogger.new }
subject { Rake::Task['my_rake_task'].execute }
describe 'one' do
context 'logs' do
let(:logs) do
[
['My message one'],
['My message two'],
]
end
after { subject }
it 'correctly' do
logs.each { |log| expect(my_log).to receive(:info).with(*log) }
end
end
end
describe 'two' do
context 'logs' do
let(:logs) do
[
['My message three'],
['My message four'],
]
end
after { subject }
it 'correctly' do
logs.each { |log| expect(my_log).to receive(:info).with(*log) }
end
end
end
end
Why is it saying MyLogger is a double? Why would it be leaking?
The reason that the error is saying that MyLogger is a double is because it is one. When you call expect(my_log).to receive or allow(my_log).to receive, you transform the instance into a partial-double.
As for why my_log is leaking: it's impossible to tell from the code that you posted. In order to cause a leak, some code either in your rake task or in the spec itself would need to be injecting my_log into some global state, like a class variable.
Mostly commonly this sort of thing is caused by storing something in a class variable. You will have to figure out where that is, and how to clear it or avoid using a class variable - it could be in your class or in a gem.
Best practice, where using a class variable or an external system is causing inter-test issues, is to clean this sort of thing between tests, if possible. For example ActionMailer::Base.deliveries and Rails.cache are common things that should be cleared. You should also clear Faker::UniqueGenerator or RequestStore if you're using those gems, and I'm sure there are more.
Once you have found the class variable, if it's in your code, and you have determined a class variable is the correct approach, you can add a reset or clear class method to the class and call it in a before(:each) RSpec block in your spec_helper.rb or rails_helper.rb.
Note that while a lot of things will clear themselves automatically between tests (such as RSpec mocks), and make you think this is all automatic, in practice it is often anything but.
Tests will only remain independent by (a) mostly making use of objects created in the tests and mostly only storing data in there and (b) ensuring anything else is cleared between tests by your explicit code or within the responsible gem.
This can be especially annoying when dealing with external third-party systems, which rarely provide an API to clear a staging environment, and hence sometimes require considerable care even when using the vcr gem.

How do RSpec's let and let! replace the original parameters?

In the process of writing tests in Rspec, if you encounter repeated required parameters {...}, you can use let to write it. This avoids writing a bunch of parameter preparations in advance for each example.
However, I don't quite understand the paradigm of Better Specs. His original code is this:
describe '#type_id' do
before { #resource = FactoryBot.create :device }
before { #type = Type.find #resource.type_id }
it 'sets the type_id field' do
expect(#resource.type_id).to eq(#type.id)
end
end
After using let it becomes the following
describe '#type_id' do
let(:resource) { FactoryBot.create :device }
let(:type) { Type.find resource.type_id }
it 'sets the type_id field' do
expect(resource.type_id).to eq(type.id)
end
end
It looks like the way to call a resource is pretty much the same, what's the benefit of using let ? What is the function of FactoryBot.create:device? And I can't see where type is being called?
The difference is that lets are lazily evaluated and then memoized for the rest of the spec.
So in the first example, first the before blocks run and set the values of #resource and #type, and then the spec runs.
In the second example, the spec runs and when it references 'resource' the let block runs and returns a value, then when 'type' is referenced is runs the let block for that. The let block for type itself references 'resource' so it gets the value for resource that was memoized from the first time resource was referenced.
For what its worth, I disagree that lets are 'better'. My team and I have found that all they do is make specs much harder to understand for very little benefit, and we have removed all use of them in all our projects.
In fact, I consider that most of 'better specs' is actually poor advice, so if you are struggling to understand why something is 'better', you are very much not alone :)

Injecting RSpec mocks to avoid allow_any_instance_of

I have a Rails project with models Partner, Department, Service. The partner and department are seeded, the service is created dynamically when #service is called on an instance of Department. I'm looking to test the #ports functionality of Service in request specs. My first attempt
before do
allow_any_instance_of(Service).to receive(:ports).and_return(ports)
end
context 'no ports' do
let(:ports) { [] }
:
it 'has status 404' do
get :show, params: params
expect(response).to have_http_status(:not_found)
end
end
Now this works, but I know that allow_any_instance_of is a code smell, and one should avoid it if possible.
I tried
let(:service) { instance_mock(Service) }
let(:department) { Department.find_by( ... ) }
before do
allow(department).to receive(:service).and_return(service)
allow(service).to receive(:ports).and_return(ports)
end
but that doesn't work, my test department is different to the one found in the model (it has a different object_id, I get that).
So I inject as follows
let(:department) { instance_double(Department) }
let(:service) { instance_double(Service) }
before do
allow(Department).to receive(:find_by).and_return(department)
allow(department).to receive(:service).and_return(service)
allow(service).to receive(:ports).and_return(ports)
end
Which works as expected. I guess properly this should make some assertions on the arguments to #find_by, but even as it stands, it seems a lot more complicated than the "smelly" version, and I'm having difficulty persuading myself that this is better. One could imagine a situation where there might be a lot of allows needed before you finally get to the object where you can inject your test data.
So my question: is this more-complex style of mock injection worth it to get rid of an allow_any_instance_of; or is there a better way that I've not yet considered?

RSpec 'Expect' Syntax and Idiomatic attribute spec

This should have an easy answer but I'm struggling to find it (have checked RSpec documentation, EverydayRails Testing with RSpec, Google results). In my model specs I like to include basic attribute specs as follows:
describe Foo do
describe "basic attributes" do
before { #foo = create(:foo) }
subject { #foo }
it { should be_valid }
it { should respond_to(:color) }
it { should respond_to(:height) }
it { should respond_to(:some_other_attribute) }
it { should respond_to(:you_get_the_idea) }
...
I like these specs because if there is some kind of error within my factory and/or model these specs help me pinpoint it quickly.
I've incorporated the expect syntax into all other specs and I like how it reads, but how to use it here? One option might be
expect(#foo).to respond_to(:color)
And another might be
expect(it).to respond_to(:color)
The former involves duplication that is avoided with the should syntax, but the latter looks strange to me (which could just be me).
I realize this question is more about style than functionality*, but we Ruby developers are conscientious about style, and I want to adhere to standard practices and have readable, idiomatic code. Any help is appreciated. Thanks.
UPDATE: Neither of my proposed options actually work, by the way. They both throw undefined method 'expect' errors. Now I'm really confused!
Having thought about the error, I realize it's because the should specs above are within one-line block. The confusion, then, is how can I write a single-line block with the expect syntax? In light of this update, the question is very much about functionality and I'll be excited to hear others' thoughts.
4/2015 UPDATE
rspec > 3.0 has added yet another way of handling these, and it sounds like rspec ~> 4.0 will do away with the should syntax. Per Myron Masters:
Some users have expressed confusion about how this should relates to the expect syntax and if you can continue using it. It will continue to be available in RSpec 3 (again, regardless of your syntax configuration), but we've also added an alternate API that is a bit more consistent with the expect syntax:
describe Post do
it { is_expected.to allow_mass_assignment_of(:title) }
end
is_expected is defined very simply as expect(subject) and also supports negative expectations via is_expected.not_to matcher.
[...]
In RSpec 3, we've kept the should syntax, and it is available by default, but you will get a deprecation warning if you use it without explicitly enabling it. This will pave the way for it being disabled by default (or potentially extracted into a seperate gem) in RSpec 4, while minimizing confusion for newcomers coming to RSpec via an old tutorial.
Myron Marston, one of the core RSpec committers, explains here that you should still use
it { should be_cool }
If you've disabled the should syntax, he offers a solution to alias expect_it to it:
RSpec.configure do |c|
c.alias_example_to :expect_it
end
RSpec::Core::MemoizedHelpers.module_eval do
alias to should
alias to_not should_not
end
With this in place, you could write this as:
describe User do
expect_it { to be_valid }
end
I don't think there's a correct answer to this question, but I've been recently rewriting my test suites to move away from should and use the expect syntax exclusively, so I'll throw my two cents in. I decided to re-write pretty much because Myron Marston, the RSpec lead maintainer, wrote:
In the future, we plan to change the defaults so that only expect is available unless you explicitly enable should. We may do this as soon as RSpec 3.0, but we want to give users plenty of time to get acquianted with it.
He also commented:
We have no plans to ever remove "should"...but expect has fewer gotchas, and is the syntax I would recommend for new projects.
I do agree with Mark Rushakoff's answer as well, but personally I don't want to have to create those aliases just to keep a single it-block style syntax. So, using your example, where originally I wrote a model spec like your example in this form:
describe Foo do
let(:foo) { create(:foo) }
subject { foo }
describe "model attributes" do
it { should be_valid }
it { should respond_to(:color) }
it { should respond_to(:height) }
it { should respond_to(:some_other_attribute) }
it { should respond_to(:you_get_the_idea) }
end
# ...
end
I now would tend to write it like:
describe Foo do
let(:foo) { create(:foo) }
specify "model attributes" do
expect(foo).to be_valid
expect(foo).to respond_to(:color)
expect(foo).to respond_to(:height)
expect(foo).to respond_to(:some_other_attribute)
expect(foo).to respond_to(:you_get_the_idea)
end
# ...
end
My opinion is that referencing foo directly in expect(foo).to respond_to(:color) is as much "duplication" as referencing a subject indirectly using it, so I'm not too phased about it, and I'm warming to the way that expect specs generally read. But, ultimately I think this comes down to just being an issue of preferred writing style.

Multiple should statements in one rspec it clause - bad idea?

Here's my rspec test:
it "can release an idea" do
james.claim(si_title)
james.release(si_title)
james.ideas.size.should eq 0
si_title.status.should eq "available"
end
Are the two should lines at the end a really bad idea? I read somewhere that you should only test for one thing per it block, but it seems silly to make a whole test just to make sure that the title status changes (the same function does both things in my code).
My interpretation of this isn't so much that there should be precisely one assertion / call to should per spec, but that there should only be one bit of behaviour tested per spec, so for example
it 'should do foo and bar' do
subject.do_foo.should be_true
subject.do_bar.should be_true
end
is bad - you're speccing 2 different behaviours at the same time.
on the other hand if your 2 assertions are just verifying different aspects of one thing then I'm ok with that, for example
it 'should return a prime integer' do
result = subject.do_x
result.should be_a(Integer)
result.foo.should be_prime
end
For me it wouldn't make too much sense to have one spec that checks that it returns an integer and a separate one that it returns a prime.
Of course in this case, the be_prime matcher could easily do both of these checks - perhaps a good rule of thumb is that multiple assertions are ok if you could sensibly reduce them to 1 with a custom matcher (whether actually doing this is actually worthwhile probably depends on your situation)
In your particular case, it could be argued that there are 2 behaviours at play - one is changing the status and other is mutating the ideas collection. I would reword your specs to say what the release method should do -
it 'should change the status to available'
it 'should remove the idea from the claimants ideas'
At the moment those things always happen simultaneously but I would argue they are separate behaviours - you could easily imagine a system where multiple people can claim/release an idea and the status only change when the last person releases the idea.
I have the same problem...
It ought to be one should per it (my boss says) by it makes testing time consuming and, as you said, silly.
Tests require good sense and flexibility or they might end up enslaving you.
Anyway, I agree with your test.
My policy is to always consider multiple assertions to be a sign of a potential problem and worth a second thought, but not necessarily wrong. Probably 1/3 of the specs I write end up having multiple assertions in them for one reason or another.
One of the problems with having multiple assertions is that when one fails, you can't see if the other one passed or not. This can sometimes be worked around by building an array of results and asserting the value of the array.
In the case you are asking about, I don't think that multiple assertions are a problem, but I do see something else that seems an issue to me. It looks like your spec may be overly coupled.
I think that you're trying to assert the behavior of whatever kind of a thing james is, but the test is also depending on behavior of si_title in order to tell us what was done to it (by way of its eventual #status value). What I would usually do instead is make si_title a test-double and use #should_receive to directly specify the messages that it should expect.
I think Frederick Cheung gave a very good answer (+1) especially the why, but I'd also like to give a comparison bit of code for you to look at which uses the its, lets, before and context syntax:
context "Given a si_title" do
let(:james) { James.new } # or whatever
before do
james.claim(si_title)
james.release(si_title)
end
context "That is valid" do
let(:si_title) { Si_title.new } # or whatever
describe "James' ideas" do
subject { james.ideas }
its(:size) { should == 0 }
its(:whatever) { should == "Whatever" }
end
describe "Si title" do
subject { si_title }
its(:status) { should == "available" }
end
end
context "That is invalid" do
# stuff here
end
end
I would even go further and make the expected values lets and then make the examples shared_examples, so that they could then be use to check the different aspects (null arguments, invalid arguments, an incorrect object… ), but I believe that this is a far better way to spec your intentions and still cut down on any repetition. Taking the example from Frederick's answer:
it 'should return a prime integer' do
result = subject.do_x
result.should be_a(Integer)
result.foo.should be_prime
end
Using RSpec's syntax to full effect you get this:
let(:result) { 1 }
subject{ result }
its(:do_x) { should be_a(Integer) }
its(:foo) { should be_prime }
Which means you can check multiple aspects of a subject.

Resources