tl;dr: Jump to the last paragraph
Recently I've been trying to use RSpec's request specs to do some more targeted testing.
This is how my testing mostly looks:
general cucumber feature specification, i.e. user goes to a post with comment, upvotes on a comment and the author gets points
model specs for when the model actually has some functinality, i.e. User#upvote(comment)
controller specs where I stub most of the things and just try to make sure the code goes the way I expect
view specs for when there is something complex in the view, such as rendering a upvote link only when the user didn't already upvote, and these are stubbed as well
The problem is when I have some specific scenario which causes a bug and everything seems to work in the model/view layer where I am unable to reproduce it.
That forces me to write an integration test, which I can also do in cucumber. The problem arises once I am able to actually reproduce it, and I need to figure out why is it happening. This usually means playing around in tests, changing different things and seeing what happens.
For example create a comment that is owned by the user who is trying to upvote, try to vote with an expired session etc. However these are really huge pain to write in Cucumber, because of the need to write a scenario and then specify each step.
At this point, I prefer to write a request spec, because it is more low level and allows me to directly do stuff. The problem is, that I'm not really sure how to properly write a request spec, or what are the rules.
A simple example here is:
visit login_path
fill_in "Username", :with => user.username
fill_in "Password", :with => user.password
click_button "Log in"
vs
post sessions_path(:username => user.username, :password => user.password)
or even something more low level like
session[:user_id] = user.id # this actually doesn't work, but the idea is there
Both of these examples achieve the same thing, they'll log a user in. I know that the answer to which one to pick is based on what I need to test, but that doesn't answer the correct, conventional way to do this.
I've been trying to find something about request specs, but they're not really described anywhere. The RSpec book doesn't cover them, the RSpec documentation doesn't say anything either.
What is a correct way to write request specs? When should I use capybara and when just the Rails' #get and #post methods instead of clicking buttons and visiting paths?
For requests spec I believe the convention is to stick to testing user behaviour and interface interactions, which would mean loading the page, filling in the form etc. A website user cant set the session or interact with variables directly so neither should your request specs.
I've often been tempted to skip page loads and form interactions by posting or setting variables in request specs (for speeds sake, especially heavy ajax specs) but it really does break the purpose of a request spec.
As the comments mentioned, you should test the specific controller / view behaviour in the other spec types.
Sermon first....
I think the natural progression of YOU the test writer goes:
Controller
Model
Requests
a mix of Request, Controller and Model specs.
I know that I started looking at the controller first, cause it was easier to grasp.
You then get into Model specs, for non-happy path things...
Then you realize rspec doesn't actually render the view, so you are starting to see dumb errors in Airbrake, so you say, shoot... I need to test the views, and the workflow. Hence Request specs.
Lastly, you get older and realize, all 3 are important and should be used sparingly, but accordingly. I'm just at step 4 now... too many request specs, and you are slogging 5 mins for the whole suite on a medium sized app. sucks.
To answer your question:
I test workflow and views i need to be "seen" (with page.should or any tricky JS (jquery ui selectors for instance) with capybara in a request spec.
If i just need to make sure the controller variables are instantiated or to do something quick with a post for a non-happy path , out of workflow thing... I use the controller. E.g., POST to IPN Controller in paypal...
You'd be surprised how much this covers. This leaves you to test the models for all the whacked out stuff you need for total all the edge cases.
Honestly though, I'd say use Fixtures and Test Unit for integration tests... still like them better, faster... stronger... etc.
Related
I read that controller specs are going out of favor and that the current recommendation is to use feature and request specs. I've found feature specs have been working really well for happy path testing, but I'm having some difficulty with exceptional cases I want to test. For instance I have an index of resources owned by a user. This list includes buttons to destroy each resource. Users only see their own resources. I want to add a test that attempting to destroy one that belongs to someone else fails. The problem is that the list won't include anybody else's items so there isn't an appropriate link to tell capybara to click. In a controller test I could directly issue a DELETE on the resource's path, but as best I can tell that is not an option in feature specs. Am I missing something? Do I need to implement request specs too if I want coverage outside the happy path? I was hoping to reserve request specs for covering my API endpoints.
From my point of view, permission specs should NOT be implemented as feature specs. Your thoughts fully makes sense and the only one thing you can really check there is that you DON'T see any entities, that were made by other users. Your logical difficulties is like +1 point, that such scenarios should be tested via another spec types. Also... feature specs are expensive and I would never do such 'deep' test in features.
I like the way Cucumber connects user specifications to integration testing. The Cucumber part is more or less clear to me.
I struggle when it comes to what should be tested with rspec (non-integration testing) and what shouldn't.
Is it right to unit-test with rspec something that has already been tested with Cucumber (e.g. unit-test will 100% fail if Cucumber test fails, and unit-test will 100% succeed if Cucumber test succeeds)?
To be specific, I have three examples I'd like to resolve.
Here is a case from the RSpec book.
They have the following Cucumber scenario:
Given I am not yet playing
When I start a new game
Then I should see "Welcome to Codebreaker!"
And I should see "Enter guess:"
They build two rspec-tests right after:
describe "#start" do
it "sends a welcome message" do
end
it "prompts for the first guess" do
end
end
Another example is testing routing or action redirect, while there is the following scenario:
Given I am at the login page
When I fill in the right username and password
Then I should be at the index page
Sometimes we test helpers that are already tested with Cucumber:
Given Mike has spent 283 minutes online
When I go to the Mike's profile page
Then I should see "4:43" for "Time online:"
I should probably test helper that breaks 283 minutes into "4:43", but it turns out that it is already tested with Cucumber.
It may not be the best examples, but it illustrates what I am talking about.
To me those tests are duplicates.
Could you please comment on the examples above?
Is there any principles or guidelines on what should be tested with rspec, when there are Cucumber tests already?
All this is only my personal opinion on this broad and open topic. Some people might disagree and funny enough they can.
As of general guidelines you should use Cucumber to test entire application stack, something that is called user experience. This application stack might be composed of many smaller independent objects but, same as user using your application wouldn't be interested in those details, your cucumber test shouldn't care for them and focus instead on outer layer of your application.
RSpec( in your setup!) should on the other hand put a main focus on those small objects, building blocks of your application.
There is a big problem with small application examples from the books: They are to small!
Boarder between outer layer of your application and its interiors is to blurred. Entire application is build with two objects! It is hard to distinguished what should test what. As your application grow in size its getting more obvious what is a user experience test(cucumber) and what is object - state test/message expectation test(RSpec).
Using your second example:
With this Cucumber story:
Given I am at the login page
When I fill in the right username and password
Then I should be at the index page
Rspec:
You will probably have some sort of User model:
test user name syntax(must start with capital for example)
test password must be 7 characters have numbers etc...
You can have some sort of authentication object:
test for valid and invalid logins etc
test for exceptions being thrown ....
What if your Authentication database is on different server?
mock connection and authentication database and test if database receive an request from authentication object.
And yada yada yada... Forever your cucumber test will guard general purpose of your application user login. Even when you add or change behaviour, under the hood, as long as this test pass you can be confident that user can login.
you should test things once(in one place)
object its responsible for testing its incoming massage(object public interface) for state(return value)
when your object depends on other object(sends its a message) don't test it for state(return value) that object is responsible for it.
when your object depends on other object(sends its a message) mock that second object and test if it receive your message.
Generally your question is too broad and you wont find a single answer, different people will have different ideas. What you should focus on is to test, as good as you can and with time you will definitely find right way for you. Not a great test suit is much better then none.
Because good design helps to write good test I would recommend Practical Object-Oriented Design in Ruby: An Agile Primer by Sandi Metz(which is one of a best book I've read)
I'm trying to get the best codecoverage/development time result
Currently I use rspec+shoulda to test my models and rspec+capybara to write my acceptance tests.
I tried writing a controller test for a simple crud but it kinda took too long and I got a confusing test in the end(my bad probably)
What`s the best pratice on controller testing with rspec?
Here is a gist on my test and my controller(one test does not pass yet):
https://gist.github.com/991687
https://gist.github.com/991685
Maybe not.
Sure you can write tests for your controller. It might help write better controllers. But if the logic in your controllers is simple, as it should be, then your controller tests are not where the battle is won.
Personally I prefer well-tested models and a thorough set of integration (acceptance) tests over controller tests any time.
That said, if you have trouble writing tests for controllers, then by all means do test them. At least until you get the hang of it. Then decide whether you want to continue or not. Same goes for every kind of test: try it until you understand it, decide afterwards.
The way I view this is that acceptance tests (i.e. Cucumber / Capybara), test the interactions that a user would normally perform on the application. This usually includes things like can a user create a specific resource with valid data and then do they see errors if they enter invalid data. A controller test is more for things that a user shouldn't be able to normally do or extreme edge cases that would be too (cu)cumbersome to test with Cucumber.
Usually when people write controller tests, they are effectively testing the same thing. The only reason to test a controller's method in a controller test are for edge cases.
Edge cases such as if a user enters an invalid ID to a show page they should be shown a 404 page. This is a very simple kind of thing to test with a controller test, and I would recommend doing that. You want to make sure that when they hit the action that they receive a 404 response, boom, simple.
Making sure that your new action responds successfully and doesn't syntax error? Please. That's what your Cucumber features would tell you. If the action suddenly develops a Case of the Whoops, your feature will break and then you will fix that.
Another way of thinking about it is do you want to test a specific action responds in a certain way (i.e. controller tests), or do you care more about that a user can go to that new action and actually go through the whole motions of creating that resource (i.e. acceptance tests)?
Writing controller tests gives your application permission to lie to you. Some reasons:
controller tests are not executed in the environment they are run in. i.e. they are not at the end of a rack middleware stack, so things like users are not available when using devise (as a single, simple example). As Rails moves more to a rack based setup, more rack middlewares are used, and your environment deviates increasingly from the 'unit' behaviour.
You're not testing the behaviour of your application, you're testing the implementation. By mocking and stubbing your way through, you're re-implementing implementation in spec form. One easy way to tell if you're doing this; if you don't change the expected behaviour of url response, but do change the implementation of the controller (maybe even map to a different controller), do your tests break? If they do, you're testing implementation not behaviour. You're also setting your self up to be lied to. When you stub and mock, there's no assurances that the mocks or stubs you've setup do what you think they do, or even if the methods they're pretending to be exists after refactoring occurs.
Calling controller methods is impossible via your applications 'public' api. The only way to get to a controller is via the stack, and the route. If you can't break it from a request via a url, is it really broken?
I use my tests as an assurance the my application is not going to break when I deploy it. Controller tests add nothing to my confidence that my application is indeed functional, and actually their presence decreases my confidence.
One other example, when testing your 'behaviour' of your application, do you care that a particular file template was rendered, or that a certain exception was raised, or instead is the behaviour of your application to return some stuff to the client with a particular status code?
Testing controllers (or views) increases the burden of tests that you impose on yourself, and means that the cost of refactoring is higher than it needs to be because of the potential to break tests.
Should you test? yes
There are gems that make testing controllers faster
http://blog.carbonfive.com/2010/12/10/speedy-test-iterations-for-rails-3-with-spork-and-guard/
Definitely test the controller. A few painfully learned rules of thumb:
mock out model objects
stub model object methods that your controller action uses
sacrifice lots of chickens.
I like to have a test on every controller method at least just to eliminate stupid syntax errors that may cause the page to blow up.
A lot of people seem to be moving towards the approach of using Cucumber for integration testing in place of writing controller and routing tests.
I'm getting started with cucumber, and I have written the following step definition, which basically I copied from the rspec test I already had.
Given /^There is a picture on the screen$/ do
describe PagesController do
describe "GET 'home'" do
it "should be successful" do
get 'home'
response.should be_success
end
end
end
end
Now I'm also quite new to Rails. But this seems a bit long to me, can't I make it any shorter?
What you're doing there is a big antipattern when writing tests with Cucumber. You haven't broken down your step far enough—when you look at the test you've got, you're actually going through at least a couple of different steps, one of which isn't something you want to test. A better test for this would be:
Given I am on the home page
Then I should see a picture
The first step you get for free when you install capybara's web steps, provided you are using the default path helper in features/support/paths.rb. The second one would look like this:
Then "I should see a picture" do
page.should have_selector("img.picture")
end
That step is going to look for an image with a 'picture' class on it—my arbitrary definition of what a picture is in the context of your application.
Notice that I'm not checking the response status here. The idea behind cucumber (even moreso than Rspec) is that you test things from the perspective of the client. Arguably your client may be a client API, so perhaps checking the status code is appropriate, but in general with a web app, you're much more concerned about the UI, and a failing status code will manifest itself in other ways, such as a broken UI. Details like status codes are generally implementation details that shouldn't be tested from a BDD point of view (though they should be covered in your unit tests.)
FWIW, I also disagree with the assertion that the length of the test is what should determine whether you should be using Rspec or Cucumber. See my blog post on the matter: http://collectiveidea.com/blog/archives/2011/04/15/language-matters/
It would strongly recommend not using Cucumber until you're more familiar with Rails/Ruby. Cucumber adds a level of abstraction that can make things very confusing, frustrating and time consuming. Just get very good with RSpec first. Then when you want to do integration tests on your Rails app, use Capybara with RSpec. When you're comfortable using those gems, you can go to Cucumber.
Cucumber is about writing user stories. You then translate those Cucumber steps into lines of RSpec and Capybara.
If you are looking for brevity, I would recommend sticking to RSpec. Cucumber's main advantage is its readability to people who do not write code, while RSpec sacrifices that in order to be shorter. I tried to use Lettuce, a python analogue to Cucumber, on one of my projects and found it to be way too much work to keep up.
So, in short: if you want to write shorter tests, use RSpec. If you want your developer to edit your tests or you are working in a team, use Cucumber.
After some time of doing Cucumber & RSpec BDD, I realized that many of my Cucumber features are just higher level view tests.
When I start writing my scenario and then go down to RSpec, I don't ever write view specs, since I could just copy and paste part of the scenario, which would be ugly dupliacation.
Take this scenario for example
Scenario: New user comes to the site
Given I am not signed in
When I go to the home page
Then I should see "Sign up free"
I know that this isn't directly testing the view, but writing separate view spec to check for the same thing seems redundant to me.
Am I approaching Cucumber wrong? What exactly should I test in view specs?
Should I write them for every single view, for example testing views for actions like
def show
#project = current_user.projects.first
end
or should I just test more complex views?
It's a widely-accepted (and in my opinion, incorrect) Cucumber philosophy that views should never be tested within RSpec. The argument goes that since the behavior of the view can be described in Cucumber, RSpec should stick to what it knows best -- Models and Controllers.
I argue that the "human-readable" aspect of Cucumber makes some aspects of view-speccing important. For instance, I find view specs to work very well when working in parallel with a front-end developer. If a JavaScript developer knows that he'll want to hook into a selector on your page, it's important that your view provides that selector.
For example:
describe 'gremlins/show.html.haml' do
context 'given it is after midnight' do
it 'has a #gremlin_warning selector' do
Time.stub!(:now).and_return(Time.parse '2010-12-16 00:01:00')
rendered.should have_selector '#gremlin_warning'
end
end
context 'it is before midnight' do
it 'does not have a #gremlin_warning selector' do
Time.stub!(:now).and_return(Time.parse '2010-12-16 23:59:00')
rendered.should_not have_selector '#gremlin_warning'
end
end
end
Note that the specs do not describe the content, they are willfully brief, and they do not describe interaction behaviors. Because the view is the portion of your application that will change the most, view specs should be used sparingly.
tl;dr: View specs are for communicating a contract to other developers and should be used sparingly (but nonetheless should be used).
Personally, I never use view specs when using Cucumber. To me, acceptance tests make a lot more sense, and my complex views are generally Javascript-focused and cannot be tested using view specs.
Don't use view specs for anything, ever. Cucumber stories -- or even RSpec integration tests -- do that better. The examples bobocopy gives are good ones for the case he postulates, but they should be rolled into Cucumber stories/integration tests, not left on their own.