I'm getting started with cucumber, and I have written the following step definition, which basically I copied from the rspec test I already had.
Given /^There is a picture on the screen$/ do
describe PagesController do
describe "GET 'home'" do
it "should be successful" do
get 'home'
response.should be_success
end
end
end
end
Now I'm also quite new to Rails. But this seems a bit long to me, can't I make it any shorter?
What you're doing there is a big antipattern when writing tests with Cucumber. You haven't broken down your step far enough—when you look at the test you've got, you're actually going through at least a couple of different steps, one of which isn't something you want to test. A better test for this would be:
Given I am on the home page
Then I should see a picture
The first step you get for free when you install capybara's web steps, provided you are using the default path helper in features/support/paths.rb. The second one would look like this:
Then "I should see a picture" do
page.should have_selector("img.picture")
end
That step is going to look for an image with a 'picture' class on it—my arbitrary definition of what a picture is in the context of your application.
Notice that I'm not checking the response status here. The idea behind cucumber (even moreso than Rspec) is that you test things from the perspective of the client. Arguably your client may be a client API, so perhaps checking the status code is appropriate, but in general with a web app, you're much more concerned about the UI, and a failing status code will manifest itself in other ways, such as a broken UI. Details like status codes are generally implementation details that shouldn't be tested from a BDD point of view (though they should be covered in your unit tests.)
FWIW, I also disagree with the assertion that the length of the test is what should determine whether you should be using Rspec or Cucumber. See my blog post on the matter: http://collectiveidea.com/blog/archives/2011/04/15/language-matters/
It would strongly recommend not using Cucumber until you're more familiar with Rails/Ruby. Cucumber adds a level of abstraction that can make things very confusing, frustrating and time consuming. Just get very good with RSpec first. Then when you want to do integration tests on your Rails app, use Capybara with RSpec. When you're comfortable using those gems, you can go to Cucumber.
Cucumber is about writing user stories. You then translate those Cucumber steps into lines of RSpec and Capybara.
If you are looking for brevity, I would recommend sticking to RSpec. Cucumber's main advantage is its readability to people who do not write code, while RSpec sacrifices that in order to be shorter. I tried to use Lettuce, a python analogue to Cucumber, on one of my projects and found it to be way too much work to keep up.
So, in short: if you want to write shorter tests, use RSpec. If you want your developer to edit your tests or you are working in a team, use Cucumber.
Related
tl;dr: Jump to the last paragraph
Recently I've been trying to use RSpec's request specs to do some more targeted testing.
This is how my testing mostly looks:
general cucumber feature specification, i.e. user goes to a post with comment, upvotes on a comment and the author gets points
model specs for when the model actually has some functinality, i.e. User#upvote(comment)
controller specs where I stub most of the things and just try to make sure the code goes the way I expect
view specs for when there is something complex in the view, such as rendering a upvote link only when the user didn't already upvote, and these are stubbed as well
The problem is when I have some specific scenario which causes a bug and everything seems to work in the model/view layer where I am unable to reproduce it.
That forces me to write an integration test, which I can also do in cucumber. The problem arises once I am able to actually reproduce it, and I need to figure out why is it happening. This usually means playing around in tests, changing different things and seeing what happens.
For example create a comment that is owned by the user who is trying to upvote, try to vote with an expired session etc. However these are really huge pain to write in Cucumber, because of the need to write a scenario and then specify each step.
At this point, I prefer to write a request spec, because it is more low level and allows me to directly do stuff. The problem is, that I'm not really sure how to properly write a request spec, or what are the rules.
A simple example here is:
visit login_path
fill_in "Username", :with => user.username
fill_in "Password", :with => user.password
click_button "Log in"
vs
post sessions_path(:username => user.username, :password => user.password)
or even something more low level like
session[:user_id] = user.id # this actually doesn't work, but the idea is there
Both of these examples achieve the same thing, they'll log a user in. I know that the answer to which one to pick is based on what I need to test, but that doesn't answer the correct, conventional way to do this.
I've been trying to find something about request specs, but they're not really described anywhere. The RSpec book doesn't cover them, the RSpec documentation doesn't say anything either.
What is a correct way to write request specs? When should I use capybara and when just the Rails' #get and #post methods instead of clicking buttons and visiting paths?
For requests spec I believe the convention is to stick to testing user behaviour and interface interactions, which would mean loading the page, filling in the form etc. A website user cant set the session or interact with variables directly so neither should your request specs.
I've often been tempted to skip page loads and form interactions by posting or setting variables in request specs (for speeds sake, especially heavy ajax specs) but it really does break the purpose of a request spec.
As the comments mentioned, you should test the specific controller / view behaviour in the other spec types.
Sermon first....
I think the natural progression of YOU the test writer goes:
Controller
Model
Requests
a mix of Request, Controller and Model specs.
I know that I started looking at the controller first, cause it was easier to grasp.
You then get into Model specs, for non-happy path things...
Then you realize rspec doesn't actually render the view, so you are starting to see dumb errors in Airbrake, so you say, shoot... I need to test the views, and the workflow. Hence Request specs.
Lastly, you get older and realize, all 3 are important and should be used sparingly, but accordingly. I'm just at step 4 now... too many request specs, and you are slogging 5 mins for the whole suite on a medium sized app. sucks.
To answer your question:
I test workflow and views i need to be "seen" (with page.should or any tricky JS (jquery ui selectors for instance) with capybara in a request spec.
If i just need to make sure the controller variables are instantiated or to do something quick with a post for a non-happy path , out of workflow thing... I use the controller. E.g., POST to IPN Controller in paypal...
You'd be surprised how much this covers. This leaves you to test the models for all the whacked out stuff you need for total all the edge cases.
Honestly though, I'd say use Fixtures and Test Unit for integration tests... still like them better, faster... stronger... etc.
I want to start diving into BDD. I have never used TDD before and am
not sure if I should start by learning RSpec and then jump to Cucumber
or just go straight to using Cucumber.
I have been reading on the internet about both and it seems to me that
Cucumber could be a 'replacement' for RSpec. Am I right or should be
one used for certain things and the other one for others?
Cucumber and RSpec are both used for BDD but have different uses.
Think of Cucumber as describing what happens from the user's perspective, through interaction with the web browser. So you can have steps like:
Given I'm not logged in
When I login
Then I should be on the user dashboard page
Pretty broad, but there's a lot going on under the hood there. Cucumber is good for making sure all these sort of high-level features and functionality are covered (e.g., that when your user logs in, they're taken to the right page). But it's not a good tool for testing lower-level code. That's where RSpec comes in.
Take the login example above. Your user may be logging in with an email address or username. You probably want to ensure the email address is valid or that the username is a certain length...or that the thing they're using to login with is unique. You'd do this in your User model with a validation.
It's a trivial example, but this is the kind of thing you'd test using RSpec (in your spec/models/user_spec.rb file). This is not something you'd test using Cucumber.
So bottom line is:
Cucumber for higher-level tests describing broad functionality as viewed from the user's perspective
RSpec for lower-level tests describing details for how your classes, methods, models, controller, etc should actually work.
This post actually does a really good job of explaining when to transition from one tool to another:
http://www.sarahmei.com/blog/2010/05/29/outside-in-bdd/
I also recommend "The RSpec Book" and "Rails Test Prescriptions" for good resources on both tools and testing in general.
P.S. - Not to confuse things, but you can actually use RSpec for the high-level stuff too. But some of that decision is a matter of which tool you prefer or maybe whether or not you're working with a non-technical client who would benefit more from Cucumber's user-friendly syntax for describing scenarios.
Yes, cucumber and rspec are both used for BDD.
I personally prefer Cucumber, but some people find it offputting, and
prefer
their tests to be less english, more code. Both are great tools, though.
Kenton did a great job with this answer. Here is how the authors of RSpec and Cucumber see it:
We use Cucumber to describe the behavior of applications and use RSpec to describe the behavior of objects.
Although we use Cucumber to focus on high-level behavior and use RSpec
to focus on more granular behavior, each can be used for either
purpose.
I have a Rails 2.3.5 app which is serving a card game. I've been a bit lax in writing tests for it (read: I haven't written any :embarrassed:), and I'd like to make a start now.
From reading other questions, I think I probably want to be using Shoulda extending Test::Unit for unit testing.
I wondered about using Capybara extending RSpec for functional testing, but the majority of users' interaction with the app is via POST, which I understand Capybara doesn't handle.
Since this is a card game, I obviously need the ability to control rand; I also definitely need the framework to handle Javascript.
Is Test::Unit + Shoulda suitable?
Am I correct that Capybara can't handle POST? Can I work around that?
Is there a better option instead of Capybara?
Can these methods handle the randomness involved in a card-game? Presumably, for Test::Unit at least, I can just call srand at some early stage to fix the generator, but I don't know if that's doable for Capybara/RSpec/anything else.
Can anyone point me to resources dealing specifically with setting up Shoulda, Capybara and RSpec for Rails 2.3.5? It's a little hard to tell what's Rails 3 specific and what isn't.
Would I gain anything from adding Watir / Firewatir into the mix?
I realise the above questions are manifold, but they basically boil down to "does this work, or have you any better suggestions?"
If the majority of the users' interactions are via POST, is it via an API (as opposed to filling out forms or something)?
Just about any combination of RSpec/Shoulda/Capybara/Test Unit/Rack::Test could work for you depending on your need. They're all capable. However, these are my recommendations:
If you've decided you want integration testing via HTML pages, use Cucumber and Capybara
If you've decided you want integration testing via HTTP API, use RSpec and Rack::Test
You probably want to fake out randomness.
You probably don't need Watir/Firewatir.
It does look like you can make POST requests via some Capybara drivers:
http://suffix.be/blog/capybara-post-requests
When Rails moved to 3.0, RSpec went to 2.0 so for at least RSpec, you'd want RSpec and RSpec Rails 1.3.2.
By "fake out randomess", I mean redefine srand in your tests so you can predictably run them.
module Kernel
def rand
YourApp.rand
end
end
module MyApp
class << self
attr_accessor :rand
end
end
Then, before you have the user press the button, run a step definition like "When random returns 6", which will set MyApp.rand = 6.
After some time of doing Cucumber & RSpec BDD, I realized that many of my Cucumber features are just higher level view tests.
When I start writing my scenario and then go down to RSpec, I don't ever write view specs, since I could just copy and paste part of the scenario, which would be ugly dupliacation.
Take this scenario for example
Scenario: New user comes to the site
Given I am not signed in
When I go to the home page
Then I should see "Sign up free"
I know that this isn't directly testing the view, but writing separate view spec to check for the same thing seems redundant to me.
Am I approaching Cucumber wrong? What exactly should I test in view specs?
Should I write them for every single view, for example testing views for actions like
def show
#project = current_user.projects.first
end
or should I just test more complex views?
It's a widely-accepted (and in my opinion, incorrect) Cucumber philosophy that views should never be tested within RSpec. The argument goes that since the behavior of the view can be described in Cucumber, RSpec should stick to what it knows best -- Models and Controllers.
I argue that the "human-readable" aspect of Cucumber makes some aspects of view-speccing important. For instance, I find view specs to work very well when working in parallel with a front-end developer. If a JavaScript developer knows that he'll want to hook into a selector on your page, it's important that your view provides that selector.
For example:
describe 'gremlins/show.html.haml' do
context 'given it is after midnight' do
it 'has a #gremlin_warning selector' do
Time.stub!(:now).and_return(Time.parse '2010-12-16 00:01:00')
rendered.should have_selector '#gremlin_warning'
end
end
context 'it is before midnight' do
it 'does not have a #gremlin_warning selector' do
Time.stub!(:now).and_return(Time.parse '2010-12-16 23:59:00')
rendered.should_not have_selector '#gremlin_warning'
end
end
end
Note that the specs do not describe the content, they are willfully brief, and they do not describe interaction behaviors. Because the view is the portion of your application that will change the most, view specs should be used sparingly.
tl;dr: View specs are for communicating a contract to other developers and should be used sparingly (but nonetheless should be used).
Personally, I never use view specs when using Cucumber. To me, acceptance tests make a lot more sense, and my complex views are generally Javascript-focused and cannot be tested using view specs.
Don't use view specs for anything, ever. Cucumber stories -- or even RSpec integration tests -- do that better. The examples bobocopy gives are good ones for the case he postulates, but they should be rolled into Cucumber stories/integration tests, not left on their own.
I use RSpec to test my lovely little web app. For integration tests I use Steak. When using Rails generators (yep, I know that this is not the Zen way of doing TDD) there are also some files in spec/requests generated. As stated on link text it is something similiar to integration test (but I couldn't find much more info).
Are those request specs still recommended when using something like Steak and Cucumber?
It all depends on what you need and want. The goal of testing is to prove that your app works once, not twice or more times.
I personally write rspec tests for models and helpers. I use cucumber to test that my views and controllers are working the way I expect them to. With this I can prove that my entire app works as I expect it to, so no, I don't use spec/requests.
Occasionally I do use spec/requests to test APIs, but you can do that with cucumber as well.
Some don't like the BDD-way cucumber works and stick with spec/requests. In the end it's all a matter of taste.