I currently use Watir-webdriver for all my front end testing, but the development team use Capybara to run their tests on Jenkins CI. We both use the same Cucumber features.
Is it worth us doing seperate tests and effectively twice?
Which one is the better tool to use? I've read all the comparisons available on the web and prefer using watir-webdriver, but Capybara allows me to use headless testing seamlessly.
So any useful thoughts and ideas? I'm at a crossroads and not sure whether to give up Watir-webdriver and just go with the rest of the dev team and use Capybara.
As to whether using two different tools for the same cucumber features depends on the test domain and which suits it best. If Watir-webdriver suits front-end testing better than Capybara-webdriver, or vice versa. You'd need to invest some time investigating Capybara (a design spike, to use Agile parlance) to compare it to what you're used to. Try to be objective, even though it's very difficult (in my experience) with a familiar framework, and then do a Cost Benefit Analysis of changing. The direct Cost may only be time/effort but it's still a cost which can impact project delivery.
Capybara is not incompatible with DRY programming - an abstraction layer (like Abe mentions in his reply) is possible with Capybara's custom selectors. Once these have been created, you can use capybara finders and matchers on your custom selectors like so;
Capybara.add_selector(:slide_show) do
xpath { ".//div[contains(#class,'slideshow')]" }
end
Capybara.add_selector(:slide_show_element) do
xpath { ".//div[contains(#class,'slideshowelem')]" }
end
allows you to use Capybara's find;
find(:slide_show_element, 'Sunset').click
From the Cucumber perspective it's possible to pass that locator string through from the step;
And I Click "Sunset" in the "Holiday Pictures" Slide Show
through a step definition like this;
Given /^(?:I |)Click "(.*?)" in the "(.*?)" Slide Show$/i do |picture,slideshow|
within(:slide_show, slideshow) do
find(:slide_show_element, picture).click
end
end
So, the question remains one of process - is having two frameworks impacting your workflow? And would reconciling the two involve an unacceptable loss of time?
We're having exactly the same sort of discussion on my current project. One of the developers is a big fan of Capybara, whereas my familiarity is with Watir-webdriver.
Capybara has a real simplicity to it, which makes it very quick to get tests up and running but I worry about maintenance issues over time, and I also worry that its simplicity may also mean a lack of flexibility.
Watir-webdriver allows me to write an abstraction layer for my web pages, identifying elements exactly how I want/need to (flexibility), which also means that the "testing" layer can still look simple to the people writing the test scenarios, like Capybara, but maintenance over time will be rapid, because the code is inherently kept DRY. There's a little more up-front cost to getting that page element layer written, but I have a sense of security that future maintenance of the framework will be rapid.
Of course, I'm biased--but open-minded. I'd love to hear a counterpoint from a Capybara fan.
If you are on Linux, you can run real browsers headless with watir-webdriver: http://watirwebdriver.com/headless/
There is probably a way to run headless browser(s) using watir-webdriver, if that is what you want to do.
Is it better to use one tool or both depends on your context. Do you have any problems with using both tools?
Related
Suppose you are building an API in Rails. Is it enough if we write request specs alone without the model specs, controller specs and the view specs? Why do we need unit testing if we do acceptance and functional testing or feature testing for front end web projects. I insisted on doing unit testing as it allows you to write decoupled code but my colleague is against it. What are the best practices on this in the ruby on rails community?
If you only have time to do one type of testing, and you are writing an API, then it might make sense to only do feature testing by simply calling your API endpoints. After all, it's pretty important that those endpoints return the expected results.
However, when your feature tests start breaking, you will potentially have a terrible time figuring out the source of the problems without unit tests. Is there a core piece of your software that most of your endpoints are using? Good luck refactoring that without a robust set of unit tests.
But that really speaks to what you have to figure out -- is your core set pretty stable? Are you really just adding features or new endpoints? If so, you can probably get away with a heavy feature test approach.
Rails and testing go together like peas and carrots. Here is a great resource that highlights (better than I ever could) the importance of using tests (of all degrees) in your Rails projects. Hope this helps!
Not sure about "the community", but in my opinion it depends on the complexity of the project. If it's a very straightforward API project, doing only feature tests may be fine.
But if the project becomes larger, unit tests allow you to better pinpoint errors in case anything breaks. I.e. you don't see "there's a bug somewhere in feature X" but "this or that class does not work when invoking a particular method with specific arguments".
I'm really getting frustrated with learning how to properly develop software using TDD. It seems that everyone does it differently and in a different order. At this point, I'd just like to know what are all the considerations? This much is what I've come up with: I should use rspec, and capybara. With that said, what are all the different types of test I need to write, to have a well built and tested application. I'm looking for a list that comprises the area of my application being tested, the framework needed to test it, and any dependencies.
For example, it seems that people advise to start by unit testing your models, but when I watch tutorials on TDD it seems like they only write integration test. Am I missing something?
Well, the theme "how do you TDD" is as much out there in the open as the theme "how do you properly test?". In Ruby, and more specifically in Rails, rspec should be the tool to start with, but not be done with. RSpec allows you to write Unit Tests for your components, to test them separately. In the Rails context, that means:
test your models
test your controllers
test your views
test your helpers
test your routes
It is a very good tool not exactly rails-bound, it is also used to test other frameworks.
After you're done with RSpec, you should jump to cucumber. Cucumber (http://cukes.info/) is the most used tool (again, for the Rails environment) to write integration tests. You can then integrate capybara on cucumber.
After you're done with cucumber, you'll be done with having tested your application backend and (part of) its HTML output. That's when you should also test your javascript code. How to do that? First, you'll have to Unit test it. Jasmine (http://pivotal.github.com/jasmine/) is one of the tools you might use for the job.
Then you'll have to test its integration in your structure. How to do that? You'll come back to cucumber and integrate selenium (http://seleniumhq.org/) with your cucumber framework, and you'll be able to test your integration "live" in the browser, having access to your javascript magic and testing it on the spot.
So, after you're done with these steps, you'll have covered most of the necessary steps to have a well-integrated test environment. Are we done? Not really. You should also set a coverage tool (one available: https://github.com/colszowka/simplecov) to check if your code is being really well tested and no loose ends are left.
After you're done with these morose steps, you should also do one last thing, in case you are not developing it all alone and the team is big enough to make it still unmanageable by itself: you'll set a test server, which will do nothing other than run all the previous steps regularly and deliver notifications about its results.
So, all of this sets a good TDD environment for the interested developer. I only named the most used frameworks in the ruby/rails community for the different types of testing, but that doesn't mean there aren't other frameworks as or more suitable for your job. It still doesn't teach you how to test properly. For that there's more theory involved, and a lot of subdebates.
In case I forgot something, please write it in a comment below.
Besides that, you should approach how you test properly. Namely, are you going for the declarative or imperative approach?
Start simple and add more tools and techniques as you need them. There are many way to TDD an app because every app is different. One way to do that is to start with an end-to-end test with Rspec and Capybara (or Cucumber and Capybara) and then add more fine-grained tests as you need them.
You know you need more fine-grained tests when it takes more than a few minutes to make a Capybara test pass.
Also, if the domain of your application is non-trivial it might be more fruitful for you to start testing the domain first.
It depends! Try different approaches and see what works for you.
End-to-end development of real-world applications with TDD is an underdocumented activity indeed. It's true that you'll mostly find schoolbook examples, katas and theoretical articles out there. However, a few books take a more comprehensive and practical approach to TDD - GOOS for instance (highly recommended), and, to a lesser extent, Beck's Test Driven Development by Example, although they don't address RoR specifically.
The approach described in GOOS starts with writing end-to-end acceptance tests (integration tests, which may amount to RSpec tests in your case) but within that loop, you code as many TDD unit tests as you need to design your lower-level objects. When writing those you can basically start where you want -from the outer layers, the inner layers or just the parts of your application that are most convenient to you. As long as you mock out any dependency, they'll remain unit tests anyway.
I also have the same question when I started learning rails, there're so many tools or methods to make the test better but after spending to much time on that, I finally realized that you could simply forget the rule that you must do something or not, test something that you think it might have problem first, then somewhere else. Well ,it needs time.
that's just my point of view.
Have you used Cucumber in a production environment?
What problems did you find using it?
Would you recommend I use it?
I'm currently learning to use Cucumber with RSpec, I'm weighing up Pro's and Con's for using it in upcoming projects and would like some input. I will be using RSpec but I'm wondering if using Cucumber will be worth the extra time taken to write the tests.
Extra Information:
I don't deal with clients directly; however I do work in a team and deal with project managers who don't have much technical knowledge so I thought the readability of Cucumber tests would prove useful
The project sizes are medium to large, projects can have multiple phases and we maintain them
Time restrictions are pretty tight however I’m thinking the extra test coverage would hopefully catch more bugs reducing time overall
Any information would be great
Cheers
Personally I don't use Cucumber.
I imagine there are great use cases but I find the idea of writing plain english 'cukes' - that I then have to essentially write ruby regular expressions for to turn the plain english tests into ruby that can be run just too much of an overhead. That said, if you have business analysts or even the customer able to write tests then Cucumber is great since they can write english and you just have to worry about making it work.
We've opted to use Steak which is built straight on top of Rspec2 and Capybara and lets us write straight ruby tests which we feel are just as descriptive without the extra overhead.
I am starting on BDD. Was wondering which would be better to start with Cucumber or Spinach. My impression is that Spinach is new off the block. Look here
Which one should I start with. The criteria would be -
Support across the board.
Flexibility of use
Third party tool and APIs integration.
Again it might be ignorant question of the newbie: Where does capybara fit into the picture.
For some context, I've been a long time user of Cucumber, but always wished it was Spinach since day one. I'm switching all my projects to Spinach despite its shortcomings because it uses the new, hot-off-the-block PORO technique (Plain Old Ruby Objects ;). Now I can expand my steps however I want, because it's just Ruby.
To answer your question, as of this writing:
Support across the board.
Cucumber
Spinach is still developing some features, including 'Background' blocks, and I'm currently trying to get it to recognize tables.
Flexibility of use
Spinach
Cucumber encourages bad step design from the start, IMO. If you create feature-specific steps, you'll trip over them later, and if you create reusable global steps, your feature definitions will be long, generic, and boring to read. I've heard people claim they can walk a balance successfully and be just specific enough but still have reusable steps; I consider myself well versed enough that if I can't do it reliably, it's too hard.
Third party tool and APIs integration
Cucumber, assuming the bullet point could be interpreted as community.
If it's really "third party tools and API integration" you're after, Spinach supports capybara and rspec, which is most of what you're after. Cucumber has 3rd party reusable step libraries, but as noted in my earlier point, I think this is bad. In regard to 3rd party & integrations, even if it's not there yet, your really can't get any better than plain old ruby objects.
Where does capybara fit into the picture
Capybara is your test interface to your site, aka a testing mouse & keyboard. You could start it up in a console and drive your app, but that'd get repetitive. Cucumber/Spinach (or rspec/test-unit/minitest) all could use capybara to automate testing your app. People prefer Cucumber/Spinach because they help you step out of the code for a bit to think like a user.
Overall, you'd probably be best off getting an rspec/cucumber book and doing what it says. Just be aware that testing takes a while to get good at, so don't stop there. Maybe check out Spinach somewhere in the process; if you like Cucumber, you might find you'll really like Spinach.
DISCLAIMER: I'm a Spinach mantainer.
If you're starting with BDD I'd highly recommend two books:
The RSpec book
The Cucumber book
I think it's important to learn all the BDD and TDD process (outside-in etc..) and then choose the tool you feel more comfortable with.
Having said that, Cucumber has a huge community, but a lot of things are also aplicable to Spinach, since what they have in common is Gherkin.
As for flexibility of use I would say both are really flexible, but I (obviously) prefer Spinach as every feature it's just a Ruby class, where you can include modules, inherit from other classes and so on (this also applies to APIs integration).
I you want, you can take a look at the spinach-rails-demo and see how everything works.
If you're stuck with Cucumber and you don't want global steps, you can work around the problem by tagging the steps with some sort of scenario ID:
# features/1_greetings.feature
Scenario: Formal greeting
Given I have an empty array [#1]
And I append my first name and my last name to it [#1]
When I pass it to my super-duper method [#1]
Then the output should contain a formal greeting [#1]
The #1 scenario id can be any value. I like to use ticket numbers for future reference.
You can then place all the steps in one step definition file. It's close enough to the look of Spinach::FeatureSteps. No regex arguments too!
# features/step_definitions/1_greetings.rb
Given 'I have an empty array [#1]' do
#...
end
And 'I append my first name and my last name to it [#1]' do
#...
end
When 'I pass it to my super-duper method [#1]' do
#...
end
Then 'the output should contain a formal greeting [#1]' do
#...
end
I posted more about the workaround at github.
I can't really speak for Spinach, as I've never used it, but Cucumber definitely has a huge community support with loads of external libraries.
Capybara allows you to easily test web applications
When I fill in "username" with "foo"
And I click on "login"
Then I should see "enter your password"
I am starting a new project for a client today.
I have done some rails projects before but never bothered writing tests for them.
I'd like to change that starting with this new project.
I am aware there are several testing tools, but am a bit confused as to which I should be using.
I heard of RSpec, Mocha, Webrat, and Cucumber.
Please keep in mind I never really wrote any regular tests, so my knowledge of testing in general is quite limited.
How would you suggest I get started?
Thanks!
Thank you for all the responses! I posted a related question that may interest those who view this question in the future. You can find it here.
I prefer using Rspec and Cucumber in conjunction. I also prefer Capybara over Webrat.
Rspec is wonderful for it's rails integration and structure. It's technically behavior testing, but in practice it ends up feeling like unit testing. It also allows you to group and define "specs", or rspec tests, in any structure you like with 'describe' blocks.
Cucumber is higher level. It is what you will translate your user stories into. For instance, you may decide that your users should be able to change their password, so you'll write something like this:
Scenario: Change password
Given I am logged in
And I am on the change password page
When I fill in "p4ssw0rd" for "old_password"
And I fill in "newp4ssw0rd$$" for "new_password"
Then my password should be "newp4ssw0rd$$"
If you are working with a QA dept, you can show your tests to them, and they'll instantly know the status of any given feature, and how the feature is supposed to work. Additionally, if you have enterprising testers, they can write tests themselves, even if they are not programmers in general.
They all serve different function and if you really care about the quality of your software, you would use them all.
RSpec is more for unit testing. Unit Testing is testing the smallest part, usually your models methods.
WebRat is more for functionality testing as end user would test it from a browser. You are actually testing the interaction and test whether it returns the correct view/data.
Cucumber is more like for behaviour driven testing or end-to-end testing. It is more to test the business logic according to many different scenarios or data.
Of course you could use all of them together in conjunction. For example, I can use Cucumber to do end-to-end testing and it would involve functional testing using WebRat and Rspec as well. Hopefully this will get the idea for you to start off.
You can have a look at the most popular testing frameworks here http://ruby-toolbox.com/
Cheers