I have been trying to determine a good way to test rails apps, covering all relevant aspects... since I have no experience writing tests at all I have been looking at railscasts.com which seems like a pretty good resource to learn basic stuff for rails. but for tests I have found several episodes, some using rails unit test classes, others use Cucumber, RSPEC, ZenTest, Autotests, Webrat, Selenium, etc.
So Im now just not sure if Im supposed to pick one of these or a combination. are some of these now old and obsolete?? are they mutually exclusive?... basically I want to learn testing for rails and would like to hear opinions on how to properly and sufficiently test rails apps and what framework to use for the different types of tests
please help, thanks in advance
ps. if someone can provide a good resource for learning the difference between unit tests, integration tests and so on I'd appreciate it.... I know I didnt do TDD or BDD cause my app is pretty much done now, and all seems to be working just fine. but I still would like to add tests for learning purposes and future developments.
http://guides.rubyonrails.org/testing.html
Related
I have to preface all my posts about rails: I'm a novice.
Is it necessary to write tests for my app to work properly or is it strictly for finding breaks?
Testing your app is not necessary for it to work, but it is highly recommended and a very good practice. Testing your application will help you develop a better piece of software and a much more solid application. In the Rails world BDD(Behavior Driven Development) is very used as a testing and development technique.
I recommend you two diferent testing suites:
The first one is Rspec that will help you with all your controllers, and models unit testing
Cucumber is a testing suite that will test your application as a whole(integration test), this one is great for a more "real life" testing approach
I encourage you to check both Rspec and Cucumber, there are also other great testing suites like Test Unit.
Remember, testing your application will give you great benefits!
NOTE: Rspec and Cucumber are not mutually exclusive, actually they are recommended to use in conjunction
Writing tests is not necessary in the sense that your application won't operate without them, but they are not only for keeping bugs away. If you're a novice, writing the tests should also help you to understand how everything works.
There's no reason not to write tests. Just write good tests, and don't waste time testing things that don't need to be tested (like generated attr_accessors).
It's not necessary at all but its considered by almost every ruby developer I know to be standard procedure.
I did a couple "rails apps" without testing, but as soon as I needed some real backend logic in ruby, testing helped me to understand what I was doing.
No, writing tests is not required for your application to run. It is a good practice, though, so if you are not used to writing tests I'd recommend that you start learning. It's easy and it will save you a lot of headaches on whatever platform you are using.
Just to chime in with everyone else, I'd suggest you find yourself an article about BDD and test first development. And then read up on mocking and stubbing. Wrapping your head around the why and how of it will probably convince you it's worth the time and effort.
When I first dove into the XP & RoR world I live in now I was daunted by what felt to me like test-mania but it really pays off in spades.
The first time someone had me write the test first THEN write the code we were testing I was mind blown. But I've never gone back to my previous evil ways.
I would like to write some automatic tests for my Rails 3 application.
I wonder how to start with that.
I've heard about Selenium/RSpec/Cucumber, and I guess there are many more options.
What are the advantages/disadvantages of these testing frameworks ? Which of them has the best documentation ? Which one is the most popular in the Ruby world ? And in the industry at all ?
I have the general knowledge of how to write tests. I just want to learn the appropriate testing framework(s) for testing Rails applications.
Please help me to decide with which testing framework to start.
There are many tools for testing rails and other webapps from many different aspects. But if you are new to testing I highly recommend you start with learning Rails own testing framework before start using other tools.
Learning, and later mastering, one testing framework makes it easier in the future to understand pros/cons with other framework and make them work in unison.
You could start with testing the following things:
Unit Testing your Models
Functional Tests for Your Controllers
Learning about Fixtures and how to load test data
I have seen many failed testing efforts, but I never saw them fail because they choose the wrong tool/framework. They fail because they don't know how to master the tools they use, and learn enough about the basics about testing.
Read more about Rails testing here.
http://guides.rubyonrails.org/testing.html
Manual Exploratory Testing
As much as I love automated testing it is, IMHO, not a substitute for manual testing. The main reason being that an automated can only do what it is told and only verify what it has been informed to view as pass/fail. A human can use it's intelligence to find faults and raise questions that appear while testing something else.
Read more about mixing Automated and Manual Testing in another of my answers here:
What test methods do you use for developing websites?
I wouldn't say there is any one best set of testing tools out there; and the community definitely has not decided on any being standard in any way.
I highly recommend The RSpec Book from the Pragmatic Programmers-- about half of the book is on testing with Rails with Cucumber, RSpec, and browser simulators like Selenium. It's a comprehensive overview of the different situations you will find yourself in with Rails and what tools you might want to use in each situation.
Selenium is a framework to automate testing of user interface. Selenium script launches a browser window, goes to a web-page and manipulates page elements.
RSpec and Cucumber are tools for what is called "behavior driven development (BDD)". BDD is a development process where you have description of a separate feature, and some script (can be a Selenium script) that checks this feature. RSpec and Cucumber basically connect these two, so that when you run a test script, you see the list of features written in plain language and their testing status.
Selenium scripts can be written in a number of languages, while Cucumber scripts use Ruby. Documentation is OK for all of them, some knowledge of programming language is required though.
Starting links:
Cucumber;
Selenium;
Testing rails applications
The easiest way to get started is using TestUnit. When you generate a model, view, controller using scaffolding it will automatically generate a testing folder for you containing a test environment setup file and tests, or you can add tests to existing models with a rails generator.
It is then a matter of reading the documentation on TestUnit, of which there is a fair amount.
For a next step, my personal preference is to use RSpec over Cucumber - I've found Cucumber hard to maintain, and whilst it is wonderful to have tests that are readable by your boss, I've found that most bosses don't really want to read or contribute to tests, and it makes for time-consuming development and is very 'bitty' - lots of small bits of text scattered in various files.
If you want to jump right in and begin using RSpec for Rails, then I'd recommend Rspec-Rails, which includes many helpers for asserting the correct behaviour of your application.
I could go on, but you've asked for 'where do I start' - there's two good places.
There are a bunch of frameworks that help you test your ruby/rails code... the great thing is the dynamic nature of Ruby gives you a lot of flexibility.
I like to use RSpec to test the internals of my code, and Cucumber to test the application's behavior. So generally, RSpec will test models and methods, and Cucumber tests user interaction through the browser.
I highly recommend this pragmatic programmers ebook on BDD using RSpec on Cucumber.
Railscast on getting started in cucumber (also called cukes)
Cucumber project
Rails project
I want to learn how to write unit tests for a Rails App. Where do I start? All the tutorials on Google are old, back from 2007, and there seem to be an abundance of choice but not a preferred solution.
Right now most people seem to be using Rspec for unit testing and Cucumber for integration testing. You can see a fairly recent poll here where 87% chose Rspec in a survey.
A great book for Rspec and Cucumber is The Rspec Book written by the current maintainer of Rspec. It goes over both Rspec and Cucumber.
Railscasts also has a few relevant screencasts. Cucmber1, Cucumber2
Many people seem to love Cucumber but it doesn't seem as useful if you don't have a client you're doing work for. Most of my projects are side projects I do myself so these days I'm looking into Steak instead of Cucumber. This will allow me to use Rspec for unit tests and Steak for integration tests and reduces some complexity/tedium that is introduced when using Cucumber.
People have said that it doesn't really matter which testing framework you pick, it's more important that you START TESTING. I agree with that but hopefully these resources will help you get started.
However, one thing you probably want to avoid from the very start are fixtures. Use factories instead and check out this Railscasts episode on it.
UPDATE: Steak is no longer necessary and the same functionality is baked into Rspec.
I'd suggest starting with the Rspec Book by David Chelimsky and friends. It goes over the various types of testing, why you should test and runs through various examples that show you good practices as well as giving you guidelines for what to avoid.
Rspec is very popular and preferred by many over the default rails test suite. Most material I've read that goes over using the default test suit ends with an introduction to Rspec and how it makes life simpler.
If you want to do it the proper way, you should use others testing frameworks like Rspec, Cucumber or shoulda. This way you will write Unit Tests in a BDD context. Those frameworks are recent so obviously the documentation that you will find about them are recent enough! Finally, I'd suggest you to read this great blog post from Sarah Mei about the outside-in approach combining Rspec/Cucumber, that's how I was convinced to use these frameworks, she really explains very well how you should approach the tests!
So yeah, I'm gettin' into this testing thing!
But I don't know what to use =/
Rspec + Shoulda?
Rspec + Steak?
Minitest?
Cucumber?
Capybara?
Coulda? (Cucumber with Shoulda mixup)
Mini_shoulda? (Minitest with Shoulda mixup)
Argh, so many choices! I'm confused <.< I do know I want something small and simple to test my future gems and sinatra & rails apps. What do you people use and why?
If you are just starting out, you should go with the defaults and follow the official guides. The important thing is that you practice testing. It's less important which testing library/framework you use - that's a stylistic choice that's secondary to actually practicing testing in the first place - and the easiest way to get starting testing your Rails app is to follow the "Golden Path" that the official guides lay out.
Once you get more comfortable with testing your Rails app, you will be in a much better position to evaluate the options out there. I personally like rspec and cucumber (at the moment - I may decide to change my preference), but I was only able to come up with this preference after already being somewhat familiar with testing and after already being somewhat familiar with Rails.
I just came across this blog post which states:
It seems that the accepted way to test in Ruby is to use Rspec for unit tests and to use Cucumber for acceptance tests (higher level functional testing).
It then goes on to ask: "if we agree that BDD is good, why don’t we write our unit tests in a format that is more amenable to BDD, that being the Cucumber format of tests?" and compares rspec unit tests against unit tests in Cucumber.
At work we use the pretty standard RSpec and Cucumber combination. I don't know how they decided on this specific combination, it was already there when I started and worked for us, so there was no need to change. Also it's a quite widely use combination so finding examples with Google etc. isn't too hard.
For my 1.9 based private projects I think I will use Minitest from now on. No external dependencies, a simple BDD DSL (require 'minitest/spec') and some other niceties like randomization. Here's a nice quick intro:
http://www.bootspring.com/2010/09/22/minitest-rubys-test-framework/
For someone just getting started I recommend just using RSpec. It doesn't do everything but it will allow you to build up reasonable sets of unit and integration tests. When you run into the limits of what RSpec handles easily then let that need guide you in choosing additional tools.
Can't offer anything more sensible than Justice, but before you ignore that wisdom :-) do check out this video (minitest author Ryan Davis at Cascadia 2011 ruby conf):
http://confreaks.net/videos/618-cascadiaruby2011-size-doesn-t-matter
slides: http://www.zenspider.com/~ryan/presentations/CascadiaRubyConf_2011_-_Size_Doesn%27t_Matter.pdf
Being able to run all your tests very quickly is a very good thing.
I'm new to rails, and I read recently on the internet (so it must to be true) that the TDD library that comes with rails is incompatible with RSpec, but also I read that RSpec is the right tool to do test.
So, my question is, if this is truth, what is the right tool to make test with rails: the rails TDD or RSpec? Or are this 2 tools total different purposes?
Thank for the clarifications!!!
TDD means test driven development. It's a methodology, not a library.
Rails ships with Ruby's Test::Unit. It is easily replacable with other libraries such as Rspec, should you wish to do so.
There is no "right way" when it comes to which tool to use. It's all down to preference. I prefer Rspec personally...
RSpec is pure Ruby and a very good way to write tests for your code. It is easy to integrate using Gems and is straightforward to work with.
There are many tools for testing rails and other webapps from many different aspects. But if you are new to testing I highly recommend you start with learning Rails own testing framework before start using other tools.
Learning, and later mastering, one testing framework makes it easier in the future to understand pros/cons with other framework and make them work in unison.
You could start with testing the following things:
Unit Testing your Models
Functional Tests for Your Controllers
Learning about Fixtures and how to load test data
I have seen many failed testing efforts, but I never saw them fail because they choose the wrong tool/framework. They fail because they don't know how to master the tools they use, and learn enough about the basics about testing.
Read more about Rails testing here.
http://guides.rubyonrails.org/testing.html
Manual Exploratory Testing
As much as I love automated testing it is, IMHO, not a substitute for manual testing. The main reason being that an automated can only do what it is told and only verify what it has been informed to view as pass/fail. A human can use it's intelligence to find faults and raise questions that appear while testing something else.
Read more about mixing Automated and Manual Testing in another of my answers here:
What test methods do you use for developing websites?