I've been searching on how to do Unit testing and find that it is quite easy, but, what I want to know is, In a asp.net mvc application, what should be REALLY important to test and which methods you guys use?
I just can't find a clear answer on about WHAT TO REALLY TEST when programming unit tests.
I just don't want to make unnecessary tests and lose development time doing overkill tests.
You should unit test as much as possible of your application.
For every line of code you write, you need to verify that it works. If you don't unit test it, you need to test it in some other fashion. Even starting up the site and clicking around is a sort of testing.
When you compare unit testing with other sorts of testing (including running the site and manually using it), unit tests tend to give the best return of investment because they are relatively easy to write and maintain, and can give you rapid feedback on whether you just introduced a regression bug or not.
I'm not saying that there's no overhead in writing unit tests - there is, but there's overhead in any sort of testing, and a big overhead in not testing at all (because regression bugs slip through quite easily).
It's still good practice to supplement unit tests with other types of tests, but a good unit test suite offers an excellent regression test suite.
Ron Jeffries says "Test everything that could possibly break."
Someone else - I think it was Kent Beck, but I can't find a reference - says, "Only test the code you want to work."
Either of these is a pretty good strategy.
I actually don't think anything needs to be tested in MVC. I think all your business logic, rules etc need testing but the Views and Controllers?
The only real reason I can see to test a Controller is for integration testing. If all your business logic is correct then that should be a simple test always returning true.
Controllers should really only get data from the view and pass data to it so....
As for views, what sort of testing can be done there other than to open the view and see what it does?
When I write my projects there is next to no code in the controller and I put all the grunt in my business engine which I have extensive tests for.
Unit testing is good for testing of services/models. But when you need in testing of application functionality the better choice will be functional tests (i.e. Selenium)
Related
Suppose you are building an API in Rails. Is it enough if we write request specs alone without the model specs, controller specs and the view specs? Why do we need unit testing if we do acceptance and functional testing or feature testing for front end web projects. I insisted on doing unit testing as it allows you to write decoupled code but my colleague is against it. What are the best practices on this in the ruby on rails community?
If you only have time to do one type of testing, and you are writing an API, then it might make sense to only do feature testing by simply calling your API endpoints. After all, it's pretty important that those endpoints return the expected results.
However, when your feature tests start breaking, you will potentially have a terrible time figuring out the source of the problems without unit tests. Is there a core piece of your software that most of your endpoints are using? Good luck refactoring that without a robust set of unit tests.
But that really speaks to what you have to figure out -- is your core set pretty stable? Are you really just adding features or new endpoints? If so, you can probably get away with a heavy feature test approach.
Rails and testing go together like peas and carrots. Here is a great resource that highlights (better than I ever could) the importance of using tests (of all degrees) in your Rails projects. Hope this helps!
Not sure about "the community", but in my opinion it depends on the complexity of the project. If it's a very straightforward API project, doing only feature tests may be fine.
But if the project becomes larger, unit tests allow you to better pinpoint errors in case anything breaks. I.e. you don't see "there's a bug somewhere in feature X" but "this or that class does not work when invoking a particular method with specific arguments".
I'm really getting frustrated with learning how to properly develop software using TDD. It seems that everyone does it differently and in a different order. At this point, I'd just like to know what are all the considerations? This much is what I've come up with: I should use rspec, and capybara. With that said, what are all the different types of test I need to write, to have a well built and tested application. I'm looking for a list that comprises the area of my application being tested, the framework needed to test it, and any dependencies.
For example, it seems that people advise to start by unit testing your models, but when I watch tutorials on TDD it seems like they only write integration test. Am I missing something?
Well, the theme "how do you TDD" is as much out there in the open as the theme "how do you properly test?". In Ruby, and more specifically in Rails, rspec should be the tool to start with, but not be done with. RSpec allows you to write Unit Tests for your components, to test them separately. In the Rails context, that means:
test your models
test your controllers
test your views
test your helpers
test your routes
It is a very good tool not exactly rails-bound, it is also used to test other frameworks.
After you're done with RSpec, you should jump to cucumber. Cucumber (http://cukes.info/) is the most used tool (again, for the Rails environment) to write integration tests. You can then integrate capybara on cucumber.
After you're done with cucumber, you'll be done with having tested your application backend and (part of) its HTML output. That's when you should also test your javascript code. How to do that? First, you'll have to Unit test it. Jasmine (http://pivotal.github.com/jasmine/) is one of the tools you might use for the job.
Then you'll have to test its integration in your structure. How to do that? You'll come back to cucumber and integrate selenium (http://seleniumhq.org/) with your cucumber framework, and you'll be able to test your integration "live" in the browser, having access to your javascript magic and testing it on the spot.
So, after you're done with these steps, you'll have covered most of the necessary steps to have a well-integrated test environment. Are we done? Not really. You should also set a coverage tool (one available: https://github.com/colszowka/simplecov) to check if your code is being really well tested and no loose ends are left.
After you're done with these morose steps, you should also do one last thing, in case you are not developing it all alone and the team is big enough to make it still unmanageable by itself: you'll set a test server, which will do nothing other than run all the previous steps regularly and deliver notifications about its results.
So, all of this sets a good TDD environment for the interested developer. I only named the most used frameworks in the ruby/rails community for the different types of testing, but that doesn't mean there aren't other frameworks as or more suitable for your job. It still doesn't teach you how to test properly. For that there's more theory involved, and a lot of subdebates.
In case I forgot something, please write it in a comment below.
Besides that, you should approach how you test properly. Namely, are you going for the declarative or imperative approach?
Start simple and add more tools and techniques as you need them. There are many way to TDD an app because every app is different. One way to do that is to start with an end-to-end test with Rspec and Capybara (or Cucumber and Capybara) and then add more fine-grained tests as you need them.
You know you need more fine-grained tests when it takes more than a few minutes to make a Capybara test pass.
Also, if the domain of your application is non-trivial it might be more fruitful for you to start testing the domain first.
It depends! Try different approaches and see what works for you.
End-to-end development of real-world applications with TDD is an underdocumented activity indeed. It's true that you'll mostly find schoolbook examples, katas and theoretical articles out there. However, a few books take a more comprehensive and practical approach to TDD - GOOS for instance (highly recommended), and, to a lesser extent, Beck's Test Driven Development by Example, although they don't address RoR specifically.
The approach described in GOOS starts with writing end-to-end acceptance tests (integration tests, which may amount to RSpec tests in your case) but within that loop, you code as many TDD unit tests as you need to design your lower-level objects. When writing those you can basically start where you want -from the outer layers, the inner layers or just the parts of your application that are most convenient to you. As long as you mock out any dependency, they'll remain unit tests anyway.
I also have the same question when I started learning rails, there're so many tools or methods to make the test better but after spending to much time on that, I finally realized that you could simply forget the rule that you must do something or not, test something that you think it might have problem first, then somewhere else. Well ,it needs time.
that's just my point of view.
Seems like the internet doesn't have a definitive answer, or set of principles to help me answer the question. So I turn to the great folk on SO to help me find answers or guiding thoughts :)
SpecFlow is very useful for BDD in .NET. But when we talk about BDD are we just talking integration/acceptance tests, or are we also talking unit tests - a total replacement for TDD?
I've only used it on small projects, but I find that even for my unit tests, SpecFlow improves code documentation and thinking in terms of language. Converseley, I can't see the full code for a test in one place - as the steps are fragmented.
Now to you..........
EDIT: I forgot to mention that I see RSpec in the RoR community which uses BDD-style syntax for unit testing.
I've recently started to use SpecFlow for my BDD testing, but also, I still use unit and integration tests.
Basically, I split the tests into seperate projects:
Specs
Integration
Unit
My unit tests are for testing a single method and do not perform any database calls, or external references whatsoever. I use integration tests for single method calls (maybe sometime two) which do interact with an external resources, such as a database, or web service, etc.
I use BDD to describe tests which mimick the business/domain requirements of the project. For example, I would have specs for the invoice generation feature of a project; or for working with a shopping basket. These tests follow the
As a user, I want, In order to
type of semantics.
My advise is to split your tests based on your needs. Avoid trying to perform unit testing using SpecFlow.
We have started using Specflow even for our unit tests.
The main reason (and benefit) for this is that we find that it forces you to write the tests from a behavior point of view, which in turn forces you to write in a more implementation agnostic way and this ultimately results in tests which are less brittle and more refactoring friendly.
Sure this can also be done with standard unit testing frameworks, but you aren't guided that way as easily as we have found we are using specflow and the gherkin syntax.
There is some overhead setting things up for specflow, but we find this is quickly repaid when you have quite a few tests (due to the significant step reusability that you can get with specflow) or you need refactor your implementation.
Plus you get nice readable specs that are easy for newcomers to the team to understand.
Given:
Unit tests are test of (small) "units of code"
The customer of most “units of codes” are other programmers.
Part of the reason for having a unit test is to provide an example of how to call the code.
Therefore:
Clearly unit tests should normally be written in the programming language that the users of the “unit of code” will be calling it with.
However:
Sometimes data tables are needed to setup the conditions a unit test runs in.
Most unit test frameworks are not good at using tables of data.
Therefore:
Specflow may be the best option for some unit test, but should not be your default choose.
I see it as an integration testing which mean it doesn't replace your unit test cases written as part of your TDD process. Someone will have different opinion about this. IMHO unit test case only test the methods/functions and all the dependencies should be mocked and injected. When in it comes to integration testing, you will be injecting real dependencies instead of mocked one. You could do the same integration testing with any of the unit testing frameworks, but the BDD provides you cleaner way of explaining the integration test use case in a Domain Specific Language which is a plain English(or any localized language).
Ta,
Rajeesh
I used specflow for BDD testing on two different good sized applications. Once we worked through the kinks of the sentence naming conventions, it worked out pretty good. BA's and QA's, and even interns could write BDD tests for the application.
However, I ALSO used it for unit tests. Heresy! I can hear some of you scream. However, there were VERY good reasons for it. The system was responsible for making many calculations or determinations based off a lot of different data. With lots of unit tests that require all this data to be input for test purposes, it makes it a LOT easier to manage that data used for the unit tests via the table format provided by specflow. Effectively mocking the data repository in table format, allowing the different components to be vigorously tested.
I don't know if I would do it in every case, but in the ones I used it for, it made laying out the volumes of data necessary for for performing the unit tests so much easier and clearer.
In the end we are trying to deliver to the customer exactly what the customer wants and as such I really don't see the need to write unit tests in addition to SpecFlow. After all, it exercises the same code base. I am fairly new to BDD/ATDD/TDD but other than being "complete" and strictly adhering to TDD I'm finding it unnecessary to write more unit tests.
Now I suppose if the team was dispersed and the developer was not able to run the entire application then separate unit tests would be necessary but where the developer(s) has access to the entire code base and is able to run the application, then why bother write more tests.
At present, my development process flows like this:
I describe the expected behaviour as an integration test using using WebRat
I write the Ruby on Rails code to provide that behaviour, so passing the test
I refactor, ensuring the tests still pass at the end of the process
I write the next integration test
It seems to me that by definition, my integration tests are testing every model, controller and view that I can create. In reality, am I missing anything by not writing unit tests too?
I'm actually pretty sympathetic to your point of view here. I love Cucumber and I love RSpec -- and I use them both, but not always on the same code. For instance, I rarely write RSpec examples for Rails controllers these days, and I almost never write view specs. Most of my controllers are very similar and don't deviate much from the "stock" controller pattern -- which is already well-tested by Rails's own unit tests. Verifying the same behavior again doesn't gain much for the time it takes and the hassle of mocking all the models. With Cucumber at an integration level I can skip that mocking and get the essential verification I'm looking for. View testing is handled much more transparently in Cucumber most of the time as well. (Then I should see "foo" and so forth.)
But that isn't to say I don't use RSpec extensively in Rails. I use it for the places where my logic lives: models, controller filters, and view helpers. I also have a couple of projects that are almost all business logic, e.g. libraries or API adapters against complex third-party interfaces. For those I usually find myself using RSpec exclusively and skipping Cucumber.
As a heuristic, I'd suggest that you should strongly consider writing a unit test any time any of the following questions can be answered "Yes":
Is the code I'm writing more than trivially complicated?
Does this code exist primarily to give answers to other code?
Is this existing code that I'm refactoring (that doesn't already have a unit test)?
Have I found a bug in this code? (If so, write a unit test before fixing it so it never sneaks in again.)
Do I have to think for more than ten seconds about the most elegant way to implement this code?
Is my Spidey Sense tingling?
If none of the above is true, then maybe you can get away with just doing integration testing. Again, there are a lot of cases where that's reasonable. But if you do run into problems later, be prepared to pay the price -- and that price should include writing unit tests at any moment if they seem called for.
Integration tests are useful to verify that different parts of code are well integrated. They may involve all layers and cover all code but... when an integration test fails, will it tell you where the bug is located? I may be wrong but I don't think so. This will just tell you that there is a problem somewhere. On the other hand, when a real unit tests (written in isolation using mocks or stubs) fails, you know exactly in which unit the problem is located (this is actually the purpose of unit testing, verifying that a unit implements the expected behavior). In other words, unit tests and integration tests are both useful but they have different purposes.
Do you have any rake tasks? Custom capistrano code? Cron methods? An API? Monkeypatches? How about flex or iPhone app integration? A job runner?
In a typical Rails application, there's lots of code that isn't exercised by the HTML UI. So no, unless your app is incredibly simple, webrat tests won't be sufficient.
Is there a proper way yet to unit test views with the aspx view engine?
I've been playing with various ways that will let me get a parseable string as a result like :
view.RenderView(viewContext);
But I'm not having any luck so far.
Most of what I've read strays into integration test territory. Integration test overlap a fair bit but don't allow me to use dummy data to check the output, which is the main thing I'm trying to achieve by unit testing views.
Have you checked out Selenium? It is directed at Web UI testing and is discussed in Hunt and Thomas' Pragmatic Unit Testing in C# with NUnit. It uses NUnit although it does so in a way that is really more of a system test than a Unit test (simply because you have to bring up the web app and navigate to your page to test it). Sorry that I didn't go down this route in my earlier answer but I was in a bit of a rush yesterday morning!
There is also NUnitAsp but that project has been essentially abandoned. It works and was widely praised in its day but I'm loathe to recommend anything that is no longer under active development.
What I have done is to create objects to "drive" the user interface and placed them all into a "BusinessObjects" dll. The interface essentially pulls its results from these objects when it is time to create the interface. I then Unit test against the Business Objects using standard NUnit unit tests. I'm afraid that your question might be a bit more specific to the MVC framework but I hope that this gives you some food for thought.
My advice is to not bother. Stick to testing the model and the controller. Checking for specific HTML is only going to lead to a fragile test base, that will break the moment you make the slightest change to the layout.