I admit that I have almost none experience of unittesting. I did a try with DUnit a while ago but gave up because there was so many dependencies between classes in my application.
It is a rather big (about 1.5 million source lines) Delphi application and we are a team that maintain it.
The testing for now is done by one person that use it before release and report bugs. I have also set up some GUI-tests in TestComplete 6, but it often fails because of changes in the application.
Bold for Delphi is used as persistance framework against the database.
We all agree that unittesting is the way to go and we plan to write a new application in DotNet with ECO as persistance framework.
I just don't know where to start with unittesting...
Any good books, URL, best practice etc ?
Well, the challenge in unit testing is not the testing itself, but in writing testable code. If the code was written not thinking about testing, then you'll probably have a really hard time.
Anyway, if you can refactor, do refactor to make it testable. Don't mix object creation with logic whenever possible (I don't know delphi, but there might be some dependency injection framework to help in this).
This blog has lots of good insight about testing. Check this article for instance (my first suggestion was based on it).
As for a suggestion, try testing the leaf nodes of your code first, those classes that don't depend on others. They should be easier to test, as they don't require mocks.
Writing unit tests for legacy code usually requires a lot of refactoring.
Excellent book that covers this is Michael Feather's "Working Effectively with Legacy Code"
One additional suggestion: use a unit test coverage tool to indicate your progress in this work. I'm not sure about what the good coverage tools for Delphi code are though. I guess this would be a different question/topic.
Working Effectively with Legacy Code
One of the more popular approaches is to write the unit-tests as you modify the code. All new codes gets unit tests, and for any code you modify you first write its test, verify it, modify it, re-verify it, and then write/fix any tests that you need due to your modifications.
One of the big advantages of having good unit test coverage is being able to verify that the changes you make don't inadvertently break something else. This approach allows you to do that, while focusing your efforts on your immediate needs.
The alternate approach I've employed is to develop my unit tests via Co-Ops :)
When you work with legacy code, mock objetcs are really usefull to build unit tests.
Take a look at this question regarding Delphi and mocks: What is your favorite Delphi mocking library?
For .Net unittesting read this : "The Art of Unit Testing: with Examples in .NET"
About best pratices :
What you said is right : Sometimes, it's difficult to write unit tests because of the dependancy between classes...
So write unit tests just after or just before ;-) the implementation of the classes. Like this, if you have some difficulties to write the tests, maybe it means you have a design problem !
Related
Seems like the internet doesn't have a definitive answer, or set of principles to help me answer the question. So I turn to the great folk on SO to help me find answers or guiding thoughts :)
SpecFlow is very useful for BDD in .NET. But when we talk about BDD are we just talking integration/acceptance tests, or are we also talking unit tests - a total replacement for TDD?
I've only used it on small projects, but I find that even for my unit tests, SpecFlow improves code documentation and thinking in terms of language. Converseley, I can't see the full code for a test in one place - as the steps are fragmented.
Now to you..........
EDIT: I forgot to mention that I see RSpec in the RoR community which uses BDD-style syntax for unit testing.
I've recently started to use SpecFlow for my BDD testing, but also, I still use unit and integration tests.
Basically, I split the tests into seperate projects:
Specs
Integration
Unit
My unit tests are for testing a single method and do not perform any database calls, or external references whatsoever. I use integration tests for single method calls (maybe sometime two) which do interact with an external resources, such as a database, or web service, etc.
I use BDD to describe tests which mimick the business/domain requirements of the project. For example, I would have specs for the invoice generation feature of a project; or for working with a shopping basket. These tests follow the
As a user, I want, In order to
type of semantics.
My advise is to split your tests based on your needs. Avoid trying to perform unit testing using SpecFlow.
We have started using Specflow even for our unit tests.
The main reason (and benefit) for this is that we find that it forces you to write the tests from a behavior point of view, which in turn forces you to write in a more implementation agnostic way and this ultimately results in tests which are less brittle and more refactoring friendly.
Sure this can also be done with standard unit testing frameworks, but you aren't guided that way as easily as we have found we are using specflow and the gherkin syntax.
There is some overhead setting things up for specflow, but we find this is quickly repaid when you have quite a few tests (due to the significant step reusability that you can get with specflow) or you need refactor your implementation.
Plus you get nice readable specs that are easy for newcomers to the team to understand.
Given:
Unit tests are test of (small) "units of code"
The customer of most “units of codes” are other programmers.
Part of the reason for having a unit test is to provide an example of how to call the code.
Therefore:
Clearly unit tests should normally be written in the programming language that the users of the “unit of code” will be calling it with.
However:
Sometimes data tables are needed to setup the conditions a unit test runs in.
Most unit test frameworks are not good at using tables of data.
Therefore:
Specflow may be the best option for some unit test, but should not be your default choose.
I see it as an integration testing which mean it doesn't replace your unit test cases written as part of your TDD process. Someone will have different opinion about this. IMHO unit test case only test the methods/functions and all the dependencies should be mocked and injected. When in it comes to integration testing, you will be injecting real dependencies instead of mocked one. You could do the same integration testing with any of the unit testing frameworks, but the BDD provides you cleaner way of explaining the integration test use case in a Domain Specific Language which is a plain English(or any localized language).
Ta,
Rajeesh
I used specflow for BDD testing on two different good sized applications. Once we worked through the kinks of the sentence naming conventions, it worked out pretty good. BA's and QA's, and even interns could write BDD tests for the application.
However, I ALSO used it for unit tests. Heresy! I can hear some of you scream. However, there were VERY good reasons for it. The system was responsible for making many calculations or determinations based off a lot of different data. With lots of unit tests that require all this data to be input for test purposes, it makes it a LOT easier to manage that data used for the unit tests via the table format provided by specflow. Effectively mocking the data repository in table format, allowing the different components to be vigorously tested.
I don't know if I would do it in every case, but in the ones I used it for, it made laying out the volumes of data necessary for for performing the unit tests so much easier and clearer.
In the end we are trying to deliver to the customer exactly what the customer wants and as such I really don't see the need to write unit tests in addition to SpecFlow. After all, it exercises the same code base. I am fairly new to BDD/ATDD/TDD but other than being "complete" and strictly adhering to TDD I'm finding it unnecessary to write more unit tests.
Now I suppose if the team was dispersed and the developer was not able to run the entire application then separate unit tests would be necessary but where the developer(s) has access to the entire code base and is able to run the application, then why bother write more tests.
I have been assigned the task to refactor a Delphi unit. Wow. 10000 lines of code, no documentation, tons of copy and paste code.
THere are many methods made with copy and paste that could be refactored, anyway I am lost in all those lines, I have the interface section where I can "find my way", but in general what do yuo suggest for tackling this kind of task?
Thanks.
Get yourself a copy of Working Effectively with Legacy Code by Michael Feathers. It has all kinds of techniques for safely refactoring code to get it running under a test framework. Examples are mostly in Java and C++ but should be easy enough to figure out.
Install a third-party refactoring tool (or multiple) such as CodeRush for Delphi(sadly no longer developed), Castalia or ModelMaker Code Explorer. Delphi has some refactoring support built in but in my experience it is too limited and tends to choke on very large code bases.
Buy a copy of Simian. It doesn't have direct support for Object Pascal but its plain text parser works well enough. If enough people request support for Object Pascal I'm sure they'd add it. I haven't found any other code duplication detection tool as capable as Simian.
I would also recommend bookmarking http://www.refactoring.com/catalog/ and http://www.industriallogic.com/xp/refactoring/catalog.html.
It also wouldn't hurt to get a copy of Clean Code: A Handbook of Agile Software Craftsmanship by Robert "Uncle Bob" Martin et al. It's easy to recognize bad code. It's much harder know when you're writing good code.
A word of caution: Focus on refactoring the code you need to work on. Its easy to start down the rabbit hole and wind up spending months refactoring code that wasn't immediately relevant to the task at hand.
And save your self some trouble. Don't try to "fix" code and refactor it at the same time. Refactor first, then fix bugs or add that new feature. Remember, refactoring is modifying without changing external behavior.
Resist the urge to attempt a complete rewrite. I learned the hard way that crappy code that meets the user's requirements is preferable to clean code that doesn't. Crappy code can always be incrementally improved until its something to be proud of.
I think the best thing you can do is to write DUnit Tests for the interface. It forces you to understand the existing code, helps during debugging and it ensures that the interface acts the same after refactoring.
The Top 12 Reasons to Write Unit Tests apply perfectly in your case:
Tests Reduce Bugs in New Features.
Tests Reduce Bugs in Existing Features.
Tests Are Good Documentation.
Tests Reduce the Cost of Change.
Tests Improve Design.
Tests Allow Refactoring.
Tests Constrain Features
Tests Defend Against Other Programmers
Testing Is Fun
Testing Forces You to Slow Down and Think
Testing Makes Development Faster
Tests Reduce Fear (Fear of change, Fear of breakage, Fear of updates)
I've faced similar situations. My condolences to you!
In my opinion, the most important thing is that you actually understand all the code as it is today. Minds better than mine may be able to simply read the code and understand it. However, I can't.
After reading the code for a general overview, I usually repeatedly single step through it in the debugger until I begin to see some patterns of operation and recognize code that I've read before. Maybe this is obvious, but thought I'd mention it.
You might also think about creating a good test suite that runs on the current code.
Does the interface section contain a bunch of class definitions? If so, create a new unit for every class and move each class to it's own unit.If you use Delphi 2007 or better, you can use the "refactor/Move" option to move those classes to the new (namespace) units.The next step is splitting the large classes into smaller classes. That's just a lot of manual work.Once your code is divided over multiple units, you can examine each unit, detect identical code and generate base classes that would be used as parent for the two classes that share similar functionality.
In addition of understanding the code etc, these tools may help refactoring and reorganizing the project:
Model Maker is powerful design, reverse-engineer and refactoring tool: http://www.modelmakertools.com/modelmaker/index.html
Model Maker Code Explorer is powerful plugin for Delphi IDE to help with refactoring, code navigation etc: http://www.modelmakertools.com/code-explorer/index.html
I would use some sort of UML tool to generate som class diagrams and other diagrams to get an overview of the system, and start splitting up and commenting like #Workshop Alex said.
Use a tool like Doxygen to help you map the code.
Help on that is here
Start out small and eventually do a partial or full rewrite. Start creating base classes to accomplish pieces of the puzzle without changing the output. Rinse-repeat until you have a new, supportable codebase.
Once you hit those copy-n-paste routines, you'll have base classes to do the work and it'll really help accelerate the task.
I've been searching on how to do Unit testing and find that it is quite easy, but, what I want to know is, In a asp.net mvc application, what should be REALLY important to test and which methods you guys use?
I just can't find a clear answer on about WHAT TO REALLY TEST when programming unit tests.
I just don't want to make unnecessary tests and lose development time doing overkill tests.
You should unit test as much as possible of your application.
For every line of code you write, you need to verify that it works. If you don't unit test it, you need to test it in some other fashion. Even starting up the site and clicking around is a sort of testing.
When you compare unit testing with other sorts of testing (including running the site and manually using it), unit tests tend to give the best return of investment because they are relatively easy to write and maintain, and can give you rapid feedback on whether you just introduced a regression bug or not.
I'm not saying that there's no overhead in writing unit tests - there is, but there's overhead in any sort of testing, and a big overhead in not testing at all (because regression bugs slip through quite easily).
It's still good practice to supplement unit tests with other types of tests, but a good unit test suite offers an excellent regression test suite.
Ron Jeffries says "Test everything that could possibly break."
Someone else - I think it was Kent Beck, but I can't find a reference - says, "Only test the code you want to work."
Either of these is a pretty good strategy.
I actually don't think anything needs to be tested in MVC. I think all your business logic, rules etc need testing but the Views and Controllers?
The only real reason I can see to test a Controller is for integration testing. If all your business logic is correct then that should be a simple test always returning true.
Controllers should really only get data from the view and pass data to it so....
As for views, what sort of testing can be done there other than to open the view and see what it does?
When I write my projects there is next to no code in the controller and I put all the grunt in my business engine which I have extensive tests for.
Unit testing is good for testing of services/models. But when you need in testing of application functionality the better choice will be functional tests (i.e. Selenium)
I'm building a test harness for my Delphi 2009 app. Testing the logic is fairly simple. Making sure the forms work properly is proving a bit more complicated. I'd like a way to simulate real user input, to open a form, make it think there's a user typing certain things and clicking in certain places, and make sure it reacts correctly. I'm sure there's a way to do this, I just don't know what it is. Does anyone know how to do it?
DUnit has GUITesting.pas whicih extends testing so you can send clicks, keys and text to controls on form, but that is about it.
Last year there where mentions of Zombie GUI testing framework that was used internaly by CodeGear developers, but nothing since Steve left for Falafel.
TestComplete is a good choice. Another commercial option for GUI testing is SmarteScript:
well for .net there's NUnitForms for GUI testing a win application.
don't know any open source for delphi though.
Test Complete can test delphi forms but it's not free.
There's 2 parts to this, firstly how do you automate the GUI, and secondly how do I then 'test' whether its working/not working.
Firtsly: To automate the GUI on windows try using AutoIT. Its a free tool for controlling windows interfaces, sending keyboard input events etc. http://www.autoitscript.com/autoit3/
Secondly: Testing is a big field, and I won't try and give you a whirlwind tour. But the mechanics of driving the GUI and testing the results could be handled using AutoIT's built in Basic like language or by using it in conjunction with a language like Ruby and TestUnit (rubys built-in unit testing framework).
If there is nothing Deliphi-specic out there and you need a quick solution try some easy to learn scripting solutions like AutoIt.
For a bit more sophisticated scripting, you might have a look on Scripted GUI Testing with Ruby.
But be aware, that you should not test too much functionality via the GUI, because such tests are very likely to break. If you end up with too much GUI testing you may need to rethink the design: Decouple logic from the GUI and test the logic directly with some xUnit framework.
Also have a look on a similar question about windows forms test automation.
It seems like DUnit has some gui-testing functionality: delphiextreme.com
Not exactly an answer to your question, but there is a very good page (IMHO of course) about GUI Architectures by Martin Fowler, featuring the "Humble View" architecture as the last entry, which is geared specifically towards test-driven software development. Worth looking into.
This will of course not help you with the task of testing whether all controls are wired correctly and handle all necessary events, but it should help to minimize the amount of GUI code that does need testing.
OpenCTF is good for you.
Quote:
OpenCTF is a test framework add-on for Embarcadero Delphi® which
performs automatic checks of all components in Forms (or DataModules).
It provides an easy way to build automatic quality checks for large
projects where many components have to pass repeated tests.
Adding OpenCTF tests to a DUnit test suite requires only a few lines of code.
Writing your own custom component tests needs only a few seconds.
OpenCTF is based on the DUnit open source test framework and extends
it by specialized test classes and helper functions.
Please head here to download.
Is there a proper way yet to unit test views with the aspx view engine?
I've been playing with various ways that will let me get a parseable string as a result like :
view.RenderView(viewContext);
But I'm not having any luck so far.
Most of what I've read strays into integration test territory. Integration test overlap a fair bit but don't allow me to use dummy data to check the output, which is the main thing I'm trying to achieve by unit testing views.
Have you checked out Selenium? It is directed at Web UI testing and is discussed in Hunt and Thomas' Pragmatic Unit Testing in C# with NUnit. It uses NUnit although it does so in a way that is really more of a system test than a Unit test (simply because you have to bring up the web app and navigate to your page to test it). Sorry that I didn't go down this route in my earlier answer but I was in a bit of a rush yesterday morning!
There is also NUnitAsp but that project has been essentially abandoned. It works and was widely praised in its day but I'm loathe to recommend anything that is no longer under active development.
What I have done is to create objects to "drive" the user interface and placed them all into a "BusinessObjects" dll. The interface essentially pulls its results from these objects when it is time to create the interface. I then Unit test against the Business Objects using standard NUnit unit tests. I'm afraid that your question might be a bit more specific to the MVC framework but I hope that this gives you some food for thought.
My advice is to not bother. Stick to testing the model and the controller. Checking for specific HTML is only going to lead to a fragile test base, that will break the moment you make the slightest change to the layout.