I came to Erlang/OTP from python world, where I'm using unittest library. Typical test environment will be presented by some TestSuite for entire application and TestCases with test methods for different modules from subpackages of application.
My first application on Erlang is cowboy-based web application. It has some modules which are required by cowboy framework and its behavior plus some set of my custom modules, let's say: parsers.erl, encoders.erl, fetchers.erl.
In the beginning of development I was writing tests inside that modules (in methods method_name_test) and then running them with eunit. But as for me it was kind of inconvenient. In a week or so I got in touch with commont_test framework. And as for newcomer from python world - CT with its suites, grouping, setup-ing, configs, execution order model looked like very familiar.
Considering my application - what is the proper way of writing test suites? Should I prepare separate suites for different modules (as for me it will create some overhead) or introduce single test suite for application and in different groups put test cases for separate modules? Would be great to read about tests organizing in real-world Erlang applications.
The stdlib in Erlang/OTP has a single Common Test suite per module in the library.
Related
I've got an Erlang project comprising a bunch of different applications. I'm using Common Test to do some of the testing.
apps/foo/suites/foo_SUITE.erl
apps/bar/suites/bar_SUITE.erl
I'm starting to see duplication of utility code in those suites.
Where should I put my utility code so that it can be shared between the two suites?
I've considered adding another application:
apps/test_stuff
...but I can't make the CT suites depend on this without making the application under test depend on this (or can I?). I don't want to do that, because test_stuff is only needed when testing.
I have a similar problem with my eunit tests, both between applications (apps/foo/test vs. apps/bar/test), and where I'm using similar functionality between the eunit and CT tests in the same application (apps/bar/suites vs apps/bar/test). Can I use the same solution for this case as well? Or do I need to ask another question about that?
Do you think ct:require/1,2 could help you so that foo and bar SUITE would require test_stuff before it gets executed? For more information http://www.erlang.org/doc/man/ct.html#require-1
It depends on how you are packaging your final releases. For example, I use rebar for relase management. I have Cowboy fetched along with other dependencies for testing purposes, but in my reltool.config, I omit it, so it doesn't get packaged with the final product. I use rebar to run Common Test, and it's able to add Cowboy to the path without having it bundled as a lib with everything else or added as a dependency to the app I'm testing.
However, if you have another process which infers your release configuration from your dependencies, you'll have to find a way to exclude your test code when you generate a release.
I am new to Erlang. It has 2 test frameworks: EUnit and Common Test. I am confused when to use one over another. Can someone explain to me what are the advantages of EUnit over Common Test, and vice versa. Seem Common Test can do everything EUnit can do and much more, not sure what I should use EUnit. Thanks!
Learn you some erlang (one of the best online resources for erlang, besides the official doc) explains both methods quite well:
EUnit
Common Test
As Pascal pointed out, EUnit is best used in white-box testing, more like internal function-by-function unit testing, light integration tests.
Common Test does the more heavy lifting: integration & system testing, black-box kind of stuff. It's also more complex, of course, and much more powerful.
While you're at it, you might try Dialyzer, another integrated testing tool in Erlang/OTP , which is great in locating dead or unreachable code, logical & type errors (it's a static type analyser). Again, learn you some erlang provides a nice introduction to it: Dialyzer.
Oh, by the way, if you chose to put the EUnit tests in separate files (which is perfectly possible), you won't be able to test unexported functions (that's expected). What's also expected is that Common Test does not test unexported functions: otherwise it wouldn't be a black-box testing tool (maybe a cheating one).
Eunit is really simple and fit very well for module test or library test at white box level. It is integrated to rebar.
Common Test is more oriented for black box testing and application and system of application test. It comes with test coverage very easily.
Edit (after Andy comment):
It is right that you can use Common test for unitary white box test, as well as it is right that you can leverage eunit to some application test using fixtures.
However eUnit is very handy for simple unitary test: you write a function myFun, you add a test function myFun_test or a test generator myFun_test_ (useful to test many patterns even is some test fails in the middle) in your test module and that's it. You can run it as often as you want (no history of test).
Common test asks you to list each test case in the all function or in a group. As far as I know it has not test generator so it is less easy to go through all test patterns of each function. It is why I think it is less adapted to unitary white box tests. On the other hand, the init_per_testcase, init_per_group ... are much more flexible than eunit fixtures to organize the tests when they need some application context to run. Common Test also keeps an history of all tests done in the log directory, It is nice, but I suggest to limit the number of run to keep it useful.
EDIT:
To avoid the problem of not exported functions, for both eunit and common test, it is possible to use defines. for example in rebar.config (because I use separate files for eunit tests):
{eunit_compile_opts, [{d,'EUNIT_TEST',true}]}.
{erl_opts, [debug_info, warn_export_all]}.
and in a module, if it is necessary:
%% export all functions when used in eunit context
-ifdef(EUNIT_TEST).
-compile(export_all).
-endif.
You can verify that it only modify the compiled code for eunit
D:\git\helper>rebar clean eunit compile
==> helper (clean)
==> helper (eunit)
Compiled test/help_list_tests.erl
Compiled test/help_ets_tests.erl
Compiled test/help_num_tests.erl
Compiled src/help_ets.erl
Compiled src/help_list.erl
Compiled src/helper.erl
src/help_num.erl:6: Warning: export_all flag enabled - all functions will be exported
Compiled src/help_num.erl
Compiled src/help_code.erl
All 31 tests passed.
Cover analysis: d:/git/helper/.eunit/index.html
==> helper (compile)
Compiled src/help_ets.erl
Compiled src/help_list.erl
Compiled src/helper.erl
Compiled src/help_num.erl
Compiled src/help_code.erl
From http://www.erlang.org/doc/apps/common_test/basics_chapter.html:
Common Test is also a very useful tool for white-box testing Erlang code (e.g. module testing), since the test programs can call exported Erlang functions directly and there's very little overhead required for implementing basic test suites and executing simple tests. For black-box testing Erlang software, Erlang RPC as well as standard O&M interfaces can for example be used.
In our Delphi2007 environment we have a SGLibrary groupproj which contains some 30 bpls. We're just starting out creating unittests for these libraries and are not sure what the most convenient way of organizing the Unittest projects would be.
We are inclined to create a test-executable for each bpl, as this will make compilation an running easy and fast. The test-exe can be set as the active project and Compilation of the bpl can be forced by setting a dependency. It is also easy to run tests, ie by setting the test-executable as the Hostapplication of the bpl.
But the downside is that the library groupproject will be expanded with another 30 items, making it a very large group (why can't we make subgroups in Delpi ???).
The opposite arrangement would be to create 1 test executable which contains all unit-tests but that would create a executable with over a hundred units, and lots of depencies which all have to be compiled before a single test can be run.
So my question ... Does anybody have any suggestions, best practices, or other ideas on how to organize this into a manageable and fast running setup?
Extra consideration: We want to have the possibility to run all tests at once, and of course this will be easier in we put all tests in one executable.
There is a little known feature of DUnit that supports running tests from a dll. You basically create a dunit exe project that has no tests of its own, rather it loads tests from dlls.
Each dll needs to export a single function:
library MyTests;
uses
TestFramework{, add your test units};
function Test: ITest;
begin
result := RegisteredTests;
end;
exports
Test;
end;
Then you just add test cases to the dll as normal. The tests are automatically registered in each unit's initialization section.
IMHO its a pity this isn't promoted as the standard way of working with DUnit. Most unit testing frameworks for other languages are organized this way. They provide a single test runner executable which dynamically loads test cases from any number of loadable modules.
In addition to letting you break up your tests for easier organization it also allows you to run the same tests under multiple scenarios. Perhaps you want to run your tests using different compiler options for your debug and release builds (or even different versions of the compiler) so you are more confident that the code behaves consistently. You can build multiple dlls from the same source and run them in the same session.
I'd probably do both, so you end up with this:
all your unit tests, group them by BPL.
a project for each of the units tests for each BPL.
a project with all the tests.
You can use the final project in your continuous integration system, and the former for testing things that are not yet checked in.
This is indeed a large number of projects, a price you pay for being able to improve the quality of your code.
--jeroen
G'day guys,
I've started a new role at a company and am working on a rails application that's used internally for particular management applications. At any rate, there are no unit tests associated with this system, and as I am building the system out I've realised there is a serious need to build unit tests so that I can have regression testing as I continue to add to this codebase.
It's a rails 1.2.7 app that I cannot migrate for reasons of compatibility, but I will be beginning to port components over it to 3.0 in the background.
There is currently only the built in tests that rails makes, and I would love some insights into how to get started with building small tests into the system and particularly at what aspects would be good to start?
This system builds a lot of individual config files, so I'm assuming a lot of the test generation would begin with testing those config files for individual aspects/elements and going from there.
I've had a look at cucumber and rspec, but just wanted to find something straightforward and easy to use that I could slowly build upon within the current system and build up a testing base over time.
Any insights, links or others would be really really appreciated.
Cheers!
A lot has changed since Rails 1.2.7, definitely one of the first things would be to right unit tests, but instead of going an all out unit test attack.. Attack the problem component,
Write tests
Upgrade the component
Run Tests ( mostly they'll fail / possibilities of errors )
Make them pass
Refactor ( May be the new features at your disposal )
I usually do the above steps with writing the integration tests, and functional tests and unit tests for the components in the integration tests.
All the best, this ain't going to be a simple task :).
How does one setup DUnit Testing in a Midas/DataSnap project in Delphi 2006
Edit
How does one set up a Dunit Test into a TRemoteDataModule
The project wizard in Delphi 2006 does not work with TRemoteDataModule
The question doesn't entirely make sense. Unit tests are performed in a separate project, not within your DataSnap server. Generally, tests which connect to a database are integration tests rather than unit tests. What is it, exactly that you want to test? If it's utility methods within, say, a TRemoteDataModule, you should extract those out into a separate class as class methods, and test them there. You should not have to instantiate an application server to perform unit tests.
Sorry for the terse answer above, the iPad posts whenever I hit return while editing a post.
dUnit is designed to perform unit testing, and what you are trying to do is NOT unit testing.
A test is not a unit test if:
It talks to the database
It communicates across the network
It touches the file system
It can't run at the same time as any of your other unit tests
You have to do special things to your environment (such as editing config files) to run it.
It can't run in isolation
If you follow the SOLID principles (especially the single responsibility principle), using dUnit to test your class (without testing the Midas/DataSnap related logic) should be reasonably simple. And really, you shouldn't need to test the Midas/DataSnap logic.
But there are ways to perform integration and behavioural tests on Delphi applications.
Personally, I wouldn't use TestComplete because it doesn't integrate well with any sort of CI server and the tests are stored in a proprietary binary format (which makes merging differences or maintaining changes in any source control system problematic).
You could try dSpec, but I'm not sure Jody Dawkins is maintaining it any more.
I have used the AutoIt BASIC scripting language directly on some projects, and also used its .NET assembly to drive a Delphi application using NUnit and C# on another. While not perfect, the NUnit / C# solution was more elegant than anything I'd seen for performing functional / behavioural testing Delphi applications. It did take some effort to get it setup though.