I've got an Erlang project comprising a bunch of different applications. I'm using Common Test to do some of the testing.
apps/foo/suites/foo_SUITE.erl
apps/bar/suites/bar_SUITE.erl
I'm starting to see duplication of utility code in those suites.
Where should I put my utility code so that it can be shared between the two suites?
I've considered adding another application:
apps/test_stuff
...but I can't make the CT suites depend on this without making the application under test depend on this (or can I?). I don't want to do that, because test_stuff is only needed when testing.
I have a similar problem with my eunit tests, both between applications (apps/foo/test vs. apps/bar/test), and where I'm using similar functionality between the eunit and CT tests in the same application (apps/bar/suites vs apps/bar/test). Can I use the same solution for this case as well? Or do I need to ask another question about that?
Do you think ct:require/1,2 could help you so that foo and bar SUITE would require test_stuff before it gets executed? For more information http://www.erlang.org/doc/man/ct.html#require-1
It depends on how you are packaging your final releases. For example, I use rebar for relase management. I have Cowboy fetched along with other dependencies for testing purposes, but in my reltool.config, I omit it, so it doesn't get packaged with the final product. I use rebar to run Common Test, and it's able to add Cowboy to the path without having it bundled as a lib with everything else or added as a dependency to the app I'm testing.
However, if you have another process which infers your release configuration from your dependencies, you'll have to find a way to exclude your test code when you generate a release.
Related
I came to Erlang/OTP from python world, where I'm using unittest library. Typical test environment will be presented by some TestSuite for entire application and TestCases with test methods for different modules from subpackages of application.
My first application on Erlang is cowboy-based web application. It has some modules which are required by cowboy framework and its behavior plus some set of my custom modules, let's say: parsers.erl, encoders.erl, fetchers.erl.
In the beginning of development I was writing tests inside that modules (in methods method_name_test) and then running them with eunit. But as for me it was kind of inconvenient. In a week or so I got in touch with commont_test framework. And as for newcomer from python world - CT with its suites, grouping, setup-ing, configs, execution order model looked like very familiar.
Considering my application - what is the proper way of writing test suites? Should I prepare separate suites for different modules (as for me it will create some overhead) or introduce single test suite for application and in different groups put test cases for separate modules? Would be great to read about tests organizing in real-world Erlang applications.
The stdlib in Erlang/OTP has a single Common Test suite per module in the library.
I have a medium-sized release with a handful of applications. I recently refactored some common functionality out into a library application within the release. This made my EUnit tests fail with undef messages whenever testing anything that required the library application.
The set up is something like this:
% In apps/utils/src/utils.erl
-module(utils).
-export([foo/0]).
foo() -> "OH HAI".
Then
% In apps/some_app/src/some_app.erl
-module(some_app).
-export([bar/0]).
bar() -> io:format("foo: ~s~n", [utils:foo()]).
% unit tests for bar()
Then the unit tests for some_app:bar() fail. I'm running them with rebar eunit skip_deps=true. I'm using skip_deps=true because my release uses some 3rd party applications (SQL, etc).
I assume that the tests start failing because EUnit is invoking the app under test without its dependencies? Is there any way to fix this? I have configured the .app file to explicitly declare the dependency. It works fine in the release, and it's been deployed for about a day now with no problem, but I'll feel a lot better if I can get the tests to pass again :)
(I could use a mocking app to stub out utils:foo/0, and I can see where that would be ideal idiomatically, but that seems like overkill in this case because utils:foo/0 (read: it's real-world counterpart) does some really simple stuff.)
I was able to get this to work by doing rebar compile eunit skip_deps=true.
The key is to have the compile in there and I have no idea why. I'm guessing that the compile step gets all of the modules into memory. I'd love to hear a good explanation.
I think you could have one of your applications load the utility by including it in the application portion of you .app file, as in:
{application,yourapp
[{description,"A description"},
{vsn,"1.0.0"},
{modules,[mod1, mod2, utils]},
SNIP
or in some other way add it to the path of the erlang node... maybe using the -pa flag on starting the node.
In our Delphi2007 environment we have a SGLibrary groupproj which contains some 30 bpls. We're just starting out creating unittests for these libraries and are not sure what the most convenient way of organizing the Unittest projects would be.
We are inclined to create a test-executable for each bpl, as this will make compilation an running easy and fast. The test-exe can be set as the active project and Compilation of the bpl can be forced by setting a dependency. It is also easy to run tests, ie by setting the test-executable as the Hostapplication of the bpl.
But the downside is that the library groupproject will be expanded with another 30 items, making it a very large group (why can't we make subgroups in Delpi ???).
The opposite arrangement would be to create 1 test executable which contains all unit-tests but that would create a executable with over a hundred units, and lots of depencies which all have to be compiled before a single test can be run.
So my question ... Does anybody have any suggestions, best practices, or other ideas on how to organize this into a manageable and fast running setup?
Extra consideration: We want to have the possibility to run all tests at once, and of course this will be easier in we put all tests in one executable.
There is a little known feature of DUnit that supports running tests from a dll. You basically create a dunit exe project that has no tests of its own, rather it loads tests from dlls.
Each dll needs to export a single function:
library MyTests;
uses
TestFramework{, add your test units};
function Test: ITest;
begin
result := RegisteredTests;
end;
exports
Test;
end;
Then you just add test cases to the dll as normal. The tests are automatically registered in each unit's initialization section.
IMHO its a pity this isn't promoted as the standard way of working with DUnit. Most unit testing frameworks for other languages are organized this way. They provide a single test runner executable which dynamically loads test cases from any number of loadable modules.
In addition to letting you break up your tests for easier organization it also allows you to run the same tests under multiple scenarios. Perhaps you want to run your tests using different compiler options for your debug and release builds (or even different versions of the compiler) so you are more confident that the code behaves consistently. You can build multiple dlls from the same source and run them in the same session.
I'd probably do both, so you end up with this:
all your unit tests, group them by BPL.
a project for each of the units tests for each BPL.
a project with all the tests.
You can use the final project in your continuous integration system, and the former for testing things that are not yet checked in.
This is indeed a large number of projects, a price you pay for being able to improve the quality of your code.
--jeroen
G'day guys,
I've started a new role at a company and am working on a rails application that's used internally for particular management applications. At any rate, there are no unit tests associated with this system, and as I am building the system out I've realised there is a serious need to build unit tests so that I can have regression testing as I continue to add to this codebase.
It's a rails 1.2.7 app that I cannot migrate for reasons of compatibility, but I will be beginning to port components over it to 3.0 in the background.
There is currently only the built in tests that rails makes, and I would love some insights into how to get started with building small tests into the system and particularly at what aspects would be good to start?
This system builds a lot of individual config files, so I'm assuming a lot of the test generation would begin with testing those config files for individual aspects/elements and going from there.
I've had a look at cucumber and rspec, but just wanted to find something straightforward and easy to use that I could slowly build upon within the current system and build up a testing base over time.
Any insights, links or others would be really really appreciated.
Cheers!
A lot has changed since Rails 1.2.7, definitely one of the first things would be to right unit tests, but instead of going an all out unit test attack.. Attack the problem component,
Write tests
Upgrade the component
Run Tests ( mostly they'll fail / possibilities of errors )
Make them pass
Refactor ( May be the new features at your disposal )
I usually do the above steps with writing the integration tests, and functional tests and unit tests for the components in the integration tests.
All the best, this ain't going to be a simple task :).
I have a project group with the main project and a test project.
When writing unit tests for a class in the main project, do you include the source file in the test project, or do you put the path to it in the search path?
Why do you do one over the other?
Are there any best practices on this?
UPDATE:
It looks like including is the preferred option, much because of the disadvantages of 'getting access' to all units in the search path vs only getting access to the units that you intend to use.
What bugs me, though, is the need to include a file in two projects all the time. I usually keep the test project as the active project, so when I need a new class, I create a new unit, that becomes a part of the test project, but store it under the main projects path. Now, I need to remember to also include it in the main project. There should be a 'create new unit, and add it to all open projects'-action....
You should probably include the units in the test project rather than rely on a search path. You will have a much better understanding of the dependencies between units, and should make any additional dependencies obvious should they occur (particularly if they are undesirable). It might also be desirable to have more than one test project if you want to make sure there is no cross dependencies between certain parts of you main application (for example if you have some shared code with another application)
I include the source as it seems to make the IDE happier WRT speed, Code Completion, etc.
If the source files have important initialization or finalization sections then it may be necessary to include them in the testing project, otherwise it is not.
In latter case do not extend the unit test project's search path too much though, rather use one output directory for dcus of the main project.