In DUnit, SetUp and TearDown are called before (and after, respectively) each test method is executed.
In SetUp, I create an object that loads data from a file. This is slow, especially if I have many tests.
Is there any way to call SetUp once, before executing ALL tests (and obviously the same for TearDown)?
From the documentation:
TTestSetup
TTestSetup can be used when you wish to set up state exactly once for
a test case class (the SetUp and TearDown methods are called once for
each test method). For example, if you were writing a suite of tests
to exercise some database code, you might subclass TTestSetup and use
it to open and close the database before executing the suite.
An an example how to use TTestSetup
Related
I am writing the Unit test cases for my sqlite database class. I have five public API in that class.
My test cases goes something like below:
+ (void)setUp { // Note, this is class method and hence called only once during start of this test suite.
// Code to delete the existing sqlite DB file.
}
- (void)testDBManagerSingletonInstance {
DBManager *dbMgr = [DBManager getSharedInstance];
DBManager *dbMgr1 = [[DBManager alloc] init];
XCTAssertEqualObjects(dbMgr, dbMgr1);
}
- (void)testSaveAndDeleteNicknameAPI {
// Multiple Assert statements in this test.
}
- (void)testAllAccountStatusAPIs {
// Multiple Assert statements in this test.
}
Each of the single unit test is executed without any errors. But it fails when whole test suite is executed.
Probably, I know the Root Cause for failing. It is because when entire test suite is executed then all test runs in parallel and there is simultaneous update-delete happens in the database. Hence, when all unit tests runs it will fail.
But I don't know how can I fix that, because this is not Async, and hence I cannot use XCExpectation class.
Need assistance to resolve & understand the problem.
Tests based on XCTests don't run in parallel - they are run sequentially. To quote the docs :
Tests execute synchronously because each test is invoked independently one after another.
Since you've shown very little code, it's hard to say what is the real problem. It is very likely that you were close with your assumption - you should either improve your setUp (maybe switch to instance version from class version) and tearDown methods or introduce mocks, and perform your tests on a mocked database if possible.
You shouldn't work with singletons in a codebase you want to test. I think that your code uses the sharedSingleton instance of DBManager internally. Each unitest will change the internal state of that instance, all following unitests are therefore corrupted. You need to reset all changes after each test case in teardown.
But my suggestion is to avoid singletons in your base code by dependency injection. When creating an instance of your class inject the singleton instance of your DBManager. That makes unit testing easier. If you inject a protocol instead of an object you may even test your code with a protocol fake implementation.
I have created a test project and noticed that XCode generates some test class named: YourProjectName-ProjectTests
This is how it looks like:
- (void)setUp {
[super setUp];
// Put setup code here. This method is called before the invocation of each test method in the class.
}
- (void)tearDown {
// Put teardown code here. This method is called after the invocation of each test method in the class.
[super tearDown];
}
- (void)testExample {
// This is an example of a functional test case.
XCTAssert(YES, #"Pass");
}
- (void)testPerformanceExample {
// This is an example of a performance test case.
[self measureBlock:^{
// Put the code you want to measure the time of here.
}];
}
Would anyone be able to explain me how to use these automated testing methods? Is there an official documentation on this?
Are those test meant to be used as UI testing or as app code testing? Or both?
You an perform UI tests through XCTest but it's designed for unit testing. Instruments has UI Automation which is designed for UI testing.
Opening up XCTestAssertions.h will give you a pretty good reference for what's included in XCTest. Generally all the assertions follow a similar pattern:
XCTAssertSomething(somethingBeingTested, #"A message to display if the test fails");
Hitting ⌘ 5 will bring up the test navigator which lets you run all of your tests, all of your failed tests, or individual tests. You can also run a test by clicking on the diamond shape to the left of the method name in the editor.
Here are a few tips I've come across when unit testing with XCTest:
The last argument in assertions is optional but in some cases it's useful to describe the behaviour you're looking for.
Tests are not run in any specific order, but your tests shouldn't rely on each other anyway.
Mock objects let you test far, far more than basic assertions. I like OCMock.
You can use instruments on your tests for debugging.
Finally, if you have access to another machine you can set up OSX Server to automatically run your tests daily or on every commit and notify you if any of the tests fail.
This is how Unit Testing is implemented in Xcode.
The setUp and TearDown methods are to prepare any mock data necessary to test a unit (which is select part of code).
The two test methods there are just an example. Usually as an author of the unit of code, you know well what the unit supposed to do and what it should not.
Using test methods you test how the assumptions and expectations are satisfied by the unit. The convention is to have a separate test method for each assumption/expectation of a behaviour of the unit.
For example given class that handles strings that has a method to to make all strings CAPS, you would test that passing any random nonCaps string would still return ALL CAPS. An putting none truing as a parameter would return nil or error.
So basically you test your units to behave as expected.
There's a whole theory behind Unit Testing which is out of scope of this thread. Just Google it.
I have a class which can be configured to do 2 slightly different things. I want to test both paths. The class is a descendant of UIViewController and most of the configuration takes place in Interface Builder. i need to verify that both Storyboard Scenes and their Outlets are wired up in the same way, but also need to check for the difference in behavior.
I'd like to use shared XCTest suites for this purpose.
One is aimed for use with the left hand, one for the right. Both appear after another when using the app. The first one (right hand) triggers a segue to the other. The last one (left hand) should trigger a different segue. This is where it differs, for example.
Now I want to verify the segues with tests. I'd like to create a BothHandSharedTests suite which both view controller instance tests use to verify everything they have in common. However, the BothHandSharedTests class is treated as a self-containing test suite, which it clearly isn't.
I came up with these strategies:
inherit from an abstract XCTest descendant, like described above (doesn't seem to be that easy),
write a test auite for the common properties and use one of the two as the Object Under Test, and add two smaller suites for the differences.
How would you solve this problem?
Here's a solution in Swift:
class AbstractTests: XCTestCase {
// Your tests here
override func perform(_ run: XCTestRun) {
if type(of: self) != AbstractTests.self {
super.perform(run)
}
}
}
I don't have a conclusive answer, but here's what I ended up doing.
I first tried the subclassing route. In the parent test (the "AbstractTestCase") I implemented all the tests that would be executed by the the AbstractTestCase subclasses, but added a macro so they don't get run by the actual parent test:
#define DONT_RUN_TEST_IF_PARENT if ([[self className] isEqualToString:#"AbstractTestCase"]) { return; }
I then added this macro to the start of every test, like so:
- (void)testSomething
{
DONT_RUN_TEST_IF_PARENT
... actual test code ...
}
This way, in the ConcreteTestCase classes which inherit from AbstractTestCase, all those tests would be shared and run automatically. You can override -setUp to perform the necessary class-specific set-up, of course.
However – This turned out to be a crappy solution for a couple reasons:
It confuses Xcodes testing UI. You don't really get to see a live representation of what's running, and tests sometimes don't show up as intended. This makes clicking through to debug test failures difficult or impossible.
It confuses XCTest itself – I found that tests would often get run even when I didn't ask them too (if I were just trying to run a single test) and the so the test output wouldn't be what I would expect.
Honestly it felt a little janky to have that macro – macros which redirect flow control are never really that good of an idea.
Instead, I'm now using a shared object, a TestCaseHelper, which is instantiated for each test class, and has has a protocol/delegate pattern common to all test cases. It's less DRY – most test cases are just duplicates of the others – but at least they are simple. This way, Xcode doesn't get confused, and debugging failures is still possible.
A better solution will likely have to come from Apple, unless you're interested in ditching your entire test suite for something else.
Is it possible to setup a unit test class to call setup and teardown methods only once for all the test cases not for each test case?
Actually I found the answer. In order to do class level setup, one needs to implement +(void) setUp and +(void) tearDown. This class methods will be called before any test methods run and after all of the test methods run.
Check this from Apple document
I am dealing with a case where my tests pass or fail based on the order of declaration. This of-course points to not properly isolated tests. But I am stumped about how to go about finding the issue.
The thing is my junit tests derive from a class that is belongs to a testing framework built on junit and has some dependency injection container. The container gets reset for every test by the base class setup and so there are no lingering objects at least in the container since the container itself is new. So I am leaning toward the following scenario.
test1 indirectly causes some classA which sets up classA.somestaticMember to xyz value. test obj does not maintain any references to classA directly- but classA is still loaded by vm with a value xyz when test1 ends.
test2 access classA and trips up on somestaticmember having xyz value.
The problem is a) I dont know if this is indeed the case- how do I go about finding that ? I cannot seem to find a reference to a static var in the code...
b) is there a way to tell junit to dump all its loaded classes and do it afresh for every test method ?
You can declare a method with #Before, like
#Before public void init()
{
// set up stuff
}
and JUnit will run it before each test. You can use that to set up a "fixture" (a known set of fresh objects, data, etc that your tests will work with independently of each other).
There's also an #After, that you can use to do any cleanup required after each test. You don't normally need to do this, as Java will clean up any objects you used, but it could be useful for restoring outside objects (stuff you don't create and control) to a known state.
(Note, though: if you're relying on outside objects in order to do your tests, what you have isn't a unit test anymore. You can't really say whether a failure is due to your code or the outside object, and that's one of the purposes of unit tests.)