I have a test suite in which there are many test classes each one having many tests.
Now if I define
+ (void)tearDown
in any class, then it will be run after all the tests of that class only.
and
- (void)tearDown
will be run after each test.
I want to define a tearDown method and want it to be run after all the tests of all the classes.
Fox eg, if I have 5 classes each having 7 tests each. So I want to run this tearDown function after all the 35 tests.
Since you're looking for a way to call __gcov_flush() after all tests have finished, see https://qualitycoding.org/ios-7-code-coverage/. The latest code from Google Toolbox for Mac shows how to do this using test observers, with versions for both SenTestingKit and XCTest.
If I understand correctly your question, you can take note of completion of each method using a global variable/flag using completion block methods like this:
+ (void)myMethod:(UIView *)exampleView completion:(void (^)(BOOL finished))completion {
if (completion) {
completion(finished);
}
}
Look at this for a better explanation.
Then create a method that checks if all taskes are executed and that runs final test when needed like this:
- (void)finalTest {
if (CHECK GLOBAL FLAG FOR COMPLETION OF OTHER TASKES) {
// do final test
} else {
// repeat check after 0.1 seconds
[self performSelector:#selector(finalTest)
withObject:nil
afterDelay:0.1];
}
}
Related
I have a number of asynchronous unit tests which work correctly on their own using expectations. However when I run all the tests in the suit, they do not wait for each other to complete - the asynchronous callbacks are still pending when the next tests start running. What I want is for each test to wait for the expectations in the previous test before running. These tests use a shared database, so having them overlap leads to annoying additional complexity, and failing tests when run as a suite.
- (void)testSignIn {
// This is an example of a functional test case.
// Use XCTAssert and related functions to verify your tests produce the correct results.
XCTestExpectation *expectation =
[self expectationWithDescription:#"Expectations"];
[_userManager signInWithUsername:kUserEmail andPassword:kUserPassword
success:^{
XCTAssertNotNil([_userManager getCurrentUser]);
XCTAssertNotNil([_userManager getCurrentUser].plan);
XCTAssertTrue([_userManager getCurrentUser].plan.liveStream == TRUE);
[expectation fulfill];
} failure:^(EDApiError *apiError) {
XCTAssertTrue(FALSE); // Should not fail
[expectation fulfill];
}];
[self waitForExpectationsWithTimeout:5.0 handler:^(NSError *error) {
if (error) {
NSLog(#"Timeout Error: %#", error);
}
}];
}
Use XCTWaiter or waitForExpectations(timeout:handler:) to stall the completion of each test until the expectations have been fulfilled.
This blog post explains some of the more complex pitfalls you might come across when writing asynchronous tests and how to prevent them: https://jeremywsherman.com/blog/2016/03/19/xctestexpectation-gotchas/
I've found you need to fulfil expectations on the main thread - important if your async completion block is maybe running on another thread. I've also found that the problem can be triggered by a faulty test that ran earlier, so it isn't always caused by the test the failure manifests in.
Also, watch if you're spawning lots of async blocks (I do this for crazy thread safety tests, or tests checking exclusive access to a resource is working as expected, and in the order expected). Once you fulfil the expectation, the testing moves on to the next test, but if you've fired off a lot of async blocks after that point, they're maybe still running.
I'm implementing the performance test for iOS app, and a strange result is derived from it and don't know why.
As Xcode performance test runs given code 10 times, test run on my machine always results such that the first attempt runs in a huge amount of time compared to other 9 attempts.
Jobs that I want to test performance is very simple, it iterates through given array and investigates with a certain conditions. There's no network connection, array manipulation, or reusing of previous variables whatsoever.
Test codes that I wrote is like the following (all of these result in the same behavior).
a. Initialize variables outside the measure block
- (void)testSomething {
// do some initialization
[self measureBlock:^{
// run code to test performance
}];
}
b. Initialize variables inside the measure block
- (void)testSomething {
[self measureBlock:^{
// do some initialization
// run code to test performance
}];
}
c. Initialize variables in -setUp, and use them in test method
- (void)setUp {
// do some initialization here, and capture the pointers as properties
}
- (void)testSomething {
[self measureBlock:^{
// run code to test performance, using properties initialized in setUp
}];
}
- (void)tearDown {
// clean up properties
}
The results are very strange, and the other 9 attempts seem not to be reliable because of the number of first attempt. Why does the first iteration take so much longer?
I'm new to UI Testing and am trying to integrate the feature into an existing project. I'm trying the most basic of tests to just see the framework in action but am having some difficulty. I've added a new UI Testing Bundle target to my project and am able to run my basic test. However I keep getting a "Failure attempting to launch" error that always occurs when I'm trying to actually launch the app in my test. It also says something about a "nil token for current process reference".
Here's the code that I'm testing out:
#implementation SKYPracticeUITesting
- (void)setUp {
[super setUp];
// Put setup code here. This method is called before the invocation of each test method in the class.
// In UI tests it is usually best to stop immediately when a failure occurs.
self.continueAfterFailure = NO;
// UI tests must launch the application that they test. Doing this in setup will make sure it happens for each test method.
XCUIApplication *app = [[XCUIApplication alloc] init];
// In UI tests it’s important to set the initial state - such as interface orientation - required for your tests before they run. The setUp method is a good place to do this.
[app launch];
}
- (void)tearDown {
// Put teardown code here. This method is called after the invocation of each test method in the class.
[super tearDown];
}
- (void)testExample {
// Use recording to get started writing UI tests.
// Use XCTAssert and related functions to verify your tests produce the correct results.
NSLog(#"Something is happening");
}
#end
Any help would be greatly appreciated, especially considering the process in Objective-C isn't very well documented.
When I run a single XCTest class, all tests within succeed.
However when I run it together with other XCTest, some tests in the class fail.
setUp and tearDown method are implemented correctly as following:
- (void)setUp {
[super setUp];
...
}
- (void)tearDown {
...
[super tearDown];
}
I set a breakpoint in the code that should be executed in the test. When I was running the tests in isolation, the breakpoint was reached; when I was running it with other tests, the breakpoint was not reached. I'm thinking that maybe XCTest has some caching behaviors? If so, how to turn it off?
Does anyone know why this might happen?
Thanks a bunch!
I was seeing the same behavior, and my issue was because I had a static variable in my method I was testing that was holding onto its value across tests.
I faced the similar issue when I run all tests together. I could able to fix it by putting Assertion in Dispatch Delay,
DispatchQueue.main.asyncAfter(deadline: .now() + 2.0) {
XCTAssert(self.response != nil)
}
I hope the above solution works for you as well.
I'm testing real web service calls with OCMock.
Right now I'm doing something like:
- (void)testWebservice
{
id mydelegatemock = [OCMockObject mockForProtocol:#protocol(MySUTDelegate)];
[[mydelegatemock expect] someMethod:[OCMArg any]];
[SUT sutWithDelegate:mydelegatemock];
// we need to wait for real result
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:2.0]];
[(OCMockObject*)mydelegatemock verify];
}
It works fine, but it implies that every such test will take 2 seconds.
Is there a way I can set a timeout of e.g. 2 seconds, and let a call to someMethod of mydelegatemock immediately verify and complete the test case?
I do this using a handy utility function I found at this link:
#import <Foundation/Foundation.h>
#import <OCMock/OCMock.h>
#interface TestUtils : NSObject
+ (void)waitForVerifiedMock:(OCMockObject *)mock delay:(NSTimeInterval)delay;
#end
And the implementation:
#import "TestUtils.h"
#implementation TestUtils
+ (void)waitForVerifiedMock:(OCMockObject *)inMock delay:(NSTimeInterval)inDelay
{
NSTimeInterval i = 0;
while (i < inDelay)
{
#try
{
[inMock verify];
return;
}
#catch (NSException *e) {}
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.5]];
i+=0.5;
}
[inMock verify];
}
#end
This allows me to to wait up to a maximum delay (in seconds) without waiting the full amount each time.
I would separate the functional testing of your web services (if you need to do that at all) from the unit testing of your class that processes the web service result.
To unit test, you should mock the web service call, providing a mock result. Then your test would verify that, for that well-defined result, your class behaves accordingly.
If you also want to do functional testing of your web service (say that it returns a specific response given some request), you don't need to mock anything--just call the service and make assertions on the result.
By separating out your tests, you have finer control over the test runs. For example, you could run your fast-running unit tests every time you change code, but run your slow-running functional tests nightly, on a dedicated server, or as needed. And when a test breaks, you'll know whether it's your code or something wrong with the web service.
You could also switch to GHUnit, like a fellow member suggests in the answer to this related question:
SenTestingKit in Xcode 4: Asynchronous testing?
You can find GHUnit here
https://github.com/gabriel/gh-unit