I'm implementing the performance test for iOS app, and a strange result is derived from it and don't know why.
As Xcode performance test runs given code 10 times, test run on my machine always results such that the first attempt runs in a huge amount of time compared to other 9 attempts.
Jobs that I want to test performance is very simple, it iterates through given array and investigates with a certain conditions. There's no network connection, array manipulation, or reusing of previous variables whatsoever.
Test codes that I wrote is like the following (all of these result in the same behavior).
a. Initialize variables outside the measure block
- (void)testSomething {
// do some initialization
[self measureBlock:^{
// run code to test performance
}];
}
b. Initialize variables inside the measure block
- (void)testSomething {
[self measureBlock:^{
// do some initialization
// run code to test performance
}];
}
c. Initialize variables in -setUp, and use them in test method
- (void)setUp {
// do some initialization here, and capture the pointers as properties
}
- (void)testSomething {
[self measureBlock:^{
// run code to test performance, using properties initialized in setUp
}];
}
- (void)tearDown {
// clean up properties
}
The results are very strange, and the other 9 attempts seem not to be reliable because of the number of first attempt. Why does the first iteration take so much longer?
Related
I have a number of asynchronous unit tests which work correctly on their own using expectations. However when I run all the tests in the suit, they do not wait for each other to complete - the asynchronous callbacks are still pending when the next tests start running. What I want is for each test to wait for the expectations in the previous test before running. These tests use a shared database, so having them overlap leads to annoying additional complexity, and failing tests when run as a suite.
- (void)testSignIn {
// This is an example of a functional test case.
// Use XCTAssert and related functions to verify your tests produce the correct results.
XCTestExpectation *expectation =
[self expectationWithDescription:#"Expectations"];
[_userManager signInWithUsername:kUserEmail andPassword:kUserPassword
success:^{
XCTAssertNotNil([_userManager getCurrentUser]);
XCTAssertNotNil([_userManager getCurrentUser].plan);
XCTAssertTrue([_userManager getCurrentUser].plan.liveStream == TRUE);
[expectation fulfill];
} failure:^(EDApiError *apiError) {
XCTAssertTrue(FALSE); // Should not fail
[expectation fulfill];
}];
[self waitForExpectationsWithTimeout:5.0 handler:^(NSError *error) {
if (error) {
NSLog(#"Timeout Error: %#", error);
}
}];
}
Use XCTWaiter or waitForExpectations(timeout:handler:) to stall the completion of each test until the expectations have been fulfilled.
This blog post explains some of the more complex pitfalls you might come across when writing asynchronous tests and how to prevent them: https://jeremywsherman.com/blog/2016/03/19/xctestexpectation-gotchas/
I've found you need to fulfil expectations on the main thread - important if your async completion block is maybe running on another thread. I've also found that the problem can be triggered by a faulty test that ran earlier, so it isn't always caused by the test the failure manifests in.
Also, watch if you're spawning lots of async blocks (I do this for crazy thread safety tests, or tests checking exclusive access to a resource is working as expected, and in the order expected). Once you fulfil the expectation, the testing moves on to the next test, but if you've fired off a lot of async blocks after that point, they're maybe still running.
I started receiving the following error today after adding some complexity to my shader:
Execution of the command buffer was aborted due to an error during execution. Discarded (victim of GPU error/recovery) (IOAF code 5)
What i discovered is that it has nothing to do with the actual added code but with fact i added more variables and function calls. I tried removing other complexities from the shader and the error was removed.
Another thing i discovered is that problem also removed when i set fast math to false.
My first guess is that there is some kind of a limit on number of variables when fast math is on. Is there such a limit? Any other ideas why such error might occur?
Most probable cause is that Metal buffers or shaders are overloaded
there is a limitations of using metal technology and it is described here
https://developer.apple.com/library/content/documentation/3DDrawing/Conceptual/OpenGLES_ProgrammingGuide/BestPracticesforShaders/BestPracticesforShaders.html
I was having this problem before and i was about to switch from Metal to OpenGL But the advantages of Metal let me try again then i discover that my issue that i was Calculating and sorting an Array of Heavy data (Mostly floats and doubles) within the renderer function
My Mistake was Here See the remarked line
- (void)renderer:(id <SCNSceneRenderer>)renderer updateAtTime:(NSTimeInterval)time
{
[self calculateMyData]; // This Is Wrong
}
- (void)calculateMyData
{
// the Heavy Calculations
}
to avoid most IOAF Errors try not to do heavy or complex calculations like sorting data or such within the renderer try use External Loops to call this calculations. This is what i have done
- (void)viewDidLoad
{
// I Use A Timer Loop Every 0.3 Sec to call the heave calculations and sort data
NSTimer *timerCounter2 = [NSTimer scheduledTimerWithTimeInterval:0.3 target:self selector:#selector(calculateMyData) userInfo:nil repeats: YES];
}
- (void)renderer:(id <SCNSceneRenderer>)renderer updateAtTime:(NSTimeInterval)time
{
//[self calculateMyData]; // Now I Avoid The Mistake
}
- (void)calculateMyData
{
// the Heavy Calculations
}
In my iOS project i call a method that contains 2 blocks say A & B.
Both blocks perform 2 different network calls, A goes first and based on some network response, executes block B. I've done so for tidiness since both do a lot of stuff & I've seen this work in my project but I'm not sure if it is correct to do so. Some illustration:
// somewhere in code
...
[self methodThatContainsTwoBlocks];
...
- (void)methodThatContainsTwoBlocks {
Block B ^(responseFromSingletonOperation) {
// do another network operation
[MyNetworkManagerSingleton performAnotherOperationAsynchronouslyWithCompletion:^{
// call some method when done
[self wereDone];
}
}
[MyNetworkManagerSingleton performAnOperationAsynchronouslyWithCompletion:^{
// do a network operation
// call block B with response
B(responseFromNetworkOperation);
}
}
The phrasing of my question may not be correct since the networking singleton instance takes a completion block, hence my saying, two blocks. I hope the illustration makes my question clearer though as I've only just started iOS programming and still dabbling my way with blocks since senior devs recommend them a lot.
Thanks.
I have a test suite in which there are many test classes each one having many tests.
Now if I define
+ (void)tearDown
in any class, then it will be run after all the tests of that class only.
and
- (void)tearDown
will be run after each test.
I want to define a tearDown method and want it to be run after all the tests of all the classes.
Fox eg, if I have 5 classes each having 7 tests each. So I want to run this tearDown function after all the 35 tests.
Since you're looking for a way to call __gcov_flush() after all tests have finished, see https://qualitycoding.org/ios-7-code-coverage/. The latest code from Google Toolbox for Mac shows how to do this using test observers, with versions for both SenTestingKit and XCTest.
If I understand correctly your question, you can take note of completion of each method using a global variable/flag using completion block methods like this:
+ (void)myMethod:(UIView *)exampleView completion:(void (^)(BOOL finished))completion {
if (completion) {
completion(finished);
}
}
Look at this for a better explanation.
Then create a method that checks if all taskes are executed and that runs final test when needed like this:
- (void)finalTest {
if (CHECK GLOBAL FLAG FOR COMPLETION OF OTHER TASKES) {
// do final test
} else {
// repeat check after 0.1 seconds
[self performSelector:#selector(finalTest)
withObject:nil
afterDelay:0.1];
}
}
I have an iPad app that I am testing in Instruments before beta testing. I have gotten rid of all memory leaks except one, and I can't find any information on it. I am baffled as to what to do, since my code never mentions the leaking object which is an instance of NSDecimalNumberPlaceHolder.
For sure I am using NSDecimalNumber. I create 2 decimals per user operation and each I time I run a cycle of the app (which performs some math operation on the two NSDecimalNumbers) I generate four instances of this NSDecimalPlaceHolder thing. Since I do not know how it gets created, I do not know how to release or dealloc it so as to not generate these 16 btye leaks over and over again.
Is it possible that these are not really leaks?
I have run the XCode Analyzer and it reports no issues.
What I'm doing is this:
I send a decimal number from my controller over to my model (analyzer_) which performs the operations and sends back the result.
[[self analyzer_] setOperand:[NSDecimalNumber decimalNumberWithString:anotherStringValue]];
The setOperand method looks like this:
-(void)setOperand:(NSDecimalNumber*)theOperand
{
NSLog(#"setOperand called");
operand_ = theOperand;
//[operand_ retain];
}
Note that if I don't retain operand_ "somewhere" I get a BAD_ACCESS crash. I currently retain and release it later where the operand and the previously provided operand (queuedOperand_) are operated upon. For example:
{
[self performQueuedOperation];
queuedOperation_ = operation;
queuedOperand_ = operand_;
}
return operand_;
[operand_ release];
where performQueuedOperation is:
-(void)performQueuedOperation
{
[operand_ retain];
if ([#"+" isEqualToString:queuedOperation_])
{
#try
{
operand_ = [queuedOperand_ decimalNumberByAdding:operand_];
}
#catch (NSException *NSDecimalNumberOverFlowException)
{
//viewController will send decimal point error message
}
<etc for the other operations>
}
Let me know if this is not clear. Thanks.
Try Heapshot in Instruments, see: When is a Leak not a Leak?
If there is still a pointer to the memory that is no longer used it is not a leak but it is lost memory. I use Heapshot often, it really works great. Also turn on recording reference counting in the Allocations tool and drill down. Here is a screenshot: