I have a number of asynchronous unit tests which work correctly on their own using expectations. However when I run all the tests in the suit, they do not wait for each other to complete - the asynchronous callbacks are still pending when the next tests start running. What I want is for each test to wait for the expectations in the previous test before running. These tests use a shared database, so having them overlap leads to annoying additional complexity, and failing tests when run as a suite.
- (void)testSignIn {
// This is an example of a functional test case.
// Use XCTAssert and related functions to verify your tests produce the correct results.
XCTestExpectation *expectation =
[self expectationWithDescription:#"Expectations"];
[_userManager signInWithUsername:kUserEmail andPassword:kUserPassword
success:^{
XCTAssertNotNil([_userManager getCurrentUser]);
XCTAssertNotNil([_userManager getCurrentUser].plan);
XCTAssertTrue([_userManager getCurrentUser].plan.liveStream == TRUE);
[expectation fulfill];
} failure:^(EDApiError *apiError) {
XCTAssertTrue(FALSE); // Should not fail
[expectation fulfill];
}];
[self waitForExpectationsWithTimeout:5.0 handler:^(NSError *error) {
if (error) {
NSLog(#"Timeout Error: %#", error);
}
}];
}
Use XCTWaiter or waitForExpectations(timeout:handler:) to stall the completion of each test until the expectations have been fulfilled.
This blog post explains some of the more complex pitfalls you might come across when writing asynchronous tests and how to prevent them: https://jeremywsherman.com/blog/2016/03/19/xctestexpectation-gotchas/
I've found you need to fulfil expectations on the main thread - important if your async completion block is maybe running on another thread. I've also found that the problem can be triggered by a faulty test that ran earlier, so it isn't always caused by the test the failure manifests in.
Also, watch if you're spawning lots of async blocks (I do this for crazy thread safety tests, or tests checking exclusive access to a resource is working as expected, and in the order expected). Once you fulfil the expectation, the testing moves on to the next test, but if you've fired off a lot of async blocks after that point, they're maybe still running.
Related
In objective C,
I am making my program to wait using while loop
doInitialize()
{
dispach_group_t loadDataGroup=dispatch_group_create();
dispatch_group_async(loadDataGroup,...get_global_queue(..),0),^{
renewauth();
}
dispatch_group_notify(loadDataGroup,...get_global_queue(..),0),^{
//Do other tasks once renew session has completed...
}
}
renewauth()
{
RenewAuthTokenInProgress=true;
startRenewThread();
**while (RenewAuthTokenInProgress);**
}
In turn startRenewThread() function also performs dispatch_async operation inside. So I have to make renewAuth() wait.
And async task in startRenewThread will update the bool variable once renewal is successful.
Is there any better approach of doing it other than dispatch_groups?
And is it good to make other threads wait with while (true) statement?
Manoj Kumar,
using a while loop to wait till the boolean variable change is not the correct approach to solve the problem. Here are few of the issues with this method
Your CPU is un-necessarily burdened with checking the variable regularly.
This will clearly show that developer isn't much equipted with basic skills of coding and features available with language.
If for any reason your variable will never change then your CPU will never stop checking the value of bool in while loop and blocks the execution of further code on the same thread.
Here are few of the correct approach :
Blocks or closures : Make use of blocks to execute the code asynchronously when the RenewAuthToken is done.
Delegates : if blocks are harder to understand, Make use of delegates and trigger the delegate when you are done with RenewAuthToken.
Notifications : Add observer for notifications in classes which needs to respond when RenewAuthToken is done and throw notification from the asynctask and let the class to catch it execute the code.
Locks : If it is necessary to block the execution of the thread till the response comes use locks to control the thread execution rather than using while loop
EDIT
As pointed out by fogmeister in comments
If you block the main thread for too long with a while(true) loop then
the app will actually be terminated by the iOS Watchdog as it will
assume it has crashed
Please have a look at the link : understand iOS watchdog termination reasons provided by fogmeister
Hope it helps.
I believe what you need it's a semaphore like:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_LOW, 0), ^{
dispatch_semaphore_t sem = dispatch_semaphore_create(0);
__block BOOL done = FALSE;
while (true) {
[self someCompletionMethod completion:^(BOOL success) {
if(success) { // Stop condition
done = TRUE;
}
// do something
dispatch_semaphore_signal(sem); // This will let a new iteration
}];
dispatch_semaphore_wait(sem, DISPATCH_TIME_FOREVER);
if(done) {
dispatch_async(dispatch_get_main_queue(), ^{
// Dispatch to main
NSLog(#"Done!");
break;
});
}
}
});
Semaphores are an old-school threading concept introduced to the world by the ever-so-humble Edsger W. Dijkstra. Semaphores are a complex topic because they build upon the intricacies of operating system functions.
You can see a tutorial here about semaphore and check it out more links: https://www.raywenderlich.com/63338/grand-central-dispatch-in-depth-part-2
I hope this can help you.
What you do is absolutely lethal. It blocks the running thread (presumably the main thread) so the UI is frozen. It runs one core at 100% load for no reason whatsoever which empties the battery rapidly and heats up the phone. This will get you some very, very unhappy customers or very, very happy ex-customers.
Anything like this has to run in the background: startRenewThread should trigger some action that sets RenewAuthTokenInProgress = NO and sets whether there is a new token or not, and then triggers further action.
This is an absolutely essential programming pattern on iOS (and Android as far as I know).
I tried to run MKNetworkKit in my project, it seemed to be fine. I then created a unit test project and call the engine to make some requests (using the MkNetworkOperation). I get no apparent error but the completionhandler block never get called, and so was the errorblock.
I examined my unit test setup, there's no apparent error, and the whole thing "ran". I just dont get any response. If i switch and do this in my main project, it works.
I have also added all the necessary framework to the unit test project.
One thing i notice is that Reachability message get printed out for the case that worked, but nothing for the case that didnt.
Any clue as to whats going on?
Sounds like you need a semaphore or something similar to block the execution until the request returns. Unit tests run until the end of the method. If they hit the end of the method without an error, they were successful. Since your using MKNetworkKit, the server request is async, so the end of the method is hit before the request is complete. I found a helper class to help with unit tests. I'll try to find it again and link to it with some sample code.
Found it: http://www.touch-code-magazine.com/unit-testing-for-blocks-based-apis/
So it's been a while since I wrote these tests. I'm pretty sure this will work, but you might have to tweak it.
-(void)tests {
//setup
NSString *key = #"myTestKey";
//test
STAssertNoThrow(
[API resetPassword:#"fakeemail#this.net callback:^(NSDictionary *result) {
STAssertNotNil(result, #"reset pass response was nil"]);
[[TestSemaphor sharedInstance] lift:key];
} error:^(NSDictionary *error) {
STFail(#"reset password failed: %#", error.description);
[[TestSemaphor sharedInstance] lift:key];
}], #"reset password failed");
[[TestSemaphor sharedInstance] waitForKey:key];
}
I have a method that at times can be invoked throughout my code. Below is a very basic example, as the code processes images and files off of the iphone photo gallery and marks them already processed when done with the method.
#property (nonatomic, assign) dispatch_queue_t serialQueue;
....
-(void)processImages
{
dispatch_async(self.serialQueue, ^{
//block to process images
NSLog(#"In processImages");
....
NSLog(#"Done with processImages");
});
}
I would think that each time this method is called I would get the below output...
"In processImages"
"Done with processImages"
"In processImages"
"Done with processImages"
etc...
but I always get
"In processImages"
"In processImages"
"Done with processImages"
"Done with processImages"
etc...
I thought a serial queue would wait till the first block is done, then start. To me it seems it is starting the method, then it gets called again and starts up before the first call even finishes, creating duplicates of images that normally would not be processed due to the fact that if it really executed serially the method would know they were already processed. Maybe my understanding of serial queues is not concrete. Any input? Thank you.
EDIT:MORE Context below, this is what is going on in the block...Could this cause the issue???
#property (nonatomic, assign) dispatch_queue_t serialQueue;
....
-(void)processImages
{
dispatch_async(self.serialQueue, ^{
//library is a reference to ALAssetsLibrary object
[library enumerateGroupsWithTypes:ALAssetsGroupSavedPhotos usingBlock:^(ALAssetsGroup *group, BOOL *stop)
{
[group enumerateAssetsUsingBlock:^(ALAsset *asset, NSUInteger index, BOOL *stop)
{
....
//Process the photos here
}];
failureBlock:^(NSError *error) { NSLog(#"Error loading images from library");
}];
});
}
-(id)init
{
self = [super init];
if(self)
{
_serialQueue = dispatch_queue_create("com.image.queue",NULL);
}
return self;
}
this object is only created once, and as far as I can tell can never be created again based off my code...I will run tests to make sure though.
UPDATE 2: WHAT I THINK IS HAPPENING, please comment on this if you agree/disagree....
Obviously my main issue is that it seems this block of code is being executed concurrently, creating duplicate entries (importing the same photo twice) when it wouldn't normally do this if it was run serially. When a photo is processed a "dirty" bit is applied to it ensuring the next time the method is invoked it skips this image, but this is not happening and some images are processed twice. Could this be due to the fact I am enumerating the objects in a second queue using enumerategroupswithtypes: within that serialQueue?
call processImages
enumerateObjects
immediately return from enumerateObjects since it is async itself
end call to processImages
processImages is not really done though due to the fact that enumerategroups is probably still running but the queue might thing it is done since it reaches the end of the block before enumerategroups is finished working. This seems like a possibility to me?
Serial Queues ABSOLUTELY will perform serially. They are not guaranteed to perform on the same thread however.
Assuming you are using the same serial queue, the problems is that NSLog is NOT guaranteed to output results in the proper order when called near simultaneously from different threads.
here is an example:
SQ runs on thread X, sends "In processImages"
log prints "In proc"
SQ on thread X, sends "Done with processImages"
SQ runs on thread Y, sends "In processImages"
log prints "essImages\n"
After 5., NSLog doesn't necessarily know which to print, 3. or 4.
If you absolutely need time ordered logging, You need a dedicated queue for logging. In practice, I've had no problems with just using the main queue:
dispatch_async(dispatch_get_main_queue(), ^{
NSLog(#"whatever");
});
If all NSlog calls are the on the same queue, you shouldn't have this problem.
enumerateGroupsWithTypes:usingBlock:failureBlock: does its work asynchronously on another thread and calls the blocks passed in when it's done (on the main thread I think). Looking at it from another perspective, if it completed all the synchronously by the time the method call was complete, it could just return an enumerator object of the groups instead, for instance, for a simpler API.
From the documentation:
This method is asynchronous. When groups are enumerated, the user may be asked to confirm the application's access to the data; the method, though, returns immediately. You should perform whatever work you want with the assets in enumerationBlock.
I'm not sure why you're trying to accomplish by using the serial queue, but if you just want to prevent simultaneous access, then you could just add a variable somewhere that keeps track of whether we're currently enumerating or not and check that at first, if you don't have to worry about synchronization issues. (If you do, perhaps you should look into using a GCD group, but it's probably overkill for this situation.)
If the question is "Can serial queue perform tasks asynchronously?" then the answer is no.
If you think that it can, you should make sure that all tasks are really performing on the same queue. You can add the following line in the block and compare the output:
dispatch_async(self.serialQueue, ^{
NSLog(#"current queue:%p current thread:%#",dispatch_get_current_queue(),[NSThread currentThread]);
Make sure that you write NSLog in the block that performs on your queue and not in the enumerateGroupsWithTypes:usingBlock:failureBlock:
Also you can try to create your queue like this
dispatch_queue_create("label", DISPATCH_QUEUE_SERIAL);
but I don't think that will change anything
EDIT:
By the way, method
enumerateGroupsWithTypes:usingBlock:failureBlock:
is asynchronous, why do you call it on another queue?
UPDATE 2:
I can suggest something like this:
dispatch_async(queue, ^{
NSLog(#"queue");
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER, *pmutex = &mutex;
pthread_mutex_lock(pmutex);
ALAssetsLibraryGroupsEnumerationResultsBlock listGroupBlock = ^(ALAssetsGroup *group, BOOL *stop) {
NSLog(#"block");
if (group) {
[groups addObject:group];
} else {
[self.tableView performSelectorOnMainThread:#selector(reloadData) withObject:nil waitUntilDone:NO];
dispatch_async(dispatch_get_current_queue(), ^{
pthread_mutex_unlock(pmutex);
});
}
NSLog(#"block end");
};
[assetsLibrary enumerateGroupsWithTypes:groupTypes usingBlock:listGroupBlock failureBlock:failureBlock];
pthread_mutex_lock(pmutex);
pthread_mutex_unlock(pmutex);
pthread_mutex_destroy(pmutex);
NSLog(#"queue end");
});
I hit an issue like this, and the answer for me was to realize that asynchronous calls from a method on the serialized queue goes to another queue for processing -- one that is not serialized.
So you have to wrap all the calls inside the main method with explicit dispatch_async(serializedQueue, ^{}) to ensure that everything is done in the correct order...
Using Swift and semaphores to illustrate an approach to serialization:
Given: a class with an asynchronous ‘run’ method that will be run on multiple objects at once, and the objective is that each not run until the one before it completes.
The issue is that the run method allocates a lot of memory and uses a lot of system resources that can cause memory pressure among other issues if too many are run at once.
So the idea is: if a serial queue is used then only one will run at a time, one after the other.
Create a serial queue in the global space by the class:
let serialGeneratorQueue: DispatchQueue = DispatchQueue(label: "com.limit-point.serialGeneratorQueue", autoreleaseFrequency: DispatchQueue.AutoreleaseFrequency.workItem)
class Generator {
func run() {
asynchronous_method()
}
func start() {
serialGeneratorQueue.async {
self.run()
}
}
func completed() {
// to be called by the asynchronous_method() when done
}
}
The ‘run’ method of this class for which very many objects will be created and run will be processed on the serial queue:
serialGeneratorQueue.async {
self.run()
}
In this case an autoreleaseFrequency is .workItem to clean up memory after each run.
The run method is of some general form:
func run() {
asynchronous_method()
}
The problem with this: the run method exits before the asynchronous_method completes, and the next run method in the queue will run, etc. So the objective is not being achieved because each asynchronous_method is running in parallel, not serially after all.
Use a semaphore to fix. In the class declare
let running = DispatchSemaphore(value: 0)
Now the asynchronous_method completes it calls the ‘completed’ method:
func completed() {
// some cleanup work etc.
}
The semaphore can be used to serialized the chain of asynchronous_method’s by add ‘running.wait()’ to the ‘run’ method:
func run() {
asynchronous_method()
running.wait()
}
And then in the completed() method add ‘running.signal()’
func completed() {
// some cleanup work etc.
running.signal()
}
The running.wait() in ‘run’ will prevent it from exiting until signaled by the completed method using running.signal(), which in turn prevents the serial queue from starting the next run method in the queue. This way the chain of asynchronous methods will indeed be run serially.
So now the class is of the form:
class Generator {
let running = DispatchSemaphore(value: 0)
func run() {
asynchronous_method()
running.wait()
}
func start() {
serialGeneratorQueue.async {
self.run()
}
}
func completed() {
// to be called by the asynchronous_method() when done
running.signal()
}
}
I thought a serial queue would wait [until] the first block is done ...
It does. But your first block simply calls enumerateGroupsWithTypes and the documentation warns us that the method runs asynchronously:
This method is asynchronous. When groups are enumerated, the user may be asked to confirm the application's access to the data; the method, though, returns immediately.
(FWIW, whenever you see a method that has a block/closure parameter, that’s a red flag that the method is likely performing something asynchronously. You can always refer to the relevant method’s documentation and confirm, like we have here.)
So, bottom line, your queue is serial, but it is only sequentially launching a series of asynchronous tasks, but obviously not waiting for those asynchronous tasks to finish, defeating the intent of the serial queue.
So, if you really need to have each tasks wait for the prior asynchronous task, there are a number of traditional solutions to this problem:
Use recursive pattern. I.e., write a rendition of processImage that takes an array of images to process and:
check to see if there are any images to process;
process first image; and
when done (i.e. in the completion handler block), remove the first image from the array and then call processImage again.
Rather than dispatch queues, consider using operation queues. Then you can implement your task as an “asynchronous” NSOperation subclass. This is a very elegant way of wrapping an asynchronous task This is illustrated in https://stackoverflow.com/a/21205992/1271826.
You can use semaphores to make this asynchronous task behave synchronously. This is also illustrated in https://stackoverflow.com/a/21205992/1271826.
Option 1 is the simplest, option 2 is the most elegant, and option 3 is a fragile solution that should be avoided if you can.
You might have more than one object, each with its own serial queue. Tasks dispatched to any single serial queue are performed serially, but tasks dispatched to different serial queues will absolutely be interleaved.
Another simple bug would be to create not a serial queue, but a concurrent queue...
I use a subclass of NSOperation to upload large files to AWS S3 using Amazon's iOS SDK (v1.3.2). This all works fine, but some beta testers experience deadlocks (iOS 5.1.1). The result is that the NSOperationQueue in which the operations are scheduled is blocked as only one operation is allowed to run at one time. The problem is that I cannot reproduce the issue whereas the beta testers experience this problem every single time.
The operation is quite complex due to how the AWS iOS SDK works. However, the problem is not related to the AWS iOS SDK as far as I know based on my testing. The operation's main method is pasted below. The idea of the operation's main method is based on this Stack Overflow question.
- (void)main {
// Operation Should Terminate
_operationShouldTerminate = NO;
// Notify Delegate
dispatch_async(dispatch_get_main_queue(), ^{
[self.delegate operation:self isPreparingUploadWithUuid:self.uuid];
});
// Increment Network Activity Count
[self incrementNetworkActivityCount];
// Verify S3 Credentials
[self verifyS3Credentials];
while (!_operationShouldTerminate) {
if ([self isCancelled]) {
_operationShouldTerminate = YES;
} else {
// Create Run Loop
[[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantFuture]];
}
}
// Decrement Network Activity Count
[self decrementNetworkActivityCount];
NSLog(#"Operation Will Terminate");
}
The method that finalizes the multipart upload sets the boolean _operationShouldTerminate to YES to terminate the operation. That method looks like this.
- (void)finalizeMultipartUpload {
// Notify Delegate
dispatch_async(dispatch_get_main_queue(), ^{
[self.delegate operation:self didFinishUploadingUploadWithUuid:self.uuid];
});
// Operation Should Terminate
_operationShouldTerminate = YES;
NSLog(#"Finalize Multipart Upload");
}
The final log statement is printed to the console, but the while loop in the main method does not seem to exit as the final log statement in the operation's main method is not printed to the console. As a result, the operation queue in which the operation is scheduled, is blocked and any scheduled operations are not executed as a result.
The operation's isFinished method simply returns _operationShouldTerminate as seen below.
- (BOOL)isFinished {
return _operationShouldTerminate;
}
It is odd that the while loop is not exited and it is even more odd that it does not happen on any of my own test devices (iPhone 3GS, iPad 1, and iPad 3). Any help or pointers are much appreciated.
The solution to the problem is both complex and simple as it turns out. What I wrongly assumed was that the methods and delegate callbacks of the operation were executed on the same thread, that is, the thread on which the operation's main method was called. This is not always the case.
Even though this was true in my test and on my devices (iPhone 3GS), which is why I did not experience the problem myself. My beta testers, however, used devices with multicore processors (iPhone 4/4S), which caused some of the code to be executed on a thread different from the thread on which the operation's main method was invoked.
The result of this is that _operationShouldTerminate was modified in the finalizeMultipartUpload method on the wrong thread. This in turn means that the while loop of the main method was not exited properly resulting in the operation deadlocking.
In short, the solution is to update _operationShouldTerminate on the same thread as the main method was invoked on. This will properly exit the while loop and exit the operation.
There are a number of problems with your code, and I can offer two solutions:
1) read up on Concurrent NSOperations in Apple's Concurrency Programming Guide. To keep the runLoop "alive" you have to add either a port or schedule a timer. The main loop should contain a autorelease pool as you may not get one (see Memory Management in that same memo). You need to implement KVO to let the operationQueue know when your operation is finished.
2) Or, you can adopt a small amount of field tested hardened code and reuse it. That Xcode project contains three classes of interest to you: a ConcurrentOperation file that does well what you are trying to accomplish above. The Webfetcher.m class shows how to subclass the concurrent operation to perform an asynchronous URL fetch from the web. And the OperationsRunner is a small helper file you can add to any kind of class to manage the operations queue (run, cancel, query, etc). All of the above are less than 100 lines of code, and provide a base for you to get your code working. The OperationsRunner.h file provide a "how to do" too.
As mentioned in title, I would like to open UIManagedDocument synchronously, i.e, I would like my execution to wait till open completes. I'm opening document on mainThread only.
Current API to open uses block
[UIManagedDocument openWithCompletionHandler:(void (^)(BOOL success))];
Locks usage mentioned at link works well on threads other than main thread. If I use locks on mainThread, it freezes execution of app.
Any advice would be helpful. Thanks.
First, let me say that I strongly discourage doing this. Your main thread just waits, and does nothing while waiting for the call to complete. Under certain circumstances, the system will kill your app if it does not respond on the main thread. This is highly unusual.
I guess you should be the one to decide when/how you should use various programming tools.
This one does exactly what you want... block the main thread until the completion handler runs. Again, I do not recommend doing this, but hey, it's a tool, and I'll take the NRA stance: guns don't kill people...
__block BOOL waitingOnCompletionHandler = YES;
[object doSomethingWithCompletionHandler:^{
// Do your work in the completion handler block and when done...
waitingOnCompletionHandler = NO;
}];
while (waitingOnCompletionHandler) {
usleep(USEC_PER_SEC/10);
}
Another option is to execute the run loop. However, this isn't really synchronous, because the run loop will actually process other events. I've used this technique in some unit tests. It is similar to the above, but still allows other stuff to happen on the main thread (for example, the completion handler may invoke an operation on the main queue, which may not get executed in the previous method).
__block BOOL waitingOnCompletionHandler = YES;
[object doSomethingWithCompletionHandler:^{
// Do your work in the completion handler block and when done...
waitingOnCompletionHandler = NO;
}];
while (waitingOnCompletionHandler) {
NSDate *futureTime = [NSDate dateWithTimeIntervalSinceNow:0.1];
[[NSRunLoop currentRunLoop] runUntilDate:futureTime];
}
There are other methods as well, but these are simple, easy to understand, and stick out like a sore thumb so it's easy to know you are doing something unorthodox.
I should also note that I've never encountered a good reason to do this in anything other than tests. You can deadlock your code, and not returning from the main run loop is a slippery slope (even if you are manually executing it yourself - note that what called you is still waiting and running the loop again could re-enter that code, or cause some other issue).
Asynchronous APIs are GREAT. The condition variable approach or using barriers for concurrent queues are reasonable ways to synchronize when using other threads. Synchronizing the main thread is the opposite of what you should be doing.
Good luck... and make sure you register your guns, and always carry your concealed weapons permit. This is certainly the wild west. There's always a John Wesley Harden out there looking for a gun fight.