I am developing a static library that needs to do some stuff in the background, without interacting with the main thread. To give you an idea, think of just logging some user events. The library must keep doing this stuff until the user exits the app or sends it to the background (pushes the home button) - in other words it needs to keep doing stuff inside a loop.
The only interaction between the main app thread and the spawned thread is that occasionally the main app thread will put some stuff (an event object) into a queue that the spawned thread can read/consume. Other than that, the spawned thread just keeps going until the app exists or backgrounds.
Part of what the spawned thread needs to do (though not all of it) involves sending data to an HTTP server. I would have thought that it would be easy to subclass NSThread, override its main method, and just make a synchronous call to NSUrlConnection with some sort of timeout on that connection so the thread doesn't hang forever. For example, in Java/Android, we just subclass Thread, override the start() method and call a synchronous HTTP GET method (say from Apache's HttpClient class). This is very easy and works fine. But from what I have seen here and elsewhere, apparently on iOS it is much more complicated than this and I'm more than a bit confused as to what the best approach is that actually works.
So should I subclass NSThread and somehow use NSUrlConnection? It seems the asynchronous NSUrlConnection does not work inside NSThread because delegate methods don't get called but what about the synchronous method? Do I somehow need to use and configure the RunLoop and set up an autorelease pool? Or should I use an NSOperation? It seems to me that what I am trying to do is pretty common - does anyone have a working example of how to do this properly?
As I understand it, to use NSURLConnection asynchronously you need a runloop. Even if you use an NSOperation you still need a runloop.
All the examples I have seen use the Main Thread to start NSURLConnection which has a runloop. The examples using NSOperation are set up so the operation is Concurrent which tells NSOperationQueue not to provide it's own thread, they then make sure that NSURLConnection is started on the main thread, for example via a call to performSelectorOnMainThread:
Here is an example:
Pulse Engineering Blog: Concurrent Downloads using NSOperationQueues
You can also search the Apple documentation for QRunLoopOperation in the LinkedImageFetcher sample which is an example class showing some ins and outs of this kind of thing.
(Although I'm not sure I actually saw any code that example showing how to run your own runloop, again this example relies on the main thread.)
I've used the grand central dispatch (GCD) methods to achieve this. Here is an example that worked for me in a simple test app (I'm not sure if it applies in a static library, but may be worth a look). I'm using ARC.
In the example, I am kicking off some background work from my viewDidLoad method, but you can kick it off from anywhere. The key is that "dispatch_async(dispatch_get_global_queue…" runs the block in a background thread. See this answer for a good explanation of that method: https://stackoverflow.com/a/12693409/215821
Here is my viewDidLoad:
- (void)viewDidLoad
{
[super viewDidLoad];
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, (unsigned long)NULL),
^(void) {
[self doStuffInBackground];
});
}
The doStuffInBackground method is running in the background at this point, so you can just use NSURLConnection synchronously. In my example here, the method loops making network calls until presumably some other code sets backgroundStuffShouldRun = false. A network call is made with a 10 second timeout. After the call, I'm updating a UI label just to show progress. Note that the UI update is performed with "dispatch_async(dispatch_get_main_queue()…". This runs the UI update on the UI thread, as required.
One potential issue with this background work: there isn't a way to cancel the http request itself. But, with a 10 second timeout, you'd be waiting a max of 10 seconds for the thread to abort itself after an outsider (likely some event in your UI) sets backgroundStuffShouldRun = false.
- (void)doStuffInBackground
{
while (backgroundStuffShouldRun) {
// prepare for network call...
NSURL* url = [[NSURL alloc] initWithString:#"http://maps.google.com/maps/geo"];
// set a 10 second timeout on the request
NSURLRequest* request = [[NSURLRequest alloc] initWithURL:url cachePolicy:NSURLCacheStorageAllowed timeoutInterval:10];
NSError* error = nil;
NSURLResponse *response = nil;
// make the request
NSData* data = [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];
// were we asked to stop the background processing?
if (!backgroundStuffShouldRun) {
return;
}
// process response...
NSString* status = #"Success";
if (error) {
if (error.code == NSURLErrorTimedOut) {
// handle timeout...
status = #"Timed out";
}
else {
// handle other errors...
status = #"Other error";
}
}
else {
// success, handle the response body
NSString *dataAsString = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
NSLog(#"%#", dataAsString);
}
// update the UI with our status
dispatch_async(dispatch_get_main_queue(), ^{
[statusLabel setText:[NSString stringWithFormat:#"completed network call %d, status = %#", callCount, status]];
});
callCount++;
sleep(1); // 1 second breather. not necessary, but good idea for testing
}
}
Related
I'm currently working on an iOS project that utilises the AWS SDK to download large media files to the device. I am using CloudFront to distribute the content and the downloads are working well, however I am having problems implementing a network queue for these operations. No matter what I try, all the files want to download at once.
I am using the AWSContent downloadWithDownloadType: method to initiate and monitor progress on the actual downloads.
I have tried using an NSOperationQueue and setting setMaxConcurrentOperationCount, and all the code blocks execute at once. :(
I have a feeling it might be configurable with AWSServiceConfiguration in the AppDelegate, but the documentation is extremely vague on what variables you can pass into that object... http://docs.aws.amazon.com/AWSiOSSDK/latest/Classes/AWSServiceConfiguration.html
Has anyone had any experience with this?
TIA
Your problem is most likely that you misunderstand an approach of asynchronous operations.
I have tried using an NSOperationQueue and setting
setMaxConcurrentOperationCount, and all the code blocks execute at
once. :(
It's difficult to say what's definitely wrong without seeing an actual code, however most likely it's tied to the following steps:
You create NSOperationQueue
You set maxConcurrentOperationsCount to 2 for example
You add 4 blocks to it with AWSContent downloadWithDownloadType:
You expect no more 2 downloads to be run simultaneously
What do you probably do wrong
The key is inside point 3. What exactly the block does? My guess is that it completes before actual download completes. So if you have something like:
NSOperationQueue *queue = [NSOperationQueue new];
queue.maxConcurrentOperationsCount = 2;
for (AWSContent *content in contentArray) { // Assume you already do have this array
[queue addOperationWithBlock:^() {
[content downloadWithDownloadType:AWSContentDownloadTypeIfNotCached
pinOnCompletion:YES
progressBlock:nil
completionHandler:^(AWSContent *content, NSData *data, NSError *error) {
// do some stuff here on completion
}];
}];
}
Your block exits before your download is finished, allowing next blocks to run on queue and starting further downloads.
What to try
You should simply add some synchronization mechanism to your block to let operation complete only on completion block. Say:
NSOperationQueue *queue = [NSOperationQueue new];
queue.maxConcurrentOperationsCount = 2;
for (AWSContent *content in contentArray) { // Assume you already do have this array
[queue addOperationWithBlock:^() {
dispatch_semaphore_t dsema = dispatch_semaphore_create(0);
[content downloadWithDownloadType:AWSContentDownloadTypeIfNotCached
pinOnCompletion:YES
progressBlock:nil
completionHandler:^(AWSContent *content, NSData *data, NSError *error) {
// do some stuff here on completion
// ...
dispatch_semaphore_signal(dsema); // it's important to call this function in both error and success cases of download to free the block from queue
}];
dispatch_semaphore_wait(dsema, DISPATCH_TIME_FOREVER); // or another dispatch_time if you want your custom timeout instead of AWS
}];
}
Effectively your answer is https://stackoverflow.com/a/4326754/2392973
You just schedule plenty of such blocks to your operation queue.
More reading
https://developer.apple.com/reference/dispatch
I have an NSOperationQueue that handles importing data from a web server on a loop. It accomplishes this with the following design.
NSURLConnect is wrapped in an NSOperation and added to the Queue
On successful completion of the download (using a block), the data from the request is wrapped in another NSOperation that adds the relevant data to Core Data. This operation is added to the queue.
On successful completion (using another block), (and after a specified delay) I call the method that started it all and return to step 1. Thus, i make another server call x seconds later.
This works great. I'm able to get data from the server and handle everything on the background. And because these are just NSOperations I'm able to put everything in the background, and perform multiple requests at a time. This works really well.
The ONLY problem that I currently have is that I'm unable to successfully cancel the operations once they are going.
I've tried something like the following :
- (void)flushQueue
{
self.isFlushingQueue = YES;
[self.operationQueue cancelAllOperations];
[self.operationQueue waitUntilAllOperationsAreFinished];
self.isFlushingQueue = NO;
NSLog(#"successfully flushed Queue");
}
where self.isFlushingQueue is a BOOL that I use to check before adding any new operations to the queue. This seems like it should work, but in fact it does not. Any ideas on stopping my Frankenstein creation?
Edit (Solved problem, but from a different perspective)
I'm still baffled about why exactly I was unable to cancel these operations (i'd be happy to keep trying possible solutions), but I had a moment of insight on how to solve this problem in a slightly different way. Instead of dealing at all with canceling operations, and waiting til queue is finished, I decided to just have a data structure (NSMutableDictionary) that had a list of all active connections. Something like this :
self.activeConnections = [NSMutableDictionary dictionaryWithDictionary:#{
#"UpdateContacts": #YES,
#"UpdateGroups" : #YES}];
And then before I add any operation to the queue, I simply ask if that particular call is On or Off. I've tested this, and I successfully have finite control over each individual server request that I want to be looping. To turn everything off I can just set all connections to #NO.
There are a couple downsides to this solution (Have to manually manage an additional data structure, and every operation has to start again to see if it's on or off before it terminates).
Edit -- In pursuit of a more accurate solution
I stripped out all code that isn't relevant (notice there is no error handling). I posted two methods. The first is an example of how the request NSOperation is created, and the second is the convenience method for generating the completion block.
Note the completion block generator is called by dozens of different requests similar to the first method.
- (void)updateContactsWithOptions:(NSDictionary*)options
{
//Hard coded for ease of understanding
NSString *contactsURL = #"api/url";
NSDictionary *params = #{#"sortBy" : #"LastName"};
NSMutableURLRequest *request = [self createRequestUsingURLString:contactsURL andParameters:params];
ConnectionCompleteBlock processBlock = [self blockForImportingDataToEntity:#"Contact"
usingSelector:#selector(updateContactsWithOptions:)
withOptions:options andParsingSelector:#selector(requestUsesRowsFromData:)];
BBYConnectionOperation *op = [[BBYConnectionOperation alloc] initWithURLRequest:request
andDelegate:self
andCompletionBlock:processBlock];
//This used to check using self.isFlushingQueue
if ([[self.activeConnections objectForKey:#"UpdateContacts"] isEqualToNumber:#YES]){
[self.operationQueue addOperation:op];
}
}
- (ConnectionCompleteBlock) blockForImportingDataToEntity:(NSString*)entityName usingSelector:(SEL)loopSelector withOptions:(NSDictionary*)options andParsingSelector:(SEL)parseSelector
{
return ^(BOOL success, NSData *connectionData, NSError *error){
//Pull out variables from options
BOOL doesLoop = [[options valueForKey:#"doesLoop"] boolValue];
NSTimeInterval timeInterval = [[options valueForKey:#"interval"] integerValue];
//Data processed before importing to core data
NSData *dataToImport = [self performSelector:parseSelector withObject:connectionData];
BBYImportToCoreDataOperation *importOperation = [[BBYImportToCoreDataOperation alloc] initWithData:dataToImport
andContext:self.managedObjectContext
andNameOfEntityToImport:entityName];
[importOperation setCompletionBlock:^ (BOOL success, NSError *error){
if(success){
NSLog(#"Import %#s was successful",entityName);
if(doesLoop == YES){
dispatch_async(dispatch_get_main_queue(), ^{
[self performSelector:loopSelector withObject:options afterDelay:timeInterval];
});
}
}
}];
[self.operationQueue addOperation:importOperation];
};
}
Cancellation of an NSOperation is just a request, a flag that is set in NSOperation. It's up to your NSOperation subclass to actually action that request and cancel it's work. You then need to ensure you have set the correct flags for isExecuting and isFinished etc. You will also need to do this in a KVO compliant manner. Only once these flags are set is the operation finished.
There is an example in the documentation Concurrency Programming Guide -> Configuring Operations for Concurrent Execution. Although I understand that this example may not correctly account for all multi-threaded edge cases. Another more complex example is provided in the sample code LinkedImageFetcher : QRunLoopOperation
If you think you are responding to the cancellation request correctly then you really need to post your NSOperation subclass code to examine the problem any further.
Instead of using your own flag for when it is ok to add more operations, you could try the
- (void)setSuspended:(BOOL)suspend
method on NSOperationQueue? And before adding a new operation, check if the queue is suspended with isSuspended?
I am interacting with a web-controlled hardware device. You send it a request via a URL (e.g., http://device/on?port=1 or http://device/off?port=3) to turn stuff on and off, and it sends back "success" or "failure". It is a simple device, however, so while it's processing a request --- i.e., until it returns the status of the request that it's processing --- it will ignore all subsequent requests. It does not queue them up; they just get lost.
So I need to send serial, synchronous requests. I.e., req#1, wait for response#1, req#2, wait for response#2, req#3, wait for response #3, etc.
Do I need to manage my own thread-safe queue of requests, have the UI thread push requests into one end of the queue, and have another thread pull the requests off, one at a time, as soon as the previous one either completes or times out, and send the results back to the UI thread? Or am I missing something in the API that already does this?
Thanks!
...R
What should work is to use an NSOperationQueue instance, and a number of NSOperation instances that perform the various URL requests.
First, set up a queue in the class that will be enqueueing the requests. Make sure to keep a strong reference to it, i.e.
#interface MyEnqueingClass ()
#property (nonatomic, strong) NSOperationQueue *operationQueue;
#end
Somewhere in the implementation, say the init method:
_operationQueue = [[NSOperationQueue alloc] init];
_operationQueue.maxConcurrentOperationCount = 1;
You want basically a serial queue, hence the maxConcurrentOperationCount of 1.
After setting this up, you'll want to write some code like this:
[self.operationQueue addOperationWithBlock:^{
NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:#"my://URLString"]];
NSError *error;
NSURLResponse *response;
NSData *responseData = [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];
if (!responseData)
{
//Maybe try this request again instead of completely restarting? Depends on your application.
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
//Do something here to handle the error - maybe you need to cancel all the enqueued operations and start again?
[self.operationQueue cancelAllOperations];
[self startOver];
}];
}
else
{
//Handle the success case;
}
}];
[self.operationQueue addOperationWithBlock:^{
//Make another request, according to the next instuctions?
}];
In this way you send synchronous NSURLRequests and can handle the error conditions, including by bailing out completely and starting all over (the lines with -cancelAllOperations called). These requests will be executed one after the other.
You can also of course write custom NSOperation subclasses and enqueuing instances of those rather than using blocks, if that serves you.
Hope this helps, let me know if you have any questions!
You can use NSOperationQueue class and also use some API which are build in on it for example AFNetworking.
On iOS, can you make a synchronous network request (off the main thread) and get progress callbacks (on a separate, non-main thread)?
I have have a serial (one-operation-at-a-time) background queue that runs all of time-consuming jobs that don't need to finish right now. I do want to show progress for the download jobs though. It doesn't look like you can instantiate an NSURLConnection and configure a delegate, start synchronous connection, and then get progress callbacks.
Is there a way to make a synchronous request on that background queue (synchronous in that the job behind it doesn't start until its done), and still get setProgress: callbacks which could be sent to update a progressbar? (Callbacks would have to be on a different queue thread, since my serial queue's thread is blocked until the request is finished.)
Apple's docs for NSURLConnection say that the synchronous request is actually built on top of the asynchronous behind the scenes. Do I have to re-implement that? I need a way to block a thread until the request finishes/fails. The best leads I have so far are NSOperationQueue's waitUntilFinished method, but I don't want to start async and continually poll on the synchronous method.
NSURLConnection Discussion
A synchronous load is built on top of the asynchronous loading code made available by the class. The calling thread is blocked while the asynchronous loading system performs the URL load on a thread spawned specifically for this load request. No special threading or run loop configuration is necessary in the calling thread in order to perform a synchronous load.
Reimplementing a synchronous request on top of an asynchronous request is not that hard. You just need to manually spin the thread's run loop until you see that the request has finished (or failed). Here's a rough example:
NSURLRequest *downloadRequest = [NSURLRequest requestWithURL:queryURL];
// Start the download and wait until it finishes
self.downloadFinished = NO;
self.downloader = [NSURLConnection connectionWithRequest:downloadRequest delegate:self];
while (!self.isDownloadFinished)
{
[[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantFuture]];
}
Here are the relevant NSURLConnectionDelegate methods:
- (void)connectionDidFinishLoading:(NSURLConnection *)connection;
{
self.downloadFinished = YES;
}
- (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error;
{
NSLog(#"An error occurred: %#", error);
self.receivedData = nil;
self.downloadFinished = YES;
}
I'm looping through a list of dates and making a request to a web server for each date in the list.
I would like each date to be processed completely before the subsequent request is sent to the server. To do this, I have set up a serial dispatch queue using GCD. Each time through the date loop, a block is added to the queue.
The problem I am having is that my NSURLConnection is set up using the standard asynchronous call. This results in requests not blocking any subsequent requests. They are thus overrunning each other.
My question: Is this a case where it would make sense for me to use the synchronous NSURLConnection (within the dispatch queue) or is there some other way to make it work using the standard asynchronous call?
There are number of ways to do this. Whatever method you choose, starting the connection needs to be tied to completion of your processing task.
In each block you add to your serial queue, use a synchronous request. This is probably the quickest solution given your current implementation as long as you're ok with the limited error handling of a synchronous request.
Don't use a serial queue. Start the first asynchronous connection and process the response. When processing is complete start the next asynchronous connection. Rinse and repeat.
I think that using the synchronous NSURLConnection API is a fine idea. You have a few other options. One would be to write a wrapper object around NSURLConnection that used the asynchronous NSURLConnection APIs, so you get the nice information that the asynchronous API callbacks provide, including download progress, you can easily continue to update your UI while the request is happening, but which presents its own synchronous method for doing whatever it is you need to do. Essentially, something like:
#implementation MyURLConnectionWrapper
- (BOOL)sendRequestWithError:(NSError **)error
{
error = error ? error : &(NSError *){ nil };
self.finishedLoading = NO;
self.connectionError = nil;
self.urlConnection = [][NSURLConnection alloc] init...]
while (!self.finishedLoading)
{
[[NSRunLoop currentRunLoop] runMode: NSDefaultRunLoopMode beforeDate: [NSDate distantFuture]];
}
if (self.connectionError != nil)
{
*error = self.connectionError;
return NO;
}
return YES;
}
#end
(This is all typed off the top of my head, and is heavily abbreviated, but should give you the basic idea.)
You could also do something like fire off each request in the completion delegate method for the previous one, forgoing the use of a serial dispatch queue altogether:
- (void)connectionDidFinishLoading:(NSURLConnection *)connection;
{
[self sendNextRequest];
}
Either way, you need to think about how to handle connection errors appropriately. I've used both approaches in different places with good success.