What are the best practices for making a serial queue of NSURLSessionTasks ?
In my case, I need to:
Fetch a URL inside a JSON file (NSURLSessionDataTask)
Download the file at that URL (NSURLSessionDownloadTask)
Here’s what I have so far:
session = [NSURLSession sharedSession];
//Download the JSON:
NSURLRequest *dataRequest = [NSURLRequest requestWithURL:url];
NSURLSessionDataTask *task =
[session dataTaskWithRequest:dataRequest
completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
//Figure out the URL of the file I want to download:
NSJSONSerialization *json = [NSJSONSerialization JSONObjectWithData:data options:kNilOptions error:nil];
NSURL *downloadURL = [NSURL urlWithString:[json objectForKey:#"download_url"]];
NSURLSessionDownloadTask *fileDownloadTask =
[session downloadTaskWithRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:playlistURL]]
completionHandler:^(NSURL *location, NSURLResponse *response, NSError *error) {
NSLog(#"completed!");
}];
[fileDownloadTask resume];
}
];
Apart from the fact that writing a completion block within another completion looks messy, I am getting an EXC_BAD_ACCESS error when I call [fileDownloadTask resume]... Even though fileDownloadTask is not nil!
So, what is the best of way of sequencing NSURLSessionTasks?
You need to use this approach which is the most straight forward: https://stackoverflow.com/a/31386206/2308258
Or use an operation queue and make the tasks dependent on each others
=======================================================================
Regarding the HTTPMaximumConnectionsPerHost method
An easy way to implement a first-in first-out serial queue of NSURLSessionTasks is to run all tasks on a NSURLSession that has its HTTPMaximumConnectionsPerHost property set to 1
HTTPMaximumConnectionsPerHost only ensure that one shared connection will be used for the tasks of that session but it does not mean that they will be processed serially.
You can verify that on the network level using http://www.charlesproxy.com/, you wil discover that when setting HTTPMaximumConnectionsPerHost, your tasks will be still be started together at the same time by NSURLSession and not serially as believed.
Expriment 1:
Declaring a NSURLSession with HTTPMaximumConnectionsPerHost to 1
With task1: url = download.thinkbroadband.com/20MB.zip
With task2: url = download.thinkbroadband.com/20MB.zip
calling [task1 resume];
calling [task2 resume];
Result: task1 completionBlock is called then task2 completionBlock is called
The completion blocks might be called in the order you expected in case the tasks take the same amount of time however if you try to download two different thing using the same NSURLSession you will discover that NSURLSession does not have any underlying ordering of your calls but only completes whatever finishes first.
Expriment 2:
Declaring a NSURLSession with HTTPMaximumConnectionsPerHost to 1
task1: url = download.thinkbroadband.com/20MB.zip
task2: url = download.thinkbroadband.com/10MB.zip (smaller file)
calling [task1 resume];
calling [task2 resume];
Result: task2 completionBlock is called then task1 completionBlock is called
In conclusion you need to do the ordering yourself, NSURLSession does not have any logic about ordering requests it will just call the completionBlock of whatever finishes first even when setting the maximum number of connections per host to 1
PS: Sorry for the format of the post I do not have enough reputation to post screenshots.
Edit:
As mataejoon has pointed out, setting HTTPMaximumConnectionsPerHost to 1 will not guarantee that the connections are processed serially. Try a different approach (as in my original answer bellow) if you need a reliable serial queue of NSURLSessionTask.
An easy way to implement a first-in first-out serial queue of NSURLSessionTasks is to run all tasks on a NSURLSession that has its HTTPMaximumConnectionsPerHost property set to 1:
+ (NSURLSession *)session
{
static NSURLSession *session = nil;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration defaultSessionConfiguration];
[configuration setHTTPMaximumConnectionsPerHost:1];
session = [NSURLSession sessionWithConfiguration:configuration];
});
return session;
}
then add tasks to it in the order you want.
NSURLSessionDataTask *sizeTask =
[[[self class] session] dataTaskWithURL:url
completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
#import "SessionTaskQueue.h"
#interface SessionTaskQueue ()
#property (nonatomic, strong) NSMutableArray * sessionTasks;
#property (nonatomic, strong) NSURLSessionTask * currentTask;
#end
#implementation SessionTaskQueue
- (instancetype)init {
self = [super init];
if (self) {
self.sessionTasks = [[NSMutableArray alloc] initWithCapacity:15];
}
return self;
}
- (void)addSessionTask:(NSURLSessionTask *)sessionTask {
[self.sessionTasks addObject:sessionTask];
[self resume];
}
// call in the completion block of the sessionTask
- (void)sessionTaskFinished:(NSURLSessionTask *)sessionTask {
self.currentTask = nil;
[self resume];
}
- (void)resume {
if (self.currentTask) {
return;
}
self.currentTask = [self.sessionTasks firstObject];
if (self.currentTask) {
[self.sessionTasks removeObjectAtIndex:0];
[self.currentTask resume];
}
}
#end
and use like this
__block __weak NSURLSessionTask * wsessionTask;
use_wself();
wsessionTask = [[CommonServices shared] doSomeStuffWithCompletion:^(NSError * _Nullable error) {
use_sself();
[self.sessionTaskQueue sessionTaskFinished:wsessionTask];
...
}];
[self.sessionTaskQueue addSessionTask:wsessionTask];
I use NSOperationQueue (as Owen has suggested). Put the NSURLSessionTasks in NSOperation subclasses and set any dependancies. Dependent tasks will wait until the task they are dependent on is completed before running but will not check the status (success or failure) so add some logic to control the process.
In my case, the first task checks if the user has a valid account and creates one if necessary. In the first task I update a NSUserDefault value to indicate the account is valid (or there is an error). The second task checks the NSUserDefault value and if all OK uses the user credentials to post some data to the server.
(Sticking the NSURLSessionTasks in separate NSOperation subclasses also made my code easier to navigate)
Add the NSOperation subclasses to the NSOperationQueue and set any dependencies:
NSOperationQueue *ftfQueue = [NSOperationQueue new];
FTFCreateAccount *createFTFAccount = [[FTFCreateAccount alloc]init];
[createFTFAccount setUserid:#"********"]; // Userid to be checked / created
[ftfQueue addOperation:createFTFAccount];
FTFPostRoute *postFTFRoute = [[FTFPostRoute alloc]init];
[postFTFRoute addDependency:createFTFAccount];
[ftfQueue addOperation:postFTFRoute];
In the first NSOperation subclass checks if account exists on server:
#implementation FTFCreateAccount
{
NSString *_accountCreationStatus;
}
- (void)main {
NSDate *startDate = [[NSDate alloc] init];
float timeElapsed;
NSString *ftfAccountStatusKey = #"ftfAccountStatus";
NSString *ftfAccountStatus = (NSString *)[[NSUserDefaults standardUserDefaults] objectForKey:ftfAccountStatusKey];
NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];
[userDefaults setValue:#"CHECKING" forKey:ftfAccountStatusKey];
// Setup and Run the NSURLSessionTask
[self createFTFAccount:[self userid]];
// Hold it here until the SessionTask completion handler updates the _accountCreationStatus
// Or the process takes too long (possible connection error)
while ((!_accountCreationStatus) && (timeElapsed < 5.0)) {
NSDate *currentDate = [[NSDate alloc] init];
timeElapsed = [currentDate timeIntervalSinceDate:startDate];
}
if ([_accountCreationStatus isEqualToString:#"CONNECTION PROBLEM"] || !_accountCreationStatus) [self cancel];
if ([self isCancelled]) {
NSLog(#"DEBUG FTFCreateAccount Cancelled" );
NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];
[userDefaults setValue:#"ERROR" forKey:ftfAccountStatusKey];
}
}
In the next NSOperation post data:
#implementation FTFPostRoute
{
NSString *_routePostStatus;
}
- (void)main {
NSDate *startDate = [[NSDate alloc] init];
float timeElapsed;
NSString *ftfAccountStatusKey = #"ftfAccountStatus";
NSString *ftfAccountStatus = (NSString *)[[NSUserDefaults standardUserDefaults] objectForKey:ftfAccountStatusKey];
if ([ftfAccountStatus isEqualToString:#"ERROR"])
{
// There was a ERROR in creating / accessing the user account. Cancel the post
[self cancel];
} else
{
// Call method to setup and run json post
// Hold it here until a reply comes back from the operation
while ((!_routePostStatus) && (timeElapsed < 3)) {
NSDate *currentDate = [[NSDate alloc] init];
timeElapsed = [currentDate timeIntervalSinceDate:startDate];
NSLog(#"FTFPostRoute time elapsed: %f", timeElapsed);
}
}
if ([self isCancelled]) {
NSLog(#"FTFPostRoute operation cancelled");
}
}
Related
As
NSURLConnection sendSynchronousRequest:returningResponse:error:&connectionError
is set deprecated I will have to replace an Importer I wrote a long time ago.
The Importer does the following:
It fetches data from API-A. The data there can be on multiple pages.
It uses the data from the first fetch (also multipage) to query data from API-B and merges
Results from the API-B query will be merged with data from API-A
I implemented this with a background-operation where I use methods for each API which called recursively if there are mulitple pages for the request.
But as NSURLSession does not support synchronous request I currently only see the option to have lot of overhead (e.g. iVars) control what's called in the completion block (e.g. next Page or start to query API-B).
So, what would be an elegant solution to bring this to NSURLSession.
NB: Just to make sure, my previous solution does not block the main thread at all. But back then it was the easiest way to control the merge of two sources.
This answer is not supposed to be best practice. It was just practical for me.
Faced with the situation when a bunch of synchronous requests is executed in background and the order of execution matters I've ended up using the following:
SyncRequestSender.h
#import <Foundation/Foundation.h>
#interface SyncRequestSender : NSObject
+ (NSData *)sendSynchronousRequest:(NSURLRequest *)request
returningResponse:(NSURLResponse **)response
error:(NSError **)error;
#end
SyncRequestSender.m
#import "SyncRequestSender.h"
#implementation SyncRequestSender
+ (NSData *)sendSynchronousRequest:(NSURLRequest *)request
returningResponse:(NSURLResponse **)response
error:(NSError **)error
{
dispatch_group_t group = dispatch_group_create();
dispatch_group_enter(group);
NSError __block *err = NULL;
NSData __block *data;
NSURLResponse __block *resp;
[[[NSURLSession sharedSession] dataTaskWithRequest:request
completionHandler:^(NSData* _data, NSURLResponse* _response, NSError* _error) {
resp = _response;
err = _error;
data = _data;
dispatch_group_leave(group);
}] resume];
dispatch_group_wait(group, DISPATCH_TIME_FOREVER);
if (response)
{
*response = resp;
}
if (error)
{
*error = err;
}
return data;
}
#end
This's an example in AFNetworking, it shows how to wait an asynchronous task.
- (NSArray *)tasksForKeyPath:(NSString *)keyPath {
__block NSArray *tasks = nil;
dispatch_semaphore_t semaphore = dispatch_semaphore_create(0);
[self.session getTasksWithCompletionHandler:^(NSArray *dataTasks, NSArray *uploadTasks, NSArray *downloadTasks) {
if ([keyPath isEqualToString:NSStringFromSelector(#selector(dataTasks))]) {
tasks = dataTasks;
} else if ([keyPath isEqualToString:NSStringFromSelector(#selector(uploadTasks))]) {
tasks = uploadTasks;
} else if ([keyPath isEqualToString:NSStringFromSelector(#selector(downloadTasks))]) {
tasks = downloadTasks;
} else if ([keyPath isEqualToString:NSStringFromSelector(#selector(tasks))]) {
tasks = [#[dataTasks, uploadTasks, downloadTasks] valueForKeyPath:#"#unionOfArrays.self"];
}
dispatch_semaphore_signal(semaphore);
}];
dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);
return tasks;
}
I'm a bit confuse of how to take advantage of the new iOS 7 NSURLSession background transfers features and AFNetworking (versions 2 and 3).
I saw the WWDC 705 - What’s New in Foundation Networking session, and they demonstrated background download that continues after the app terminated or even crashes.
This is done using the new API application:handleEventsForBackgroundURLSession:completionHandler: and the fact that the session's delegate will eventually get the callbacks and can complete its task.
So I'm wondering how to use it with AFNetworking (if possible) to continue downloading in background.
The problem is, AFNetworking conveniently uses block based API to do all the requests, but if the app terminated or crashes those block are also gone. So how can I complete the task?
Or maybe I'm missing something here...
Let me explain what I mean:
For example my app is a photo messaging app, lets say that I have a PhotoMessage object that represent one message and this object has properties like
state - describe the state of the photo download.
resourcePath - the path to the final downloaded photo file.
So when I get a new message from the server, I create a new PhotoMessage object, and start downloading its photo resource.
PhotoMessage *newPhotoMsg = [[PhotoMessage alloc] initWithInfoFromServer:info];
newPhotoMsg.state = kStateDownloading;
self.photoDownloadTask = [[BGSessionManager sharedManager] downloadTaskWithRequest:request progress:nil destination:^NSURL *(NSURL *targetPath, NSURLResponse *response) {
NSURL *filePath = // some file url
return filePath;
} completionHandler:^(NSURLResponse *response, NSURL *filePath, NSError *error) {
if (!error) {
// update the PhotoMessage Object
newPhotoMsg.state = kStateDownloadFinished;
newPhotoMsg.resourcePath = filePath;
}
}];
[self.photoDownloadTask resume];
As you can see, I use the completion block to update that PhotoMessage object according to the response I get.
How can I accomplish that with a background transfer? This completion block won't be called and as a result, I can't update the newPhotoMsg.
A couple of thoughts:
You have to make sure you do the necessary coding outlined in the Handling iOS Background Activity section of the URL Loading System Programming Guide says:
If you are using NSURLSession in iOS, your app is automatically relaunched when a download completes. Your app’s application:handleEventsForBackgroundURLSession:completionHandler: app delegate method is responsible for recreating the appropriate session, storing a completion handler, and calling that handler when the session calls your session delegate’s URLSessionDidFinishEventsForBackgroundURLSession: method.
That guide shows some examples of what you can do. Frankly, I think the code samples discussed in the latter part of the WWDC 2013 video What’s New in Foundation Networking are even more clear.
The basic implementation of AFURLSessionManager will work in conjunction with background sessions if the app is merely suspended (you'll see your blocks called when the network tasks are done, assuming you've done the above). But as you guessed, any task-specific block parameters that are passed to the AFURLSessionManager method where you create the NSURLSessionTask for uploads and downloads are lost "if the app terminated or crashes."
For background uploads, this is an annoyance (as your task-level informational progress and completion blocks you specified when creating the task will not get called). But if you employ the session-level renditions (e.g. setTaskDidCompleteBlock and setTaskDidSendBodyDataBlock), that will get called properly (assuming you always set these blocks when you re-instantiate the session manager).
As it turns out, this issue of losing the blocks is actually more problematic for background downloads, but the solution there is very similar (do not use task-based block parameters, but rather use session-based blocks, such as setDownloadTaskDidFinishDownloadingBlock).
An alternative, you could stick with default (non-background) NSURLSession, but make sure your app requests a little time to finish the upload if the user leaves the app while the task is in progress. For example, before you create your NSURLSessionTask, you can create a UIBackgroundTaskIdentifier:
UIBackgroundTaskIdentifier __block taskId = [[UIApplication sharedApplication] beginBackgroundTaskWithExpirationHandler:^(void) {
// handle timeout gracefully if you can
[[UIApplication sharedApplication] endBackgroundTask:taskId];
taskId = UIBackgroundTaskInvalid;
}];
But make sure that the completion block of the network task correctly informs iOS that it is complete:
if (taskId != UIBackgroundTaskInvalid) {
[[UIApplication sharedApplication] endBackgroundTask:taskId];
taskId = UIBackgroundTaskInvalid;
}
This is not as powerful as a background NSURLSession (e.g., you have a limited amount of time available), but in some cases this can be useful.
Update:
I thought I'd add a practical example of how to do background downloads using AFNetworking.
First define your background manager.
//
// BackgroundSessionManager.h
//
// Created by Robert Ryan on 10/11/14.
// Copyright (c) 2014 Robert Ryan. All rights reserved.
//
#import "AFHTTPSessionManager.h"
#interface BackgroundSessionManager : AFHTTPSessionManager
+ (instancetype)sharedManager;
#property (nonatomic, copy) void (^savedCompletionHandler)(void);
#end
and
//
// BackgroundSessionManager.m
//
// Created by Robert Ryan on 10/11/14.
// Copyright (c) 2014 Robert Ryan. All rights reserved.
//
#import "BackgroundSessionManager.h"
static NSString * const kBackgroundSessionIdentifier = #"com.domain.backgroundsession";
#implementation BackgroundSessionManager
+ (instancetype)sharedManager {
static id sharedMyManager = nil;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
sharedMyManager = [[self alloc] init];
});
return sharedMyManager;
}
- (instancetype)init {
NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:kBackgroundSessionIdentifier];
self = [super initWithSessionConfiguration:configuration];
if (self) {
[self configureDownloadFinished]; // when download done, save file
[self configureBackgroundSessionFinished]; // when entire background session done, call completion handler
[self configureAuthentication]; // my server uses authentication, so let's handle that; if you don't use authentication challenges, you can remove this
}
return self;
}
- (void)configureDownloadFinished {
// just save the downloaded file to documents folder using filename from URL
[self setDownloadTaskDidFinishDownloadingBlock:^NSURL *(NSURLSession *session, NSURLSessionDownloadTask *downloadTask, NSURL *location) {
if ([downloadTask.response isKindOfClass:[NSHTTPURLResponse class]]) {
NSInteger statusCode = [(NSHTTPURLResponse *)downloadTask.response statusCode];
if (statusCode != 200) {
// handle error here, e.g.
NSLog(#"%# failed (statusCode = %ld)", [downloadTask.originalRequest.URL lastPathComponent], statusCode);
return nil;
}
}
NSString *filename = [downloadTask.originalRequest.URL lastPathComponent];
NSString *documentsPath = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES)[0];
NSString *path = [documentsPath stringByAppendingPathComponent:filename];
return [NSURL fileURLWithPath:path];
}];
[self setTaskDidCompleteBlock:^(NSURLSession *session, NSURLSessionTask *task, NSError *error) {
if (error) {
// handle error here, e.g.,
NSLog(#"%#: %#", [task.originalRequest.URL lastPathComponent], error);
}
}];
}
- (void)configureBackgroundSessionFinished {
typeof(self) __weak weakSelf = self;
[self setDidFinishEventsForBackgroundURLSessionBlock:^(NSURLSession *session) {
if (weakSelf.savedCompletionHandler) {
weakSelf.savedCompletionHandler();
weakSelf.savedCompletionHandler = nil;
}
}];
}
- (void)configureAuthentication {
NSURLCredential *myCredential = [NSURLCredential credentialWithUser:#"userid" password:#"password" persistence:NSURLCredentialPersistenceForSession];
[self setTaskDidReceiveAuthenticationChallengeBlock:^NSURLSessionAuthChallengeDisposition(NSURLSession *session, NSURLSessionTask *task, NSURLAuthenticationChallenge *challenge, NSURLCredential *__autoreleasing *credential) {
if (challenge.previousFailureCount == 0) {
*credential = myCredential;
return NSURLSessionAuthChallengeUseCredential;
} else {
return NSURLSessionAuthChallengePerformDefaultHandling;
}
}];
}
#end
Make sure app delegate saves completion handler (instantiating the background session as necessary):
- (void)application:(UIApplication *)application handleEventsForBackgroundURLSession:(NSString *)identifier completionHandler:(void (^)())completionHandler {
NSAssert([[BackgroundSessionManager sharedManager].session.configuration.identifier isEqualToString:identifier], #"Identifiers didn't match");
[BackgroundSessionManager sharedManager].savedCompletionHandler = completionHandler;
}
Then start your downloads:
for (NSString *filename in filenames) {
NSURL *url = [baseURL URLByAppendingPathComponent:filename];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
[[[BackgroundSessionManager sharedManager] downloadTaskWithRequest:request progress:nil destination:nil completionHandler:nil] resume];
}
Note, I don't supply any of those task related blocks, because those aren't reliable with background sessions. (Background downloads proceed even after the app is terminated and these blocks have long disappeared.) One must rely upon the session-level, easily recreated setDownloadTaskDidFinishDownloadingBlock only.
Clearly this is a simple example (only one background session object; just saving files to the docs folder using last component of URL as the filename; etc.), but hopefully it illustrates the pattern.
It shouldn't make any difference whether or not the callbacks are blocks or not. When you instantiate an AFURLSessionManager, make sure to instantiate it with NSURLSessionConfiguration backgroundSessionConfiguration:. Also, make sure to call the manager's setDidFinishEventsForBackgroundURLSessionBlock with your callback block - this is where you should write the code typically defined in NSURLSessionDelegate's method:
URLSessionDidFinishEventsForBackgroundURLSession:(NSURLSession *)session. This code should invoke your app delegate's background download completion handler.
One word of advice regarding background download tasks - even when running in the foreground, their timeouts are ignored, meaning you could get "stuck" on a download that's not responding. This is not documented anywhere and drove me crazy for some time. The first suspect was AFNetworking but even after calling NSURLSession directly, the behaviour remained the same.
Good luck!
AFURLSessionManager
AFURLSessionManager creates and manages an NSURLSession object based on a specified NSURLSessionConfiguration object, which conforms to <NSURLSessionTaskDelegate>, <NSURLSessionDataDelegate>, <NSURLSessionDownloadDelegate>, and <NSURLSessionDelegate>.
link to documentation here documentation
I have an app that currently uses NSURLConnection for the vast majority of its networking. I would like to move to NSURLSession because Apple tells me that is the way to go.
My app just uses the synchronous version of NSURLConnection by way of the + (NSData *)sendSynchronousRequest:(NSURLRequest *)request returningResponse:(NSURLResponse **)response error:(NSError **)error class method. I do this within a NSBlockOperation running on an NSOperationQueue so I am not needlessly blocking the main queue. The big advantage to doing things this way is that I can make the operations dependent on one another. For example, I can have the task that is requesting data be dependent on the login task finishing.
I have not seen any support for synchronous operations within NSURLSession. All I can find are articles deriding me for even thinking of using it synchronously and that I am a horrible person for blocking the threads. Fine. But I see no way to make NSURLSessionTasks dependent on each other. Is there a way to do that?
Or is there a description of how I would do such a thing in a different way?
The harshest criticisms of synchronous network requests are reserved for those who do it from the main queue (as we know that one should never block the main queue). But you're doing it on your own background queue, which addresses the most egregious problem with synchronous requests. But you're losing some wonderful features that asynchronous techniques provide (e.g. cancelation of requests, if needed).
I'll answer your question (how to make NSURLSessionDataTask behave synchronously) below, but I'd really encourage you to embrace the asynchronous patterns rather than fighting them. I'd suggest refactoring your code to use asynchronous patterns. Specifically, if one task is dependent upon another, simply put the initiation of the dependent task in the completion handler of the prior task.
If you have problems in that conversion, then post another Stack Overflow question, showing us what you tried, and we can try to help you out.
If you want to make an asynchronous operation synchronous, a common pattern is to use a dispatch semaphore so your thread that initiated the asynchronous process can wait for a signal from the completion block of the asynchronous operation before continuing. Never do this from the main queue, but if you're doing this from some background queue, it can be a useful pattern.
You can create a semaphore with:
dispatch_semaphore_t semaphore = dispatch_semaphore_create(0);
You can then have the completion block of the asynchronous process signal the semaphore with:
dispatch_semaphore_signal(semaphore);
And you can then have the code outside of the completion block (but still on the background queue, not the main queue) wait for that signal:
dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);
So, with NSURLSessionDataTask, putting that all together, that might look like:
[queue addOperationWithBlock:^{
dispatch_semaphore_t semaphore = dispatch_semaphore_create(0);
NSURLSession *session = [NSURLSession sharedSession]; // or create your own session with your own NSURLSessionConfiguration
NSURLSessionTask *task = [session dataTaskWithURL:url completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
if (data) {
// do whatever you want with the data here
} else {
NSLog(#"error = %#", error);
}
dispatch_semaphore_signal(semaphore);
}];
[task resume];
// but have the thread wait until the task is done
dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);
// now carry on with other stuff contingent upon what you did above
]);
With NSURLConnection (now deprecated), you have to jump through some hoops to initiate requests from a background queue, but NSURLSession handles it gracefully.
Having said that, using block operations like this means that the operations won't respond to cancellation events (while they're running, at least). So I generally eschew this semaphore technique with block operations and just wrap the data tasks in asynchronous NSOperation subclass. Then you enjoy the benefits of operations, but you can make them cancelable, too. It's more work, but a much better pattern.
For example:
//
// DataTaskOperation.h
//
// Created by Robert Ryan on 12/12/15.
// Copyright © 2015 Robert Ryan. All rights reserved.
//
#import Foundation;
#import "AsynchronousOperation.h"
NS_ASSUME_NONNULL_BEGIN
#interface DataTaskOperation : AsynchronousOperation
/// Creates a operation that retrieves the contents of a URL based on the specified URL request object, and calls a handler upon completion.
///
/// #param request A NSURLRequest object that provides the URL, cache policy, request type, body data or body stream, and so on.
/// #param dataTaskCompletionHandler The completion handler to call when the load request is complete. This handler is executed on the delegate queue. This completion handler takes the following parameters:
///
/// #returns The new session data operation.
- (instancetype)initWithRequest:(NSURLRequest *)request dataTaskCompletionHandler:(void (^)(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error))dataTaskCompletionHandler;
/// Creates a operation that retrieves the contents of a URL based on the specified URL request object, and calls a handler upon completion.
///
/// #param url A NSURL object that provides the URL, cache policy, request type, body data or body stream, and so on.
/// #param dataTaskCompletionHandler The completion handler to call when the load request is complete. This handler is executed on the delegate queue. This completion handler takes the following parameters:
///
/// #returns The new session data operation.
- (instancetype)initWithURL:(NSURL *)url dataTaskCompletionHandler:(void (^)(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error))dataTaskCompletionHandler;
#end
NS_ASSUME_NONNULL_END
and
//
// DataTaskOperation.m
//
// Created by Robert Ryan on 12/12/15.
// Copyright © 2015 Robert Ryan. All rights reserved.
//
#import "DataTaskOperation.h"
#interface DataTaskOperation ()
#property (nonatomic, strong) NSURLRequest *request;
#property (nonatomic, weak) NSURLSessionTask *task;
#property (nonatomic, copy) void (^dataTaskCompletionHandler)(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error);
#end
#implementation DataTaskOperation
- (instancetype)initWithRequest:(NSURLRequest *)request dataTaskCompletionHandler:(void (^)(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error))dataTaskCompletionHandler {
self = [super init];
if (self) {
self.request = request;
self.dataTaskCompletionHandler = dataTaskCompletionHandler;
}
return self;
}
- (instancetype)initWithURL:(NSURL *)url dataTaskCompletionHandler:(void (^)(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error))dataTaskCompletionHandler {
NSURLRequest *request = [NSURLRequest requestWithURL:url];
return [self initWithRequest:request dataTaskCompletionHandler:dataTaskCompletionHandler];
}
- (void)main {
NSURLSessionTask *task = [[NSURLSession sharedSession] dataTaskWithRequest:self.request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
self.dataTaskCompletionHandler(data, response, error);
[self completeOperation];
}];
[task resume];
self.task = task;
}
- (void)completeOperation {
self.dataTaskCompletionHandler = nil;
[super completeOperation];
}
- (void)cancel {
[self.task cancel];
[super cancel];
}
#end
Where:
//
// AsynchronousOperation.h
//
#import Foundation;
#interface AsynchronousOperation : NSOperation
/// Complete the asynchronous operation.
///
/// This also triggers the necessary KVO to support asynchronous operations.
- (void)completeOperation;
#end
And
//
// AsynchronousOperation.m
//
#import "AsynchronousOperation.h"
#interface AsynchronousOperation ()
#property (nonatomic, getter = isFinished, readwrite) BOOL finished;
#property (nonatomic, getter = isExecuting, readwrite) BOOL executing;
#end
#implementation AsynchronousOperation
#synthesize finished = _finished;
#synthesize executing = _executing;
- (instancetype)init {
self = [super init];
if (self) {
_finished = NO;
_executing = NO;
}
return self;
}
- (void)start {
if ([self isCancelled]) {
self.finished = YES;
return;
}
self.executing = YES;
[self main];
}
- (void)completeOperation {
self.executing = NO;
self.finished = YES;
}
#pragma mark - NSOperation methods
- (BOOL)isAsynchronous {
return YES;
}
- (BOOL)isExecuting {
#synchronized(self) {
return _executing;
}
}
- (BOOL)isFinished {
#synchronized(self) {
return _finished;
}
}
- (void)setExecuting:(BOOL)executing {
#synchronized(self) {
if (_executing != executing) {
[self willChangeValueForKey:#"isExecuting"];
_executing = executing;
[self didChangeValueForKey:#"isExecuting"];
}
}
}
- (void)setFinished:(BOOL)finished {
#synchronized(self) {
if (_finished != finished) {
[self willChangeValueForKey:#"isFinished"];
_finished = finished;
[self didChangeValueForKey:#"isFinished"];
}
}
}
#end
#Rob I would encourage you to post your reply as a solution, in view of the following documentation note from NSURLSession.dataTaskWithURL(_:completionHandler:):
This method is intended as an alternative to the
sendAsynchronousRequest:queue:completionHandler: method of
NSURLConnection, with the added ability to support custom
authentication and cancellation.
If semaphore based approach doesn't work, try polling based approach.
var reply = Data()
/// We need to make a session object.
/// This is key to make this work. This won't work with shared session.
let conf = URLSessionConfiguration.ephemeral
let sess = URLSession(configuration: conf)
let task = sess.dataTask(with: u) { data, _, _ in
reply = data ?? Data()
}
task.resume()
while task.state != .completed {
Thread.sleep(forTimeInterval: 0.1)
}
FileHandle.standardOutput.write(reply)
Polling based approach works very reliably, but effectively limits maximum throughput to polling interval. In this example, it's been limited to 10 times/sec.
I made a Swift package for this.
Semaphore based approach has been worked well so far, but since Xcode 11 era, it's getting broken. (maybe only for me?)
A data task does not finish if I wait for semaphores. If I wait for semaphore on different thread, it task fails with an error.
nw_connection_copy_protocol_metadata [C2] Client called nw_connection_copy_protocol_metadata on unconnected nw_connection error.
It seems something has been changed in the implementation as Apple is moving Network.framework.
Edit to Question: 21/10/2013 20:10 GMT
Added how the method is called below and also the object that is leaking is "allDBObjects" If i remove this and change the dictionary below to "NSMutableDictionary *objectsById" there is no leak.
Edit to Question: 21/10/2013
After a few days not on this issue I have come back to it. I believe this is all down to "The Deallocation Problem" (https://developer.apple.com/library/ios/technotes/tn2109/_index.html#//apple_ref/doc/uid/DTS40010274-CH1-SUBSECTION11). I have tested with both MKNetworkKit and AFNetworking 1.3.3 (Changing 1 method to use AFNetworking instead of MKNetwork Kit) and am still getting these objects leaking in my completion block. I have no references to self within my block and using AFNetworking I can see the completionBlock is set to nil and I have tried to manually break the retain cycle by setting the [weakOp setCompletionBlock:nil].
EDIT: The code sample below I tried to use properties and reference them as weakSelf. I have now changed these to local variables and they still leak.
Any ideas?
Original Question
I have taken over a project using MKNetworkKit and Core Data, after running the project through Leaks in instruments I can see a lot of leaked objects in various places in the app.
After debugging the code I can see the objects that are leaking are 2 fetch requests that are happening in the callback of a MKNetworkKit request (setCompletionBlock:). The fetch requests need to be done to check whether the data needs to be inserted or updated.
Some further information. Inside the completion block I am getting an instance of the ManagedObjectContext and creating it with concurrency type of "NSPrivateQueueConcurrencyType" and to perform the insert I am correctly calling "performBlock:" on the moc.
Please advise.
James
Sample Code of Block:
Please note: I have commented out the 2 fetch requests are there are no leaks and putting them back in causes the leaks of hundreds of objects, also the weakSelf properties I am setting the NSDictionary and NSArray are (nonatomic, strong).
- (void) updateDbObjects: (int) page withCallback: (CompletionResultsNumberBlock) callback {
#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
dispatch_queue_t callerQueue = dispatch_get_current_queue();
#pragma GCC diagnostic warning "-Wdeprecated-declarations"
__weak typeof(self) weakSelf = self;
NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:[self createFullPath:urlStr]]];
AFJSONRequestOperation *operation = [AFJSONRequestOperation JSONRequestOperationWithRequest:request success:^(NSURLRequest *request, NSHTTPURLResponse *response, id JSON) {
NSManagedObjectContext *moc = [weakSelf managedObjectContextForCurrentThread];
DataRoot *dataRoot = [DataRoot sharedInstanceInMoc:moc];
NSArray *returnJSON = JSON[#"object"];
__block int count = returnJSON.count;
if (!count)
{
dispatch_async(callerQueue, ^{
callback(0);
});
return;
}
NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:#"DBObjects"];
NSError *error;
NSArray *allDBObjects = [moc executeFetchRequest:fetchRequest error:&error];
NSMutableDictionary *objectsById = [NSMutableDictionary dictionaryWithObjects:allTeamsArray forKeys:[allTeamsArray valueForKey: GoalTeamObjectAttributes.teamId]];
for (NSDictionary *rootDict in returnJSON)
{
GoalTeamObject *dbObject = objectsById[rootDict[#"id"]];
if (dbObject == nil)
{
dbObject = [DBObjects insertInManagedObjectContext:dataRoot.managedObjectContext];
}
[weakSelf importStandardParametersFrom:rootDict into:dbObject withPrefix:#""];
}
returnJSON = nil;
objectsById = nil;
[dataRoot saveContext];
NSError *childError = nil;
if ([moc save:&childError]) {
NSError *parentError = nil;
if (![moc.parentContext save:&parentError]) {
NSLog(#"Error saving parent");
}
dispatch_async(callerQueue, ^{
callback(count);
});
} else {
NSLog(#"Error saving child");
}
} failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error, id JSON) {
}];
[operation start];
}
This is how this code is called: It is called recursively in a loop as there are many pages of data.
__block int page = 1;
__weak typeof(self) weakSelf = self;
CompletionResultsNumberBlock loadData;
__block CompletionResultsNumberBlock block_loadData = loadData = ^(int results)
{
if (results < 100)
{
dispatch_async(callerQueue, callback);
} else {
[weakSelf updateDbObjects:++page withCallback:block_loadData];
}
};
[self updateDbObjects:page withCallback: loadData];
This doesn't look right:
__block CompletionResultsNumberBlock block_loadData = loadData = ^...
Under ARC, the block would hold a strong reference to itself. Under ARC, you should do:
__block __weak CompletionResultsNumberBlock block_loadData = loadData = ^...
You use weakSelf in some places within the block, and self in others. That means that the block will still capture self in a strong fashion.
I'd try replacing all references to self with weakSelf within your block first.
I am downloading four plist files asynchronously over the internet. I need to wait until all four files are downloaded, until I either on the first run, push a UIViewController, or on all subsequent runs, refresh the data, and reload all my UITableViews.
On the first run, everything works perfectly. When refreshing though, all four url requests are called, and started, but never call their completion or failure blocks, and the UI freezes. Which is odd since I preform all operations in a background thread. I have not been able to figure out why this is happening.
The first load and the refresh methods call the four "update" methods in the same way, and use NSCondition in the same way.
For the first run:
- (void)loadContentForProgram:(NSString *)programPath
{
NSLog(#"Start Load Program");
AppDelegate *myDelegate = (AppDelegate *)[UIApplication sharedApplication].delegate;
hud = [[MBProgressHUD alloc] initWithView:myDelegate.window];
[myDelegate.window addSubview:hud];
hud.labelText = #"Loading...";
hud.detailsLabelText = #"Loading Data";
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
//Do stuff here to load data from files
//Update From online files
hud.detailsLabelText = #"Updating Live Data";
resultLock = NO;
progressLock = NO;
recallLock = NO;
stageLock = NO;
condition = [[NSCondition alloc] init];
[condition lock];
[self updateCurrentCompsText];
[self updateCompetitionResults];
[self updateCompetitionRecalls];
[self updateCompetitionProgress];
while (!resultLock) {
[condition wait];
}
NSLog(#"Unlock");
while (!stageLock) {
[condition wait];
}
NSLog(#"Unlock");
while (!recallLock) {
[condition wait];
}
NSLog(#"Unlock");
while (!progressLock) {
[condition wait];
}
NSLog(#"Unlock");
[condition unlock];
updateInProgress = NO;
//Reset Refresh controls and table views
self.refreshControlsArray = [[NSMutableArray alloc] init];
self.tableViewsArray = [[NSMutableArray alloc] init];
NSLog(#"Finished Loading Program");
[[NSNotificationCenter defaultCenter] postNotificationName:#"WMSOFinishedLoadingProgramData" object:nil]; //Pushes view controller
dispatch_async(dispatch_get_main_queue(), ^{
[MBProgressHUD hideHUDForView:myDelegate.window animated:YES];
});
});
}
When refreshing data:
- (void)updateProgramContent
{
if (!updateInProgress) {
updateInProgress = YES;
for (int i = 0; i < self.refreshControlsArray.count; i++) {
if (!((UIRefreshControl *)self.refreshControlsArray[i]).refreshing) {
[self.refreshControlsArray[i] beginRefreshing];
[self.tableViewsArray[i] setContentOffset:CGPointMake(0.0, 0.0) animated:YES];
}
}
resultLock = NO;
stageLock = NO;
recallLock = NO;
progressLock = NO;
dispatch_sync(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
condition = [[NSCondition alloc] init];
[condition lock];
[self updateCompetitionProgress];
[self updateCompetitionRecalls];
[self updateCompetitionResults];
[self updateCurrentCompsText];
while (!resultLock) {
[condition wait];
}
NSLog(#"Unlock");
while (!stageLock) {
[condition wait];
}
NSLog(#"Unlock");
while (!recallLock) {
[condition wait];
}
NSLog(#"Unlock");
while (!progressLock) {
[condition wait];
}
NSLog(#"Unlock");
[condition unlock];
});
for (int i = 0; i < self.refreshControlsArray.count; i++) {
[self.refreshControlsArray[i] performSelector:#selector(endRefreshing) withObject:nil afterDelay:1.0];
[self.tableViewsArray[i] performSelector:#selector(reloadData) withObject:nil afterDelay:1.0];
}
updateInProgress = NO;
}
}
The block below that appears in each loading method above, corresponds to a method that will download and update a specific piece of data.
[self updateCompetitionProgress];
[self updateCompetitionRecalls];
[self updateCompetitionResults];
[self updateCurrentCompsText];
which runs:
- (void)updateCompetitionResults
{
__block NSDictionary *competitionResultsData = nil;
NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:[NSString stringWithFormat:#"Some URL",[self.programName stringByReplacingOccurrencesOfString:#" " withString:#"%20"]]] cachePolicy:NSURLCacheStorageNotAllowed timeoutInterval:20.0];
AFPropertyListRequestOperation *operation = [AFPropertyListRequestOperation propertyListRequestOperationWithRequest:request success:^(NSURLRequest *request, NSHTTPURLResponse *response, id propertyList) {
competitionResultsData = (NSDictionary *)propertyList;
[competitionResultsData writeToFile:[#"SOME LOCAL PATH"] atomically:NO];
[self updateCompetitionResultsWithDictionary:competitionResultsData];
} failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error, id propertyList) {
competitionResultsData = [NSDictionary dictionaryWithContentsOfFile:[#"SOME LOCAL PATH"]];
NSLog(#"Failed to retreive competition results: %#", error);
[self updateCompetitionResultsWithDictionary:competitionResultsData];
}];
[operation start];
}
and the completion and failure blocks call the same method to update the data
- (void)updateCompetitionResultsWithDictionary:(NSDictionary *)competitionResultsData
{
//Do Stuff with the data here
resultLock = YES;
[condition signal];
}
So, Why does this work on the first run, but not any of the subsequent runs?
As I mentioned in my comments, above, the most obvious problem is that you're invoking methods that use condition before you initialize condition. Make sure initialize condition before you start calling updateCompetitionResults, etc.
In terms of a more radical change, I might suggest retiring NSCondition altogether, and use operation queues:
I might use NSOperationQueue (or you can use dispatch groups, too, if you want, but I like the operation queue's ability to configure how many concurrent operations you can operate ... also if you get to a point that you want to cancel operations, I think NSOperationQueue offers some nice features there, too). You can then define each download and processing as a separate NSOperation (each of the downloads should happen synchronously, because they're running in an operation queue, you get the benefits of asynchronous operations, but you can kick off the post-processing immediately after the download is done). You then just queue them up to run asynchronously, but define a final operation which is dependent upon the other four will kick off as soon as the four downloads are done. (By the way, I use NSBlockOperation which provides block-functionality for NSOperation objects, but you can do it any way you want.)
And whereas your updateProgramContent might download asynchronously, it processes the four downloaded files sequentially, one after another. Thus, if the first download takes a while to download, it will hold up the post-processing of the others. Instead, I like to encapsulate both the downloading and the post processing of each of the four plist files in a single NSOperation, each. Thus, we enjoy maximal concurrency of not only the downloading, but the post-processing, too.
Rather than using the AFNetworking (which I'm generally a big fan of) plist-related method, I might be inclined to use NSDictionary and NSArray features that allow you to download a plist from the web and load them into the appropriate structure. These dictionaryWithContentsOfURL and arrayWithContentsOfURL run synchronously, but because we're doing this in a background operation, everything runs asynchronously like you want. This also bypasses the saving them to files. If you wanted them saved to files in your Documents directory, you can do that easily, too. Clearly, if you're doing something sophisticated in your downloading of the plist files (e.g. your server is engaging in some challenge-response authentication), you can't use the convenient NSDictionary and NSArray methods. But if you don't need all of that, the simple NSDictionary and NSArray methods, ___WithContentsOfURL make life pretty simple.
Pulling this all together, it might look like:
#interface ViewController ()
#property (nonatomic, strong) NSArray *competitions;
#property (nonatomic, strong) NSDictionary *competitionResults;
#property (nonatomic, strong) NSDictionary *competitionRecalls;
#property (nonatomic, strong) NSDictionary *competitionProgress;
#end
#implementation ViewController
- (void)viewDidLoad
{
[super viewDidLoad];
[self transfer];
}
- (void)allTransfersComplete
{
BOOL success;
if (self.competitions == nil)
{
success = FALSE;
NSLog(#"Unable to download competitions");
}
if (self.competitionResults == nil)
{
success = FALSE;
NSLog(#"Unable to download results");
}
if (self.competitionRecalls == nil)
{
success = FALSE;
NSLog(#"Unable to download recalls");
}
if (self.competitionProgress == nil)
{
success = FALSE;
NSLog(#"Unable to download progress");
}
if (success)
{
NSLog(#"all done successfully");
}
else
{
NSLog(#"one or more failed");
}
}
- (void)transfer
{
NSURL *baseUrl = [NSURL URLWithString:#"http://insert.your.base.url.here/competitions"];
NSURL *competitionsUrl = [baseUrl URLByAppendingPathComponent:#"competitions.plist"];
NSURL *competitionResultsUrl = [baseUrl URLByAppendingPathComponent:#"competitionresults.plist"];
NSURL *competitionRecallsUrl = [baseUrl URLByAppendingPathComponent:#"competitionrecalls.plist"];
NSURL *competitionProgressUrl = [baseUrl URLByAppendingPathComponent:#"competitionprogress.plist"];
NSOperationQueue *queue = [[NSOperationQueue alloc] init];
queue.maxConcurrentOperationCount = 4; // if your server doesn't like four concurrent requests, you can ratchet this back to whatever you want
// create operation that will be called when we're all done
NSBlockOperation *completionOperation = [NSBlockOperation blockOperationWithBlock:^{
// any stuff that can be done in background should be done here
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
// any user interface stuff should be done here; I've just put this in a separate method so this method doesn't get too unwieldy
[self allTransfersComplete];
}];
}];
// a variable that we'll use as we create our four download/process operations
NSBlockOperation *operation;
// create competitions operation
operation = [NSBlockOperation blockOperationWithBlock:^{
// download the competitions and load it into the ivar
//
// note, if you *really* want to download this to a file, you can
// do that when the download is done
self.competitions = [NSArray arrayWithContentsOfURL:competitionsUrl];
// if you wanted to do any post-processing of the download
// you could do it here.
NSLog(#"competitions = %#", self.competitions);
}];
[completionOperation addDependency:operation];
// create results operation
operation = [NSBlockOperation blockOperationWithBlock:^{
self.competitionResults = [NSDictionary dictionaryWithContentsOfURL:competitionResultsUrl];
NSLog(#"competitionResults = %#", self.competitionResults);
}];
[completionOperation addDependency:operation];
// create recalls operation
operation = [NSBlockOperation blockOperationWithBlock:^{
self.competitionRecalls = [NSDictionary dictionaryWithContentsOfURL:competitionRecallsUrl];
NSLog(#"competitionRecalls = %#", self.competitionRecalls);
}];
[completionOperation addDependency:operation];
// create progress operation
operation = [NSBlockOperation blockOperationWithBlock:^{
self.competitionProgress = [NSDictionary dictionaryWithContentsOfURL:competitionProgressUrl];
NSLog(#"competitionProgress = %#", self.competitionProgress);
}];
[completionOperation addDependency:operation];
// queue the completion operation (which is dependent upon the other four)
[queue addOperation:completionOperation];
// now queue the four download and processing operations
[queue addOperations:completionOperation.dependencies waitUntilFinished:NO];
}
#end
Now, I don't know which of your plist's are arrays and which are dictionaries (in my example, I made competitions an array and the rest were dictionaries keyed by the competition id), but hopefully you get the idea of what I was shooting for. Maximize concurrency, eliminate NSCondition logic, really make the most of NSOperationQueue, etc.
This may be all to much to take in, but I only mention it as an alternative to NSCondition. If your current technique works, that's great. But the above outlines how I would tackle a challenge like this.