How to handling more than 1000 api request using NSUrlsession - ios

-(instancetype)initWithDataTaskWithUrlRequest:(NSURLRequest *)request WithIdentifierIndex:(NSString*)identifier
{
self = [super init];
if (self) {
NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:[NSString stringWithFormat:#"%#",[self uuid]]];
configuration.HTTPMaximumConnectionsPerHost = 1;
operationSession = [NSURLSession sessionWithConfiguration:[NSURLSessionConfiguration ephemeralSessionConfiguration] delegate:self delegateQueue:nil];
_sessionDataTask = [operationSession dataTaskWithRequest:request];
}
return self;
}
I have a operation queue. And In each NSOperation i have used session to downloading the content. I have ephemeralSessionConfiguration and DefaultSessionConfiguration. It is working fine for 200 0r 300 operations. But when the operation increase to 1000 to 1000+ it will give me time out, handshaking error. Please help me for best solution.

In the same NSURLSessionConfiguration you can add/Increase timeout as :
configuration.timeoutIntervalForRequest = KTimeOutValue; //Your timeout value

Related

NSURLSessionTask creation fail in extension with backgroundSessionConfiguration

Context:
I try to call a create a task (download or upload) from an action extension, with a backgroundSessionConfiguration.
To do this I fallow the exemple in apple documention
-(void)downloadTest
{
NSURLSession *mySession = [self configureMySession];
NSURL *url = [NSURL URLWithString:#"http://www.sellcell.com/blog/wp-content/uploads/2014/03/dog-apps.jpg"];
NSURLSessionTask *myTask = [mySession downloadTaskWithURL:url];
[myTask resume];
}
- (NSURLSession *) configureMySession {
if (!_mySession) {
NSURLSessionConfiguration* config = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:#"com.mycompany.myapp.backgroundsession"];
// To access the shared container you set up, use the sharedContainerIdentifier property on your configuration object.
config.sharedContainerIdentifier = #"group.com.mycompany.appname";
_mySession = [NSURLSession sessionWithConfiguration:config delegate:self delegateQueue:nil];
}
return _mySession;
}
My problem is that when I call [mySession downloadTaskWithURL:url]; it returns nil.
If I change the configuration to NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration defaultSessionConfiguration]; then a task is created.
I don't see what I'm doing wrong , I have created an app group and I use it the app and in the extension.
I use the group name that I have created in config.sharedContainerIdentifier but I'm not sure it's necessary.
NOTE: I have the same problem with uploadTask.
Are you using ARC? If not, make sure your session is being retained properly. (It looks like you're using an ivar directly.)
Is that shared container identifier correct? If the container isn't in your entitlements or doesn't exist yet, your session will be immediately invalidated. Add an invalidation delegate method and see if it is getting called. If so, that's probably the issue.

iOS NSURLSession Listen to Timeout

I created a simple NSURLSessionDownloadTask to download from a URL, with its class having the NSURLSession delegates:
#interface DownloadManager : NSObject <NSURLSessionDataDelegate, NSURLSessionDelegate, NSURLSessionDownloadDelegate, NSURLSessionTaskDelegate>
//...
NSURLSessionConfiguration *sessionConfiguration = [NSURLSessionConfiguration defaultSessionConfiguration];
[sessionConfiguration setTimeoutIntervalForRequest:30.0];
[sessionConfiguration setTimeoutIntervalForResource:60.0];
NSURLSession *session = [NSURLSession sessionWithConfiguration:sessionConfiguration];
NSURLSessionDownloadTask *downloadTask = [session downloadTaskWithURL:self.url];
[downloadTask resume];
However, I could not find a protocol method that listens to the download task timing out. Is there a way to listen to the timeout (ex. - I wanted to close a progress dialog box when 30.0 seconds have passed and no data is still received)
I've already scavenged Google but haven't found any information so far, so I'll leave this question here while I search for more info.
Thanks so much!
The timeout is one of the errors NSURLSession will give you in completionHandler block. It's NSURLErrorTimedOut = -1001.
in delegate method
- URLSession:task:didCompleteWithError:
check the NSError if it's NSURLErrorTimedOut do what you want
https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Miscellaneous/Foundation_Constants/#//apple_ref/doc/constant_group/URL_Loading_System_Error_Codes

Handling redirects with custom NSURLProtocol and HTTP proxy

I have a custom URLProtocol where I want to redirect all traffic via a proxy server.
My current working code looks like that:
+(BOOL)canInitWithRequest:(NSURLRequest*)request
{
if ([NSURLProtocol propertyForKey:protocolKey inRequest:request])
return NO;
NSString *scheme = request.URL.scheme.lowercaseString;
return [scheme isEqualToString:#"http"] || [scheme isEqualToString:#"https"];
}
-(void)startLoading
{
NSMutableURLRequest *request = self.request.mutableCopy;
[NSURLProtocol setProperty:#YES forKey:protocolKey inRequest:request];
NSURLSessionConfiguration *config = [NSURLSessionConfiguration defaultSessionConfiguration];
config.requestCachePolicy = NSURLRequestReloadIgnoringLocalCacheData;
config.connectionProxyDictionary = #
{
(id)kCFNetworkProxiesHTTPEnable:#YES,
(id)kCFNetworkProxiesHTTPProxy:#"1.2.3.4",
(id)kCFNetworkProxiesHTTPPort:#8080
};
m_session = [NSURLSession sessionWithConfiguration:config delegate:self delegateQueue:[NSOperationQueue currentQueue]];
[[m_session dataTaskWithRequest:request] resume];
}
This works great so far. The problem is that there are some url's which use redirection - and I want the redirection to be performed by the proxy server as well, rather than by the device. I've tried to add the following code, but it didn't help:
-(void)URLSession:(NSURLSession*)session task:(NSURLSessionTask*)task willPerformHTTPRedirection:(NSHTTPURLResponse*)response newRequest:(NSURLRequest*)newRequest completionHandler:(void (^)(NSURLRequest*))completionHandler
{
NSMutableURLRequest *request = newRequest.mutableCopy;
[NSURLProtocol removePropertyForKey:protocolKey inRequest:request];
[self.client URLProtocol:self wasRedirectedToRequest:request redirectResponse:response];
[task cancel];
[self.client URLProtocol:self didFailWithError:[NSError errorWithDomain:NSCocoaErrorDomain code:NSUserCancelledError userInfo:nil]];
}
The problem is that the new request is not being sent to the proxy server but instead being redirected by the device itself.
Thanks.
It turned out the problem was with the redirection for an HTTPS server, while there is no HTTPS proxy defined. To use HTTPS proxy, the code should looks like:
config.connectionProxyDictionary = #
{
#"HTTPEnable":#YES,
(id)kCFStreamPropertyHTTPProxyHost:#"1.2.3.4",
(id)kCFStreamPropertyHTTPProxyPort:#8080,
#"HTTPSEnable":#YES,
(id)kCFStreamPropertyHTTPSProxyHost:#"1.2.3.4",
(id)kCFStreamPropertyHTTPSProxyPort:#8080
};
Source: How to programmatically add a proxy to an NSURLSession

Why is NSURLSession slower than cURL when downloading many files?

I've been using cURL to download about 1700+ files -- which total to about ~290MB -- in my iOS app. It takes about 5-7 minutes on my Internet connection to download all of them using cURL. But since not everyone has fast internet connection (especially when on the go), I decided to allow the files to be downloaded in the background, so that the user can do other things while waiting for the download to finish. This is where NSURLSession comes in.
Using NSURLSession, it takes about 20+ minutes on my Internet connection to download all of them while the app is in foreground. I don't mind it being slow when the app is in background, because I understand that it is up to the OS to schedule the downloads. But it's a problem when it's slow even when it's in foreground. Is this the expected behaviour? Is it because of the quantity of the files?
In case I'm not using NSURLSession correctly, here's a snippet of how I'm using it:
// Initialization
NSURLSessionConfiguration *sessionConfiguration = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:#"<my-identifier>"];
sessionConfiguration.HTTPMaximumConnectionsPerHost = 40;
backgroundSession = [NSURLSession sessionWithConfiguration:sessionConfiguration
delegate:self
delegateQueue:nil];
// ...
// Creating the tasks and starting the download
for (int i = 0; i < 20 && queuedRequests.count > 0; i++) {
NSDictionary *requestInfo = [queuedRequests lastObject];
NSURLSessionDownloadTask *downloadTask = [backgroundSession downloadTaskWithURL:[NSURL URLWithString:requestInfo[#"url"]]];
ongoingRequests[#(downloadTask.taskIdentifier)] = requestInfo;
[downloadTask resume];
[queuedRequests removeLastObject];
NSLog(#"Begin download file %d/%d: %#", allRequests.count - queuedRequests.count, allRequests.count, requestInfo[#"url"]);
}
// ...
// Somewhere in (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask didFinishDownloadingToURL:(NSURL *)location
// After each download task is completed, grab a file to download from
// queuedRequests, and create another task
if (queuedRequests.count > 0) {
requestInfo = [queuedRequests lastObject];
NSURLSessionDownloadTask *newDownloadTask = [backgroundSession downloadTaskWithURL:[NSURL URLWithString:requestInfo[#"url"]]];
ongoingRequests[#(newDownloadTask.taskIdentifier)] = requestInfo;
[newDownloadTask resume];
[queuedRequests removeLastObject];
NSLog(#"Begin download file %d/%d: %#", allRequests.count - queuedRequests.count, allRequests.count, requestInfo[#"url"]);
}
I've also tried using multiple NSURLSession, but it's still slow. The reason I tried that is because when using cURL, I create multiple threads (around 20), and each thread will download a single file at a time.
It's also not possible for me to reduce the number of files by zipping it, because I need the app to be able to download individual files since I will update them from time to time. Basically, when the app starts, it will check if there are any files that have been updated, and only download those files. Since the files are stored in S3, and S3 doesn't have zipping service, I could not zip them into a single file on the fly.
As mentioned by Filip and Rob in the comments, the slowness is because when NSURLSession is initialized with backgroundSessionConfigurationWithIdentifier:, the download tasks will be executed in the background regardless if the app is in the foreground. So I solved this issue by having 2 instances of NSURLSession: one for foreground download, and one for background download:
NSURLSessionConfiguration *foregroundSessionConfig = [NSURLSessionConfiguration defaultSessionConfiguration];
foregroundSessionConfig.HTTPMaximumConnectionsPerHost = 40;
foregroundSession = [NSURLSession sessionWithConfiguration:foregroundSessionConfig
delegate:self
delegateQueue:nil];
[foregroundSession retain];
NSURLSessionConfiguration *backgroundSessionConfig = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:#"com.terato.darknessfallen.BackgroundDownload"];
backgroundSessionConfig.HTTPMaximumConnectionsPerHost = 40;
backgroundSession = [NSURLSession sessionWithConfiguration:backgroundSessionConfig
delegate:self
delegateQueue:nil];
[backgroundSession retain];
When the app is switched to background, I simply call cancelByProducingResumeData: on each of the download tasks that's still running, and then pass it to downloadTaskWithResumeData::
- (void)switchToBackground
{
if (state == kDownloadManagerStateForeground) {
[foregroundSession getTasksWithCompletionHandler:^(NSArray *dataTasks, NSArray *uploadTasks, NSArray *downloadTasks) {
for (NSURLSessionDownloadTask *downloadTask in downloadTasks) {
[downloadTask cancelByProducingResumeData:^(NSData *resumeData) {
NSURLSessionDownloadTask *downloadTask = [backgroundSession downloadTaskWithResumeData:resumeData];
[downloadTask resume];
}];
}
}];
state = kDownloadManagerStateBackground;
}
}
Likewise, when the app is switched to foreground, I do the same but switched foregroundSession with backgroundSession:
- (void)switchToForeground
{
if (state == kDownloadManagerStateBackground) {
[backgroundSession getTasksWithCompletionHandler:^(NSArray *dataTasks, NSArray *uploadTasks, NSArray *downloadTasks) {
for (NSURLSessionDownloadTask *downloadTask in downloadTasks) {
[downloadTask cancelByProducingResumeData:^(NSData *resumeData) {
NSURLSessionDownloadTask *downloadTask = [foregroundSession downloadTaskWithResumeData:resumeData];
[downloadTask resume];
}];
}
}];
state = kDownloadManagerStateForeground;
}
}
Also, don't forget to call beginBackgroundTaskWithExpirationHandler: before calling switchToBackground when the app is switched to background. This is to ensure that the method is allowed to complete while in background. Otherwise, it will only be called once the app enters foreground again.

Best practices for making a queue of NSURLSessionTasks

What are the best practices for making a serial queue of NSURLSessionTasks ?
In my case, I need to:
Fetch a URL inside a JSON file (NSURLSessionDataTask)
Download the file at that URL (NSURLSessionDownloadTask)
Here’s what I have so far:
session = [NSURLSession sharedSession];
//Download the JSON:
NSURLRequest *dataRequest = [NSURLRequest requestWithURL:url];
NSURLSessionDataTask *task =
[session dataTaskWithRequest:dataRequest
completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
//Figure out the URL of the file I want to download:
NSJSONSerialization *json = [NSJSONSerialization JSONObjectWithData:data options:kNilOptions error:nil];
NSURL *downloadURL = [NSURL urlWithString:[json objectForKey:#"download_url"]];
NSURLSessionDownloadTask *fileDownloadTask =
[session downloadTaskWithRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:playlistURL]]
completionHandler:^(NSURL *location, NSURLResponse *response, NSError *error) {
NSLog(#"completed!");
}];
[fileDownloadTask resume];
}
];
Apart from the fact that writing a completion block within another completion looks messy, I am getting an EXC_BAD_ACCESS error when I call [fileDownloadTask resume]... Even though fileDownloadTask is not nil!
So, what is the best of way of sequencing NSURLSessionTasks?
You need to use this approach which is the most straight forward: https://stackoverflow.com/a/31386206/2308258
Or use an operation queue and make the tasks dependent on each others
=======================================================================
Regarding the HTTPMaximumConnectionsPerHost method
An easy way to implement a first-in first-out serial queue of NSURLSessionTasks is to run all tasks on a NSURLSession that has its HTTPMaximumConnectionsPerHost property set to 1
HTTPMaximumConnectionsPerHost only ensure that one shared connection will be used for the tasks of that session but it does not mean that they will be processed serially.
You can verify that on the network level using http://www.charlesproxy.com/, you wil discover that when setting HTTPMaximumConnectionsPerHost, your tasks will be still be started together at the same time by NSURLSession and not serially as believed.
Expriment 1:
Declaring a NSURLSession with HTTPMaximumConnectionsPerHost to 1
With task1: url = download.thinkbroadband.com/20MB.zip
With task2: url = download.thinkbroadband.com/20MB.zip
calling [task1 resume];
calling [task2 resume];
Result: task1 completionBlock is called then task2 completionBlock is called
The completion blocks might be called in the order you expected in case the tasks take the same amount of time however if you try to download two different thing using the same NSURLSession you will discover that NSURLSession does not have any underlying ordering of your calls but only completes whatever finishes first.
Expriment 2:
Declaring a NSURLSession with HTTPMaximumConnectionsPerHost to 1
task1: url = download.thinkbroadband.com/20MB.zip
task2: url = download.thinkbroadband.com/10MB.zip (smaller file)
calling [task1 resume];
calling [task2 resume];
Result: task2 completionBlock is called then task1 completionBlock is called
In conclusion you need to do the ordering yourself, NSURLSession does not have any logic about ordering requests it will just call the completionBlock of whatever finishes first even when setting the maximum number of connections per host to 1
PS: Sorry for the format of the post I do not have enough reputation to post screenshots.
Edit:
As mataejoon has pointed out, setting HTTPMaximumConnectionsPerHost to 1 will not guarantee that the connections are processed serially. Try a different approach (as in my original answer bellow) if you need a reliable serial queue of NSURLSessionTask.
An easy way to implement a first-in first-out serial queue of NSURLSessionTasks is to run all tasks on a NSURLSession that has its HTTPMaximumConnectionsPerHost property set to 1:
+ (NSURLSession *)session
{
static NSURLSession *session = nil;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration defaultSessionConfiguration];
[configuration setHTTPMaximumConnectionsPerHost:1];
session = [NSURLSession sessionWithConfiguration:configuration];
});
return session;
}
then add tasks to it in the order you want.
NSURLSessionDataTask *sizeTask =
[[[self class] session] dataTaskWithURL:url
completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
#import "SessionTaskQueue.h"
#interface SessionTaskQueue ()
#property (nonatomic, strong) NSMutableArray * sessionTasks;
#property (nonatomic, strong) NSURLSessionTask * currentTask;
#end
#implementation SessionTaskQueue
- (instancetype)init {
self = [super init];
if (self) {
self.sessionTasks = [[NSMutableArray alloc] initWithCapacity:15];
}
return self;
}
- (void)addSessionTask:(NSURLSessionTask *)sessionTask {
[self.sessionTasks addObject:sessionTask];
[self resume];
}
// call in the completion block of the sessionTask
- (void)sessionTaskFinished:(NSURLSessionTask *)sessionTask {
self.currentTask = nil;
[self resume];
}
- (void)resume {
if (self.currentTask) {
return;
}
self.currentTask = [self.sessionTasks firstObject];
if (self.currentTask) {
[self.sessionTasks removeObjectAtIndex:0];
[self.currentTask resume];
}
}
#end
and use like this
__block __weak NSURLSessionTask * wsessionTask;
use_wself();
wsessionTask = [[CommonServices shared] doSomeStuffWithCompletion:^(NSError * _Nullable error) {
use_sself();
[self.sessionTaskQueue sessionTaskFinished:wsessionTask];
...
}];
[self.sessionTaskQueue addSessionTask:wsessionTask];
I use NSOperationQueue (as Owen has suggested). Put the NSURLSessionTasks in NSOperation subclasses and set any dependancies. Dependent tasks will wait until the task they are dependent on is completed before running but will not check the status (success or failure) so add some logic to control the process.
In my case, the first task checks if the user has a valid account and creates one if necessary. In the first task I update a NSUserDefault value to indicate the account is valid (or there is an error). The second task checks the NSUserDefault value and if all OK uses the user credentials to post some data to the server.
(Sticking the NSURLSessionTasks in separate NSOperation subclasses also made my code easier to navigate)
Add the NSOperation subclasses to the NSOperationQueue and set any dependencies:
NSOperationQueue *ftfQueue = [NSOperationQueue new];
FTFCreateAccount *createFTFAccount = [[FTFCreateAccount alloc]init];
[createFTFAccount setUserid:#"********"]; // Userid to be checked / created
[ftfQueue addOperation:createFTFAccount];
FTFPostRoute *postFTFRoute = [[FTFPostRoute alloc]init];
[postFTFRoute addDependency:createFTFAccount];
[ftfQueue addOperation:postFTFRoute];
In the first NSOperation subclass checks if account exists on server:
#implementation FTFCreateAccount
{
NSString *_accountCreationStatus;
}
- (void)main {
NSDate *startDate = [[NSDate alloc] init];
float timeElapsed;
NSString *ftfAccountStatusKey = #"ftfAccountStatus";
NSString *ftfAccountStatus = (NSString *)[[NSUserDefaults standardUserDefaults] objectForKey:ftfAccountStatusKey];
NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];
[userDefaults setValue:#"CHECKING" forKey:ftfAccountStatusKey];
// Setup and Run the NSURLSessionTask
[self createFTFAccount:[self userid]];
// Hold it here until the SessionTask completion handler updates the _accountCreationStatus
// Or the process takes too long (possible connection error)
while ((!_accountCreationStatus) && (timeElapsed < 5.0)) {
NSDate *currentDate = [[NSDate alloc] init];
timeElapsed = [currentDate timeIntervalSinceDate:startDate];
}
if ([_accountCreationStatus isEqualToString:#"CONNECTION PROBLEM"] || !_accountCreationStatus) [self cancel];
if ([self isCancelled]) {
NSLog(#"DEBUG FTFCreateAccount Cancelled" );
NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];
[userDefaults setValue:#"ERROR" forKey:ftfAccountStatusKey];
}
}
In the next NSOperation post data:
#implementation FTFPostRoute
{
NSString *_routePostStatus;
}
- (void)main {
NSDate *startDate = [[NSDate alloc] init];
float timeElapsed;
NSString *ftfAccountStatusKey = #"ftfAccountStatus";
NSString *ftfAccountStatus = (NSString *)[[NSUserDefaults standardUserDefaults] objectForKey:ftfAccountStatusKey];
if ([ftfAccountStatus isEqualToString:#"ERROR"])
{
// There was a ERROR in creating / accessing the user account. Cancel the post
[self cancel];
} else
{
// Call method to setup and run json post
// Hold it here until a reply comes back from the operation
while ((!_routePostStatus) && (timeElapsed < 3)) {
NSDate *currentDate = [[NSDate alloc] init];
timeElapsed = [currentDate timeIntervalSinceDate:startDate];
NSLog(#"FTFPostRoute time elapsed: %f", timeElapsed);
}
}
if ([self isCancelled]) {
NSLog(#"FTFPostRoute operation cancelled");
}
}

Resources