Download Multiple Items from Amazon S3 Bucket iOS - ios

Question

I've recently implemented the new AWS 2.0 iOS SDK in my application (yay, cocoapods!), and using the sample code from Amazon managed to properly configure access and downloads. I can successfully download a single item without issue, but I need to be able to download multiple files dynamically generated based on the current tableview. There doesn't appear to be a way to set up a batch download, so I'm simply trying to loop through an array of objects and trigger a download with each one. It works, but if the list includes more than a few items, it starts randomly misfiring. For example, if my dynamically created list has 14 items in it, 12 will be downloaded, and the other 2 aren't even attempted. The request just vanishes. In my testing, I added a sleep(1) timer, and then all 14 are triggered and downloaded, so I'm guessing that I'm overwhelming the download requests and they are getting dropped unless I slow it down. Slowing it down is not ideal... perhaps there is another way? Here is the code:
- (IBAction)downloadAllPics:(UIBarButtonItem *)sender {
if (debug==1) {
NSLog(#"Running %# '%#'", self.class, NSStringFromSelector(_cmd));
}
CoreDataHelper *cdh =
[(AppDelegate *)[[UIApplication sharedApplication] delegate] cdh];
// for loop iterates through all of the items in the tableview
for (Item *item in self.frc.fetchedObjects) {
NSString *docDir = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
NSString *downloadingFilePath1 = [NSString stringWithFormat:#"%#/%##2x.jpg",docDir, item.imageName];
NSURL *downloadingFileURL1 = [NSURL fileURLWithPath:downloadingFilePath1];
NSFileManager *fileManager = [NSFileManager defaultManager];
NSError *error = nil;
if ([fileManager fileExistsAtPath:downloadingFilePath1]) {
fileAlreadyExists = TRUE;
if (![fileManager removeItemAtPath:downloadingFilePath1
error:&error]) {
NSLog(#"Error: %#", error);
}
}
__weak typeof(self) weakSelf = self;
self.downloadRequest1 = [AWSS3TransferManagerDownloadRequest new];
self.downloadRequest1.bucket = S3BucketName;
// self.downloadRequest1.key = S3KeyDownloadName1;
self.downloadRequest1.key = [NSString stringWithFormat:#"images/%##2x.jpg", item.imageName];
self.downloadRequest1.downloadingFileURL = downloadingFileURL1;
self.downloadRequest1.downloadProgress = ^(int64_t bytesWritten, int64_t totalBytesWritten, int64_t totalBytesExpectedToWrite){
// update progress
dispatch_sync(dispatch_get_main_queue(), ^{
weakSelf.file1AlreadyDownloaded = totalBytesWritten;
weakSelf.file1Size = totalBytesExpectedToWrite;
});
};
// this launches the actual S3 transfer manager - it is successfully launched with each pass of loop
[self downloadFiles];
}
[cdh backgroundSaveContext];
}
That launches the downloadFiles method:
- (void) downloadFiles {
//if I add this sleep, all 14 download. If I don't usually 11-13 download.
sleep(1);
AWSS3TransferManager *transferManager = [AWSS3TransferManager defaultS3TransferManager];
__block int downloadCount = 0;
[[transferManager download:self.downloadRequest1] continueWithExecutor:[BFExecutor mainThreadExecutor] withBlock:^id(BFTask *task) {
if (task.error != nil){
if(task.error.code != AWSS3TransferManagerErrorCancelled && task.error.code != AWSS3TransferManagerErrorPaused){
NSLog(#"%s Errorx: [%#]",__PRETTY_FUNCTION__, task.error);
}
} else {
self.downloadRequest1 = nil;
}
return nil;
}];
}
There has got to be a way to download a dynamic list of files from an Amazon S3 bucket, right? Maybe there is a transfer manager that allows an array of files instead of doing them individually?
Any and all help is appreciated.
Zack

Answer 1

Sounds like request timeout interval setting issue.
First, when you configure AWSServiceConfiguration *configuration = ... try to configure the timeoutIntervalForRequest property. Also, maxRetryCount as well. maxRetryCount will attempt to download if failure for downloading each operation.
AWSServiceConfiguration *configuration = [[AWSServiceConfiguration alloc] initWithRegion:DefaultServiceRegionType
credentialsProvider:credentialsProvider];
[configuration setMaxRetryCount:2]; // 10 is the max
[configuration setTimeoutIntervalForRequest:120]; // 120 seconds
Second, for the multiple items downloading try to collect each AWSTask into one array and get the result at the end of group operation. ex)
// task collector
NSMutableSet *uniqueTasks = [NSMutableSet new];
// Loop
for (0 -> numOfDownloads) {
AWSS3TransferManagerDownloadRequest *downloadRequest = [AWSS3TransferManagerDownloadRequest new];
[downloadRequest setBucket:S3BucketNameForProductImage];
[downloadRequest setKey:filename];
[downloadRequest setDownloadingFileURL:sourceURL];
[showroomGroupDownloadRequests addObject:downloadRequest];
AWSTask *task = [[AWSS3TransferManager defaultS3TransferManager] download:downloadRequest];
[task continueWithBlock:^id(AWSTask *task) {
// handle each individual operation
if (task.error == nil) {
}
else if (task.error) {
}
// add to the tasks
[uniqueTasks addObject:task];
return nil;
}
[[AWSTask taskForCompletionOfAllTasks:tasks] continueWithBlock:^id(AWSTask *task) {
if (task.error == nil) {
// all downloads succeess
}
else if (task.error != nil) {
// failure happen one of download
}
return nil;
}];

Answer 2

The reason some requests seem to vanish is that you define AWSS3TransferManagerDownloadRequest as a property. self.downloadRequest1 = nil; is executed on the background thread, and it is possible that when [transferManager download:self.downloadRequest1] is executed, self.downloadRequest1 is nil.
You should remove the property and simply pass an instance of AWSS3TransferManagerDownloadRequest as an argument for - downloadFiles:.