I have an iOS app written in Swift 3. in one of the modules i need to upload several files into a Web Server concurrently (files are chosen by the user from camera or gallery). i also have to display progress indicator for each file that indicates when upload is in progress and when upload is completed. the files are about 3-4MB each and the number of files is unknown (depends on the user) .
My Question:
should i use the low level GCD api or the higher NSOperation/NSOperationQueue abstraction ?
I have searched a lot but some got mixed answers (some say GCD and other say NSOperationQueue).i'v been struggling with this question a lot and i don't want to start coding only to find out in the end that i picked the wrong approach. (multi-threaded code is challenging anyhow)
Thank you very much!
I think you should go for NSOperationQueue because it will give you more control over your task. NSOperationQueue will provide objects upon which you can also add dependencies if required.
https://developer.apple.com/documentation/foundation/nsoperation?language=objc
Related
I have an app on the app store that uses AFNetworking 2.x to download large files in the background with NSURLSession-based downloads, because the user will often put the app in the background (it gets terminated after a while of course, but the downloads finish all the same. Wonderful). This app is working well. Usually users are only downloading a few files at a time.
Now I need to make another similar app, but this time instead of a few large files, it is very likely that the user will want to download a large number of smallish files: for example, 500 files that are 1-5mb each. Again, the app will often be put in the background, so I want to stay with NSURLSessionDownloadTask unless there's a really good reason not to.
My question is, can I simply create 500 NSURLSessionDownloadTasks all at once? Does AFNetworking do some clever throttling so as not to overload the system? Or does iOS do it? Or does nothing do it, and I have to painfully track & organize the state of transfers across restarts of my app (ie. because it gets put in the background eventually terminated) ?
If anyone knows the limits of how many NSURLSessionDownloadTasks you can create reliably simultaneously, that would be awesome...
thanks!
p.s. I greatly prefer obj-c to swift, thx :)
Last I checked (haven't looked at the iOS 9 betas), task creation was unexpectedly expensive and also superlinear. On my test runs:
50 tasks -> ~1.5s
200 tasks -> ~11.5s
500 tasks -> ~55s
Since my file count was often a 5-digit number, scheduling everything at once wasn't a solution for me. My approach (which isn't in production yet, I stopped working on the feature in favour of other things), combines persistence with NSURLSessionDownloadTask and uses the session identifiers to sort out which logical download a particular file belongs to. Further downloads are scheduled from one of the delegates depending on whether I'm on the normal lifecycle or coming from -application:handleEventsForBackgroundURLSession:completionHandler: (debugging this situation can get painful; NSUserDefaults is your friend). The theory seems sound, I can see that tasks do get scheduled, but I'm currently stuck getting the iOS downloader daemon to conform to my will.
If the server-side zip as suggested by Benjamin Jimenez is an option for you, do yourself a favour and use that instead.
The Apple staff member "eskimo" on apple developer forums helped me find the answer, which you can see in this forum post:
https://forums.developer.apple.com/thread/11621
Pasting here the relevant parts:
(me) I've read through this thread and the one you linked to here
(https://devforums.apple.com/message/938057#938057) and I have a
question about best practices to download 10,000-20,000 files via
NSURLSessionDownloadTasks. (Disclaimer, i'm using AFNetworking 2.x).
I'm targeting iOS 8 and newer, so answers do not have to work on iOS
7. How can we compute a reasonable batch (group) size ? I understand the resume-rate limiter means one wants the batch size to be higher,
but there's an unknown max limit of simultaneous task requests that
will crash the daemon.
(me) My assumption here is that when the user opens my app and it runs
for some time in the foreground, then the rate limiter is "reset" or
similar -- so now things will flow nicely again. Is this assumption
correct?
(eskimo) Yes. Also, starting with iOS 8, if the user brings your app
to the front then iOS will automatically give tasks a 'kick'. I've
forgotten the exact mechanics of this but I'm pretty sure it's covered
in WWDC 2014 Session 707 What's New in Foundation Networking.
this is an open question aiming at understanding what is the best practice or the most common solution for a problem that I think might be common.
Let's say I have a list of URLs to download; the list is itself hosted on a server, so I start a NSURLConnection that downloads it. The code in connectionDidFinishLoading will use the list of URLs to instantiate one new NSURLConnection, asynchronously, per each URL; this in turn will trigger even more NSURLConnections, and so on - until there are no more URLs. See it as a tree of connections.
What is the best way to detect when all connections have finished?
I'm aiming the question to iOS7, but comments about other versions are welcome.
A couple of thoughts:
In terms of triggering the subsequent downloads after you retrieve the list from the server, just put the logic to perform those subsequent downloads inside the completion handler block (or completion delegate method) of the first request.
In terms of downloading a bunch of files, if targeting iOS 7 and later, you might consider using NSURLSession instead of NSURLConnection.
First, the downloading of files with a nice modest memory footprint is enabled by initiating "download" tasks (rather than "data" tasks).
Second, you can do the downloads using a background NSURLSessionConfiguration, which will let the downloads continue even if the user leaves the app. See the Downloading Content in the Background section of the App Programming Guide for iOS. There are a lot of i's that need dotting and t's that need crossing if you do this, but it's a great feature to consider implementing.
See WWDC 2013 What's New in Foundation Networking for an introduction to NSURLSession. Or see the relevent chapter of the URL Loading System Programming Guide.
In terms of keeping track of whether you're done, as Wain suggests, you can just keep track of the number of requests issued and the number of requests completed/failed, and in your "task completion" logic, just compare these two numbers, and initiate the "all done" logic if the number of completions matches the number of requests. There are a bunch of ways of doing this, somewhat dependent upon the details of your implementation, but hopefully this illustrates the basic idea.
Instead of using GCD you should consider using NSOperationQueue. You should also limit the number of concurrent operations, certainly on mobile devices, to perhaps 4 so you don't flood the network with requests.
Now, the number of operations on the queue is the remaining count. You can add a block to the end of each operation to check the queue count and execute any completion logic.
As Rob says in his answer you might want to consider NSURLSession rather than doing this yourself. It has a number of advantages.
Other options are building your own download manager class, or using a ready-made third party framework like AFNetworking. I've only worked with AFNetworking a little bit but from what I've seen its elegant, powerful, and easy to use.
Our company wrote an async download manager class based on NSURLConnection for a project that predates both AFNetworking and NSURLSession. It's not that hard, but it isn't as flexible as either NSURLSession or AFNetworking.
I have a problem regarding the best approach to make an App that has to download and show pdf's, It is fed by a JSON that has links to 147 pdf files, sized between 1 and 2 MB.
Questions:
What is the best approach to download all the files to an iPad?
Shall I use AFNetworking 2.0?
Is NSFileManager the way to save all the files?
Problems I may encounter:
With an asynchronous download, if lost connection or no more space on the iPad, what are the counter mesures?
Are there tutorials or examples that deal with this situation?
Sorry for all the questions but I'm new to this.
Best Regards.
What is the best approach to download all the files to an iPad?
This is really broad as #rmaddy suggested. Specific questions are more easily answered. There are lots of ways you could download a file via an HTTP request to your device each with pros/cons depending on your situation.
Shall I use AFNetworking 2.0?
Sure. You'll get no argument from me. This is a widely used and solid API to interface with HTTP-based resources.
Is NSFileManager the way to save all the files?
Yes. NSFileManager is the class you use to read/write files from/to your app's sandbox.
With an asynchronous download, if lost connection or no more space on the iPad, what are the counter measures?
I'm not 100% certain so I can't speak to exactly what happens in this case. AFNetworking may provide some help by writing to a temporary file during a download, etc....
Are there tutorials or examples that deal with this situation?
I have a sample project on Github that shows a table of files that you can download. You can watch the progress of your downloads, pause each request, resume and cancel as well. When you're done you can view the file you downloaded. It uses AFNetworking and might be useful to you:
https://github.com/chefnobody/StreamingDownloadTest
When downloading large files, the main counsel would be to avoid trying to load these into memory as you download them. Instead, make sure you stream them directly to persistent storage. In terms of handling space-specific errors, just make sure you check NSError objects that are returned to you in completion handlers or the appropriate delegate methods.
If using AFNetworking, you can specify the outputStream of the AFURLConnectionOperation to reference a NSOutputStream that you create, referencing some path in your persistent storage.
See Memory pressure issue while downloading multiple files using AFNetworking for example.
Alternatively, you can use NSURLSession (whether via AFNetworking or you do it yourself) and instantiate a NSURLSessionDownloadTask, which does the same sort of thing.
Google "NSURLSessionDownloadTask example" and you'll find tons of references. The block-based rendition of downloadTaskWithURL is incredibly simple. To do background downloads is a little more complicated and requires delegate-based implementation (see Downloading Files and Handling iOS Background Activity sections of URL Loading System Programming Guide: Using NSURLSession or watch the WWDC 2013 video, What’s New in Foundation Networking.)
Either way, you avoid some of the memory consumption challenges associated with downloading large files.
I have created a simple testing app to learn how to use NSURLSession. This App has to download images from a webservice and present them into a UITableView.
I've already written the first part of the App that reads a list of images urls from the web service, now, I want to display this list.
My doubt is:
given that the list of images could be a really long list, is it ok to create a NSURLSessionDownloadTask for each image?
I thought to create the session in the cellForRowAtIndexPath function and store the NSURLSessions in a NSDictionary using as key the IndexPath of the cell (and probably relying on NSURLCache to avoid to download the same images more than once).
Other solutions:
I can see three more solutions:
Using GCD with dispatch_async
Subclassing NSOperation and essentially store an NSOperation for any image I need to download.
Using a third party library like AFNetwork... but since it is a learning purpose app I prefer to go completely with my code
.
If the multiple NSURLSession isn't a good solution, I'd choose one of those options.
What do you think about this approach?
NSURLSessionTask is fine for a large number of downloads. One advantage of it over some of the other methods you mentioned is that downloads can be cancelled or paused. It also correctly implements concurrency for network operations, which is more difficult than many cats on the internet will lead you to believe (if you don't believe me, view the eskimo's 2010 WWDC session and sample code. NSOperation for network connections is not trivial).
NSURLSessionTask and friends are designed for exactly the kinds of problems you are trying to solve, and it's very well tested.
For a tableview, start the task in tableView:willDisplayCell:forRowAtIndexPath: and cancel (or pause) a task in tableView:didEndDisplayingCell:forRowAtIndexPath:. That will limit the active downloads to the currently visible cells.
Suggestion:
I also came across a similar situation were I need to download about 2000 Image files and 100 Video files. For that purpose I implemented a custom download manager using NSOperationQueue and blocks.
I have added this library to GitHub, please feel free to check the implementation.
IMO whilst it is ok to create an NSURLSessionTask for each image a standard first in first out implementation will cause problems when scrolling through your cells. The reason for this is that downloads will be queued on your NSURLSession and tasks will be executed in the order they've been added to the queue, in other words in a FIFO manner. Imagine a scenario where you've scrolled through a vast number of cells and you have to wait for all downloads to complete in order. You would not only have to wait a long time, you would be making unnecessary network requests for image assets that may no longer be relevant to your user.
Nick Lockwood created a great NSOperationQueue subclass called NSOperationStack that reverses the order of operations so that the the last operation is executed first (LIFO). IMO for a large number of downloads a LIFO implementation is a must.
NSOsperationStack is available here
If you combine this with an implementation that uses cellForRowAtIndexPath to initiate and NSURLCache to store downloads, you should end up with a very streamlined and efficient solution.
I would use (or at least take a look at) SDWebImage's SDWebImageManager.
Besides downloading you can set priority and continue in the background options which I think you'll want to have.
I have an iPhone app which pretty much is a mobile app for a website. Pretty much everything it does is call API methods from our server. The app retrieves the user's information, and keeps updating the server using the API.
My colleague and I had a discussion whether to introduce GCD to the downloading aspect on the app. My colleague argues that since the UI needs to wait for the download to complete before it can display the pictures, text or whatever, there is absolutely no need for GCD. My argument is that we should keep the main thread busy with UI rendering (even if there is no data), and introduce GCD to the app to create other threads for download.
Which argument is right here? In my case, if the UI renders with no data, will there be some sort of lag? Which will yield a cleaner, sleeker and faster app?
One argument would be : what will happen when the download fails and times out because there is a problem at the server end ?
Without GCD the app will remain blocked and will crash after a time
out since the UI can not be blocked for longer than 20 seconds.
With GCD the application remains functional but there will be no data
being downloaded and the application will not crash.
Other aspects to take into account are :
the thread safety of the objects that you are using
how you handle downloads that are no longer necessary because the user navigates away from the page
I don't think doing time consuming operations in the main thread is a good idea.
Even if user have to wait for the data te be downloaded before he can do anything meaningful, still he will not hope UI is blocked.
Let's assume you have a navigator view, and after user tap some button, you push a new view to it and start download something. If user suddenly decides he don't want to wait anymore, he tap the "back" button. If your downloading operation blocks UI, user will have to wait it to end, it's really bad.
A more appropriate question would perhaps be if you should download asynchronously or on the main thread for your app, since there are several different methods to download asynchronously on iOS (e.g. using NSThread, NSOperation or indeed GCD). An easy approach to achieve your goals could be to use the AFNetworking library. It makes multithreaded networking / internet code very easy to implement and understand.
Personally I'm very fond of GCD and recommend you learn it someday soon, though in itself it is not as suitable for asynchronous downloading compared to a library like AFNetworking (that uses GCD under the hood, I believe).
Here is a good read on using NSOperationQueues (that uses GCD behind the scenes) to download images. There is also some Github code you can check out. They have an elegant solution to pause downloads and enqueue new downloads when the user moves to different parts of your app.
http://eng.alphonsolabs.com/concurrent-downloads-using-nsoperationqueues/?utm_medium=referral&utm_source=pulsenews
Use GCD / NSOperationQueues as opposed to using NSThreads. You will have a good learning on core fundamentals and at the same time create a well architectured app. :)