Can anyone suggest a solution for downloading multiple files say 100s, in background. The important thing being that the download url has a life span of 15 minutes, so it's required that we fetch a download url and then start downloads. We can't prefetch all urls and add it to download task, as that may lead to download failures of expired url after few successful downloads.
Thanks in advance.
you can do the following:
var data : NSData?{
didSet{
//Parse the data to any thing you want
}
}
var urlFetchedAsString : String? {
didSet{
if(urlFetchedAsString == nil)
return
let url : NSURL = NSURL(string: urlFetchedAsString!)!
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), {
() -> Void in
data = NSData(contentsOfURL: url)
})
}
}
func fetchURL() ->String{
//Fetched Your url and return
}
override func viewDidLoad(){
super.viewDidLoad()
urlFetchedAsString = fetchURL()
}
Explanation
The OS will execute the didSet block in the variable urlFetchedAsString each time it is set
The didSet block will fetch the data from the url and save them as NSData
The dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0) means that fetching data from url will be done on the different thread to prevent blocking the UI thread
After done saving you, the variable data will be set and the didSet for this variable will be executed. In this block you can implement your parsing algorithm.
Fetching the url itself is up to you because you didn't clarify where they are or how will u get them
Note
Here i assumed that you don't need to copy of all the urls because as you said they will expire in 15 mins
Related
I'm working on an app that has to load some images and data from server on every launch (to make sure it's using up-to-date info). I'm using Firestore as a DB and currently storing images in it as an URL to Firebase storage.
Is it somehow possible to store an actual image in Firestore? And how can I cache loaded image? Either from
UIImage(contentsOf: URL)
or from Firestore?
Try this Asynchronous image downloader with cache support as a UIImageView category - http://cocoadocs.org/docsets/SDWebImage
It is called sdwebimage really easy to use
I don't know if that's the most efficient way of solving my problem but I did it the following way:
In my Firestore DB I stored references to images in Cloud Storage. Then when app starts for the first time, it loads those files from Firestore DB using default methods AND saves those images in app's container (Documents folder) using Swift's FileManager().
Next time the app starts, it goes through references array and skips the files which are already in app's container.
You could use the bytes type in Firestore (see a list of types) to save whatever binary data you want (use NSData on iOS), but this is almost certainly not what you actually want to do. The limit for the size of an entire document is 1 MB, and images can easily exceed that. Also, you'll be paying the cost of downloading that image to the client any time that document is read, which could be wasteful.
You'll be far better off storing the actual file data in Cloud Storage (using the Firebase SDK on the client), then storing a reference or URL to that in the document, and fetch it from there only when needed.
You could use https://github.com/pinterest/PINRemoteImage, this framework use https://github.com/pinterest/PINCache
import PINRemoteImage
extension UIImageView {
public func setImageFrom(urlString: String!, animated: Bool = false) {
guard let urlString = urlString else {
return
}
guard let url = URL(string: urlString) else {
return
}
layer.removeAllAnimations()
pin_cancelImageDownload()
image = nil
if !animated {
pin_setImage(from: url)
} else {
pin_setImage(from: url, completion: { [weak self] result in
guard let _self = self else { return }
_self.alpha = 0
UIView.transition(with: _self, duration: 0.5, options: [], animations: { () -> Void in
_self.image = result.image
_self.alpha = 1
}, completion: nil)
})
}
}
}
....
UIImageView(). setImageFrom(urlString: "https://ssssss")
I have an API that generates signed download links that expire after a short amount of time. I'd like to add the ability to resume downloads, but the URLSession APIs don't provide the native ability to resume downloads if the URL for the asset changes.
My attempt at solving this was to track the bytes downloaded at the time of pausing, store the data blob that was downloaded, fetch a new signed download url, resume downloading using Range headers, and then concatenate all the data blobs together when the download is completed.
Here's the code used to start the download:
let session = URLSession(configuration: URLSessionConfiguration.default, delegate: self, delegateQueue: nil)
let task = session.downloadTask(with: signedURL)
self.sessionDownloadRequest = task
The problem that I am facing is that the resume data var doesn't appear to actually contain the data that was downloaded.
self.sessionDownloadRequest.cancel(byProducingResumeData: { (data) in
print(data.count) //This surprisingly always returns the same count
}
It appears that the size of that data blob is always the same regardless of how long I let the download continue for before pausing. Where/How can I access the chunk of data that was downloaded?
Thanks!
The resume data that is returned by:
- (void)cancelByProducingResumeData:(void (^)(NSData *resumeData))completionHandler;
is actually a plist that includes:
NSURLSessionDownloadURL
NSURLSessionResumeBytesReceived
NSURLSessionResumeCurrentRequest
NSURLSessionResumeEntityTag
NSURLSessionResumeInfoTempFileName
NSURLSessionResumeInfoVersion
NSURLSessionResumeOriginalRequest
NSURLSessionResumeServerDownloadDate
You can access the plist with the following code:
if let resumeDictionary = try? PropertyListSerialization.propertyList(from: self, options: PropertyListSerialization.MutabilityOptions.mutableContainersAndLeaves, format: nil), let plist = resumeDictionary as? [String: Any] {
print(plist)
}
You don't actually need to store and concatenate the data blobs as you initially suggested. You can replace the current request stored in the plist (NSURLSessionResumeCurrentRequest) with a new one with your updated signed URL. After this, create a new resumeData instance to use instead of the original.
guard let bytesReceived = plist["NSURLSessionResumeBytesReceived"] as? Int
else {
return nil
}
let headers = ["Range":"bytes=\(bytesReceived)"]
let newReq = try! URLRequest(url: signedURL, method: .get, headers: headers)
let archivedData = NSKeyedArchiver.archivedData(withRootObject: newReq)
if let updatedResumeData = try? PropertyListSerialization.data(fromPropertyList: plist, format: PropertyListSerialization.PropertyListFormat.binary, options: 0) {
return updatedResumeData
}
From there you can manipulate the plist and actually create a new one to pass it thru to the instance method:
- (NSURLSessionDownloadTask *)downloadTaskWithResumeData:(NSData *)resumeData;
NOTE: If you are working with iOS 10 and macOS10.12.*, there is a bug that prevents the resume ability to work as the plist is corrupted. Check this article out for a fix. You may need to fix the plist before accessing certain properties on it.
Resume NSUrlSession on iOS10
I am trying to update a progress bar as my images load. I've read several answers here and tried formatting my code many different ways. I'm trying to read in the images and update the progress bar. Then, will all the images are loaded call the code to process them. The best result I've got was some code that works most of the time. However, if I'm dealing with a situation where it is pulling in a lot of images, I get weird errors. I think it is going ahead and running the continue code before all the images are fully loaded. When I remove the dispatch_async, the code works fine but the progress bar does not update.
func imageLocXML(il:String) {
dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0)) {
let url:NSURL? = NSURL(string: il)
let data:NSData? = NSData(contentsOfURL: url!)
let image = UIImage(data: data!)
pieceImages.append(image!)
self.numImagesLoaded += 1
self.updateProgressBar()
if self.numImagesLoaded == self.numImagesToLoad {
self.continueLoad()
}
}
}
There a number of issues:
This code isn't thread safe, because you have race condition on numImagesLoaded. This could, theoretically, result in continueLoad to be called more than once. You can achieve thread safety by synchronizing numImagesLoaded by dispatching updates to this (and other model objects) back to the main queue.
Like DashAndRest said, you have to dispatch the UI update to the main queue, as well.
When you made this asynchronous, you introduced a network timeout risk when you initiate a lot of requests. You can solve this by refactoring the code to use operation queues instead of dispatch queues and specify maxConcurrentOperationCount.
The images are being added to an array:
Because these tasks run asynchronously, they're not guaranteed to complete in any particular order, and thus the array won't be in order. You should save the images in a dictionary, in which case the order no longer matters.
Just like numImagesLoaded, the pieceImages isn't thread safe.
You are using a lot of forced unwrapping, so if any requests failed, this would crash.
But to address this, we have to step back and look at the routine calling this method. Let's imagine that you have something like:
var pieceImages = [UIImage()]
func loadAllImages() {
for imageUrl in imageURLs {
imageLocXML(imageUrl)
}
}
func imageLocXML(il:String) {
dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0)) {
let url:NSURL? = NSURL(string: il)
let data:NSData? = NSData(contentsOfURL: url!)
let image = UIImage(data: data!)
self.pieceImages.append(image!)
self.numImagesLoaded += 1
self.updateProgressBar()
if self.numImagesLoaded == self.numImagesToLoad {
self.continueLoad()
}
}
}
I'm suggesting that you replace that with something like:
var pieceImages = [String: UIImage]()
func loadAllImages() {
let queue = NSOperationQueue()
queue.maxConcurrentOperationCount = 4
let completionOperation = NSBlockOperation {
self.continueLoad()
}
for imageURL in imageURLs {
let operation = NSBlockOperation() {
if let url = NSURL(string: imageURL), let data = NSData(contentsOfURL: url), let image = UIImage(data: data) {
NSOperationQueue.mainQueue().addOperationWithBlock {
self.numImagesLoaded += 1
self.pieceImages[imageURL] = image
self.updateProgressBar()
}
}
}
queue.addOperation(operation)
completionOperation.addDependency(operation)
}
NSOperationQueue.mainQueue().addOperation(completionOperation)
}
Having said that, I think there are deeper issues here:
Should you be loading images in advance like this at all? We would generally advise lazy loading of images, only loading them when needed.
If you're going to load images into a structure like this, you should gracefully handle memory pressure, purging it upon low memory warning. You then need to gracefully handle what to do when you go to retrieve an image and it's been purged due to memory pressure (leading you right back to a just-in-time lazy loading pattern).
We'd generally advise against synchronous network requests (the NSData(contentsOfURL:_)). We'd generally use NSURLSession which is cancellable, offers richer error handling, etc. Admittedly, that complicates the above code even further (probably leading me down the road of asynchronous NSOperation subclass), but you should at least be aware of the limitations of contentsOfURL.
Try DISPATCH_QUEUE_PRIORITY_DEFAULT this for background queue as:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)) {
let url:NSURL? = NSURL(string: il)
let data:NSData? = NSData(contentsOfURL: url!)
let image = UIImage(data: data!)
self.pieceImages.append(image!)
self.numImagesLoaded += 1
dispatch_async(dispatch_get_main_queue(), {
//UI must be updated on main thread queue
self.updateProgressBar()
})
if self.numImagesLoaded == self.numImagesToLoad {
self.continueLoad()
}
}
UI must be updated on MAIN thread queue!
You also have to use self for accessing pieceImages.
So I have an app that pulls up movie results when I type in a search. (Like IMDb.) I use a free API from themoviedb.org to load the results. I load them in a TableViewController. I load the posters for the results using a mod on the .dataTaskWithRequest method. to make it synchronous. Other than that, it's just basic API sending and receiving for the titles, genres, and years of the movies or TV Shows.
Now my app lags when I type too fast, this isn't completely because of the synchronous loading, because it still happens when I don't load images at all, but image loading makes the app lag, too. Now this is an issue in and of itself, but the problem is that when the app loads the words on to the screen, and is done with the lag, the results are the results of part of the word I have on screen. For example, if I type "The Simpsons" too fast, I get results for "The Sim", but if I backspace once, and retype "The Simpsons", the results reload correctly. Something that complicates things even more, is that sometimes I get the top result only being one of the old, partial results, and the rest are normal and loaded underneath.
Here is a video explaining the situation. The first time i type down "the simpsons", you can see the lag. I typed it all really fast, but it lags past the word "the". When it is done loading, it loads up a beowulf result that shouldn't even be there. I have no idea what's going on and it's driving me nuts. Even when I don't load images, and the typing doesn't lag, the results still don't update.
Here are the relevant code snippets, if you want any more, feel free to ask. I just don't want to bombard you with too much code at once:
This updates search results when text is typed in search bar:
extension SearchTable : UISearchResultsUpdating {
func updateSearchResultsForSearchController(searchController: UISearchController) {
//To Handle nils
var searchBarText = searchController.searchBar.text
if (searchBarText == nil) {
searchBarText = ""
}
searchBarText! = searchBarText!.condenseWhitespace()
//To Handle Disallowed Characters
searchBarText = searchBarText!.stringByAddingPercentEncodingWithAllowedCharacters(NSCharacterSet.URLQueryAllowedCharacterSet())
//Find Results from themoviedb
let urlString = "https://api.themoviedb.org/3/search/multi?query=" + searchBarText! + "&api_key= (I can't post the api key publicly online, sorry)"
let results = NSURL(string: urlString)
if (results == nil) {
//Server Error
}
//Wire Up Results with matchingItems Array
let task = NSURLSession.sharedSession().dataTaskWithURL(results!) { (data, response, error) -> Void in
if let jsonData = data {
do {
let jsonData = try NSJSONSerialization.JSONObjectWithData(jsonData, options: NSJSONReadingOptions.MutableContainers)
if var results = jsonData["results"] as? [NSDictionary] {
if results.count > 0 {
//Clean out non-english results:
//I wrote the function, it shouldn't be the source of the lag, but I can still provide it.
self.cleanArray(&results)
self.matchingItems = results
} else {
self.matchingItems = []
}
}
} catch {
//JSON Serialization Error
}
}
}
task.resume()
self.tableView.reloadData()
}
}
Then, after I get the results, I reload the table using the two required methods from a TableViewDataSource:
//Table Data Source
extension SearchTable {
override func tableView(tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
return matchingItems.count
}
override func tableView(tableView: UITableView, cellForRowAtIndexPath indexPath: NSIndexPath) -> UITableViewCell {
let cell = tableView.dequeueReusableCellWithIdentifier("cell")! as! CustomCell
//Safe-Guard. This shouldn't be needed if I understood what I was doing
if (indexPath.row < matchingItems.count) {
cell.entry = matchingItems[indexPath.row] //404
//Name & Type & Year
//This is only for TV Shows, I removed the rest for simplicity
cell.title.text = matchingItems[indexPath.row]["name"] as? String
cell.icon.image = UIImage(named: "tv.png")
let date = (matchingItems[indexPath.row]["first_air_date"] as? String)
cell.year.text = date == nil ? "" : "(" + date!.substringToIndex(date!.startIndex.advancedBy(4)) + ")"
//Genre
//Code here removed for simplicity
//Poster
cell.poster.image = UIImage(named: "Placeholder.jpg")
if let imagePath = matchingItems[indexPath.row]["poster_path"] as? String {
let url = NSURL(string: "http://image.tmdb.org/t/p/w185" + imagePath)
let urlRequest = NSURLRequest(URL: url!)
let session = NSURLSession.sharedSession()
//Synchronous Request
let semaphore = dispatch_semaphore_create(0)
let task = session.dataTaskWithRequest(urlRequest) { data, response, error in
if let poster = UIImage(data: data!) {
cell.poster.image = poster
}
dispatch_semaphore_signal(semaphore)
}
task.resume()
dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER)
}
}
return cell
}
}
Thanks!
First of all, I strongly recommend you don't use synchronous request, mainly because it blocks your UI until it finish and this is a very bad responsiveness for an app.
In your case you can place a placeholder for the UIImage and when the request finish substitute it for the correct image.
Regarding your issue of typing faster, it's called throttle or debounce, Apple recommends:
Performance issues. If search operations can be carried out very rapidly, it is possible to update the search results as the user is typing by implementing the searchBar:textDidChange: method on the delegate object. However, if a search operation takes more time, you should wait until the user taps the Search button before beginning the search in the searchBarSearchButtonClicked: method. Always perform search operations a background thread to avoid blocking the main thread. This keeps your app responsive to the user while the search is running and provides a better user experience.
But if you until want it to handle yourself you can see this two good answers explaining how to handle it correctly:
How to throttle search (based on typing speed) in iOS UISearchBar?
How can I debounce a method call?
I recommend you handle it as Apple recommends or you can change your philosophy and adopt some libraries that handle it for your automatically like:
Bond
RxSwift
The first one in more easy to learn, the second one needs to learn Reactive Programming and concepts of Functional Programming, It's up to you.
I hope this help you.
Just for people who may be struggling in the future with this same issue. First of all, read my comment to Victor Sigler's answer.
Here were the issues:
1 - I searched for the results online using .dataTaskWithURL() This is an asynchronous method which ran in the background while the code continued. So on occasion, the table would reload before the new results were in. See this thread for more information. I highly recommend checking this tutorial on concurrency if you are serious about learning Swift:
www.raywenderlich.com/79149/grand-central-dispatch-tutorial-swift-part-1
2 - The images lagged because of the search being synchronous, as Victor said. His answer pretty much handles the rest, so read it!
I am trying to download images but it is crucial that the images I download are in a specific order. I am using the following code to download the image:
func downloadImage(url: NSURL, completionHandler: (response: UIImage) -> ()){
print("Started downloading \"\(url.URLByDeletingPathExtension!.lastPathComponent!)\".")
manager.getDataFromUrl(url) { (data, response, error) in
dispatch_async(dispatch_get_main_queue()) { () -> Void in
guard let data = data where error == nil else { return }
print("Finished downloading \"\(url.URLByDeletingPathExtension!.lastPathComponent!)\".")
completionHandler(response: UIImage(data: data)!)
}
}
}
and I am using this code to call downloadImage
self.downloadImage(NSURL(string: self.url)!, completionHandler: { response in
dispatch_async(dispatch_get_main_queue()) {
self.images.append(response)
}
})
The problem is that the images start downloading in the correct order however the response isn't (this is obviously because the size of the images are different and one comes in faster). So what is happening is that all images start downloading, and whichever comes first appends to the images : [UIImage] array. How can I make it so that the images in the images array is in order?
I've also tried to remove the main block when calling downloadImage function
self.downloadImage(NSURL(string: self.url)!, completionHandler: { response in
self.images.append(response)
})
You cannot control the download order in the sense that all the requests to the same server will be pipelined no matter what the order you create the URL objects in. Also, some URLs may be cached while others may need to go to the remote server. What you need to do is maintain a mutable array or dictionary that contains the url to actual data mapping, then wait until all the urls have been completely downloaded and then iterate in a known order.
The simplest method is that you can save every image in Dictionary with their url after downloading. Like var imageData = [String: NSData](). Later you can sort it or use it by keys(url).