I have a requirement to download large number of files - previously only one file could be downloaded at a time. The current design is such that when the user downloads a single file, a URLSession task is created and the progress/completion/fail is recorded using the delegate methods for urlsession. My question is, how can I leave a dispatch group in this delegate method? I need to download 10 files at a time, start the next 10 when the previous ten finishes. Right now, if I leave the dispatch group in the delegate method, the dispatch group wait waits forever. Here's what I've implemented so far:
self.downloadAllDispatchQueue.async(execute: {
self.downloadAllDispatchGroup = DispatchGroup()
let maximumConcurrentDownloads: Int = 10
var concurrentDownloads = 0
for i in 0..<files.count
{
if self.cancelDownloadAll {
return
}
if concurrentDownloads >= maximumConcurrentDownloads{
self.downloadAllDispatchGroup.wait()
concurrentDownloads = 0
}
if let workVariantPart = libraryWorkVariantParts[i].workVariantPart {
concurrentDownloads += 1
self.downloadAllDispatchGroup.enter()
//call method for download
}
}
self.downloadAllDispatchGroup!.notify(queue: self.downloadAllDispatchQueue, execute: {
DispatchQueue.main.async {
}
})
})
In the delegates:
func downloadDidFinish(_ notification: Notification){
if let dispatchGroup = self.downloadAllDispatchGroup {
self.downloadAllDispatchQueue.async(execute: {
dispatchGroup.leave()
})
}
}
Is this even possible? If not, how can I achieve this?
If downloadAllDispatchQueue is a serial queue, the code in your question will deadlock. When you call wait, it blocks that current thread until it receives the leave call(s) from another thread. If you try to dispatch the leave to a serial queue that is already blocked with a wait call, it will deadlock.
The solution is to not dispatch the leave to the queue at all. There is no need for that. Just call it directly from the current thread:
func downloadDidFinish(_ notification: Notification) {
downloadAllDispatchGroup?.leave()
}
When downloading a large number of files, we often use a background session. See Downloading Files in the Background. We do this so downloads continue even after the user leaves the app.
When you start using background session, there is no need to introduce this “batches of ten” logic. The background session manages all of these requests for you. Layering on a “batches of ten” logic only introduces unnecessary complexities and inefficiencies.
Instead, we just instantiate a single background session and submit all of the requests, and let the background session manage the requests from there. It is simple, efficient, and offers the ability to continue downloads even after the user leaves the app. If you are downloading so many files that you feel like you need to manage them like this, it is just as likely that the end user will get tired of this process and may want to leave the app to do other things while the requests finish.
Related
In the accepted answer for Timer.scheduledTimer not firing, it is emphasised to start the timer on the main thread to ensure that it fires. However, if I do that then I often end up with the timer being slow to initialise, and therefore failing in its purpose as a debouncer. Just wondering if there is something I am doing wrong, or a better way of doing this.
My problem (pseudocode at the bottom):
I use a JWT to authenticate my server calls, and I check this locally to see if it's expired before submitting it. However, I don't want several network calls to notice the expired JWT all at once and submit several refresh requests, so I use a semaphore to ensure only one call at a time is checking/renewing the JWT. I also use a dispatchGroup to delay the original network call until after the checking/renewing is done. However, if the refresh fails I want to avoid all the queued calls then trying it again. I don't want to block all refresh calls forever more with a boolean, so I thought I would create a scheduledTimer to block it. However, if I create it on the main thread, there's a delay before it's created and the released network calls submit a few more refresh attempts before they're blocked.
Questions
Should I just create the timer on the local thread to ensure there's no delay (I presume the main thread is occupied with some UI tasks which is why the timer doesn't get created instantly?)
More generally, is there a better way of doing this? I suspect there is - I tried playing with adding items to a queue, and then cancelling them, but then I began getting worried about creating work items with out of date values of functions, and capturing things in closures etc (it was a while ago, I can't remember the details), so I went with my current bodge.
This might all be easier if I was using await/async, but our app supports all the way back to iOS12, so I'm stuck with nests of completion handlers.
Hopefully this pseudocode is accurate enough to be helpful!
private static let requestQueue: DispatchQueue = DispatchQueue(label: "requestQueue", qos: .userInteractive, attributes: .concurrent)
public static let jwtValidityCheckSemaphore: DispatchSemaphore = DispatchSemaphore(value: 1)
private static var uglyHackTimer: Timer?
#objc private class func clearUglyHackTimer(){
uglyHackTimer?.invalidate()
uglyHackTimer = nil
}
class func myNetworkCall(for: myPurposes){
let group = DispatchGroup()
jwtValidityCheckSemaphore.wait()
if (uglyHackTimer?.isValid ?? false){
jwtValidityCheckSemaphore.signal()
return
}
group.enter()
if jwtIsInvalid(){
refreshJWT(){success in
if !success{
DispatchQueue.main.async{
self.uglyHackTimer = Timer.scheduledTimer(timeInterval: TimeInterval(2), target: self, selector: #selector(clearUglyHackTimer), userInfo: nil, repeats: false)
}
}
group.leave()
jwtValidityCheckSemaphore.signal()
}
}else{
group.leave()
jwtValidityCheckSemaphore.signal()
}
// Make the original network call
newNetworkRequest = DispatchWorkItem{
// Blah, blah
}
group.notify(queue: requestQueue, work: newNetworkRequest)
}
I'd like a user's upload operation that's started in the foreground to continue when they leave the app. Apple's article Extending Your App's Background Execution Time has the following code listing
func sendDataToServer( data : NSData ) {
// Perform the task on a background queue.
DispatchQueue.global().async {
// Request the task assertion and save the ID.
self.backgroundTaskID = UIApplication.shared.
beginBackgroundTask (withName: "Finish Network Tasks") {
// End the task if time expires.
UIApplication.shared.endBackgroundTask(self.backgroundTaskID!)
self.backgroundTaskID = UIBackgroundTaskInvalid
}
// Send the data synchronously.
self.sendAppDataToServer( data: data)
// End the task assertion.
UIApplication.shared.endBackgroundTask(self.backgroundTaskID!)
self.backgroundTaskID = UIBackgroundTaskInvalid
}
}
The call to self.sendAppDataToServer( data: data) is unclear. Is this where the upload operation would go, wrapped in Dispatch.global().sync { }?
You have stumbled across a less-than-stellar code sample in Apple’s documentation.
First, if you perform a synchronous network request, you definitely should dispatch it to a background queue. If you don't, you risk having the watchdog process kill your app. But you shouldn’t dispatch the network request synchronously to the global queue, but rather asynchronously, or else you just end up with the same problem, namely blocking the main thread.
That having been said, one really should never perform network requests synchronously. You should perform them asynchronously and end the background task in the completion handler.
In that example, they use NSData, which we don’t use anymore.
Also UIBackgroundTaskInvalid doesn’t exist anymore. It is now UIBackgroundTaskIdentifier.invalid.
In Apple’s defense, the point of this code sample is the background task, not the network code. They really didn’t want to get into the weeds of the implementation of this network code and were trying to keep it simple. That having been said, it really is a horrible and outdated code example.
See this answer for a better example of how one might use background tasks. Also, if the upload might take more than 30 seconds, we wouldn’t use the background task at all, but a proper (but more complicated) background URLSession. (See Downloading files in the background. Upload tasks follow the same basic pattern outlined there, though make sure to upload from a file, not a Data.)
I have array of photos that needs to be downloaded, but download function can download only one photo at a time. I neeed to make sure that download function is completed before I call it for other photo. I create .initialyInactive DispatchQueue and queue is activated in completion block of download function. This is woking, and photos are downloaded, but my question is how to cancel download process? Can I somehow to remove not activated queues?
My code logic looks something like this..
func downloadPhoto(photo: Photo, completion: (_ success: Bool) -> Void) {
... completion(true) ..
}
for photo in photos {
let queue = DispatchQueue(label: "Download queue\(counter)", qos: .userInitiated, attributes: .initiallyInactive)
self.inactiveQueues[counter] = queue
queue.async {
self.downloadPhoto(photo: photo) { (success) in
nextQueue?.activate()
if success {
...
} else {
...
}
}
}
Ant other solution to my problem is great too. Thanks.
There are several options:
Historically we’d use a custom, asynchronous, Operation subclass. See https://stackoverflow.com/a/38469350/1271826 (or see https://stackoverflow.com/a/48104095/1271826 for alternate AsynchronousOperation implementation). That gives us a cancelable operation and we can easily control the degree of concurrency.
If you want to download them one at a time, you could set the queue’s maxConcurrentOperationCount to 1. Then you can just add your download operations to a single OperationQueue, and if you want to cancel them, you’d call queue.cancelAllOperations().
If you are targeting iOS 13 and later, Combine makes this even easier. See https://stackoverflow.com/a/61195234/1271826 for example of downloading a series of images using Combine (both sequentially as well as with a controlled degree of concurrency).
All of this having been said, I’d encourage you to reconsider the choice to download the images sequentially. In my random test, increasing the max allowed concurrency to 6 resulted in downloads that were, overall, twice as fast as the serial behavior.
I have a singleton that manages an array. This singleton can be accessed from multiple threads, so it has its own internal DispatchQueue to manage read/write access across threads. For simplicity we'll say it's a serial queue.
There comes a time where the singleton will be reading from the array and updating the UI. How do I handle this?
Which thread my internal dispatch queue is not known, right? It's just an implementation detail I'm to not worry about? In most cases this seems fine, but in this one specific function I need to be sure it uses the main thread.
Is it okay to do something along the lines of:
myDispatchQueue.sync { // Synchronize with internal queue to ensure no writes/reads happen at the same time
DispatchQueue.main.async { // Ensure that it's executed on the main thread
for item in internalArray {
// Pretend internalArray is an array of strings
someLabel.text = item
}
}
}
So my questions are:
Is that okay? It seems weird/wrong to be nesting dispatch queues. Is there a better way? Maybe something like myDispatchQueue.sync(forceMainThread: true) { ... }?
If I DID NOT use DispatchQueue.main.async { ... }, and I called the function from the main thread, could I be sure that my internal dispatch queue will execute it on the same (main) thread as what called it? Or is that also an "implementation detail" where it could be, but it could also be called on a background thread?
Basically I'm confused that threads seem like an implementation detail you're not supposed to worry about with queues, but what happens on the odd chance when you DO need to worry?
Simple example code:
class LabelUpdater {
static let shared = LabelUpdater()
var strings: [String] = []
private let dispatchQueue: dispatchQueue
private init {
dispatchQueue = DispatchQueue(label: "com.sample.me.LabelUpdaterQueue")
super.init()
}
func add(string: String) {
dispatchQueue.sync {
strings.append(string)
}
}
// Assume for sake of example that `labels` is always same array length as `strings`
func updateLabels(_ labels: [UILabel]) {
// Execute in the queue so that no read/write can occur at the same time.
dispatchQueue.sync {
// How do I know this will be on the main thread? Can I ensure it?
for (index, label) in labels.enumerated() {
label.text = strings[index]
}
}
}
}
Yes, you can nest a dispatch to one queue inside a dispatch to another queue. We frequently do so.
But be very careful. Just wrapping an asynchronous dispatch to the main queue with a dispatch from your synchronizing queue is insufficient. Your first example is not thread safe. That array that you are accessing from the main thread might be mutating from your synchronization queue:
This is a race condition because you potentially have multiple threads (your synchronization queue’s thread and the main thread) interacting with the same collection. Rather than having your dispatched block to the main queue just interact objects directly, you should make a copy of of it, and that’s what you reference inside the dispatch to the main queue.
For example, you might want to do the following:
func process(completion: #escaping (String) -> Void) {
syncQueue.sync {
let result = ... // note, this runs on thread associated with `syncQueue` ...
DispatchQueue.main.async {
completion(result) // ... but this runs on the main thread
}
}
}
That ensures that the main queue is not interacting with any internal properties of this class, but rather just the result that was created in this closure passed to syncQueue.
Note, all of this is unrelated to it being a singleton. But since you brought up the topic, I’d advise against singletons for model data. It’s fine for sinks, stateless controllers, and the like, but not generally advised for model data.
I’d definitely discourage the practice of initiating UI controls updates directly from the singleton. I’d be inclined to provide these methods completion handler closures, and let the caller take care of the resulting UI updates. Sure, if you want to dispatch the closure to the main queue (as a convenience, common in many third party API), that’s fine. But the singleton shouldn’t be reaching in and update UI controls itself.
I’m assuming you did all of this just for illustrative purposes, but I added this word of caution to future readers who might not appreciate these concerns.
Try using OperationQueues(Operations) as they do have states:
isReady: It’s prepared to start
isExecuting: The task is currently running
isFinished: Once the process is completed
isCancelled: The task canceled
Operation Queues benefits:
Determining Execution Order
observe their states
Canceling Operations
Operations can be paused, resumed, and cancelled. Once you dispatch a
task using Grand Central Dispatch, you no longer have control or
insight into the execution of that task. The NSOperation API is more
flexible in that respect, giving the developer control over the
operation’s life cycle
https://developer.apple.com/documentation/foundation/operationqueue
https://medium.com/#aliakhtar_16369/concurrency-in-swift-operations-and-operation-queue-part-3-a108fbe27d61
I have cells that have buttons that trigger the downloading of their respective PDF from online. I want it so that only one download can occur at a time, and the other ones (if their button is clicked) wait for it to finish.
I cannot use any sort of queue, because the queue operation calls the download methods but does not wait for them to complete before moving on.
Is there any way that I can only move on once the did finish download function says that it is ready by passing a boolean or something? I am pretty lost here so any direction is greatly appreciated.
I cannot use any sort of queue, because the queue operation calls the download methods but does not wait for them to complete before moving on.
This can be accomplished using NSOperation Queues. The key is that your download tasks have to be async NSOperation subclasses where you mark the operation as finished when the download finishes. More importantly, these operations should be queued on a serial queue. Then, operations will be executed only one at a time in FIFO order.
However, it takes a bit of boilerplate to get NSOperations setup this way. Another good way to do it is using Dispatch Groups.
// A serial queue ensures only one operation is executed at a time, FIFO
let downloadsQueue = dispatch_queue_create("com.youapp.pdfdownloadsqueue", DISPATCH_QUEUE_SERIAL)
let downloadGroup = dispatch_group_create()
func queueDownload(from url: NSURL) {
// Register this download task with the group
dispatch_group_enter(downloadGroup)
// Async dispatch the download task to our serial queue,
// so that it returns control back without blocking the main thread
dispatch_async(downloadsQueue) {
downloadPDF(with: url) { (pdf, error) in
// handle PDF data / error
// { .. }
// leave the dispatch group in the completion method,
// notifying the group that this task is finished
dispatch_group_leave(downloadGroup)
}
}
}
func downloadPDF(with url: NSURL, completion: (pdf: NSData?, error: ErrorType?) -> ()) {
// make network request
// call completion with PDF data or error when the download request returns
}