I have a handful of production crashes on iOS that look like this:
Crashed: com.superdupertango.Server.CallbackRunQueue
0 SDTServer 0x102b78ed0 specialized Dictionary.subscript.getter + 4378562256 (<compiler-generated>:4378562256)
1 SDTServer 0x102a22fec closure #1 in Server.processError(_:) + 4377161708 (<compiler-generated>:4377161708)
2 SDTServer 0x10257ce90 thunk for #escaping #callee_guaranteed () -> () + 4372287120 (<compiler-generated>:4372287120)
This is the offending code (all code in this area is Swift5):
private let callbackRunQueue = DispatchQueue(label: "com.superdupertango.Server.CallbackRunQueue", qos: DispatchQoS.userInitiated, target: nil)
var bodyCompletionBlocks: [String: ((Data?, Error?, String) -> Void)] = [:]
...
private init() {
NotificationCenter.default.addObserver(self, selector: #selector(self.processError(_:)), name: Notification.Name.SDTServerWriteFailed, object: nil)
}
...
#objc private func processError(_ notification: Notification) {
guard let userInfo = notification.userInfo, let requestId = userInfo["requestId"] as? String else {
return
}
self.callbackRunQueue.async {
DLog("Server.processError (\(requestId)): Got Error for request. Shutting down request.")
guard let bodyCompletionBlock = self.bodyCompletionBlocks[requestId] else {
DLog("Server.processError (\(requestId)): Failed getting body completion block. Returning")
return
}
bodyCompletionBlock(Data(), nil, requestId)
self.bodyCompletionBlocks.removeValue(forKey: requestId)
self.initialCompletionBlocks.removeValue(forKey: requestId)
}
}
Note that the callbackRunQueue is a serial queue.
The only dictionary value that's being retrieved inside the callbackRunQueue is the self.bodyCompletionBlocks:
guard let bodyCompletionBlock = self.bodyCompletionBlocks[requestId] else {
I've found a few references to this error, but they all say that this may be a multi-thread access issue.
However, in my code, there are only 3 places where there is access to self.bodyCompletionBlocks, and they are all within callbackRunQueue.async or callbackRunQueue.sync blocks. Also note that some of this code is running inside threads within GCDWebServer, but as I mentioned, I'm always ensuring the code runs in my callbackRunQueue queue, so I don't think this is GCDWebServer related.
I thought that enclosing thread sensitive code inside a serial queue's async or sync blocks would protect against multi-thread access problems like this.
Any ideas?
Thanks!
Related
Does the thread from which a method is being used matter in terms of completionHandlers? When running on the Main thread, I have no problems; the (data) in completionHandler is called and thus the function's completionHandler within that.
The following is my code:
static func getInfo(user: User,completion: (([String:String]) -> Void)? = nil){
print(user.apiUid)
var enrolledCourses: [String:String] = [:]
NetworkingManager.staticGeneralRequest(withEndpoint: someURL, oauthToken: user.oauth_token, oauthTokenSecret: user.oauth_token_secret) { (data) in
let decodedData: [String:String] = data.getData()
completion?(decodedData)
}
}
//NOTE: According to the compiler,
//the use of an optional completionHandler means that it's `#escaping` by default.
Is this an issue pretaining to threading, and what are some best practices to ensure that code works well across threads?
Your closure was deallocated before you get the response. Adding #escaping like this: static func getInfo(user: User, completion: #escaping (([String:String]) -> Void)? = nil) to keep it in scope.
Swift closures strongly capture reference types.
DispatchGroup is a reference type.
My questions have to do with the following code:
func getUsername(onDone: #escaping (_ possUsername: String?) -> ())
{
//Post request for username that calls onDone(retrievedUsername)...
}
func getBirthdate(using username: String?, onDone: #escaping (_ possBday: String?) -> ())
{
//Post request for token that calls onDone(retrievedToken)...
}
func asyncTasksInOrder(onDone: #escaping (_ resultBDay: String?) -> ())
{
let thread = DispatchQueue(label: "my thread", qos: .userInteractive, attributes: [],
autoreleaseFrequency: .workItem, target: nil)
thread.async { [weak self, onDone] in
guard let self = self else {
onDone(nil)
return
}
let dg = DispatchGroup() //This is a reference type
var retrievedUsername: String?
var retrievedBday: String?
//Get username async first
dg.enter()
self.getUsername(onDone: {[weak dg](possUsername) in
retrievedUsername = possUsername
dg?.leave() //DG is weak here
})
dg.wait()
//Now that we've waited for the username, get bday async now
dg.enter()
self.getBirthdate(using: retrievedUsername, onDone: {[weak dg](possBday) in
retrievedBday = possBday
dg?.leave() //DG is also weak here
})
dg.wait()
//We've waited for everything, so now call the return callback
onDone(retrievedBday)
}
}
So the two closures inside of asyncTasksInOrder(onDone:) each capture dg, my DispatchGroup.
Is it even necessary to capture my dispatch group?
If I don't capture it, how would I even know I've got a retain cycle?
What if the dispatch group evaporates during one of the callback executions? Would it even evaporate since it's waiting?
Is it unnecessarily expensive to instantiate a DispatchQueue like this often (disregarding the .userInteractive)? I'm asking this particular question because spinning up threads in Android is extremely expensive (so expensive that JetBrains has dedicated lots of resources towards Kotlin coroutines).
How does dg.notify(...) play into all of this? Why even have a notify method when dg.wait() does the same thing?
I feel like my understanding of GCD is not bulletproof, so I'm asking to build some confidence. Please critique as well if there's anything to critique. The help is truly appreciated.
1) No, the dispatch group is captured implicitly. You don't even need to capture self in async because GCD closures don't cause retain cycles.
2) There is no retain cycle.
3) Actually you are misusing DispatchGroup to force an asynchronous task to become synchronous.
4) No, GCD is pretty lightweight.
5) The main purpose of DispatchGroup is to notify when all asynchronous tasks – for example in a repeat loop – are completed regardless of the order.
A better solution is to nest the asynchronous tasks. With only two tasks the pyramid of doom is manageable.
func asyncTasksInOrder(onDone: #escaping (String?) -> Void)
{
let thread = DispatchQueue(label: "my thread", qos: .userInteractive, autoreleaseFrequency: .workItem)
thread.async {
//Get username async first
self.getUsername { [weak self] possUsername in
guard let self = self else { onDone(nil); return }
//Now get bday async
self.getBirthdate(using: possUsername) { possBday in
//Now call the return callback
onDone(possBday)
}
}
}
}
I have an Operation subclass and Operation queue with maxConcurrentOperationCount = 1.
This performs my operations in a sequential order that i add them which is good but now i need to wait until all operations have finished before running another process.
i was trying to use notification group but as this is run in a for loop as soon as the operations have been added to the queue the notification group fires.. How do i wait for all operations to leave the queue before running another process?
for (index, _) in self.packArray.enumerated() {
myGroup.enter()
let myArrayOperation = ArrayOperation(collection: self.outerCollectionView, id: self.packArray[index].id, count: index)
myArrayOperation.name = self.packArray[index].id
downloadQueue.addOperation(myArrayOperation)
myGroup.leave()
}
myGroup.notify(queue: .main) {
// do stuff here
}
You can use operation dependencies to initiate some operation upon the completion of a series of other operations:
let queue = OperationQueue()
let completionOperation = BlockOperation {
// all done
}
for object in objects {
let operation = ...
completionOperation.addDependency(operation)
queue.addOperation(operation)
}
OperationQueue.main.addOperation(completionOperation) // or, if you don't need it on main queue, just `queue.addOperation(completionOperation)`
Or, in iOS 13 and later, you can use barriers:
let queue = OperationQueue()
for object in objects {
queue.addOperation(...)
}
queue.addBarrierBlock {
DispatchQueue.main.async {
// all done
}
}
A suitable solution is KVO
First before the loop add the observer (assuming queue is the OperationQueue instance)
queue.addObserver(self, forKeyPath:"operations", options:.new, context:nil)
Then implement
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
if object as? OperationQueue == queue && keyPath == "operations" {
if queue.operations.isEmpty {
// Do something here when your queue has completed
self.queue.removeObserver(self, forKeyPath:"operations")
}
} else {
super.observeValue(forKeyPath: keyPath, of: object, change: change, context: context)
}
}
Edit:
In Swift 4 it's much easier
Declare a property:
var observation : NSKeyValueObservation?
and create the observer
observation = queue.observe(\.operationCount, options: [.new]) { [unowned self] (queue, change) in
if change.newValue! == 0 {
// Do something here when your queue has completed
self.observation = nil
}
}
Since iOS13 and macOS15 operationCount is deprecated. The replacement is to observe progress.completedUnitCount.
Another modern way is to use the KVO publisher of Combine
var cancellable: AnyCancellable?
cancellable = queue.publisher(for: \.progress.completedUnitCount)
.filter{$0 == queue.progress.totalUnitCount}
.sink() { _ in
print("queue finished")
self.cancellable = nil
}
I use the next solution:
private let queue = OperationQueue()
private func addOperations(_ operations: [Operation], completionHandler: #escaping () -> ()) {
DispatchQueue.global().async { [unowned self] in
self.queue.addOperations(operations, waitUntilFinished: true)
DispatchQueue.main.async(execute: completionHandler)
}
}
Set the maximum number of concurrent operations to 1
operationQueue.maxConcurrentOperationCount = 1
then each operation will be executed in order (as if each was dependent on the previous one) and your completion operation will execute at the end.
Code at the end of the queue
refer to this link
NSOperation and NSOperationQueue are great and useful Foundation framework tools for asynchronous tasks. One thing puzzled me though: How can I run code after all my queue operations finish? The simple answer is: use dependencies between operations in the queue (unique feature of NSOperation). It's just 5 lines of code solution.
NSOperation dependency trick
with Swift it is just easy to implement as this:
extension Array where Element: NSOperation {
/// Execute block after all operations from the array.
func onFinish(block: () -> Void) {
let doneOperation = NSBlockOperation(block: block)
self.forEach { [unowned doneOperation] in doneOperation.addDependency($0) }
NSOperationQueue().addOperation(doneOperation)
}}
My solution is similar to that of https://stackoverflow.com/a/42496559/452115, but I don't add the completionOperation in the main OperationQueue but into the queue itself. This works for me:
var a = [Int](repeating: 0, count: 10)
let queue = OperationQueue()
let completionOperation = BlockOperation {
print(a)
}
queue.maxConcurrentOperationCount = 2
for i in 0...9 {
let operation = BlockOperation {
a[i] = 1
}
completionOperation.addDependency(operation)
queue.addOperation(operation)
}
queue.addOperation(completionOperation)
print("Done 🎉")
I am taking user input to download files from the server. The downloading task may include requesting web services.
I am expecting something like this:
1) Whenever the user selects a file to download or requesting for web
service, then it should be treated as one block of operation or task
and should go in the queue which will be managed globally at the app
level.
2) At the same time if the queue is empty then it should
automatically start executing the current task.
3) If queue contains
any operation then it should execute all old operation in
synchronously then execute the last one.
Can any one suggest how this can be done by the optimized way?
Take a look what I tried:
class func downloadChaptersFromDownloadQueue() {
let gbm = GlobalMethods()
for chapterDetail in gbm.downloadOpertationQueue.array.enumerated() {
if chapterDetail.element.chapterdata.state == .non || chapterDetail.element.chapterdata.state == .paused || chapterDetail.element.chapterdata.state == .downloading {
gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .downloading
let s = DispatchSemaphore(value: 0)
self.downloadActivty(courseId: chapterDetail.element.courseId, mod: chapterDetail.element.chapterdata, selectedIndexpath: chapterDetail.element.cellForIndexpath, success: { (result) in
if (result) {
if (WC_SQLite.shared.updateChapterState(courseId: chapterDetail.element.courseId, chapterId: chapterDetail.element.chapterdata.id, state: .downloaded)) {
s.signal()
gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .downloaded
NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": 1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
}
else {
s.signal()
gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .non
NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": -1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
}
}
else {
_ = WC_SQLite.shared.updateChapterState(courseId: chapterDetail.element.courseId, chapterId: chapterDetail.element.chapterdata.id, state: .non)
s.signal()
gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .non
NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": -1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
}
})
s.wait()
}
}
}
Create an async queue. First use a dispatch group to track how many requests are completed and get notified when all are finished (completely async).
Next, enqueue ALL your requests. Each request should have a unique identifier so you know which request completed or failed (the chapterId & pageNumber in your case should be enough).
Execute all the requests at once (again it is async) and you will be notified when each one completes (on the main queue through your completion block). The completion block should be called with all the requests responses and their unique identifiers.
Example:
class NetworkResponse {
let data: Data?
let response: URLResponse?
let error: Error?
init(data: Data?, response: URLResponse?, error: Error?) {
self.data = data
self.response = response
self.error = error
}
}
class NetworkQueue {
static let instance = NetworkQueue()
private let group = DispatchGroup()
private let lock = DispatchSemaphore(value: 0)
private var tasks = Array<URLSessionDataTask>()
private var responses = Dictionary<String, NetworkResponse>()
private init() {
}
public func enqueue(request: URLRequest, requestID: String) {
//Create a task for each request and add it to the queue (we do not execute it yet). Every request that is created, we enter our group.
self.group.enter();
let task = URLSession.shared.dataTask(with: request) { (data, response, error) in
//Only one thread can modify the array at any given time.
objc_sync_enter(self)
self.responses.updateValue(NetworkResponse(data: data, response: response, error: error), forKey: requestID)
objc_sync_exit(self)
//Once the request is complete, it needs to leave the group.
self.group.leave()
}
//Add each task to the queue.
self.tasks.append(task)
}
public func execute(completion: #escaping (_ responses: Dictionary<String, NetworkResponse>) -> Void) {
//Get notified on the main queue when every single request is completed (they all happen asynchronously, but we get one notification)
self.group.notify(queue: DispatchQueue.main) {
//Call our completion block with all the responses. Might be better to use a sorted dictionary or something here so that the responses are in order.. but for now, a Dictionary with unique identifiers will be fine.
completion(self.responses)
}
//Execute every task in the queue.
for task in self.tasks {
task.resume()
}
//Clear all executed tasks from the queue.
self.tasks.removeAll()
}
}
EDIT (Using your own code):
class func downloadChaptersFromDownloadQueue() {
let gbm = GlobalMethods()
let group = DispatchGroup()
let lock = NSLock()
//Get notified when ALL tasks have completed.
group.notify(queue: DispatchQueue.main) {
print("FINISHED ALL TASKS -- DO SOMETHING HERE")
}
//Initially enter to stall the completion
group.enter()
defer {
group.leave() //Exit the group to complete the enqueueing.
}
for chapterDetail in gbm.downloadOpertationQueue.array.enumerated() {
if chapterDetail.element.chapterdata.state == .non || chapterDetail.element.chapterdata.state == .paused || chapterDetail.element.chapterdata.state == .downloading {
gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .downloading
//Enter the group for each downloadOperation
group.enter()
self.downloadActivty(courseId: chapterDetail.element.courseId, mod: chapterDetail.element.chapterdata, selectedIndexpath: chapterDetail.element.cellForIndexpath, success: { (result) in
lock.lock()
defer {
group.leave() //Leave the group when each downloadOperation is completed.
}
if (result) {
if (WC_SQLite.shared.updateChapterState(courseId: chapterDetail.element.courseId, chapterId: chapterDetail.element.chapterdata.id, state: .downloaded)) {
gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .downloaded
lock.unlock()
NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": 1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
}
else {
gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .non
lock.unlock()
NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": -1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
}
}
else {
_ = WC_SQLite.shared.updateChapterState(courseId: chapterDetail.element.courseId, chapterId: chapterDetail.element.chapterdata.id, state: .non)
gbm.downloadOpertationQueue[chapterDetail.offset].chapterdata.state = .non
lock.unlock()
NotificationCenter.default.post(name: NSNotification.Name(("DownloadChapter")), object: self, userInfo: ["progress": -1.0, "notifIdentifier":(chapterDetail.element.cellForIndexpath)])
}
})
}
}
}
Again, this is asynchronous because you don't want the user waiting forever to download 100 pages..
For tasks like these the first thing you need to do is to do them asynchronously using dispatch_async, so that they are on a different thread and don't impact the performance of the application (or freeze it.)
Whenever your downloads succeeds/fails, you can always control what happens next in it's completion block. (I suggest you use recursion for what you're trying to achieve as it suits your needs).
Hope this helps!
My build completes successfully. Then after some time this error pops up:
"Ambiguous use of 'addObjectsDidChangeNotificationObserver(handler:)'"
What I do not understand is why this happens because the addObjectsDidChangeNotificationObserver method is only declared once in the project and the second occurence shown by Xcode is the use of the method itself.
Here is the code where the error is shown and which Xcode also shows me as first candidate:
public init?(object: Managed, changeHandler: #escaping (ChangeType) -> ()) {
guard let moc = object.managedObjectContext else { return nil }
objectHasBeenDeleted = !type(of: object).defaultPredicate.evaluate(with: object)
token = moc.addObjectsDidChangeNotificationObserver(handler: {
[unowned self] note in
guard let changeType = self.changeType(of: object, in: note) else { return }
self.objectHasBeenDeleted = changeType == .delete
changeHandler(changeType)
})
}
and the implementation of addObjectsDidChangeNotificationObserver(), which Xcode shows me as second candidate:
extension NSManagedObjectContext {
public func addObjectsDidChangeNotificationObserver(handler: #escaping (ObjectsDidChangeNotification) -> ()) -> NSObjectProtocol {
let nc = NotificationCenter.default
return nc.addObserver(forName: .NSManagedObjectContextObjectsDidChange, object: self, queue: nil) { note in
let wrappedNote = ObjectsDidChangeNotification(note: note)
handler(wrappedNote)
}
}
}
Ok, the problem seems to be solved now.
Apparently, I messed up with the access modifiers, but good to know that something like that can cause an ambiguous error