So I'm beginning to learn the basics of Grand Central Dispatch and the whole concept of multithreading with iOS applications. Every tutorial will tell you that you must run UI events on the main thread, but I don't completely understand why.
Here's a problem I came across yesterday, and finally fixed it by running a segue on the main thread, but I still don't understand why running it off the main thread was a problem:
I had a custom initial VC (barcode scanner) and a segue to a new view controller with a UIWebView attached. As soon as the VC found a barcode, it called a handler, and in that closure, I had a performSegueWithIdentifier. However, I got a EXC_BAD_ACCESS because of this (it didn't happen when the second VC had a label or a UIImageView, just with UIWebView). I finally realized that for some reason, the closure was called off the main thread, and thus the segue was being performed off the main thread. Why exactly would performing the segue on another thread throw a memory error? Is it because self in self.performSegueWithIdentifier was somehow nil? And why wouldn't Swift automatically dispatch a segue event on the main thread?
Interesting question! The crash isn't related to UIKit. It's a crash specific to UIWebView. Looking at the stack trace, the exception happens in the WebCore::FloatingPointEnvironment::saveMainThreadEnvironment function, which is part of the WebKit init process. Since WebKit manages a threaded execution environment of its own, it makes sense that it needs a definite starting point (i.e. the main thread) to build this environment.
UIKit operations (like presenting a view controller) performed on threads other than main will not cause an exception, but they will be delayed (depending on the QoS of the dispatching queue).
As for why the UIKit operations aren't automatically dispatched on the main queue, I can only speculate that adding extra checks inside the library calls would add too much redundant work that can be avoided simply by following a convention.
For a larger discussion on UIKit and the main thread, see this answer: Why must UIKit operations be performed on the main thread?
The short answer is that all operations that modify the UI of your app need to come together in one place to be evaluated to generate the next frame at regular intervals (the V-Sync interval specifically). Keeping track of all of the mutated state requires all changes to happen synchronously, and for performance reasons, all of these operations are generally batched up and executed once per frame (while also coordinating with the GPU).
Making things thread-safe is difficult, and has major performance implications, all of which can be avoided by requiring access to come from a single (main) thread. You want your UI frameworks to be as fast (i.e. responsive) as possible. If you declare that all your UI objects are main-thread only, then you don't have to slow them down with a bunch of synchronization overhead. I've heard it asked, 'if UIKit is really main-thread only, then why don't they just check the current thread and abort() if it's not the main thread?' and the answer is the same: Even a simple check like that would result in an appreciable performance hit given the number of methods that would need the check and the frequency with which they're called. It's all about speed.
Also, remember, not too long ago, all applications were single threaded.
Related
Why is it the responsibility of the programmer to call UI related methods on the main thread with:
DispatchQueue.main.async {}
Theoretically, couldn’t this be left up to the compiler or some other agent to determine?
The actual answer is developer inertia and grandfathering.
The Cocoa UI API is huge—nay, gigantic. It has also been in continuous development since the 1990's.
Back when I was a youth and there were no multi-core, 64-bit, anything, 99.999% of all applications ran on the main thread. Period. (The original Mac OS, pre-OS X, didn't even have threads.)
Later, a few specialized tasks could be run on background threads, but largely apps still ran on the main thread.
Fast forward to today where it's trivial to dispatch thousands of tasks for background execution and CPUs can run 30 or more current threads, it's easy to say "hey, why doesn't the compiler/API/OS handle this main-thread thing for me?" But what's nigh on impossible is re-engineering four decades of Cocoa code and apps to make that work.
There are—I'm going to say—hundreds of millions of lines of code that all assume UI calls are executing concurrently on the main thread. As others have pointed out, there is no cleaver switch or pre-processor that's going to undo all of those assumptions, fix all of those potential deadlocks, etc.
(Heck, if the compiler could figure this kind of stuff out we wouldn't even have to write multi-threaded code; you'd just let the compiler slice up your code so it runs concurrently.)
Finally, such a change just isn't worth the effort. I do Cocoa development full time and the number of times I have to deal with the "update control from a background thread problem" occurs, at most, once a week or so. There's no development cost-benefit analysis that's going to dedicate a million man-hours to solving a problem that already has a straight forward solution.
Now if you were developing a new, modern, UI API from scratch, you'd simply make the entire UI framework thread safe and whole question goes away. And maybe Apple has a brand new, redesigned-from-the-ground-up, UI framework in a lab somewhere that does that. But that's the only way I see something like this happening.
You would be substituting one kind of frustration for another.
Suppose that all UI-related methods that require invocation on the main thread did so by:
using DispatchQueue.main.async: You would be hiding asynchronous behaviour, with no obvious way to "follow up" on the result. Code like this would now fail:
label.text = "new value"
assert(label.text == "new value")
You would have thought that the property text just harmlessly assigned some value. In fact, it enqueued a work item to asynchronously execute on the main thread. In doing so, you've broken the expectation that your system has reached its desired state by the time you've completed that line.
using DispatchQueue.main.sync: You would be hiding a potential for deadlock. Synchronous code on the main queue can be very dangerous, because it's easy to unintentionally block (on the main thread) yourself waiting for such work, causing deadlock.
I think one way this could have been achieved is by having a hidden thread dedicated to UI. All UI-related APIs would switch to that thread to do their work. Though I don't know how expensive that would be (each switch to that thread is probably no faster than waiting on a lock), and I could imagine there's lots of "fun" ways that'll get you to write deadlocking code.
Only on rare instances would the UI call anything in the main thread, except for user login timeouts for security. Most UI related methods for any particular window are called within the thread that was started when the window was initialized.
I would rather manage my UI calls instead of the compiler because as a developer, I want control and do not want to rely on third party 'black boxes'.
check https://developer.apple.com/documentation/code_diagnostics/main_thread_checker
and UPDATE UI FROM MAIN THREAD ONLY!!!
I'm developing a sort of fast image scan application.
In the -captureOutput:didOutputSampleBuffer:fromConnection: method I pick up the CVPixelBuffer and add them into an NSOperation subclass. The NSOperation takes the buffer and transform it into an image saved on the filesystem.
The -isConcurrent method of NSOperation returns YES. After the operation is created, is added to an NSOperationQueue.
Everything runs fine except for one issue that is affecting the scan frame rate.
Using time profiler I've found that some of the NSOperation are run on the same thread of the AVCaputureVideoOutput delegate that I've created:
dispatch_queue_t queue = dispatch_queue_create("it.CloudInTouchLabs.avsession", DISPATCH_QUEUE_SERIAL);
[dataOutput setSampleBufferDelegate:(id)self queue:queue];
When an operation is running on the same thread of the AV session queue, it affects the frame rate, it happens only with some them, probably GCD is deciding under the hood dispatching on active threads. The only method I've found to solve that issue, is to create a separate thread and pass it to the the single operations forcing them to run on it.
Is there another way, to force operations run on a different thread?
[EDIT]
Following the #impcc suggestions I made some test.
Results are pretty interesting even if sort of inconsistent.
Test platform is an iPhone 5 connected in debugging mode through an MBP with 16gb of RAM quad core i7. The session has a 60fps output tested with the algorithm in RosyWriter apple sample code.
AVSession queue sync targeted to high priority queue
26 fps where sometimes the thread of the queue is shared with the one
of the operation. The time taken inside the delegate methods has an
average of 0.02s
14 fps with a thread created just for the operations, if the main
method is not called on this thread, the start will be forced to perform
on that specific thread. This thread is create one time and kept
alive with a dummy port.. The time taken inside the delegate is
0.008.
AVSession queue concurrent targeted to high priority queue
13.5 fps where sometimes the thread of the queue is shared with the one of the operation. The time taken inside the delegate methods has
an average of 0.02s, substacially equal to the counterpart with sync
queue.
14 fps with a thread created just for the operations, if the main
method is not called on this thread, the start will be forced to perform
on that specific thread. This thread is create one time and kept
alive with a dummy port. The time taken inside the delegate is
0.008.
Conclusion
Concurrent or serial queue doesn't seems to make a big difference, however concurrent for me is not ok cause of the timestamps that I need to take to preserve the sequence of single picture. The fact that is surprising me is the frame drop using an ad-hoc thread, even if the time taken inside the delegate method is considerably less, the frame rate drops about 10fps. Just trusting in GCD the frame rate is better but the delegate methods takes more than 2 times to finish, this probably due to the fact that sometimes the AV queue is used by also the nsoperations. I can't really get why the time taken inside the delegate doesn't seems to be correlated to the fps. Shouldn't be the faster the better?
By further investigations it really seems that what is stealing time si in the process of adding and probably executing operation in queue.
I think you might be misunderstanding the meaning of isConcurrent. This is totally understandable, since it's incredibly poorly named. When you return YES from -isConcurrent what it really means is "I will handle any concurrency needs associated with this operation, and will otherwise behave asynchronously." In this situation, NSOperationQueue is free to call -start on your operation synchronously on the thread from which you add the operation. The NSOperationQueue is expecting that, because you've declared that you will manage your own concurrency, that your -start method will simply kick off an asynchronous process and return right away. I suspect this is the source of your problem.
If you implemented your operation by overriding -main then you almost certainly want to be returning NO from isConcurrent. To make matters more complicated, the behaviors associated with isConcurrent have changed over the years (but all that is covered in the official docs.) A fantastic explanation of how to properly implement a returns-YES-from-isConcurrent NSOperation can be found here.
The way I'm reading your question here, it doesn't sound like your NSOperation subclass actually needs to "manage its own concurrency", but rather that you simply want it to execute asynchronously, "on a background thread", potentially "concurrently" with other operations.
In terms of assuring the best performance for your AVCaptureVideoDataOutputSampleBufferDelegate callbacks, I would suggest making the queue you pass in to -setSampleBufferDelegate:queue: be (itself) concurrent and that it target the high-priority global concurrent queue, like this:
dispatch_queue_t queue = dispatch_queue_create("it.CloudInTouchLabs.avsession", DISPATCH_QUEUE_CONCURRENT);
dispatch_set_target_queue(queue, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0));
[dataOutput setSampleBufferDelegate:(id)self queue:queue];
Then you should make the delegate callback methods as lightweight as possible -- nothing more than packaging up the information you need to make the NSOperation and add it to the NSOperationQueue.
This should ensure that the callbacks always take precedence over the NSOperations. (It's my understanding that NSOperationQueue targets either the main queue (for the NSOperationQueue that's associated with the main thread and run loop) or the default priority background queue.) This should allow your callbacks to fully keep up with the frame rate.
Another important thing to take away here (that another commenter was getting at) is that if you're doing all your concurrency using GCD, then there is only one special thread -- the main thread. Other than that, threads are just a generic resource that GCD can (and will) use interchangeably with one another. The fact that a thread with a thread ID of X was used at one point to service your delegate callback, and at another point was used to do your processing is not, in and of itself, indicative of a problem.
Hope this helps!
The app I am working on fetches a bunch of different newsfeeds when it first starts up and updates any expired ones. While this is happening the interface often freezes up and you can't click anything. The actual network calls are being done on a separate thread, but the database operations are being done on the main thread. Would this cause the interface to freeze?
I have been told that I need to make it to where only two feeds to update are inserted into the network operation queue at a time so that it won't try all of them at once, but it's already set up to only do so many network calls at once. I don't understand how having less things in a queue at a time would cause it to go faster if they're just going to be put in there sequentially anyways. Please correct me if I am wrong, I'm still pretty new to this.
Any kind of help regarding what could cause the UI to freeze up during startup like this would be much appreciated!
It is always a good idea to move time consuming operation away from the main thread.
Fortunately it is pretty simple to do on iOS. If the time-consuming task is fairly simple you could consider using performSelectorInBackground
e.g:
[self performSelectorInBackground:#selector(myFunction:)
withObject:myParam];
It is however important to remberber, that you must not access the GUI from the background thread. To get objects back to the main thread use performSelectorOnMainThread
e.g:
[self performSelectorOnMainThread:#selector(myFunction:) myParamwaitUntilDone:YES];
Try applying this strategy to your database calls. Depending on your scenario you might want to wrap it up in a NSOperation or use a Thread when the cause of the freeze is found.
We can have many handlers: touches handler, UIControl handler (buttons, sliders), performSelector, CADisplayLink, NSTimer events, Gesture Recognizer, accelerometer handler, and UIView animation completion block, and some other ones.
Are all of them in the same thread? That is, only one of them can be running at the same time?
Can some other method or handler be part of another thread and therefore can create race conditions?
In general, you'll find that most simple applications on iOS tend to perform almost every action on the main thread. As you noted, the instant that you bring multithreading into the picture you add another set of tricky issues to watch out for. Many developers don't want to bother with this added complexity, or are unfamiliar with GCD or threading in general, so they avoid doing anything on a background thread or GCD queue.
Several of the items you list in your question involve interactions with UIKit, and in general those interactions must occur on the main thread (iOS 4.x added the ability to perform some drawing functions in the background, though). You receive touch and other related events on the main thread. If you wish to update most aspects of an interface, the safe way to do that is by performing these updates on the main thread.
Timers (NSTimer, CADisplayLink) can have their updates be fired on a background thread by attaching them to an NSRunLoop operating on that background thread. You rarely see people do this, but it can be done. Usually, people configure timers on the main run loop, which causes callbacks to be delivered on the main thread.
When performing animations, the animations themselves will run on a background thread (you see that they don't stop while you're blocking the main thread with something else), but you'll almost always receive a completion block or callback on the main thread when they're done. If I remember correctly, there are one or two exceptions to this and they are noted as such in Apple's documentation. Having these callbacks trigger on the main thread is a safe approach when dealing with developers who might not realize what's going on behind the scenes.
All that said, there are very good reasons to want to add multithreading to your application. Because all user interface updates and touch interactions occur on the main thread, if you have something that is computationally expensive or that simply will take a lot of time to perform, if you run this on your main thread you'll appear to have frozen your application. This is a terrible user experience, so you want to move this task onto a background thread so that the user can keep interacting with your application while this is going on. Additionally, more and more iOS devices are shipping every day with multiple cores in them, and balancing your work load across these cores and being efficient with this hardware requires some degree of concurrent processing.
People have written books about best practices when making code multithreaded, and you can find a lot of questions about this here, so I won't go into too much detail. What I can tell you is that you should read Apple's Concurrency Programming Guide and watch the WWDC videos from the last two years that deal with Grand Central Dispatch. With GCD, Apple has made it a lot easier to add multithreading to your application in an efficient and (relatively) safe manner, and I encourage you to look into this for your own applications.
For example, I have an open source iOS application that performs detailed rendering of molecular structures. I render each frame on a background GCD queue because sometimes they take more than 1/60th of a second to process, and in those cases they'd cause touch events to be dropped and the interface to stutter if this was all on the main thread. Additionally, I've seen up to a 40% performance boost by doing this when running on the newer multicore devices. To avoid race conditions, I wrap interactions with shared data structures and contexts in serial dispatch queues so that only one action can be using a resource at a time, no matter what thread a particular block is running on. This only required the addition of a few lines of code, but the performance and user experience benefits were huge.
I am working on an app, which uploads native contacts to server then get responses(JSON, a contact list that already installed the app). When native contacts are large enough, server response will be slow and unstable. And user cannot do other things. so I put network request into background thread. every time I will upload 100 contacts, do some tasks , then next 100 contacts until loop finish.
But in running, the result is not as expected. background thread is running, it keeps to request server. UI thread is blocked, I still cannot do anything.
is this cause a long loop in background thread? Although I have 2 thread, but they will compete CPU resources(test device is iPod, 1 core. And I think this may not related core numbers)?
Could anyone tell me hints on how to handle this kind of scenario? Thanks in advance!
Update:
I have found the root cause. A global variable in App delegate is set to wrong value, therefore UI behavior is weird. I found this by comment all network request method. So this problem is not related with multiple threading. Sorry for the bother.
I think there needs to be some clarification as to how you are performing the network operations.
1st, NSOperatiomQueue deals with NSOperations, so you are presumably wrapping your network code in an NSOperation subclass.
2nd, are you using NSURLConnections for your networking code?
3rd, is the blocking part the NSURLConnection or you delegate callback for NSURLConnection?
1 thing to note is that plain ol' NSURLConnections are implemented under the hood multithreaded. The object is placed into your main threads run loop by default (when run from the main thread), but the object is just a wrapper that handles callbacks to the delegate from the lower level networking code (BSD sockets) which happens on another thread.
You really shouldn't be able to block your UI with NSURLConnections on the main thread, unless A) you are blocking the thread with expensive code in the delegate callback methods or B) you are overwhelming your run loop with too many simultaneous URL connections (which is where NSOperationQueue's setMaxConcurrentOperationsCount: comes into play)