Note, related but not the same: iPhone - Grand Central Dispatch main thread
I've failed at this question many times, so here's source code:
While on the main thread
dispatch_async( dispatch_get_main_queue(), ^{ NSString * str = #"Interrupt myself to do something."} );
I'm just curious, when a thread switches, it stores its registers in Thread Local Storage, switches context, runs from its new spot in the Program Counter (which I assume is within a copy of the program that simply uses a different stack and register), then it "goes back" to the main thread.
When it interrupts itself, I'm just wondering what decides when it should, and what happens to the Thread Local stuff.
I've read up on this a little, but I'm still wrapping my head around the fact that programs are not continuous. They're just "something to do in small chunks when the OS decides to run a chunk of a process, or its chunks (threads).
I am self-taught, which might add to my lack of register/asm knowledge that may be standard to any scholar.
Thanks. The code should help, this is iOS specific, but I believe the answer/question is related to any language going from main-to-main.
Since every past attempt has resulted in lengthy answers that ignore the reason I'm asking this, I will iterate one last time....
This is for the SAME thread. Main-to-main. Does it really just stop itself, move the program counter elsewhere, go, then end at the block? Also don't these things usually change at branches (if/for and blocks too).
Pointing me in the right direction works too, but like I said, previously the question was misread.
It is hard to answer your question specifically without having access to the internals of GCD, but generically, the answer is no, simply adding a unit of work to a dispatch queue will not immediately interrupt the executing code.
As you suggest context switches are expensive, not only in terms of state saving & restoration but also the processor will need to dump the instruction pipeline resulting in wasted cycles.
Typically the operating system will keep executing the current task until it suspends (e.g. waits on a network or other IO operation) or perhaps is interrupted by some external event (pressing the home key on the phone), but there are also time limits to prevent a runaway task from locking the whole device (This is pre-emptive multi-tasking, as opposed to co-operative multitasking where the task needs to relinquish the CPU)
With dispatch_async there is no guarantee of when the code will execute in relation to the current code block. The code block may not even be next in the queue - other threads may have added other units of work to the queue before this one.
I think the thing that's confusing you is the use of dispatch_async( dispatch_get_main_queue()), which submits code to run on a queue on the main thread.
Using dispatch_async on the main queue:
When you call dispatch_async( dispatch_get_main_queue()), it adds a unit of work to the main queue, which runs it's jobs from the main thread.
If you run this call from the main thread, the results are the same. The work gets added to the main queue for later processing.
When you make this call from the main thread, the system doesn't check the main queue for work to do until your code returns.
Think of this as a one-cook kitchen. As the cook works, he puts trays of dishes in the dishwashing area. He doesn't stop to do dishes until he gets to a breaking point in what he's currently doing. At that point he takes a tray of dishes, loads it into the dishwasher, and then goes back to cooking.
The cook knows that he has to check for dishes each time he gets to a breaking point, and then completes a dishwashing task before returning to cooking.
Using dispatch_async on a background queue:
A dispatch_async call to a background queue is like a 2-person kitchen. There is a dishwasher working at the same time. The cook puts a tray of dishes into the dishwashing station (the queue) and the dishwasher (the other thread) picks up that task as soon as it's finished with it's previous tasks, while the cook continues to work on cooking.
The above assumes a machine with multiple processors, which is the norm these days. Each processor can do work at the same time without having to juggle multiple tasks.
If you are running on a single-core system with preemptive multitasking, submitting tasks to separate threads/background queues has the same effect as if there were multiple processors, but now the OS has to do a juggling act. There's only one person in the kitchen, but he wears multiple hats. The person is doing the cook job, and the OS shouts "Switch!" The cook jots down notes on what he was doing (saves state) and then jumps into the dish-pit and starts washing dishes, and keeps washing dishes until the OS yells "Switch!" again, and the worker again saves state, switches to the next role, and picks up that role (cook) where it was left off.
Multi-tasking is more costly on a single-core system because each time the worker switches roles, it has to save the current state, then load the saved state for the other role, and continue. Those context switches take time.
Related
In the introductory section of the Concurrency chapter of "The Swift Programming Language" I read:
When an asynchronous function resumes, Swift doesn’t make any
guarantee about which thread that function will run on.
This surprised me. It seems odd, comparing for example with waiting on semaphore in pthreads, that execution can jump threads.
This leads me to the following questions:
Why doesn't Swift guarantee resuming on the same thread?
Are there any rules by which the resuming thread could be
determined?
Are there ways to influence this behaviour, for example make sure it's resumed on the main thread?
EDIT: My study of Swift concurrency & subsequent questions above were triggered by finding that a Task started from code running on the main thread (in SwiftUI) was executing it's block on another thread.
It helps to approach Swift concurrency with some context: Swift concurrency attempts to provide a higher-level approach to working with concurrent code, and represents a departure from what you may already be used to with threading models, and low-level management of threads, concurrency primitives (locking, semaphores), and so on, so that you don't have to spend any time thinking about low-level management.
From the Actors section of TSPL, a little further down on the page from your quote:
You can use tasks to break up your program into isolated, concurrent pieces. Tasks are isolated from each other, which is what makes it safe for them to run at the same time…
In Swift Concurrency, a Task represents an isolated bit of work which can be done concurrently, and the concept of isolation here is really important: when code is isolated from the context around it, it can do the work it needs to without having an effect on the outside world, or be affected by it. This means that in the ideal case, a truly isolated task can run on any thread, at any time, and be swapped across threads as needed, without having any measurable effect on the work being done (or the rest of the program).
As #Alexander mentions in comments above, this is a huge benefit, when done right: when work is isolated in this way, any available thread can pick up that work and execute it, giving your process the opportunity to get a lot more work done, instead of waiting for particular threads to be come available.
However: not all code can be so fully isolated that it runs in this manner; at some point, some code needs to interface with the outside world. In some cases, tasks need to interface with one another to get work done together; in others, like UI work, tasks need to coordinate with non-concurrent code to have that effect. Actors are the tool that Swift Concurrency provides to help with this coordination.
Actors help ensure that tasks run in a specific context, serially relative to other tasks which also need to run in that context. To continue the quote from above:
…which is what makes it safe for them to run at the same time, but sometimes you need to share some information between tasks. Actors let you safely share information between concurrent code.
… actors allow only one task to access their mutable state at a time, which makes it safe for code in multiple tasks to interact with the same instance of an actor.
Besides using Actors as isolated havens of state as the rest of that section shows, you can also create Tasks and explicitly annotate their bodies with the Actor within whose context they should run. For example, to use the TemperatureLogger example from TSPL, you could run a task within the context of TemperatureLogger as such:
Task { #TemperatureLogger in
// This task is now isolated from all other tasks which run against
// TemperatureLogger. It is guaranteed to run _only_ within the
// context of TemperatureLogger.
}
The same goes for running against the MainActor:
Task { #MainActor in
// This code is isolated to the main actor now, and won't run concurrently
// with any other #MainActor code.
}
This approach works well for tasks which may need to access shared state, and need to be isolated from one another, but: if you test this out, you may notice that multiple tasks running against the same (non-main) actor may still run on multiple threads, or may resume on different threads. What gives?
Tasks and Actors are the high-level tools in Swift concurrency, and they're the tools that you interface with most as a developer, but let's get into implementation details:
Tasks are actually not the low-level primitive of work in Swift concurrency; Jobs are. A Job represents the code in a Task between await statements, and you never write a Job yourself; the Swift compiler takes Tasks and creates Jobs out of them
Jobs are not themselves run by Actors, but by Executors, and again, you never instantiate or use an Executor directly yourself. However, each Actor has an Executor associated with it, that actually runs the jobs submitted to that actor
This is where scheduling actually comes into play. At the moment there are two main executors in Swift concurrency:
A cooperative, global executor, which schedules jobs on a cooperative thread pool, and
A main executor, which schedules jobs exclusively on the main thread
All non-MainActor actors currently use the global executor for scheduling and executing jobs, and the MainActor uses the main executor for doing the same.
As a user of Swift concurrency, this means that:
If you need a piece of code to run exclusively on the main thread, you can schedule it on the MainActor, and it will be guaranteed to run only on that thread
If you create a task on any other Actor, it will run on one (or more) of the threads in the global cooperative thread pool
And if you run against a specific Actor, the Actor will manage locks and other concurrency primitives for you, so that tasks don't modify shared state concurrently
With all of this, to get to your questions:
Why doesn't Swift guarantee resuming on the same thread?
As mentioned in the comments above — because:
It shouldn't be necessary (as tasks should be isolated in a way that the specifics of "which thread are we on?" don't matter), and
Being able to use any one of the available cooperative threads means that you can continue making progress on all of your work much faster
However, the "main thread" is special in many ways, and as such, the #MainActor is bound to using only that thread. When you do need to ensure you're exclusively on the main thread, you use the main actor.
Are there any rules by which the resuming thread could be determined?
The only rule for non-#MainActor-annotated tasks are: the first available thread in the cooperative thread pool will pick up the work.
Changing this behavior would require writing and using your own Executor, which isn't quite possible yet (though there are some plans on making this possible).
Are there ways to influence this behaviour, for example make sure it's resumed on the main thread?
For arbitrary threads, no — you would need to provide your own executor to control that low-level detail.
However, for the main thread, you have several tools:
When you create a Task using Task.init(priority:operation:), it defaults to inheriting from the current actor, whatever actor this happens to be. This means that if you're already running on the main actor, the task will continue using the current actor; but if you aren't, it will not. To explicitly annotate that you want the task to run on the main actor, you can annotate its operation explicitly:
Task { #MainActor in
// ...
}
This will ensure that regardless of what actor the Task was created on, the contained code will only run on the main actor.
From within a Task: regardless of the actor you're currently on, you can always submit a job directly onto the main actor with MainActor.run(resultType:body:). The body closure is already annotated as #MainActor, and will guarantee execution on the main thread
Note that creating a detached task will never inherit from the current actor, so guaranteed that a detached task will be implicitly scheduled through the global executor instead.
My study of Swift concurrency & subsequent questions above were triggered by finding that a Task started from code running on the main thread (in SwiftUI) was executing it's block on another thread.
It would help to see specific code here to explain exactly what happened, but two possibilities:
You created a non-explicitly #MainActor-annotated Task, and it happened to begin execution on the current thread. However, because you weren't bound to the main actor, it happened to get suspended and resumed by one of the cooperative threads
You created a Task which contained other Tasks within it, which may have run on other actors, or were explicitly detached tasks — and that work continued on another thread
For even more insight into the specifics here, check out Swift concurrency: Behind the scenes from WWDC2021, which #Rob linked in a comment. There's a lot more to the specifics of what's going on, and it may be interesting to get an even lower-level view.
If you want insights into the threading model underlying Swift concurrency, watch WWDC 2021 video Swift concurrency: Behind the scenes.
In answer to a few of your questions:
Why doesn't Swift guarantee resuming on the same thread?
Because, as an optimization, it can often be more efficient to run it on some thread that is already running on a CPU core. As they say in that video:
When threads execute work under Swift concurrency they switch between continuations instead of performing a full thread context switch. This means that we now only pay the cost of a function call instead. …
You go on to ask:
Are there any rules by which the resuming thread could be determined?
Other than the main actor, no, there are no assurances as to which thread it uses.
(As an aside, we’ve been living with this sort of environment for a long time. Notably, GCD dispatch queues, other than the main queue, make no such guarantee that two blocks dispatched to a particular serial queue will run on the same thread, either.)
Are there ways to influence this behaviour, for example make sure it's resumed on the main thread?
If we need something to run on the main actor, we simply isolate that method to the main actor (with #MainActor designation on either the closure, method, or the enclosing class). Theoretically, one can also use MainActor.run {…}, but that is generally the wrong way to tackle it.
Is there a way to know when an iOS app becomes responsive to user interaction? For example, a user taps a button and the app performs work, this work may dispatch other work asynchronously to the main thread. In hopes of using it as a performance metric, I want to know the precise moment at which the app is again able to process touch events in a responsive manner. With this I would want data like "On average, the app becomes responsive 55ms after a user interaction".
Currently, immediately after a user interaction, I watch the main queue and have a heuristic for submitting samples to it in order to estimate responsiveness based on the main queue's responsiveness, with the assumption that the main queue's responsiveness directly correlates with the apps' responsiveness. The sampling only occurs until the queue is again consistently responsive for some time again (ex. 100ms). Is there any downside to this method? Is there any other method I could/should be using to do this?
Using MetricKit to watch for Hang Time is not an option as I cannot those results to a specific interaction (i.e. knowing how different interactions affect hang time).
You said:
For example, a user taps a button and the app performs work. I want to know the precise moment at which the app is again able to process touch events in a responsive manner.
The main thread should never be blocked. It should always be responsive. (You can disable the UI if your app requires that, but never block the main thread, regardless.)
So, with that in mind, if you are starting some process that takes a little time, you should:
If you want the app to let the user know that a time consuming process is about to start, add that chrome to the UI (e.g. UIActivityIndicatorView, aka a “spinner”, or whatever);
Start that task, asynchronously, on a background queue (so that it does not block the main thread);
Give that task a “completion handler” closure that it will call when the background work is done;
In that completion handler, the caller can supply the code to remove any chrome added in the first step, above.
In short, rather than worrying about “how does the app know when the main thread is free again”, you should focus on eliminating anything that would ever block the main thread in the first place. See Understand and eliminate hangs from your app.
Here is the problem that I got. I have several tasks to complete in background when application is running. When I run these tasks in background by pushing them to concurrent dispatch queue it takes more then 10 seconds to complete all of them. They basically load data from disk and parse it and represent the result to the user. That is they are just cached results and hugely improve the user experience.
This cached results are used in a particular functionality inside the app, and when that functionality is not used immediately after opening the application, it is not a problem that it takes 10 seconds to load the data that supports that functionality, because when user decides to use it, that data will already be loaded.
But when user immediately enters that function in the app after opening it, it takes considerable time (from the point of view of the user) to load the data. Also the whole data is not needed at the same moment, but rather the piece of it at a given moment.
That's why we need concurrently load the data, and if possible bring the results as soon as possible. That's why I decided to break the data into chunks, and when user requests the data, we should load the corresponding chunk by background thread and give that thread the highest priority. I'll explain what I mean.
Imagine there are 100 pieces of data and it takes more than 10 seconds to load them all. Whenever user queries the data first time, the app determines which chunk of the data user needs and starts loading that chunk. After that part is loaded the remaining data will also be loaded in the background, in order to make later queries faster (without the lag of loading the cache). But here a problem occurs, when user decides to change the query immediately after he has already entered one, and that change occurs for instance on the 2nd second of data loading process (remember it takes more than 10 seconds to load the data and we still have more than 8 seconds to complete the loading process), then in the extreme case user will receive his data waiting until all data will be loaded. That's way I need somehow manage the execution of the background tasks. That is, when user changes the input, I should change the priorities of execution, and give the thread that loads the corresponding chunk the highest priority without stopping it, so it will receive more processor time, and will finish sooner, and deliver results to the user faster, than it would if I have left the priorities the same. I know I can assign priorities to queues. But is there a way that I can change them dynamically while they are still executing?
Or do I need to implement custom thread management, in order to implement these behaviour? I really don't want to dive into thread management, and will be glad if it is possible to implement using only dispatch or operation queues.
I hope I've described the problem well. If not please comment bellow what is unclear, I'll explain.
Thank you so much for reading so far :) And special thanks to one who will provide an answer. And very special thanks to one, who will give me solution using dispatch or operation queues :)))
I think you need to move away from thinking about the priority at which the queues are running (which actually doesn't sound very important for the scenario you are describing) and more towards how you can use Dispatch I/O or an even simpler Dispatch source to control how the data is being read in. As you say, it takes 10 seconds the load the data and if the user suddenly changes their query immediately after asking, you need to essentially stop reading the data for the previous request and do whatever needs to be done to fulfill the most recent query. Using Dispatch I/O to chunk the data (asynchronously) and update the UI also asynchronously will allow you to change your mind mid-stream (using some sort of semaphore or cancellation flag) and either continue to trickle the data in (you don't say whether or not that data will remain useful if the user changes their mind or not), suspend the reading process, or cancel it altogether and start a new operation. Eithe way, being able to suspend/resume a source and also have it fire callbacks for reasonably small chunks of data will certainly enable you to make decisions on a much more granular chunk of time than 8 seconds!
I'm afraid the only way to do that is to cancel running operation before starting new one.
You cannot remove it from queue until it's done or canceled.
As an improvement for your problem I would suggest to load things even user doesn't need them in background - so you can load them from cache after it's there.
You can create 2 NSOperationQueue with 2 different priorities and download things in background whenever user is idle on LowPriorityQueue. For important operations you can have high priority queue - which you will cancel each time search term changes.
On top of that you just need to cache results from both of those queues.
I am trying to understand multi-threading on iOS in more detail. I went through some of the class references like NSThread, NSRunLoop, NSTask..
First of all as indicated on the following link:
use of runloop
Runloop runs within a Thread.
So why do we need to define our own Runloop in our app? In the case of NSThread it is useful because some of time-consuming processes can run in a separate thread so that the app will still be responsive on the main thread.
Interacting with the thread's run loop may be useful if you have a thread whose work you want to continue periodically. That is, a run loop would do some work, and then when it is finished with that work, it would put the thread to rest for some time, then resume work at a later time -- effectively preventing the thread from exiting. You won't need to interact with them or configure/create them yourself regularly (only a small percentage of apps would qualify, if you are using high level abstractions such as Foundation because Foundation would set them up on your behalf in most scenarios).
If your secondary thread just does a specified task and does not need to wait for some external event (e.g. a download to finish), you would (typically) not need to interact with the run loop.
You might consider looking at using NSOperationQueues, NSOperations and NSBlockOperations instead as these will manage themselves, will allow for cancellation of tasks and can be scheduled on main and background threads.
I have an important operation that is executed rarely. In some cases, it might take minutes to execute. My app is getting killed after a 50 second operation. How to avoid that?
Should I put it in a background thread? Could anyone please point me in the right direction here. I have not found any useful information about the so called watchdog. Is a background thread the way to go?
Yes, you need to move this task to a background thread. You should never jam up the main thread with any task that takes longer than a fraction of a second to perform. Ignoring the watchdog timer, which only kicks in under extreme conditions, your application is completely unresponsive to touch or other events during this lengthy operation, and you're unable to provide feedback to the user as to the progression of this operation.
The watchdog timer will kill an application that jams up the main thread for an extremely long period of time, making the application unresponsive to input (I believe this duration is currently 20 seconds on startup, but I'm not sure what it is for when the application is running). You should never let your application get to the point where the watchdog is killing it, because that's pointing to a real problem in the way your application is handling things.
Moving a long-running task to a background thread is a lengthy topic by itself, which is why I recommend reading Apple's Concurrency Programming Guide (updated) as well as watching some of their WWDC videos on the subject before starting.
However, in my opinion the most elegant way to deal with long-running tasks is to use Grand Central Dispatch, where something like
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
// Do your long-running task here
dispatch_async(dispatch_get_main_queue(), ^{
// Do callbacks to any UI updates here, like for a status indicator
});
});
will fire off your task to be performed in a background thread on one of the global concurrent queues. The little section of code within the main block shows how you might update any UI elements, such as a progress bar, from within this background task. Generally, UI updates must be performed on the main thread (there are some exceptions as of iOS 4.0, but it's still a good practice in general).
I also highly recommend adding some kind of visual indication of the status of this long-running task as it proceeds. Your users will really appreciate this, and it will make your application appear faster, even though it may run for the same duration.
Can you occasionally hit the watchdog during your process? Watchdog timers are just there to detect whether something crashed. They aren't really concerned with the system being busy.
Is the phone still able to respond to the user doing stuff like pressing the home button during your process?
EDIT: This guys recommends using a background thread