As we know, dart is a single-threaded language. So according to the document, we can use Futrure/Stream to implement a async opetation. It sends the time-consuming operation to the Event Queue.
What confused me is where the Event Queue working on. It is working on the dart threat? if yes, it will block the app.
Another question is Event Queue a FIFO queue. If i have two opertion, one is a 1mins needed networking request, the other is a click event. The two operation will send to the Event Queue.
So if the click event will blocked by the networking request? Because the queue is a FIFO queue?
So where is the event queue working on?
Thank you very much!
One thing to note is that asynchronous and multithreading are two different things. Dart uses Futures and async/await to achieve asynchronicity, but Dart is still inherently a single-threaded language.
The way it works is when a Future is created (either manually or via calling an async method), that process is added to an event queue, as you read. Then, in the middle of all the synchronous execution, whenever there is a lull, the event queue can take priority. It can then go through the processes and figure out if any of the Futures have been completed. If so, the result is passed along to any other asynchronous processes that are waiting on that resource, if any.
This also means that, yes, if your program hangs in the middle of an asynchronous operation (with the easy example of an endless loop via while (true) {}), it will freeze the entire program, including the synchronous code and other asynchronous processes still waiting to resolve (even if the conditions allowing them to resolve have already occurred).
However, in your case, this won't be an issue. If you fire an asynchronous process in the form of a network request followed by another in the form of a "click event" (not sure what you're referring to, but I'll assume it's asynchronous as well), they will both be added to the event queue in that order. But if the click event resolves before the network request, the event queue will merely recognize that the network request Future has not yet resolved and will move on to the click event that has.
As a side note, it's worth noting that Dart does have a multi-threading capability, albeit in a fairly roundabout way. Dart has something called an Isolate, which isn't a thread but a completely separate child program. This means that the Isolate cannot access any of the same data in memory as the root program itself. However, data can be passed between the two using SendPorts and ReceivePorts. This makes using Isolates slightly more complicated than threads, but it also means that, if no memory is shared, it virtually eliminates race conditions based on which thread accesses the memory first.
Related
I tried multiple ways of wrapping a file read within a synchronous method call (including using multiple queues, specifying target queues, setting up an NSThread and signalling with NSCondition's, even moving the allocation of the UIDocument to the background thread in the end, and also trying dispatch_sync on the background queue as well).
What ended up consistently happening is the completion handler for UIDocument.openWithCompletionHandler wasn't executing, although the documentation indicates that shall happen on the same queue that initiated the openWithCompletionHandler call.
I figured this has ultimately something to do with the control not being returned by the outer/top-level method call to the run loop. It would seem that regardless of what other queues or threads are being set up, the dispatch system expects me to return from the outermost method call, or things get blocked. This would however defeat the whole synchronous design approach.
My use case requires synchronous file reads (with very small data sizes), and I'd prefer the convenience of UIDocument over moving to lower level methods, or looking at ways to introduce async patterns. I reckon UIDocument was designed for more conventional cases, I understand well enough the ubiquity - and in most cases user friendliness and efficiency of async patterns, but in this case it would present a cumbersome situation for both development and user experience.
I wonder if there is something else that could be tried with dispatch queues that could still be explored (like manually consuming events from a queue, creating a custom run loop) that could avoid this seemingly global synchronization effect.
EDIT: this is for an Audio Unit app extension. Instantiation is controlled by the platform, and a "half-initialized" state could become a problematic situation. It is pretty much standard in the industry to fully load the plugin before even allowing the host app to start playing any audio for example, not to mention starting to stream MIDI/automation events. (That's not to say there aren't extensions with crazy load times that could take another look at their design, but in most cases these are well justified in this domain).
I might have the wrong idea of Isolate and Future. Please help me to clear it up. Here is my understanding of both subjects.
Isolate: Isolates run code in its own event loop, and each event may run smaller tasks in a nested microtask queue.
Future: A Future is used to represent a potential value, or error, that will be available at some time in the future.
My confusions are:
The doc says Isolate has it own loop? I feel like having its own event queue makes more sense to me, am I wrong?
Is future running asynchronously on the main Isolate? I'm assuming future task actually got placed at the end of event queue so if it will be execute by loop in the future. Correct me if I'm wrong.
Why use Isolate when there is future? I saw some examples using Isolate for some heavy task instead of Future. But why? It only makes sense to me when future execute asynchronously on the main isolate queue.
A Future is a handle that allows you to get notified when async execution is completed.
Async execution uses the event queue and code is executed concurrently within the same thread.
https://webdev.dartlang.org/articles/performance/event-loop
Dart code is by default executed in the root isolate.
You can start up additional isolates that usually run on another thread.
An isolate can be either loaded from the same Dart code the root isolate was started with (with a different entry-point than main() https://api.dartlang.org/stable/2.0.0/dart-isolate/Isolate/spawn.html) or with different Dart code (loaded from some Dart file or URL https://api.dartlang.org/stable/2.0.0/dart-isolate/Isolate/spawnUri.html).
Isolates don't share any state and can only communicate using message passing (SendPort/ReceivePort). Each isolate has its own event queue.
https://webdev.dartlang.org/articles/performance/event-loop
An Isolate runs Dart code on a single thread. Synchronous code like
print('hello');
is run immediately and can't be interrupted.
An Isolate also has an Event Loop that it uses to schedule asynchronous tasks on. Asynchronous doesn't mean that these tasks are run on a separate thread. They are still run on the same thread. Asynchronous just means that they are scheduled for later.
The Event Loop runs the tasks that are scheduled in what is called an Event Queue. You can put a task in the Event Queue by creating a future like this:
Future(() => print(hello));
The print(hello) task will get run when the other tasks ahead of it in the Event Queue have finished. All of this is happening on the same thread, that is, the same Isolate.
Some tasks don't get added to the Event Queue right away, for example
Future.delayed(Duration(seconds: 1), () => print('hello'));
which only gets added to the queue after a delay of one second.
So far everything I've been talking about gets done on the same thread, the same Isolate. Some work may actually get done on a different thread, though, like IO operations. The underlying framework takes care of that. If something expensive like reading from disk were done on the main Isolate thread then it would block the app until it finished. When the IO operation finishes the future completes and the update with the result is added to the Event Queue.
When you need to do CPU intensive operations yourself, you should run them on another isolate so that it doesn't cause jank in your app. The compute property is good for this. You still use a future, but this time the future is returning the result from a different Isolate.
Further study
Futures - Isolates - Event Loop
Dart asynchronous programming: Isolates and event loops
Are Futures in Dart threads?
The Event Loop and Dart
Flutter/Dart non-blocking demystify
The Engine architecture
Single Thread Dart, What? — Part 1
Single Thread Dart, What? — Part 2
Flutter Threading: Isolates, Future, Async And Await
The Fundamentals of Zones, Microtasks and Event Loops in the Dart Programming Language
An introduction to the dart:io library
What thread / isolate does flutter run IO operations on?
In one sentence we could say,
Isolates: Dart is single-threaded but it is capable of doing multi-threading stuff using Isolates (many processes).
Future: Future is a result which is returned when dart has finished an asynchronous work. The work is generally done in that single-thread.
Isolate could be compared to Thread even if dart is not multithreaded. It has it's own memory and event loop indeed, when Futures shares the same memory
Dart is able to spawn standalone processes, called Isolates (web workers in dart2js), which do not share memory when the main program, but are able to asynchronously, in another process (effectively a thread of sorts) is able to do computations without blocking the main thread.
A Future is run inside the Isolate that called it, not necesserally the main isolate.
I recommend this article which has better explanation than me.
TLDR: https://medium.com/flutter-community/isolates-in-flutter-a0dd7a18b7f6
Let's understand async-await first and then go into isolates.
void main() async {
// Read some data.
final fileData = await _readFileAsync();
final jsonData = jsonDecode(fileData);
// Use that data.
print('Number of JSON keys: ${jsonData.length}');
}
Future<String> _readFileAsync() async {
final file = File(filename);
final contents = await file.readAsString();
return contents.trim();
}
We want to read some data from a file and then decode that JSON and print the JSON Keys length. We don’t need to go into the implementation details here but can take the help of the image below to understand how it works.
When we click on this button Place Bid, it sends a request to _readFileAsync, all of which is dart code that we wrote. But this function _readFileAsync, executes code using Dart Virtual Machine/OS to perform the I/O operation which in itself is a different thread, the I/O thread.
This means, the code in the main function runs inside the main isolate. When the code reaches the _readFileAsync, it transfers the code execution to I/O thread and the Main Isolate waits until the code is completely executed or an error occurs. This is what await keyword does.
Now, once the contents of the files are read, the control returns back to the main isolate and we start parsing the String data as JSON and print the number of keys. This is pretty straight forward. But let’s suppose, the JSON parsing was a very big operation, considering a very huge JSON and we start manipulating the data to conform to our needs. Then this work is happening on the Main Isolate. At this point of time, the UI could hang, making our users fustrated.
Now let's get back to isolates.
Dart uses Isolate model for concurrency. Isolate is nothing but a wrapper around thread. But threads, by definition, can share memory which might be easy for the developer but makes code prone to race conditions and locks. Isolates on the other hand cannot share memory and instead rely on message passing mechanism to talk with each other.
Using isolates, Dart code can perform multiple independent tasks at once, using additional cores if they’re available. Each Isolate has its own memory and a single thread running an event loop.
Hope this help solve someone's doubt.
One of the reasons async was praised for ASP.NET was to follow Nodejs async platform which led to more scalability with the freeing up of threads to handle subsequent requests etc.
However, I've since read that wrapping CPU-bound code in Task.Run will have the opposite effect i.e. add even more overhead to a server and use more threads.
Apparently, only true async operations will benefit e.g. making a web-request or a call to a database.
So, my question is as follows. Is there any clear guidance out there as to when action methods should be async?
Mr Cleary is the one who opined about the fruitlessness of wrapping CPU-bound operations in async code.
Not exactly, there is a difference between wrapping CPU-bound async code in an ASP.NET app and doing that in a - for example - WPF desktop app. Let me use this statement of yours to build my answer upon.
You should categorize the async operations in your mind (in its simplest form) as such:
ASP.NET async methods, and among those:
CPU-bound operations,
Blocking operations, such as IO-operations
Async methods in a directly user-facing application, among those, again:
CPU-bound operations,
and blocking operations, such as IO-operations.
I assume that by reading Stephen Cleary's posts you've already got to understand that the way async operations work is that when you are doing a CPU-bound operation then that operation is passed on to a thread pool thread and upon completion, the program control returns to the thread it was started from (unless you do a .ConfigureAwait(false) call). Of course this only happens if there actually is an asynchronous operation to do, as I was wondering myself as well in this question.
When it comes to blocking operations on the other hand, things are a bit different. In this case, when the thread from which the code performed asynchronously gets blocked, then the runtime notices it, and "suspends" the work being done by that thread: saves all state so it can continue later on and then that thread is used to perform other operations. When the blocking operation is ready - for example, the answer to a network call has arrived - then (I don't exactly know how it is handled or noticed by the runtime, but I am trying to provide you with a high-level explanation, so it is not absolutely necessary) the runtime knows that the operation you initiated is ready to continue, the state is restored and your code can continue to run.
With these said, there is an important difference between the two:
In the CPU-bound case, even if you start an operation asynchronously, there is work to do, your code does not have to wait for anything.
In the IO-bound case or blocking case, however, there might be some time during which your code simply cannot do anything but wait and therefore it is useful that you can release that thread that has done the processing up until that point and do other work (perhaps process another request) meanwhile using it.
When it comes to a directly-user-facing application, for example, a WPF app, if you are performing a long-running CPU-operation on the main thread (GUI thread), then the GUI thread is obviously busy and therefore appears unresponsive towards the user because any interaction that is normally handled by the GUI thread just gets queued up in the message queue and doesn't get processed until the CPU-bound operation finishes.
In the case of an ASP.NET app, however, this is not an issue, because the application does not directly face the user, so he does not see that it is unresponsive. Why you don't gain anything by delegating the work to another thread is because that would still occupy a thread, that could otherwise do other work, because, well, whatever needs to be done must be done, it just cannot magically be done for you.
Think of it the following way: you are in a kitchen with a friend (you and your friend are one-one threads). You two are preparing food being ordered. You can tell your friend to dice up onions, and even though you free yourself from dicing onions, and can deal with the seasoning of the meat, your friend gets busy by dicing the onions and so he cannot do the seasoning of the meat meanwhile. If you hadn't delegated the work of dicing onions to him (which you already started) but let him do the seasoning, the same work would have been done, except that you would have saved a bit of time because you wouldn't have needed to swap the working context (the cutting boards and knives in this example). So simply put, you are just causing a bit of overhead by swapping contexts whereas the issue of unresponsiveness is invisible towards the user. (The customer neither sees nor cares which of you do which work as long as (s)he receives the result).
With that said, the categorization I've outlined at the top could be refined by replacing ASP.NET applications with "whatever application has no directly visible interface towards the user and therefore cannot appear unresponsive towards them".
ASP.NET requests are handled in thread pool threads. So are CPU-bound async operations (Task.Run).
Dispatching async calls to a thread pool thread in ASP.NET will result in returning a thread pool thread to the thread pool and getting a another. one to run the code and, in the end, returning that thread to the thread pool and getting a another one to get back to the ASP.NET context. There will be a lot of thread switching, thread pool management and ASP.NET context management going on that will make that request (and the whole application) slower.
Most of the times one comes up with a reason to do this on a web application ends up being because the web applications is doing something that it shouldn't.
I am working on an app, which uploads native contacts to server then get responses(JSON, a contact list that already installed the app). When native contacts are large enough, server response will be slow and unstable. And user cannot do other things. so I put network request into background thread. every time I will upload 100 contacts, do some tasks , then next 100 contacts until loop finish.
But in running, the result is not as expected. background thread is running, it keeps to request server. UI thread is blocked, I still cannot do anything.
is this cause a long loop in background thread? Although I have 2 thread, but they will compete CPU resources(test device is iPod, 1 core. And I think this may not related core numbers)?
Could anyone tell me hints on how to handle this kind of scenario? Thanks in advance!
Update:
I have found the root cause. A global variable in App delegate is set to wrong value, therefore UI behavior is weird. I found this by comment all network request method. So this problem is not related with multiple threading. Sorry for the bother.
I think there needs to be some clarification as to how you are performing the network operations.
1st, NSOperatiomQueue deals with NSOperations, so you are presumably wrapping your network code in an NSOperation subclass.
2nd, are you using NSURLConnections for your networking code?
3rd, is the blocking part the NSURLConnection or you delegate callback for NSURLConnection?
1 thing to note is that plain ol' NSURLConnections are implemented under the hood multithreaded. The object is placed into your main threads run loop by default (when run from the main thread), but the object is just a wrapper that handles callbacks to the delegate from the lower level networking code (BSD sockets) which happens on another thread.
You really shouldn't be able to block your UI with NSURLConnections on the main thread, unless A) you are blocking the thread with expensive code in the delegate callback methods or B) you are overwhelming your run loop with too many simultaneous URL connections (which is where NSOperationQueue's setMaxConcurrentOperationsCount: comes into play)
I'd like to delay the handling for some captured events in ActionScript until a certain time. Right now, I stick them in an Array when captured and go through it when needed, but this seems inefficient. Is there a better way to do this?
Well, to me this seems a clean and efficient way of doing that.
What do you mean by delaying? you mean simply processing them later, or processing them after a given time?
You can always set a timout to the actual processing function in your event handler (using flash.utils.setTimeout), to process the event at a precise moment in time. But that can become inefficient, since you may have many timeouts dangeling about, that need to be handled by the runtime.
Maybe you could specify your needs a little more.
edit:
Ok, basically, flash player is single threaded - that is bytecode execution is single threaded. And any event, that is dispatched, is processed immediatly, i.e. dispatchEvent(someEvent) will directly call all registered handlers (thus AS bytecode).
Now there are events, which actually are generated in the background. These come either from I/O (network, userinput) or timers (TimerEvents). It may happen, that some of these events actually occur, while bytecode is executed. This usually happens in a background thread, which passes the event (in the abstract sense of the term) to the main thread through a (de)queue.
If the main thread is busy executing bytecode, then it will ignore these messages until it is done (notice: nearly any bytecode execution is always the implicit consequence of an event (be it enter frame, or input, or timer or load operation or whatever)). When it is idle, it will look in all queues, until it finds an available message, wraps the information into an ActionScript Event object, and dispatches it as previously described.
Thus this queueing is a very low level mechanism, that comes from thread-to-thread communication (and appears in many multi-threading scenarios), and is inaccessible to you.
But as I said before, your approach both is valid and makes sense.
Store them into Vector instead of Array :p
I think it's all about how you structure your program, maybe you can assign the captured event under the related instance? So that it's all natural to process the captured event with it instead of querying from a global vector