After I profile an ASP.NET Core application, JetBrain's dotTrace tool by default opens up the All Calls tab and shows a list of threads.
Could you please clarify some of my questions?
During my profiling session, I sent around 100K requests to the web application. Are these threads (apart from Main and some system threads like Finalizer thread) the only ones which were used by the thread pool to serve all those requests based on the image below?
Ideally I would prefer to know the details of the function that took lot of CPU (i.e hottest method in the profiling session), so I keep wondering how this threads view actually helps?
Do you think there could have been many more threads from the thread pool which could have been used and returned back to the thread pool, but the profiler was only able to 'capture' the information of particular thread pool threads because those thread pool threads were executing during that time? I say this because for ~100K requests the number of threads shown here seem less, but I could be wrong. (Probably its based on number of concurrent requests executed?)
The Threads view is usefull when you use real Threads in your application.
For example, in a WPF the main thread just for dispatcher and your custom thread for background working.
In your case, you still can expand the dispatch to see what's executed inside.
You can also use Tracing mode to see the number of request inside a pool worker.
All the threads actually used during your profiling recording are displayed, each thread will execute lots of requests
For this kind of use, I prefer the Timeline mode, so you can filter all your requests and see all the threads where is was executed.
Related
One of the reasons async was praised for ASP.NET was to follow Nodejs async platform which led to more scalability with the freeing up of threads to handle subsequent requests etc.
However, I've since read that wrapping CPU-bound code in Task.Run will have the opposite effect i.e. add even more overhead to a server and use more threads.
Apparently, only true async operations will benefit e.g. making a web-request or a call to a database.
So, my question is as follows. Is there any clear guidance out there as to when action methods should be async?
Mr Cleary is the one who opined about the fruitlessness of wrapping CPU-bound operations in async code.
Not exactly, there is a difference between wrapping CPU-bound async code in an ASP.NET app and doing that in a - for example - WPF desktop app. Let me use this statement of yours to build my answer upon.
You should categorize the async operations in your mind (in its simplest form) as such:
ASP.NET async methods, and among those:
CPU-bound operations,
Blocking operations, such as IO-operations
Async methods in a directly user-facing application, among those, again:
CPU-bound operations,
and blocking operations, such as IO-operations.
I assume that by reading Stephen Cleary's posts you've already got to understand that the way async operations work is that when you are doing a CPU-bound operation then that operation is passed on to a thread pool thread and upon completion, the program control returns to the thread it was started from (unless you do a .ConfigureAwait(false) call). Of course this only happens if there actually is an asynchronous operation to do, as I was wondering myself as well in this question.
When it comes to blocking operations on the other hand, things are a bit different. In this case, when the thread from which the code performed asynchronously gets blocked, then the runtime notices it, and "suspends" the work being done by that thread: saves all state so it can continue later on and then that thread is used to perform other operations. When the blocking operation is ready - for example, the answer to a network call has arrived - then (I don't exactly know how it is handled or noticed by the runtime, but I am trying to provide you with a high-level explanation, so it is not absolutely necessary) the runtime knows that the operation you initiated is ready to continue, the state is restored and your code can continue to run.
With these said, there is an important difference between the two:
In the CPU-bound case, even if you start an operation asynchronously, there is work to do, your code does not have to wait for anything.
In the IO-bound case or blocking case, however, there might be some time during which your code simply cannot do anything but wait and therefore it is useful that you can release that thread that has done the processing up until that point and do other work (perhaps process another request) meanwhile using it.
When it comes to a directly-user-facing application, for example, a WPF app, if you are performing a long-running CPU-operation on the main thread (GUI thread), then the GUI thread is obviously busy and therefore appears unresponsive towards the user because any interaction that is normally handled by the GUI thread just gets queued up in the message queue and doesn't get processed until the CPU-bound operation finishes.
In the case of an ASP.NET app, however, this is not an issue, because the application does not directly face the user, so he does not see that it is unresponsive. Why you don't gain anything by delegating the work to another thread is because that would still occupy a thread, that could otherwise do other work, because, well, whatever needs to be done must be done, it just cannot magically be done for you.
Think of it the following way: you are in a kitchen with a friend (you and your friend are one-one threads). You two are preparing food being ordered. You can tell your friend to dice up onions, and even though you free yourself from dicing onions, and can deal with the seasoning of the meat, your friend gets busy by dicing the onions and so he cannot do the seasoning of the meat meanwhile. If you hadn't delegated the work of dicing onions to him (which you already started) but let him do the seasoning, the same work would have been done, except that you would have saved a bit of time because you wouldn't have needed to swap the working context (the cutting boards and knives in this example). So simply put, you are just causing a bit of overhead by swapping contexts whereas the issue of unresponsiveness is invisible towards the user. (The customer neither sees nor cares which of you do which work as long as (s)he receives the result).
With that said, the categorization I've outlined at the top could be refined by replacing ASP.NET applications with "whatever application has no directly visible interface towards the user and therefore cannot appear unresponsive towards them".
ASP.NET requests are handled in thread pool threads. So are CPU-bound async operations (Task.Run).
Dispatching async calls to a thread pool thread in ASP.NET will result in returning a thread pool thread to the thread pool and getting a another. one to run the code and, in the end, returning that thread to the thread pool and getting a another one to get back to the ASP.NET context. There will be a lot of thread switching, thread pool management and ASP.NET context management going on that will make that request (and the whole application) slower.
Most of the times one comes up with a reason to do this on a web application ends up being because the web applications is doing something that it shouldn't.
I want to understand something about GCD and Threads.
I have a for loop in my view controller which asks my model to do some async network request.
So if the loop runs 5 times, the model sends out 5 network requests.
Is it correct to state that 5 threads have been created by my model considering the fact that I'm using NSURLConnection's sendAsyncRequest and the completion handlers will be called on an additional 5 threads ?
Now, If I ask my view controller to execute this for loop on a different thread and in every iteration of the loop, the call to the model should be dependent on the previous iteration, would I be creating an "Inception" of threads here ?
Basically, I want the subsequent async requests to my server only if the previous thread has completed entirely (By entirely I mean all of its sub threads should have finished executing too.)
I can't even frame the question properly because I'm massively confused myself.
But if anybody could help with anything, that would be helpful.
It is not correct to state that five threads have been created in the general case.
There is no one-to-one mapping between threads and blocks. GCD is an implementation of thread pooling.
A certain number of threads are created according to the optimal setup for that device — the cost of creating and maintaing threads under that release of the OS, the number of processor cores available, the number of threads it already has but which are presently blocked and any other factors Apple cares to factor in may all be relevant.
GCD will then spread your blocks over those threads. Or it may create new threads. But it won't necessarily.
Beyond that queues are just ways of establishing the sequencing between blocks. A serial dispatch queue does not necessarily own its own thread. All concurrent dispatch queues do not necessarily own their own threads. But there's no reason to believe that any set of queues shares any threads.
The exact means of picking threads for blocks has changed between versions of the OS. E.g. iOS 4 was highly profligate in thread creation, in a way that iOS 5+ definitely haven't been.
GCD will just try to do whatever is best in the circumstances. Don't waste your time trying to second guess it.
"Basically, I want the subsequent async requests to my server only if the previous thread has completed entirely (By entirely I mean all of its sub threads should have finished executing too.)"
Only focusing on the above statement to avoid confusion. Simple solution would be create a queue. feed the queue with 5 loops. Each loop will be making network request synchronously(you can use sendSynchronousRequest: method available in NSURLConnection), performing the operations after request completion and then start the next loop. queue following FIFO will execute the your requests subsequently.
GCD : Think of this as a simple queue that can accept tasks. Tasks are blocks of your code. You can put in as many tasks as you want in a queue (permitting system limits). Queues come in different flavors. Concurrent vs Serial. Main vs Global. High Priority vs Low Priority. A queue is not a thread.
Thread : It is a single line of execution of code in sequence. You can have multiple threads working on your code at the same time. A thread is not a queue.
Once you separate the 2 entities things start become clear.
GCD basically uses the threads in the process to work on tasks. In a serial queue everything is processed in sequence. So you don't need to have synchronization mechanisms in your code, the very nature of serial queue ensures synchronization. If this is a concurrent queue (i.e. 2 or more tasks being processed at the same time, then you need to ensure critical sections of your code are protected with synchronization).
Here is how you queue work to be done.
dispatch_async(_yourDispatchQueue, ^() {
NSLog (#"work queued");
});
The above NSLog will now get executed in a background thread in a near future time, but in a background thread.
If you notice when we put a request in we use dispatch_async. The other variation is dispatch_sync. The different between the 2 is after you put the request in the queue, the async variation will move on. The sync variation will not !!
If you are going to use a GCD for NSURLConnection you need to be careful on which thread you start the connection. Here is a SO link for more info. GCD with NSURLConnection
I Have a quick question about async Controller and Actions in ASP.Net MVC 4+ (using the async/await programming model returning a Task ).
What do i risk if all my actions are async even if the underline operations are not IO Bound (for example slow web service or network communication ) and can be CPU Bound. I mean all my actions will be async no matter what code is int it . I Hope i'am clear.
Will be performance problems due to synchronisation context overhead or any other significant overhead for a public website that can have a lot of simultanous users ?
Thank you for your futur answers.
What do i risk if all my actions are async even if the underline operations are not IO Bound (for example slow web service or network communication ) and can be CPU Bound.
This is exactly what you don't want to do.
Consider what happens with a synchronous action: the request comes in, ASP.NET allocates a thread for that request, and that thread executes the action. When the action is complete, that same thread sends the response.
Now consider what happens if you "offload" an action's CPU-bound work to the thread pool (e.g., Task.Run). The request comes in, ASP.NET allocates a thread for that request, and the thread starts executing the action. When the thread hits Task.Run, it allocates another thread from the thread pool to execute the CPU-bound code. Then the asynchronous action method hits the await for that task, so the original thread is returned to the ASP.NET runtime. The other thread then finishes the work and continues on to send the response.
So, you're doing more work for every request if you have an asynchronous action that pushes CPU-bound work to the thread pool. You should never do this on ASP.NET.
I explain this in more detail on my blog.
I don't think you will run into any problems for a simple app since the async workers are running off a pool of threads with a limit number of threads. But what you may run into is a condition where the client HTTP threads are waiting on the asyn response and it exceeds a network gateway timeout. For instance, amazon ELB have a 60 second timeout. So clients can be disconnected while the async task is still running. If that happens a lot then you could end up with a lot of async tasks running and completing with no client to respond to. That would be an unfortunate condition because your clients are not getting data and your server is working for nothing.
One thing I would consider is whether or not you need async calls. I would suggest tuning up your service calls and making sure they are fast enough to address load instead of making the front end async as a workaround for the latency. For instance, using caching. Just a suggestion.
Hope it helps.
beside heavy processing, should multithreading mainly be used when you have a not quite responsive UI? or does it have other considerations?
How can I know if my application should have multithreading or not?
One of the Important Application of thread in ios is during network communication.Whole your app is communication with server and if you want to show busy view on UR UI you need to create thread in such scenario to perform network communication in background thread.
In IOS 5,You can opt for GCD(Grand Central Dispatch)Instead of thread to perform same functionality..
Basically in iOS development Threads are used when you don't want to affect you UI by a process which will take long time to complete. for example when you make a connection to parse xml,json,image data etc then you don't want to stop user interaction at that time you can use threads.
you can start a thread by using NSThread.
Things to have in mind before using threads -
You should never do a graphical change in a thread. If you need to
that in a thread then you can do only on main thread.
Never use a NSTimer in a secondary thread, because your thread may
complete before timer execution so timer may not run.
whenever you want to perform a long process then you can use thread.
The use of threading in ios is to ensure hussle-free and seamless experience by the end-users.
You can implement thread whenever you want to extract some resource over the network such as parsing or data retrieval and you don't want the ui to be affected as application would run on main thread and the web-operation on your custom thread.
You may want to use the thread when you need to have concurrent operations or simultaneous such as in game when you hae to have multiple animations on same object at same time.There can be quite a large number of scenarios which may need threading.
You may read Concurrency Programming Guide By Apple
and Thread Management
but threads may be an overhead in the application as it needs memory allocation and large operations on thread may affect the performance so use it when it can't be avoided.
You can use NSThread,NSOperations to create threads .GCD is deprecated now.
I am working on an app, which uploads native contacts to server then get responses(JSON, a contact list that already installed the app). When native contacts are large enough, server response will be slow and unstable. And user cannot do other things. so I put network request into background thread. every time I will upload 100 contacts, do some tasks , then next 100 contacts until loop finish.
But in running, the result is not as expected. background thread is running, it keeps to request server. UI thread is blocked, I still cannot do anything.
is this cause a long loop in background thread? Although I have 2 thread, but they will compete CPU resources(test device is iPod, 1 core. And I think this may not related core numbers)?
Could anyone tell me hints on how to handle this kind of scenario? Thanks in advance!
Update:
I have found the root cause. A global variable in App delegate is set to wrong value, therefore UI behavior is weird. I found this by comment all network request method. So this problem is not related with multiple threading. Sorry for the bother.
I think there needs to be some clarification as to how you are performing the network operations.
1st, NSOperatiomQueue deals with NSOperations, so you are presumably wrapping your network code in an NSOperation subclass.
2nd, are you using NSURLConnections for your networking code?
3rd, is the blocking part the NSURLConnection or you delegate callback for NSURLConnection?
1 thing to note is that plain ol' NSURLConnections are implemented under the hood multithreaded. The object is placed into your main threads run loop by default (when run from the main thread), but the object is just a wrapper that handles callbacks to the delegate from the lower level networking code (BSD sockets) which happens on another thread.
You really shouldn't be able to block your UI with NSURLConnections on the main thread, unless A) you are blocking the thread with expensive code in the delegate callback methods or B) you are overwhelming your run loop with too many simultaneous URL connections (which is where NSOperationQueue's setMaxConcurrentOperationsCount: comes into play)