If i'm looking to use TPL async at my data layer, must i also use Task<T> on my MVC controller?
In other words, for async to work with .NET MVC, must it be implemented from the time the request begins in order for it to work on deeper execution layers? Or is there still a benefit to having Task<T> at my DAL/web request level even if i'm using a sync controller?
If you don't use an async controller you will have to wait on a task at some point. At this point the main advantage is gone: Reducing the number of blocked threads.
Of course this is not true if you run multiple async activities at the same time. That would reduce the number of blocked threads from N to one. (If N = 1 there is no benefit, just damage).
Note that async is not faster by default. Its main purpose in ASP.NET is to gain scalability at the extreme end - with 100s of concurrent requests. Only then will it be faster or scale higher.
So if you have a "usual" number of concurrent requests (like < 100), just go synchronous and don't worry about all of this.
Related
I want to make this Action Asynchronous,Can Anyone Help On this.
[HttpGet]
public ActionResult LoadAddressDetail(int? whId)
{
try
{
return Json(new
{
GridData = wareHouseMasterService.LoadAddressGrid(GetCommonProperty(), whId ?? 0, DataManager.CustomerType, DataManager.CustomerTypeCode.WAREHOUSE.ToString()),
Status = true
}, JsonRequestBehavior.AllowGet);
}
catch (Exception e) { this.loggerService.Error(e); return Json(false, JsonRequestBehavior.AllowGet); }
}
Since I am Using Three httpget Methods executed at one page load and the screen flickers Three Times.So I need parallel Processing.
Not sure exactly what you're looking for here. Making an action async is as simple as adding the async keyword and returning Task<ActionResult>:
public async Task<ActionResult> LoadAddressDetail(int? whId)
Of course, then, you must await something. The only thing I see that seems to do any work is wareHouseMasterService.LoadAddressGrid, but you've provided no details about what this does. Generally speaking, you would need to make this method async, as well, or add an async version of it. Then, you would simple do:
GridData = await wareHouseMasterService.LoadAddressGridAsync(...),
All that said, there's a big gotcha to keep in mind. Async, in the context of a web request, isn't the same thing as parallel processing, and it also does not make anything happen "faster". Additionally, there's a minor performance impact from using async, as there's overhead that goes along with that. In other words, simply making this action action async does not really in fact serve any other your mentioned goals.
The true reason to use async is to allow your web server to handle load more efficiently. When the thread handling the request is in a wait-state, async allows the thread to be returned to the pool to field other requests. Since there's a ceiling to the amount of threads a web server can spawn, and therefore the number of requests it can handle simultaneously, async gives you some extra head room when the server is under load. That's pretty much it, though.
That said, if you're trying all this out in development, using IIS Express, it's important to realize that IIS Express is single-threaded. In other words, requests are queued and handled serially, since raw performance is not a real factor in development. Therefore, in this situation, your three AJAX requests are being processed one at a time, rather than in parallel as they likely would be in full IIS, and that may be the source of what you're experiencing. Async may help here, but not necessarily. Regardless, it's only an issue in development.
Short of all that, you need to look at your JavaScript and ensure that you are doing things in an optimized way such that the AJAX requests are processed efficiently. There's also ways to minimize the flash of unstyled content issues apart from simply making everything happen quicker. However, since you've posted no code in this regard, there's not more that can be said.
From the wikibook on F# there is a small section where it says:
What does let! do?#
let! runs an async<'a> object on its own thread, then it immediately
releases the current thread back to the threadpool. When let! returns,
execution of the workflow will continue on the new thread, which may
or may not be the same thread that the workflow started out on.
I have not found anywhere else in books or on the web where this fact (highlighted in bold) is stated.
Is this true for all let!/do! regardless of what the async object contains (e.g. Thread.Sleep()) and how it is started (e.g. Async.Start)?
Looking in the F# source code on github, I wasn't able to find the place where a call to bind executes on a new (TP) thread. Where in the code is the magic happening?
Which part of that statement do you find surprising? That parts of a single async can execute on different threadpool threads, or that a threadpool thread is necessarily being released and obtained on each bind?
If it's the latter, then I agree - it sounds wrong. Looking at the code, there are only a few places where a new work item is being queued on the threadpool (namely, the few Async module functions that use queueAsync internally), and Async.SwitchToNewThread spawns a non-threadpool thread and runs the continuation there. A bind alone doesn't seem to be enough to switch threads.
The spirit of the statement however seems to be about the former - no guarantees are made that parts of an async block will run on the same thread. The exact thread that you run on should be treated as an implementation detail, and when you yield control and await some result, you can be pretty sure that you'll land on a different thread at least some of the time.
No. An async operations might execute synchronously on the current thread, or it might wind up completing on a different thread. It depends entirely on how the async API in question is implemented.
See Do the new C# 5.0 'async' and 'await' keywords use multiple cores? for a decent explanation. The implementation details of F# and C# async are different, but the overall principles are the same.
The builder that implements the F# async computation expression is here.
Examples:
Asynchronous method with its own dispatching:
// Library
func asyncAPI(callback: Result -> Void) {
dispatch_async(self.queue) {
...
callback(result)
}
}
// Caller
asyncAPI() { result in
...
}
Synchronous method with exposed dispatch queue:
// Library
func syncAPI() -> Result {
assert(isRunningOnCorrectQueue())
...
return result
}
// Caller
dispatch_async(api.queue) {
let result = api.syncAPI()
...
}
These two examples behave the same but I am looking to learn whether one of these ends up complicating a larget codebase more than the other, especially when there is a lot of asynchrony.
I would argue against both of the patterns you propose.
For the first pattern (where the API manages it's own backgrounding) I see little or no benefit to doing it this way, as opposed to leaving it to the caller. If you want to use a private, serial queue to protect data (or any other sort of critical section) internal to your API, that's fine, but that queue should be private, and it should specifically not target any public, non-global-concurrent queue (Note: it should especially not target the main queue). Ideally, the primary implementation of your API would also take a second parameter, so callers can specify on which queue to invoke the callback. (People can work around the lack of such a parameter by passing a callback block that re-dispatches to their desired queue, but I think that's clunkier than having an extra, optional parameter.) This puts the API consumer in complete control of the concurrency, while preserving your freedom to use queues internally to protect state.
As to the second approach, it's my opinion that we all should avoid creating new synchronous, blocking API. When you provide a synchronous, blocking API and don't provide a callback-based version, that means that you have denied consumers of your API any opportunity to avoid blocking. When you only provide synchronous, blocking API, then if someone wants to call your API in the background, at least one thread (in addition to any additional threads that your API consumes behind the scenes) will be consumed from the finite number of threads available to each process. (In the worst case this can lead to starvation conditions that are effectively deadlocks.)
Another red flag with this second example is that it vends a queue; Any time an API vends a queue, something is amiss. As mentioned, if you want to use a private serial queue to protect state or other critical sections internal to your API, go for it, but don't expose that queue to the outside world. If nothing else, it unnecessarily exposes details of your implementation. In looking at the system framework headers, I couldn't find a single case where a dispatch_queue_t was vended where it wasn't immediately obvious that the intent was for the API consumer to push in the queue, and not read it out.
It's also worth mentioning that these patterns are problematic regardless of whether your workload is CPU-bound or IO-bound. If it's CPU-bound, then not managing your own dispatch gives consumers of the API explicit control over how this CPU work is executed. If your workload is IO-bound, then you should use the OS- and libdispatch-provided asynchronous IO mechanisms (dispatch_io, dispatch_sources, kevent, etc) to avoid consuming a thread (or more than one) for the duration of your work.
Another answer here implied that forcing consumers to manage their own concurrency leads to "boilerplate" code. If you feel that the burden of API consumers potentially having to wrap calls to your API with dispatch_async is too great, then feel free to provide a convenience overload that dispatches to the default global concurrent queue, but please always leave the version that allows API consumers the ability to explicitly manage their own concurrency.
If, on the other hand, all this is internal to the implementation, and not part of the public API, then do whatever is most expedient, knowing that you can refactor the implementation behind the public API any time in the future.
As you said, the 2 generally accomplish the same thing but the first is more preferable in most scenarios. There are several benefits to using the first method.
The API is simpler. You simply call the method and provide code for the callback block.
Less boilerplate code, No typing dispatch_async every time you want to call it as it is just included in the method itself.
Less room for bugs/errors. By wrapping the asynchronous logic inside the method itself, you ensure that it is called on the right queue internally without the caller having to worry about any of that.
Touching on the last point, you also have finer control over the queue itself. Let's say you are trying to perform certain tasks on a particular queue. It is way simpler to simply wrap the code in a GCD call on that queue a single time rather than having to remember to reuse that same queue every time you want to call the method.
We have a website that is struggling with concurrent users right now.
Here is the very high-level background of the project:
Legacy ASP.NET MVC 3 project (.NET 4)
Can't do any major rewriting of core code
Main entry point that takes the longest time to execute is the SubmitSearch action on the Search controller. Average time to respond is 5-10 seconds.
So as the second point outlines, we don't want to spend too much time on this project rewriting large sections. However, we want to attempt to increase concurrent users. We're not looking to change anything else or increase performance since it would require much more work.
What we are seeing is that as more people hit SubmitSearch, the web site in general slows down. That's most likely due to all the IIS threads being locked up executing the search.
We are looking to implement AsyncController and making the SubmitSearch action execute on a normal CLR thread. Here's how we wanted to implement it:
Assume this is the original SubmitSearch method:
/// <summary>
/// Submits a search for execution.
/// </summary>
/// <param name="searchData">The search data</param>
/// <returns></returns>
public virtual ActionResult SubmitSearch(SearchFormModel searchData)
{
//our search code
}
The quickest way we were hoping to convert to AsyncController is to simply do this:
/// <summary>
/// Submits a search for execution.
/// </summary>
/// <param name="searchData">The search data</param>
/// <returns></returns>
protected virtual ActionResult SubmitSearch(SearchFormModel searchData)
{
//our search code
}
/// <summary>
/// Asynchronous Search entry point
/// </summary>
/// <param name="searchData"></param>
public void SubmitSearchAsync(SearchFormModel searchData)
{
AsyncManager.OutstandingOperations.Increment();
System.Threading.Tasks.Task.Factory.StartNew(() =>
{
ActionResult result = SubmitSearch(searchData);
AsyncManager.Parameters["result"] = result;
AsyncManager.OutstandingOperations.Decrement();
});
return;
}
/// <summary>
/// Called when the asynchronous search has completed
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
public ActionResult SubmitSearchCompleted(ActionResult result)
{
//Just return the action result
return result;
}
Of course this didn't work because all through-out the code, we are referencing HttpContext.Current, which we know ends up being null in this approach.
So we were then hoping to do this with SubmitSearchAsync:
/// <summary>
/// Asynchronous Search entry point
/// </summary>
/// <param name="searchData"></param>
public void SubmitSearchAsync(SearchFormModel searchData)
{
AsyncManager.OutstandingOperations.Increment();
System.Threading.Tasks.Task.Factory.StartNew(() =>
{
ActionResult result = null;
AsyncManager.Sync(() =>
{
result = SubmitSearch(searchData);
});
AsyncManager.Parameters["result"] = result;
AsyncManager.OutstandingOperations.Decrement();
});
return;
}
This fixes the issue.
So here's my concern:
Does wrapping the execution of SubmitSearch in the AsyncManager.Sync method defeat the purpose of using this model? In other words, when we are within the AsyncManager.Sync method, are we back on the IIS threads, which puts us back at square one?
Thanks
Does wrapping the execution of SubmitSearch in the AsyncManager.Sync method defeat the purpose of using this model? In other words, when we are within the AsyncManager.Sync method, are we back on the IIS threads, which puts us back at square one?
More or less, yes. But unfortunately, in your case, using Task.Factory.StartNew also defeats the purpose of using an async controller. With the approach you're trying to use, you can't win.
IIS threads, threads started by ThreadPool.QueueUserWorkItem, and Task threads, are all taken from the same thread pool.
In order to gain any benefit from async controllers, you need true async methods. In other words, methods like Stream.ReadAsync or WebRequest.GetResponseAsync. These specially-named methods use I/O completion ports instead of normal threads, which use hardware interrupts and operate on a different thread pool.
I wrote about this a long time ago in my answer here: Using ThreadPool.QueueUserWorkItem in ASP.NET in a high traffic scenario. Tasks and awaiters are pretty sweet, but they don't change the fundamental dynamics of the .NET thread pool.
One thing to note is that there is an option, TaskCreationOptions.LongRunning, that you can specify when creating a Task, which essentially informs the framework that the task will be doing a lot of waiting, and in theory the TPL will attempt to avoid scheduling it in the thread pool. In practice, this probably won't be very practical on a high-traffic site because:
The framework doesn't actually guarantee that it won't use the thread pool. That's an implementation detail, and the option is simply a hint that you provide.
Even if it does avoid the pool, it still needs to use a thread, which is essentially like using new Thread - if not literally then at least effectively so. What this means is heavy context-switching, which absolutely kills performance and is the main reason why thread pools exist in the first place.
A "search" command clearly implies some kind of I/O, which means there's probably a real asynchronous method you can use somewhere, even if it's the old-style BeginXyz/EndXyz. There are no shortcuts here, no quick fixes; you'll have to re-architect your code to actually be asynchronous.
The .NET framework can't inspect what's going on inside your Task and magically convert it into an interrupt. It simply cannot make use of an I/O completion port unless you refer directly to the specific methods that are aware of them.
Next web or middleware application you work on, try to consider this ahead of time and avoid synchronous I/O like the plague.
I think #Aaronaught has the best answer so far: you need true asynchronous processing in order to scale (i.e., Begin/End, not just using thread pool threads), and that there are no shortcuts or quick fixes to asynchronous code - it will take a re-architecting of at least that portion.
Which brings me to this part of your question:
we don't want to spend too much time on this project rewriting large sections.
The best bang for your buck is probably to purchase more memory and stick that in the server. (You should check with a profiler first just to make sure it is a memory issue - memory usually is the limiting factor on ASP.NET but it's best to check first).
As much as we developers love to solve problems, the truth is we can burn a lot of hours, e.g., changing synchronous code to asynchronous. FYI, the Task-based Asynchronous Pattern (async/await) in .NET 4.5 will allow you to change synchronous code to asynchronous much more easily.
So for now I say buy a couple RAM chips and make a mental note to do the (much easier) upgrade to async after you change to .NET 4.5.
I would start by looking at the performance of the server itself and then consider using the profiling tools in visual studio to identify exactly what and where the bottleneck is. Consider looking at the mini profiler a discussion of which can be found here http://www.hanselman.com/blog/NuGetPackageOfTheWeek9ASPNETMiniProfilerFromStackExchangeRocksYourWorld.aspx. Generally agree with the comment above about thread consumption.
There are dozens of reasons that can cause the server to slow down. If we only talk about threads, threads consume a minimum of 1/4 memory each, that means the more threads drawn up from the thread pool, the more memory is consumed. This can be one of the problems causing the server to slow down.
If the response from server takes over 10 seconds, consider using Asynchronous. Like in your code, make SubmitSearchAsync function Asynchronously, it will avoid blocking a thread, and also releases the thread back to thread pool. However, like the code you provided, when a request is received from the SubmitSearchAsync action, a thread is drawn from the thread pool to executed its body.
The SubmitSearch is a synchronous action, it waits until the implementation is finished, and it blocks the thread until the implementation finishes.
In other word, you released one thread, but you also blocked another thread. If you need to synchronize code from an asynchronous thread, use the AsyncManager.Sync method. But in your case, AsyncManager.Sync might not help much. I suggest two possible solutions:
1) manually spawning a thread:
public virtual ActionResult SubmitSearch(SearchFormModel searchData){
new Thread(() => {
//your search code
}).Start();
}
In this case, your search code might take longer, but the execution of the search will be done on a thread not a part of the pool.
2) change the SubmitSearch function asynchronously along with using Parallelism:
protected virtual async Task<ActionResult> SubmitSearch(SearchFormModel searchData){
// Make your search code using Parallel task like below.
var task1 = DoingTask1Async(searchData);
var task2 = DoingTask2Async(searchData)
await Task.WhenAll(task1,task2);
}
Aside from above suggestion, consider using Cancellation tokens, it further reduces thread usage.
Hope it helps.
What we are seeing is that as more people hit SubmitSearch, the web site in general slows down. That's most likely due to all the IIS threads being locked up executing the search.
If it was the threads locked up then it wouldn't be a slow down but probably http errors were returned. Can I ask how many parallel hits causes the slow down? The threadpool in .Net4 is quite big. Also, if your search takes 10 seconds that means your database is doing the heavy-lifting. I would have a look at the DB performance: if other parts of your site are also DB dependent then several parallel DB intensive searches will slow down your application.
If for some reason you can't/don't want to perfmon the database then here is a simple test: change the database search call to a sleep call for X seconds (around 10 in this case). Then run your parallel requests and see if the site responsiveness drops or not. Your request thread numbers are the same so if that was the reason then it should have the same effect.
Lets say I wanted to scrape a webpage, and extract some data. I'd most likely write something like this:
let getAllHyperlinks(url:string) =
async { let req = WebRequest.Create(url)
let! rsp = req.GetResponseAsync()
use stream = rsp.GetResponseStream() // depends on rsp
use reader = new System.IO.StreamReader(stream) // depends on stream
let! data = reader.AsyncReadToEnd() // depends on reader
return extractAllUrls(data) } // depends on data
The let! tells F# to execute the code in another thread, then bind the result to a variable, and continue processing. The sample above uses two let statements: one to get the response, and one to read all the data, so it spawns at least two threads (please correct me if I'm wrong).
Although the workflow above spawns several threads, the order of execution is serial because each item in the workflow depends on the previous item. Its not really possible to evaluate any items further down the workflow until the other threads return.
Is there any benefit to having more than one let! in the code above?
If not, how would this code need to change to take advantage of multiple let! statements?
The key is we are not spawning any new threads. During the whole course of the workflow, there are 1 or 0 active threads being consumed from the ThreadPool. (An exception, up until the first '!', the code runs on the user thread that did an Async.Run.) "let!" lets go of a thread while the Async operation is at sea, and then picks up a thread from the ThreadPool when the operation returns. The (performance) advantage is less pressure against the ThreadPool (and of course the major user advantage is the simple programming model - a million times better than all that BeginFoo/EndFoo/callback stuff you otherwise write).
See also http://cs.hubfs.net/forums/thread/8262.aspx
I was writing an answer but Brian beat me to it. I fully agree with him.
I'd like to add that if you want to parallelize synchronous code, the right tool is PLINQ, not async workflows, as Don Syme explains.