From Pro Asynchrnous Programming with .Net:
for (int nTry = 0; nTry < 3; nTry++)
{
try
{
AttemptOperation();
break;
}
catch (OperationFailedException) { }
Thread.Sleep(2000);
}
While sleeping, the thread doesn’t consume any CPU-based resources,
but the fact that the thread is alive means that it is still consuming
memory resources. On a desktop application this is probably no big
deal, but on a server application, having lots of threads sleeping is
not ideal because if more work arrives on the server, it may have to
spin up more threads, increasing memory pressure and adding additional
resources for the OS to manage.
Ideally, instead of putting the thread to sleep, you would like to
simply give it up, allowing the thread to be free to serve other
requests. When you are ready to continue using CPU resources again,
you can obtain a thread ( not necessarily the same one ) and continue
processing. You can solve this problem by not putting the thread to
sleep, but rather using await on a Task that is deemed to be completed
in a given period.
for (int nTry = 0; nTry < 3; nTry++)
{
try
{
AttemptOperation();
break;
}
catch (OperationFailedException) { }
await Task.Delay(2000);
}
I don't follow author's reasoning. While it's true that calling await Task.Delay will release this thread ( which is processing a request ), but it's also true that task created by Task.Delay will occupy some other thread to run on. So does this code really enable server to process more simultaneous requests or is the text wrong?!
Task.Delay does not occupy some other thread. It gives you a task without blocking. It starts a timer that completes that task in its callback. The timer does not use any thread while waiting.
It is a common myth that async actions like delays or IO just push work to a different thread. They do not. They use OS facilities to truly use zero threads while the operation is in progress. (They obviously need to use some thread to initiate and complete the operation.)
If async was just pushing work to a different thread it would be mostly useless. It's value would be just to keep the UI responsive in client apps. On the server it would only cause harm. It is not so.
The value of async IO is to reduce memory usag (less thread stacks), context switching and thread-pool utilization.
The async version of the code you posted would scale to literally tens of thousands of concurrent requests (if you increase the ASP.NET limits appropriately, which is a simple web.config change) with small memory usage.
Related
Does async operation in iOS, internally create a new thread, and allocate task to it ?
An async operation is capable to internally create a new thread and allocate task to it. But in order for this to happen you need to run an async operation which creates a new thread and allocates task to it. Or in other words: There is no direct correlation.
I assume that by async you mean something like DispatchQueue.main.async { <#code here#> }. This does not create a new thread as main thread should already be present. How and why does this work can be (if oversimplified) explained with an array of operations and an endless loop which is basically what RunLoop is there for. Imagine the following:
Array<Operations> allOperations;
int main() {
bool continueRunning = true;
for(;continueRunning;) {
allOperations.forEach { $0.run(); }
allOperations.clear();
}
return 0;
}
And when you call something like DispatchQueue.main.async it basically creates a new operation and inserts it into allOperations. The same thread will eventually go into a new loop (within for-loop) and call your operation asynchronously. Again keep in mind that this is all over-simplified just to illustrate the idea behind all of it. You can from this also imagine how for instance timers work; the operation will evaluate if current time is greater then the one of next scheduled execution and if so it will trigger the operation on timer. That is also why timers can not be very precise since they depend on rest of execution and thread may be busy.
A new thread on the other hand may be spawned when you create a new queue DispatchQueue(label: "Will most likely run on a new thread"). When(if) exactly will a thread be made is not something that needs to be fixed. It may vary from implementations and systems being run on. The tool will only guarantee to perform what it is designed for but not how it will do it.
And then there is also Thread class which can generate a new thread. But the deal is same as for previous one; it might internally instantly create a new thread or it might do it later, lazily. All it guarantees is that it will work for it's public interface.
I am not saying that these things change over time, implementation or system they run on. I am only saying that they potentially could and they might have had.
I am working with an external device that I receive data from. I want to handle its data read/write queue asynchronously, in a thread.
I've got it mostly working: There is a class that simply manages the two streams, using the NSStreamDelegate to respond to incoming data, as well as responding to NSStreamEventHasSpaceAvailable for sending out data that's waiting in a buffer after having failed to be sent earlier.
This class, let's call it SerialIOStream, does not know about threads or GCD queues. Instead, its user, let's call it DeviceCommunicator, uses a GCD queue in which it initializes the SerialIOStream class (which in turn creates and opens the streams, scheduling them in the current runloop):
ioQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT,0);
dispatch_async(ioQueue, ^{
ioStreams = [[SerialIOStream alloc] initWithPath:[#"/dev/tty.mydevice"]];
[[NSRunLoop currentRunLoop] run];
});
That way, the SerialIOStreams stream:handleEvent: method runs in that GCD queue, apparently.
However, this causes some problems. I believe I run into concurrency issues, up to getting crashes, mainly at the point of feeding pending data to the output stream. There's a critical part in the code where I pass the buffered output data to the stream, then see how much data was actually accepted into the stream, and then removing that part from my buffer:
NSInteger n = self.dataToWrite.length;
if (n > 0 && stream.hasSpaceAvailable) {
NSInteger bytesWritten = [stream write:self.dataToWrite.bytes maxLength:n];
if (bytesWritten > 0) {
[self.dataToWrite replaceBytesInRange:NSMakeRange(0, bytesWritten) withBytes:NULL length:0];
}
}
The above code can get called from two places:
From the user (DeviceCommunicator)
From the local stream:handleEvent: method, after being told that there's space in the output stream.
Those may be (well, surely are) running in separate thread, and therefore I need to make sure they do not run concurrently this code.
I thought I'd solve this by using the following code in DeviceCommunicator when sending new data out:
dispatch_async (ioQueue, ^{
[ioStreams writeData:data];
});
(writeData adds the data to dataToWrite, see above, and then runs the above code that sends it to the stream.)
However, that doesn't work, apparently because ioQueue is a concurrent queue, which may decide to use any available thread, and therefore lead to a race condition when writeData get called by the DeviceCommunicator while there's also a call to it from stream:handleEvent:, on separate threads.
So, I guess I am mixing expectations of threads (which I'm a bit more familiar with) into my apparent misunderstandings with GCD queues.
How do I solve this properly?
I could add an NSLock, protecting the writeData method with it, and I believe that would solve the issue in that place. But I am not so sure that that's how GCD is supposed to be used - I get the impression that'd be a cludge.
Shall I rather make a separate class, using its own serial queue, for accessing and modifying the dataToWrite buffer, perhaps?
I am still trying to grasp the patterns that are involved with this. Somehow, it looks like a classic producer / consumer pattern, but on two levels, and I'm not doing this right.
Long story, short: Don't cross the streams! (haha)
NSStream is a RunLoop-based abstraction (which is to say that it intends to do its work cooperatively on an NSRunLoop, an approach which pre-dates GCD). If you're primarily using GCD to support concurrency in the rest of your code, then NSStream is not an ideal choice for doing I/O. GCD provides its own API for managing I/O. See the section entitled "Managing Dispatch I/O" on this page.
If you want to continue to use NSStream, you can either do so by scheduling your NSStreams on the main thread RunLoop or you can start a dedicated background thread, schedule it on a RunLoop over there, and then marshal your data back and forth between that thread and your GCD queues. (...but don't do that; just bite the bullet and use dispatch_io.)
I watched this video : https://channel9.msdn.com/Events/TechDays/Techdays-2012-the-Netherlands/2287. So i tried to implement the usage of async/await in a controller. SO this is basicaly what I did :
public class HomeController : Controller
{
private static WebClient _webClient = new WebClient();
public async Task<ActionResult> IndexAsync()
{
var data = await _webClient.DownloadStringTaskAsync("http://stackoverflow.com/");
return View("Index", (object)data);
}
public ActionResult Index()
{
var data = _webClient.DownloadString("http://stackoverflow.com/");
return View("Index", (object)data);
}
}
Then I used Apache Benchmark and did the two following tests :
ab -n 100 -c 100 http://localhost:53446/Home/index
and
ab -n 100 -c 100 http://localhost:53446/Home/indexasync
And I got the exact same performance (I have 8 CPU core). Why is that ?
Async is not about performance. That's just categorically incorrect. In fact, an async request will often be less performant that sync, simply because there's additional overhead involved with async.
The reason to use async is about efficient resource-management and scale. A typical web server process will have around 1000 threads. This is often called the "max requests", as one thread general equals one request. If you have an 8 core CPU, you should ideally have a process per core (in IIS those are called "web workers"). So, theoretically, you'd have around 8000 threads total to work with.
That's quite a lot actually, though a modern web page consumes more requests than most people think. The page itself is one request, but that page will have images and external JS and CSS files, all of which generate a request, and will often utilize AJAX, for further requests. The point is that while 8000+ threads is still quite a lot to have in your pool, you could still very well run out if the server is under significant load.
Async merely gives you breathing room above that limit. In situations where the thread enters a wait-state, it can be returned to the pool to field other requests while whatever external action is being completed. The alternative is that the thread would just sit there idle (sync). That's really all there is to it. It's entirely about tasking those otherwise idle threads with some other bit of work, which could mean the difference between requests queuing up and timing out or being handled, even if slowly.
Running a load test that exhausts the thread pool is difficult to do on a local box. It's a lot easier to pretend the thread pool is exhausted by artificially restricting it, as I do in my gist:
protected void Application_Start()
{
int workerThreads, ioThreads;
ThreadPool.GetMaxThreads(out workerThreads, out ioThreads);
ThreadPool.SetMaxThreads(Environment.ProcessorCount, ioThreads);
...
}
There are a couple of reasons that stand out.
From Using Asynchronous Methods in ASP.NET MVC 4
the number of threads in the thread pool is limited (the default maximum for .NET 4.5 is 5,000). In large applications with high concurrency of long-running requests, all available threads might be busy. This condition is known as thread starvation.
So, running 100 request at a time will not even begin to starve your threads.
Also, a simple GET request will run very quickly. A test that performs an action that takes multiple seconds or even minutes would bear more obvious performance gains.
I have a specific task routine which performs some operations in a specific order, and these operations handle few volatile variables. There is a specific interrupt which updates these volatile variables asynchronously. Hence, the task routine should restart if such an interrupt occurs. Normally FreeRTOS will resume the task, but this will result in wrong derived values, hence the requirement for restarting the routine. I also cannot keep the task routine under critical section, because I should not be missing any interrupts.
Is there a way in FreeRTOS with which I can achieve this? Like a vtaskRestart API. I could have deleted the task and re-created it, but this adds a lot of memory management complications, which I would like to avoid. Currently my only option is to add checks in the routine on a flag to see if a context switch have occured and if yes, restart, else continue.
Googling did not fetch any clue on this. Seems like people never faced such a problem or may be its that this design is poor. In FreeRTOS forum, few who asked for a task-restart didn't seem to have this same problem. stackOverflow didn't have a result on freertos + task + restart. So, this could be the first post with this tag combination ;)
Can someone please tell me if this is directly possible in FreeRTOS?
You can use semaphore for this purpose. If you decide using semaphore, you should do the steps below.
Firstly, you should create a binary semaphore.
The semaphore must be given in the interrupt routine with
xSemaphoreGiveFromISR( Example_xSemaphore, &xHigherPriorityTaskWoken
);
And, you must check taking semaphore in the task.
void vExample_Task( void * pvParameters )
{
for( ;; )
{
if (xSemaphoreTake( Example_xSemaphore, Example_PROCESS_TIME)==pdTRUE)
{
}
}
}
For this purpose you should use a queue and use the queue peek function to yield at your volatile data.
I'm using it as I have a real time timer and this way I make the time available to all my task, without any blocking.
Here it how it goes:
Declare the queue:
xQueueHandle RTC_Time_Queue;
Create the queue of 1 element:
RTC_Time_Queue = xQueueCreate( 1, sizeof(your volatile struct) );
Overwrite the queue everytime your interrupt occurs:
xQueueOverwriteFromISR(RTC_Time_Queue, (void*) &time);
And from other task peek the queue:
xQueuePeek(RTC_GetReadQueue(), (void*) &TheTime, 0);
The 0 at the end of xQueuePeek means you don't want to wait if the queue is empty. The queue peek won't delete the value in the queue so it will be present every time you peek and the code will never stop.
Also you should avoid having variable being accessed from ISR and the RTOS code as you may get unexpected corruption.
I've seen 2 flavors of working with asyncronous operations in mvc controllers.
First:
public void GetNewsAsync()
{
AsyncManager.OutstandingOperations.Increment();
using (ManualResetEvent mre = new ManualResetEvent(false))
{
//Perform the actual operation in a worker thread
ThreadPool.QueueUserWorkItem((object _mre) =>
{
//do some work in GetFeed that takes a long time
var feed = GetFeed();
AsyncManager.Parameters["Feed"] = feed;
AsyncManager.OutstandingOperations.Decrement();
mre.Set();
}, mre);
//Wait for the worker thread to finish
mre.WaitOne(TimeSpan.FromSeconds(SomeNumberOfSecondsToWait));
}
}
Second:
public void GetNewsAsync()
{
AsyncManager.OutstandingOperations.Increment();
//Perform the actual operation in a worker thread
ThreadPool.QueueUserWorkItem((object x) =>
{
//do some work in GetFeed that takes a long time
var feed = GetFeed();
AsyncManager.Parameters["Feed"] = feed;
AsyncManager.OutstandingOperations.Decrement();
}, null);
}
The first blocks GetNewsAsync for SomeNumberOfSecondsToWait, the second does not. Both perform the work inside a of a worker thread and the results passed to GetNewsCompleted.
So my question is, which is the correct way to handle an Ajax call to GetNews; Wait, or don't wait?
I don't know where did you see the first example but that's a total anti-pattern that completely defeats the purpose of an asynchronous controller. The whole point of an asynchronous operation is to execute asynchronously and free the main thread as fast as possible.
This being said if GetFeed is a blocking call (which is what its name supposes it is) you get strictly 0 benefit from an asyncrhonous controller so the second example is also wrong for me. You could use a standard synchronous controller action in this case. With the second example you draw a thread from the pool and instead of blocking inside the main thread you block inside the other thread so the net effect is almost the same (in reality it's worse) if you had used a standard synchronous controller action.
So both those examples will bring more overhead than any benefit.
Where asynchronous controllers are useful is when you have some I/O intensive API such as a database or web service call where you could take advantage of IO Completion Ports. The following article provides a good example of this scenario. The newsService used there is providing real asynchronous methods and there is no blocking during the I/O network call. No worker thread being jeopardized.
I would also recommend you reading the following article. Even if it is for classic WebForms it still contains some very useful information.