I have an MVC project, and I'm looking at speeding things up. One thing I'm scratching my head over is the BeginProcesRequest() which I have no control over. Using New Relic I found that this method is, on average consuming 90% of the time required for the transaction to complete.
The code in my controller is pretty simple. It has a look for an action session for the user and redirects to their dashboard if it finds one. there isn't any database calls on the actual page. The only written is:
if (Session["UserID"] != null)
// Perform actions
The BeginProcessRequest() method takes almost 4 seconds as you can see in the screenshot
This can't be something unique to my site? I'm using a small EC2 instance for the server, and although there are other applications running on the site the CPU and memory stay pretty much at 0 throughout the request.
EDIT - Reviewed the following post:
What happens in BeginProcessRequest()?
However as my application is idle when the most time consuming requests take place I can't see how it could be related to competing threads.
I think the issue was with IIS, as after I changed the property of idle time-out in the application pool to be one day it now seems to load much faster on initial start.
I also explicitly disabled the session state on my home controller, and ensured that SQL Server's auto close parameter was set to off.
Related
I use ASP.Net MVC 5 and I have a long running action which have to poll webservices, process data and store them in database.
For that I want to use TPL library to start the task async.
But I wonder how to do 3 things :
I want to report progress of this task. For this I think about SignalR
I want to be able to left the page where I start this task from and be able to report the progression across the website (from a panel on the left but this is ok)
And I want to be able to cancel this task globally (from my panel on the left)
I know quite a few about all of technologies involved. But I'm not sure about the best way to achieve this.
Is someone can help me about the best solution ?
The fact that you want to run long running work while the user can navigate away from the page that initiates the work means that you need to run this work "in the background". It cannot be performed as part of a regular HTTP request because the user might cancel his request at any time by navigating away or closing the browser. In fact this seems to be a key scenario for you.
Background work in ASP.NET is dangerous. You can certainly pull it off but it is not easy to get right. Also, worker processes can exit for many reasons (app pool recycle, deployment, machine reboot, machine failure, Stack Overflow or OOM exception on an unrelated thread). So make sure your long-running work tolerates being aborted mid-way. You can reduce the likelyhood that this happens but never exclude the possibility.
You can make your code safe in the face of arbitrary termination by wrapping all work in a transaction. This of course only works if you don't cause non-transacted side-effects like web-service calls that change state. It is not possible to give a general answer here because achieving safety in the presence of arbitrary termination depends highly on the concrete work to be done.
Here's a possible architecture that I have used in the past:
When a job comes in you write all necessary input data to a database table and report success to the client.
You need a way to start a worker to work on that job. You could start a task immediately for that. You also need a periodic check that looks for unstarted work in case the app exits after having added the work item but before starting a task for it. Have the Windows task scheduler call a secret URL in your app once per minute that does this.
When you start working on a job you mark that job as running so that it is not accidentally picked up a second time. Work on that job, write the results and mark it as done. All in a single transaction. When your process happens to exit mid-way the database will reset all data involved.
Write job progress to a separate table row on a separate connection and separate transaction. The browser can poll the server for progress information. You could also use SignalR but I don't have experience with that and I expect it would be hard to get it to resume progress reporting in the presence of arbitrary termination.
Cancellation would be done by setting a cancel flag in the progress information row. The app needs to poll that flag.
Maybe you can make use of message queueing for job processing but I'm always wary to use it. To process a message in a transacted way you need MSDTC which is unsupported with many high-availability solutions for SQL Server.
You might think that this architecture is not very sophisticated. It makes use of polling for lots of things. Polling is a primitive technique but it works quite well. It is reliable and well-understood. It has a simple concurrency model.
If you can assume that your application never exits at inopportune times the architecture would be much simpler. But this cannot be assumed. You cannot assume that there will be no deployments during work hours and that there will be no bugs leading to crashes.
Even if using http worker is a bad thing to run long task I have made a small example of how to manage it with SignalR :
Inside this example you can :
Start a task
See task progression
Cancel task
It's based on :
twitter bootstrap
knockoutjs
signalR
C# 5.0 async/await with CancelToken and IProgress
You can find the source of this example here :
https://github.com/dragouf/SignalR.Progress
I have a particularly long running method that I need to execute from my controller. The Method is in it's own Model. I am using an async controller, and I have the method setup using asyncFunc library to make it asynchronous. I have also tried invoking it on it's own process. The problem is I want to controller to go ahead and return a view so the user can continue doing other things as the method will notify the user it is completed or has any errors via e-mail.
The problem is even thogh it is an asynchronous method the controller will not move forward to return the view until the process is done. 15+ mins. and if you navigate to a different page the method stops trying to execute.
so how can I get the method to execute as a worker and free up the controller?
Any Help would be greatly appreciated.
all the best,
Chase Q, Aucoin
Use ThreadPool.QueueUserWorkItem() as a fire-and-forget approach in the ASPX page.
Do the long-running work in the WaitCallback you pass to QUWI.
when the work is complete, that WaitCallback can send an email, or whatever it wants.
You need to take care to handle the case that the w3wp.exe is stopped during the 15 minute run. What will you do if the work is 2/3 complete? Some options are, making the work restartable, or just allowing the interrupted work to be forgotten.
Making it restartable might mean, when w3wp.exe restarts, your ASP.NET logic makes sure to begin again, any work that was interrupted. It might mean that your ASP.NET logic sets "syncpoints" so that it knows where to restart.
If you want the restartable option, you might think about Workflow, which is specifically designed for this purpose - maintaining state of long-running workflows, restarting automatically, and so on. If you use Workflow, you can set it to run asynchronously, and you may decide you do not need QueueUserWorkItem.
see also:
Moving a time taking process away from my asp.net application
the Workflow Foundation tag
This will help > http://msdn.microsoft.com/en-us/library/ms227433.aspx
It is the standard way of running a background process on the server in the .NET stack.
I don't know why, but I still live in conviction that this should not be done. Executing background threads in ASP.NET smells. You will also steal threads from ASP.NET thread pool which is controlled by IIS. It can decide that something is wrong with your worker process and restart it any time just to keep memory consumption, processing time consumption or thread consumption low. If you need background logic create custom NT service and call the process on that service either via old .NET remoting or WCF.
Btw. approach I described is used frequently in commercial applications and those which doesn't use it often self-host the whole web server.
I'm going to create a view counter for articles. I have some questions:
Should I ignore article's author
when he opens the article?
I don't want to update database each
time. I can store in a
Dictionary<int, int> (articleId, viewCount) how many times
each article was viewed. After 100
hits I can update the database.
I should only count the hit once per
hour for each user and article. (if
the user opens one article many
times during one hour the view count
should be incremented only once).
For each question I want to know your suggestions how to do it right.
I'm especially interested how to do #3. Should I store the time when the user opened the article in a cookie? Does it mean that I should create a new cookie for each page?
I think I know the answer - they are analyzing the IIS log as Ope suggested.
Hidden image src is set to
http://stackoverflow.com/posts/3590653/ivc/[Random code]
[Random code] is needed because many people may share the same IP (in a network, for example) and the code is used to distinguish users.
Sure - I think that is a good idea
and 3. are related: The issue is where would you actually store this dictionary and logic.
An ASP.NET application or session scope are of course the easiest choice, but there you really need to understand the logic of application pools. ASP.NET applications are recycled from time to time: when there is no action on the site for a certain period or in special situations - e.g. if the process starts to take too much memory the application is shut down and a new one is started in the next request. There are events for session and application shut-down, but at least some years ago they were not really reliable: In many special cases they did not always fire. Perhaps they are better now, but it is painful to test. And 1 hour is really a long time: Usually sessions are kept alive only like 20 minutes after last request.
A reliable way would be to have a separate Windows service (a lot of work to program) or always storing to database with double-view analyses (quite a lot of overhead for such a small feature).
Do you have access to IIS logs? How about analyzing IIS logs e.g. every 30 minutes with some kind of timer process and taking the count from there? Or then just store all the hits to the database with user information and calculate the unique hits with a similar timed process.
One final question: Are you really sure none of the thousands of counter applications/services in the Internet wouldn't do the job close enough to your requirements?
Good luck!
This is the screenshot of this page in Firebug. You can see that there is a request which returns 204 status code (No Content).
This is stackoverflow's view counter. They are using a hidden image which point to a controller's action.
I have many articles. How to track which articles the user visited already?
P.S. BTW, why is this request made two times?
I have an ASP.NET MVC 2 Beta application where I need to block incoming requests for a specific action until I have some data available to return or just release the request after 30 seconds with no new data available.
In order to accomplish this, I'm using AutoResetEvent.WaitOne(30000);
The big issue is that IIS does not seem to be accepting any new request while the thread is blocked at the WaitOne instruction. New requests get hung till the thread releases.
I need to be able to parallelize the requests while still keeping the WaitOne behavior.
Async handlers are what you're looking for. If you're building a comet solution, you may want to check out our .NET implementation of a comet server here, it'll save you some time. If you're wanting to roll your own, you'll definately need to use the async handlers to avoid hitting upper concurrency limits by the time you get past 60 or 70 users, but even with the async handlers, you'll still have to do some fancy footwork. Basically, you're still going to hit some upper limits in the threadpool unless you hand off the requests into a bounded thread pool that can basically manage all the incoming requests for you.
Good luck!
You should not be blocking incoming requests at all. If the data you need are not ready, then return an empty response, or perhaps return an error code.
For a web application, it is more advisable (not a hard rule) to return a message to tell the users to retry again later due to whatever reason you want to call it.
Stalling/blocking the requests by 'waiting' doesn't really help much as the wait is undeterministic, unless of course you have a mechanism to make it so.
I do not know the nature/context/traffic pattern of your website. 30 seconds can be a number that works for you. Perhaps my points above are not really relevant, just my 2 cents.
Actually, it turns out that this behavior only happens with ASP.NET MVC 2 Beta. I had this working fine with MVC 2 Preview 2 and rolled back to this version to re-test and confirmed that the application worked fine with that version.
Now, the question is: Why am I seeing this different behavior between these two MVC release versions, and what is the correct behavior I should expect to get in this scenario?
I used System.Timers.timer in global.as in asp.net to set a timer for scheduling execute a
function
let' say transferMoney().
But it seems that this timer might stop after several hours unexpected.
And this cause that all the actions are pending.
I want to know whether there are any better methods to set up a timer in asp.net, MVC 1.0?
Thanks in advance!
It might just be because the application got recycled. Global.ashx is not really the right place to do long running tasks because if your AppDomain gets recycled your timer will die. I suggest making a job windows service instead.
Edit: Well, it's fairly easy to create a windows service project in Visual Studio just do [File] > [Add] > [New Project...] > [Windows] > [Windows Service] and you will get the stub code for the project.
It's hard to come up with a complete example so i suggest you google it. ;) There are tons of samples out there for you to look at.
This article on CodeProject seems to be a good introduction to Windows Services.
Any timer you'll use in ASP.NET apps will eventually "terminate", but this a very expected behavior due to process recycling.
The timer will never work because IIS will reschedule the worker process regularly based on Application Pool settings, so when it recycles your timer will get destroyed and you might need to reopen it.
You can put a check on whether timer object is still available or not, if not available then create it !!, using any other timer object will not work. But this still has a problem, because if you dont have any web request for particular period of time, it will still get destroyed. Best is to setup a ping monitor from other place which can keep your website alive.
You can't reliably run a timer in ASP.NET. If there are no requests coming in, the IIS can shut down the application, and it will not start until the next request arrives.
Why do you think that you need a timer? In most web applications this is not needed at all to do periodical updates unless they depend on an external source.
If you are just moving data around inside your application, the actual transactions doesn't have to happen at an exact interval, you only have to calculate what the result would be if they had happened. Whenever a request comes in, you calculate how many transactions would have happened since the last request, and do them to catch up to the current state.
If your transactions rely on an external source so that they actually has to run at a specific time, you simply can't do it with ASP.NET alone. You need an application that runs outside IIS, for example started periodically by the windows scheduler.
You could try the system.threading.timer
http://msdn.microsoft.com/en-us/library/system.threading.timer.aspx