Is it possible to commit a transaction in the background while already returning the view to the user when using nHibernate in an ASP.NET MVC application?
So upon reaching "ActionExecuted", which is normally the point at which the transaction is committed when using the session-per-request pattern, I want to continue right away while nHibernate starts committing. This would allow the user to see the resulting view earlier.
Instead of committing in the background (which you can do with a Thread, as long as you make sure the session gets cleaned afterwards), why not switch to a queue-based architecture?
Advantages:
It's actually designed for what you want, not a hack
You can scale out as far as you want (same app, different app, different server, different datacenter...)
If you build it correctly, it can offer even more reliability than a straight-to-db approach
Of course, there is a cost, which is creating the DTOs for the queue and then building the actual transaction. And also, the request is not really finished when you return control to the user (this is non-deterministic. The next request might find the data in the db or not)
Related
I have an old SL4/ria app, which I am looking to replace with breeze. I have a question about memory use and caching. My app loads lists of Jobs (a typical user would have access to about 1,000 of these jobs). Additionally, there are quite a few lookup entity types. I want to make sure these are cached well client-side, but updated per session. When a user opens a job, it loads many more related entities (anywhere from 200 - 800 additional entities) which compose multiple matrix-style views for the jobs. A user can view the list of jobs, or navigate to view 1 job at a time.
I feel that I should be concerned with memory management, especially not knowing how browsers might deal with this. Originally I felt this should all be 1 EntityManager and I would detachEntities when user navigates away from a job, but I'm thinking this might benefit from multiple managers by intended lifetime. Or perhaps I should create a new dataservice & EntityManager each time the user navigates to a new hash '/#/' area, since comments on clear() seems to indicate that this would be faster? If I did this, I suppose I will be using pub/sub to notify other viewmodels of changes to entities? This seems complex and defeating some of the benefits of breeze as the context.
Any tips or thoughts about this would be greatly appreciated.
I think I understand the question. I think I would use a multi-manager approach:
Lookups Manager - holds once-per session reference (lookup) entities
JobsView Manager - "readonly" list of Jobs in support of the JobsView
JobEditor Manager - One per edit session.
The Lookups Manager maintains the canonical copy of reference entities. You can fill it once with a single call to server (see docs for how). This Lookups Manager will Breeze-export these reference entities to other managers which Breeze-import them as they are created. I am assuming that, while numerous and diverse, the total memory footprint of reference entities is pretty low ... low enough that you can afford to have more than one copy in multiple managers. There are more complicated solutions if that is NOT so. But let that be for now.
The JobsView Manager has the necessary reference entities for its display. If you only displayed a projection of the Jobs, it would not have Jobs in cache. You might have an array and key map instead. Let's keep it simple and assume that it has all the Jobs but not their related entities.
You never save changes with this manager! When editing or creating a Job, your app always fires up a "Job Editor" view with its own VM and JobEditor Manager. Again, you import the reference entities you need and, when editing an existing Job, you import the Job too.
I would take this approach anyway ... not just because of memory concerns. I like isolating my edit sessions into sandboxes. Eases cancellation. Gives me a clean way to store pending changes in browser storage so that the user won't lose his/her work if the app/browser goes down. Opens the door to editing several Jobs at the same time ... without worrying about mutually dependent entities-with-changes. It's a proven pattern that we've used forever in SL apps and should apply as well in JS apps.
When a Job edit succeeds, You have to tell the local client world about it. Lots of ways to do that. If the ONLY place that needs to know is the JobsView, you can hardcode a backchannel into the app. If you want to be more clever, you can have a central singleton service that raises events specifically about Job saving. The JobsView and each new JobEditor communicate with this service. And if you want to be hip, you use an in-process "Event Aggregator" (your pub/sub) for this purpose. I'd probably be using Durandal for this app anyway and it has an event aggregator in the box.
Honestly, it's not that complicated to use and importing/exporting entities among managers is a ... ahem ... breeze. Well worth it compared to refreshing the Jobs List every time you return to it (although you'll want a "refresh button" too because OTHER users could be adding/changing those Jobs). You retain plenty of Breeze benefits: querying, validation, change-tracking, batch saves, entity navigation (those reference lists work "for free" in Breeze).
As a refinement, I don't know that I would automatically destroy the JobEditor view/viewmodel/manager when I returned to the JobsView. In my experience, people often return to the same Job that they just left. I might hold on to a view so you could go back and forth quickly. But now I'm getting tricky.
I'm running a db4o server with multiple clients accessing it. I just ran into the issue of one client not seeing the changes from another client. From my research on the web, it looks like there are basically two ways to solve it.
1: Call Refresh() on the object (from http://www.gamlor.info/wordpress/2009/11/db4o-client-server-and-concurrency/):
const int activationDeph = 4;
client2.Ext().Refresh(objFromClient2, activationDeph);
2: Instead of caching the IObjectContainer, open a new IObjectContainer for every DB request.
Is that right?
Yes, #1 is more efficient, but is that really realistic to specify which objects to refresh? I mean, when a DB is involved, every time a client accesses it, it should get the latest information. That's why I'm leaning towards #2. Plus, I don't have major efficiency concerns.
So, am I right that those are the two approaches? Or is there another?
And, wait a sec... what happens when your object goes out of scope? On a timer, I call a method that gets an object from the DB server. That method instantiates the object. Since the object went out of scope, it's not there to refresh. And when I call the DB, I don't see the changes from the client. In this case, it seems like the only option is to open a new IObjectContainer. No?
** Edit **
I thought I'd post some code using the solution I finally decided to use. Since there were some serious complexities with using a new IObjectContainer for every call, I'm simply going to do a Refresh() in every method that accesses the DB (see Refresh() line below). Since I've encapsulated my DB access into logic classes, I can make sure to do the Refresh() there, every time. I just tested this and it seems to be working.
Note: The Database variable below is the the db4o IObjectContainer.
public static ApplicationServer GetByName(string serverName)
{
ApplicationServer appServer = (from ApplicationServer server in Database
where server.Name.ToUpperInvariant() == serverName.ToUpperInvariant()
select server).FirstOrDefault();
Database.Ext().Refresh(appServer, 10);
return appServer;
}
1) As you said, the major problem with this that you usually really don't know what objects to refresh.
You can use the committed event to refresh objects as soon as any client has committed. db4o will distribute that event. Note that this also consumes some network traffic & time to send the events. And there will be a time frame where your objects have a stale state.
2) It actually the cleanest method, but not for every db request. Use a object container for every logical unit of work. Any operation which is one 'atomic' unit of work in your business-operations.
Anyway in general. db4o was never build with the client server scenario as first priority, and it shows in the concurrent scenarios. You cannot avoid working with stale (and even inconsistent) object state and there is no concurrency control options (except the low level semaphores).
My recommendation: Use a client container per unit of work. Be aware that even then you might get stale data, which then might lead to a inconsistent view & update. When there are rarely any contentions & races in your application scenario and you can tolerate a mistake once in a while, then this is fine. However if you really need to ensure correctness, then I recommend to use a database which has a better concurrency store =(
The context: a web application written in asp.net MVC + NHibernate. This is a card game where players play at the same time so their actions might modify the same field of an entity at the same time (they all do x = x + 1 on a field). Seems pretty classical but I don't know how to handle it.
Needless to say I can't present a popup to the user saying "The entity has been modified by another player. Merge or cancel ?". When you think that this is related to an action to a card, I can't interfere like this. My application has to internally handle this scenario. Since the field is in an entity class and each session has it own instance of the entity, I can't simply take a CLR lock. Does it mean I should use pessimistic concurrency so that each web request acting on this entity is queued until a player finished his action? In practical terms in means that each PlayCard request should use a lock?
Please, don't send me to NH doc about concurrency or alike. I'm after the technique that should be used in this case, not how to implement it in NH.
Thanks
It may make sense depending on your business logic to try second level caching. This may be a good depending on the length of the game and how it is played. Since the second level cache exists on the session factory level, the session factory will have to be managed according to the life time of the game. An Nh session can be created per request, but being spawned by a session factory configured for second level cache means data of interest is cached across all sessions. The advantage of using second level cache is that you can configure this on a class by class basis - caching only the entities your require. It also provides a variety of concurrency strategies depending on the cache provider. Even though this may shift the concurrency issue from the DB level to the NH session, this may give you a better option for dealing with your situation. There are gotchas to using this but it's suitability all depends on your business logic.
You can try to apply optimistic locking in this way:
DB entity will have a column tracking entity version (nhibernate.info link).
If you get "stale version" exception while saving entity ( = modified by another user) - reload the entity and try again. Then send the updated value to the client.
As I understand your back-end receives request from the client, then opens session, does some changes and updates entities closing session. In this case no thread will hold one entity in memory for too long and optimistic locking conflicts shouldn't happen too often.
This way you can avoid having many locked threads waiting for operation to complete.
On the other hand, if you expect retries to happen too often you can try SELECT FOR UPDATE locking when loading your entity (using LockMode.Upgrade in NH Get method). Although I found the thread that discourages me from using this with SQL Server: SO link.
In general the solution depends on the logic of the game and whether you can resolve concurrency conflicts in your code without showing messages to users. I'd also made UI updating itself with the latest data often enough to avoid players acting on obsolete game situation and then be surprised with the outcome.
I have a particularly long running method that I need to execute from my controller. The Method is in it's own Model. I am using an async controller, and I have the method setup using asyncFunc library to make it asynchronous. I have also tried invoking it on it's own process. The problem is I want to controller to go ahead and return a view so the user can continue doing other things as the method will notify the user it is completed or has any errors via e-mail.
The problem is even thogh it is an asynchronous method the controller will not move forward to return the view until the process is done. 15+ mins. and if you navigate to a different page the method stops trying to execute.
so how can I get the method to execute as a worker and free up the controller?
Any Help would be greatly appreciated.
all the best,
Chase Q, Aucoin
Use ThreadPool.QueueUserWorkItem() as a fire-and-forget approach in the ASPX page.
Do the long-running work in the WaitCallback you pass to QUWI.
when the work is complete, that WaitCallback can send an email, or whatever it wants.
You need to take care to handle the case that the w3wp.exe is stopped during the 15 minute run. What will you do if the work is 2/3 complete? Some options are, making the work restartable, or just allowing the interrupted work to be forgotten.
Making it restartable might mean, when w3wp.exe restarts, your ASP.NET logic makes sure to begin again, any work that was interrupted. It might mean that your ASP.NET logic sets "syncpoints" so that it knows where to restart.
If you want the restartable option, you might think about Workflow, which is specifically designed for this purpose - maintaining state of long-running workflows, restarting automatically, and so on. If you use Workflow, you can set it to run asynchronously, and you may decide you do not need QueueUserWorkItem.
see also:
Moving a time taking process away from my asp.net application
the Workflow Foundation tag
This will help > http://msdn.microsoft.com/en-us/library/ms227433.aspx
It is the standard way of running a background process on the server in the .NET stack.
I don't know why, but I still live in conviction that this should not be done. Executing background threads in ASP.NET smells. You will also steal threads from ASP.NET thread pool which is controlled by IIS. It can decide that something is wrong with your worker process and restart it any time just to keep memory consumption, processing time consumption or thread consumption low. If you need background logic create custom NT service and call the process on that service either via old .NET remoting or WCF.
Btw. approach I described is used frequently in commercial applications and those which doesn't use it often self-host the whole web server.
A bit of backstory: I am working on an web application that requires quite a bit of time to prep / crunch data before giving it to the user to edit / manipulate. The data request task ~ 15 / 20 secs to complete and a couple secs to process. Once there, the user can manipulate vaules on the fly. Any manipulation of values will require the data to be reprocessed completely.
Update: To avoid confusion, I am only making the data call 1 time (the 15 sec hit) and then wanting to keep the results in memory so that I will not have to call it again until the user is 100% done working with it. So, the first pull will take a while, but, using Ajax, I am going to hit the in-memory data to constantly update and keep the response time to around 2 secs or so (I hope).
In order to make this efficient, I am moving the intial data into memory and using Ajax calls back to the server so that I can reduce processing time to handle the recalculation that occurs w/ this user's updates.
Here is my question, with performance in mind, what would be the best way to storing this data, assuming that only 1 user will be working w/ this data at any given moment.
Also, the user could potentially be working in this process for a few hours. When the user is working w/ the data, I will need some kind of failsafe to save the user's current data (either in a db or in a serialized binary file) should their session be interrupted in some way. In other words, I will need a solution that has an appropriate hook to allow me to dump out the memory object's data in the case that the user gets disconnected / distracted for too long.
So far, here are my musings:
Session State - Pros: Locked to one user. Has the Session End event which will meet my failsafe requirements. Cons: Slowest perf of the my current options. The Session End event is sometimes tricky to ensure it fires properly.
Caching - Pros: Good Perf. Has access to dependencies which could be a bonus later down the line but not really useful in current scope. Cons: No easy failsafe step other than a write based on time intervals. Global in scope - will have to ensure that users do not collide w/ each other's work.
Static - Pros: Best Perf. Easies to maintain as I can directly leverage my current class structures. Cons: No easy failsafe step other than a write based on time intervals. Global in scope - will have to ensure that users do not collide w/ each other's work.
Does anyone have any suggestions / comments on what I option I should choose?
Thanks!
Update: Forgot to mention, I am using VB.Net, Asp.Net, and Sql Server 2005 to perform this task.
I'll vote for secret option #4: use the database for this. If you're talking about a 20+ second turnaround time on the data, you are not going to gain anything by trying to do this in-memory, given the limitations of the options you presented. You might as well set this up in the database (give it a table of its own, or even a separate database if the requirements are that large).
I'd go with the caching method of for storing the data across any page loads. You can name the cache you want to store the data in to avoid conflicts.
For tracking user-made changes, I'd go with a more old-school approach: append to a text file each time the user makes a change and then sweep that file at intervals to save changes back to DB. If you name the files based on the user/account or some other session-unique indicator then there's no issue with conflict and the app (or some other support app, which might be a better idea in general) can sweep through all such files and update the DB even if the session is over.
The first part of this can be adjusted to stagger the write out more: save changes to Session, then write that to file at intervals, then sweep the file at larger intervals. you can tune it to performance and choose what level of possible user-change loss will be possible.
Use the Session, but don't rely on it.
Simply, let the user "name" the dataset, and make a point of actively persisting it for the user, either automatically, or through something as simple as a "save" button.
You can not rely on the session simply because it is (typically) tied to the users browser instance. If they accidentally close the browser (click the X button, their PC crashes, etc.), then they lose all of their work. Which would be nasty.
Once the user has that kind of control over the "persistent" state of the data, you can rely on the Session to keep it in memory and leverage that as a cache.
I think you've pretty much just answered your question with the pros/cons. But if you are looking for some peer validation, my vote is for the Session. Although the performance is slower (do you know by how much slower?), your processing is going to take a long time regardless. Do you think the user will know the difference between 15 seconds and 17 seconds? Both are "forever" in web terms, so go with the one that seems easiest to implement.
perhaps a bit off topic. I'd recommend putting those long processing calls in asynchronous (not to be confused with AJAX's asynchronous) pages.
Take a look at this article and ping me back if it doesn't make sense.
http://msdn.microsoft.com/en-us/magazine/cc163725.aspx
I suggest to create a copy of the data in a new database table (let's call it EDIT) as you send the initial results to the user. If performance is an issue, do this in a background thread.
As the user edits the data, update the table (also in a background thread if performance becomes an issue). If you have to use threads, you must make sure that the first thread is finished before you start updating the rows.
This allows a user to walk away, come back, even restart the browser and commit whenever she feels satisfied with the result.
One possible alternative to what the others mentioned, is to store the data on the client.
Assuming the dataset is not too large, and the code that manipulates it can be handled client side. You could store the data as an XML data island or JSON object. This data could then be manipulated/processed and handled all client side with no round trips to the server. If you need to persist this data back to the server the end resulting data could be posted via an AJAX or standard postback.
If this does not work with your requirements I'd go with just storing it on the SQL server as the other comment suggested.