I've run into an apparent caching issue where what's being returned by NHibernate is not matching up with what's in the DB.
I believe this is level2 cache data. It looks like I can use the Evict to do this, but when should the Evict method actually be called? For my particular application, the data is going to be unique to the user and chances are the data provided will only be used once*.
Can I disable Level2 caching for these sets of objects completely?
UPDATE 10/31
My scenario is this: I have a shopping cart where the customer is going to be adding and removing items. What I am doing is the following: before the updates to the cart are processed, I evict the Cart and CartProduct entities. Once this is done, I retrieve the list of CartProducts from the provider and return the view (this is happening in a .NET MVC Controller).
UPDATE 11/3
The shopping cart has since been finished, and I ran into an issue that appeared to be related to the same NHibernate issue, but in fact was an MVC issue. Deeper digging revealed that the HTML Helper extensions were overriding the value I supposed and replacing with what was in the Model state. So a double whammy on this one. Hope this helps someone.
No, you cannot disable the cache for certain entities.
You do have several options:
Use ISession.Clear() to clear all entities from NHibernate's cache.
http://nhibernate.info/doc/nh/en/index.html#performance-sessioncache
Use ISession.Evict( obj ) to clear a specific entity from the cache.
http://nhibernate.info/doc/nh/en/index.html#performance-sessioncache
Use ISessionFactory.Evict( typeof( obj ) ) to evict all entities/objects of a particular type from the cache. This would be the closest to what you are wanting in my opinion.
http://nhibernate.info/doc/nh/en/index.html#performance-sessioncache
Use the IStatelessSession to fetch the objects/entities from the database as this completely bypasses all caches.
http://nhibernate.info/doc/nh/en/index.html#batch-statelesssession
Related
I am experiencing some bizarre problems with Nhibernate within my MVC web application.
There is not 1 consistent error, I keep getting loads of random ones:
Transaction not successfully started
New request is not allowed to start because it should come with valid transaction descriptor
Unexpected row count: -1; expected: 1
To give a little context to the setup, I am using Ninject to DI the sessions and other Nhibernate related objects, currently I am using RequestScope however I have tried SingletonScope. I have a large and complicated data model, which is read out as a whole, but persisted back in separate parts, as these can all be edited and saved individually.
An example would be having a Customer object, which contains a address object, a contact object, friends object, previous orders object etc etc...
So the whole object is read out, then mapped to the UI domain models and then displayed in different partials within the page. Each partial can be updated individually via ajax, so you may update 1 section or you could update them all together. It seems mainly to give me the problems when I try to persist them all together (so 2-4 simultanious ajax requests to persist chunks of the model).
Now I have integration tests that work fine, which just test the persistence and retrieval of entities. As a whole and individually and all pass fine, however in the web app they just seem to keep throwing random exceptions, and originally refused to persist outside of the Nhibernate cache. I found a way round this by wrapping most units of work within transactions, which got the data persisting but started adding new errors to the mix.
Originally I was thinking of just scrapping Nhibernate from the project, as although I really want its persistance/caching layer, it just didnt seem to be flexible enough for my domain, which seems odd as I have used it before without much problem, although it doesn't like 1-1 mappings.
So has anyone else had flakey transaction/nhibernate issues like this within an ASP MVC app... I know this may be a bit vague as the errors dont point to one thing, and it doesn't always error, so its like stabbing in the dark, but I am out of ideas so any help would be great!
-- Update --
I cannot post all relevant code as the project is huge, but the transaction bit looks like:
using (var transaction = sessionManager.Session.BeginTransaction(IsolationLevel.ReadUncommitted))
{
try
{
// Do unit of work
transaction.Commit();
}
catch (Exception)
{
transaction.Rollback();
throw;
}
}
Some of the main problems I have had on this project have stemmed from:
There are some 1-1 relationships with composite keys, but logically it makes sense
The Nhibernate domain entities go through a mapping layer to become the UI domain entities, then vice versa when saving. Problem here is that with the 1-1 mappings, when persisting the example Address I have to make a Surrogate Customer object with the correct Id then merge.
There is ALOT of Ajax that deals with chunks of the overall model (I talk like there is one single model, but there are quite a few top level models, just one that is most important)
Some notes that may help. I use windsor but imagine the concepts are the same. Sounds like there may be a combination of things.
SessionFactory should be created as singleton and session should be per web request. Something like:
Bind<ISessionFactory>()
.ToProvider<SessionFactoryBuilder>()
.InSingletonScope();
Bind<ISession>()
.ToMethod( context => context.Kernel.Get<ISessionFactory>().OpenSession() )
.InRequestScope();
Be careful of keeping transactions open for too long, keep them as short lived as possible to avoid deadlocks.
Check your queries are running as as expected by using a tool like NHProf. Often people load up too much of the graph which impacts performance and can create deadlocks.
Check your mappings for things like not.lazyload() and see if you actually need the additional data in the queries and keep results returned to a min. Check your queries execution plans and ensure adequate indexes are in place.
I have had issues with mvc3 action filters being cached, which meant transactions were not always started, but would attempt to be closed causing issues. Moved all my transaction commits into ActionResults in the controllers to keep transaction as short as possible and close to the action.
Check your cascades in your mappings and keep the updates to a minimum.
Entity Framework 4, POCO objects and ASP.Net MVC2. I have a many to many relationship, lets say between BlogPost and Tag entities. This means that in my T4 generated POCO BlogPost class I have:
public virtual ICollection<Tag> Tags {
// getter and setter with the magic FixupCollection
}
private ICollection<Tag> _tags;
I ask for a BlogPost and the related Tags from an instance of the ObjectContext and send it to another layer (View in the MVC application). Later I get back the updated BlogPost with changed properties and changed relationships. For example it had tags "A" "B" and "C", and the new tags are "C" and "D". In my particular example there are no new Tags and the properties of the Tags never change, so the only thing which should be saved is the changed relationships. Now I need to save this in another ObjectContext. (Update: Now I tried to do in the same context instance and also failed.)
The problem: I can't make it save the relationships properly. I tried everything I found:
Controller.UpdateModel and Controller.TryUpdateModel don't work.
Getting the old BlogPost from the context then modifying the collection doesn't work. (with different methods from the next point)
This probably would work, but I hope this is just a workaround, not the solution :(.
Tried Attach/Add/ChangeObjectState functions for BlogPost and/or Tags in every possible combinations. Failed.
This looks like what I need, but it doesn't work (I tried to fix it, but can't for my problem).
Tried ChangeState/Add/Attach/... the relationship objects of the context. Failed.
"Doesn't work" means in most cases that I worked on the given "solution" until it produces no errors and saves at least the properties of BlogPost. What happens with the relationships varies: usually Tags are added again to the Tag table with new PKs and the saved BlogPost references those and not the original ones. Of course the returned Tags have PKs, and before the save/update methods I check the PKs and they are equal to the ones in the database so probably EF thinks that they are new objects and those PKs are the temp ones.
A problem I know about and might make it impossible to find an automated simple solution: When a POCO object's collection is changed, that should happen by the above mentioned virtual collection property, because then the FixupCollection trick will update the reverse references on the other end of the many-to-many relationship. However when a View "returns" an updated BlogPost object, that didn't happen. This means that maybe there is no simple solution to my problem, but that would make me very sad and I would hate the EF4-POCO-MVC triumph :(. Also that would mean that EF can't do this in the MVC environment whichever EF4 object types are used :(. I think the snapshot based change tracking should find out that the changed BlogPost has relationships to Tags with existing PKs.
Btw: I think the same problem happens with one-to-many relations (google and my colleague say so). I will give it a try at home, but even if that works that doesn't help me in my six many-to-many relationships in my app :(.
Let's try it this way:
Attach BlogPost to context. After attaching object to context the state of the object, all related objects and all relations is set to Unchanged.
Use context.ObjectStateManager.ChangeObjectState to set your BlogPost to Modified
Iterate through Tag collection
Use context.ObjectStateManager.ChangeRelationshipState to set state for relation between current Tag and BlogPost.
SaveChanges
Edit:
I guess one of my comments gave you false hope that EF will do the merge for you. I played a lot with this problem and my conclusion says EF will not do this for you. I think you have also found my question on MSDN. In reality there is plenty of such questions on the Internet. The problem is that it is not clearly stated how to deal with this scenario. So lets have a look on the problem:
Problem background
EF needs to track changes on entities so that persistance knows which records have to be updated, inserted or deleted. The problem is that it is ObjectContext responsibility to track changes. ObjectContext is able to track changes only for attached entities. Entities which are created outside the ObjectContext are not tracked at all.
Problem description
Based on above description we can clearly state that EF is more suitable for connected scenarios where entity is always attached to context - typical for WinForm application. Web applications requires disconnected scenario where context is closed after request processing and entity content is passed as HTTP response to the client. Next HTTP request provides modified content of the entity which has to be recreated, attached to new context and persisted. Recreation usually happends outside of the context scope (layered architecture with persistance ignorace).
Solution
So how to deal with such disconnected scenario? When using POCO classes we have 3 ways to deal with change tracking:
Snapshot - requires same context = useless for disconnected scenario
Dynamic tracking proxies - requires same context = useless for disconnected scenario
Manual synchronization.
Manual synchronization on single entity is easy task. You just need to attach entity and call AddObject for inserting, DeleteObject for deleting or set state in ObjectStateManager to Modified for updating. The real pain comes when you have to deal with object graph instead of single entity. This pain is even worse when you have to deal with independent associations (those that don't use Foreign Key property) and many to many relations. In that case you have to manually synchronize each entity in object graph but also each relation in object graph.
Manual synchronization is proposed as solution by MSDN documentation: Attaching and Detaching objects says:
Objects are attached to the object
context in an Unchanged state. If you
need to change the state of an object
or the relationship because you know
that your object was modified in
detached state, use one of the
following methods.
Mentioned methods are ChangeObjectState and ChangeRelationshipState of ObjectStateManager = manual change tracking. Similar proposal is in other MSDN documentation article: Defining and Managing Relationships says:
If you are working with disconnected
objects you must manually manage the
synchronization.
Moreover there is blog post related to EF v1 which criticise exactly this behavior of EF.
Reason for solution
EF has many "helpful" operations and settings like Refresh, Load, ApplyCurrentValues, ApplyOriginalValues, MergeOption etc. But by my investigation all these features work only for single entity and affects only scalar preperties (= not navigation properties and relations). I rather not test this methods with complex types nested in entity.
Other proposed solution
Instead of real Merge functionality EF team provides something called Self Tracking Entities (STE) which don't solve the problem. First of all STE works only if same instance is used for whole processing. In web application it is not the case unless you store instance in view state or session. Due to that I'm very unhappy from using EF and I'm going to check features of NHibernate. First observation says that NHibernate perhaps has such functionality.
Conclusion
I will end up this assumptions with single link to another related question on MSDN forum. Check Zeeshan Hirani's answer. He is author of Entity Framework 4.0 Recipes. If he says that automatic merge of object graphs is not supported, I believe him.
But still there is possibility that I'm completely wrong and some automatic merge functionality exists in EF.
Edit 2:
As you can see this was already added to MS Connect as suggestion in 2007. MS has closed it as something to be done in next version but actually nothing had been done to improve this gap except STE.
I have a solution to the problem that was described above by Ladislav. I have created an extension method for the DbContext which will automatically perform the add/update/delete's based on a diff of the provided graph and persisted graph.
At present using the Entity Framework you will need to perform the updates of the contacts manually, check if each contact is new and add, check if updated and edit, check if removed then delete it from the database. Once you have to do this for a few different aggregates in a large system you start to realize there must be a better, more generic way.
Please take a look and see if it can help http://refactorthis.wordpress.com/2012/12/11/introducing-graphdiff-for-entity-framework-code-first-allowing-automated-updates-of-a-graph-of-detached-entities/
You can go straight to the code here https://github.com/refactorthis/GraphDiff
I know it's late for the OP but since this is a very common issue I posted this in case it serves someone else.
I've been toying around with this issue and I think I got a fairly simple solution,
what I do is:
Save main object (Blogs for example) by setting its state to Modified.
Query the database for the updated object including the collections I need to update.
Query and convert .ToList() the entities I want my collection to include.
Update the main object's collection(s) to the List I got from step 3.
SaveChanges();
In the following example "dataobj" and "_categories" are the parameters received by my controller "dataobj" is my main object, and "_categories" is an IEnumerable containing the IDs of the categories the user selected in the view.
db.Entry(dataobj).State = EntityState.Modified;
db.SaveChanges();
dataobj = db.ServiceTypes.Include(x => x.Categories).Single(x => x.Id == dataobj.Id);
var it = _categories != null ? db.Categories.Where(x => _categories.Contains(x.Id)).ToList() : null;
dataobj.Categories = it;
db.SaveChanges();
It even works for multiple relations
The Entity Framework team is aware that this is a usability issue and plans to address it post-EF6.
From the Entity Framework team:
This is a usability issue that we are aware of and is something we have been thinking about and plan to do more work on post-EF6. I have created this work item to track the issue: http://entityframework.codeplex.com/workitem/864 The work item also contains a link to the user voice item for this--I encourage you to vote for it if you have not done so already.
If this impacts you, vote for the feature at
http://entityframework.codeplex.com/workitem/864
All of the answers were great to explain the problem, but none of them really solved the problem for me.
I found that if I didn't use the relationship in the parent entity but just added and removed the child entities everything worked just fine.
Sorry for the VB but that is what the project I am working in is written in.
The parent entity "Report" has a one to many relationship to "ReportRole" and has the property "ReportRoles". The new roles are passed in by a comma separated string from an Ajax call.
The first line will remove all the child entities, and if I used "report.ReportRoles.Remove(f)" instead of the "db.ReportRoles.Remove(f)" I would get the error.
report.ReportRoles.ToList.ForEach(Function(f) db.ReportRoles.Remove(f))
Dim newRoles = If(String.IsNullOrEmpty(model.RolesString), New String() {}, model.RolesString.Split(","))
newRoles.ToList.ForEach(Function(f) db.ReportRoles.Add(New ReportRole With {.ReportId = report.Id, .AspNetRoleId = f}))
This is something that has been pulling at me for a while. Consider a (MVC type) web application with an ORM (e.g. Nhiberate) as the data access layer.
On one hand - the OOP/Rich domain model hand - I feel I should be passing around (references to) the real objects I am talking about.
On the other hand - the DB/Web App hand - I feel that it is easier and more efficient just to pass the integer Ids of the objects rather than the object themselves.
Consider an ecommerce catalogue type application:
The user is logged in and navigates to a product page.
They post a comment.
The controller action tasked with persisting this comment has 3 pieces of information: a) The user id (from the auth cookie or wherever), b) The product id (probably from the querystring), and c) the comment text.
Now, what what is best practice here? Is it really worth inflating the user and product objects (e.g. by getting them from the repository, with all the DB work that entails) when we know that all they will be used for is so the ORM can read their IDs and set the appropriate foreign keys in the DB table that stores the comments?
What are peoples views on this? Perhaps web apps should be given a little more leway than other apps, due to their stateless nature? I imagine there will be 'it depends' answers, but maybe some people are purists about the issue.
This is a general question which probably is applicable to many platforms, but if giving examples I would prefer them to be ASP.NET MVC if possible.
Thank you.
NHibernate has the load operation (as opposed to doing a get) exactly for this reason.
session.Save(
new Comment
{
Text = commentTextFromScreen,
User = session.Load<User>(userID),
Product = session.Load<Product>(productID)
}
};
In the above example, you are telling NHibernate: I know these already exist in the database, so don't bother selecting them right now. NHibernate will return proxy objects for them and a select won't happen against the database as long as you don't attempt to access any properties on the objects.
For more info check out Ayende's blog post: The difference between Get, Load, and query by id.
Using ASP.NET MVC, I've implemented an autocomplete textbox using the approach very similar to the implementation by Ben Scheirman as shown here: http://flux88.com/blog/jquery-auto-complete-text-box-with-asp-net-mvc/
What I haven't been able to figure out is if it's a good idea to cache the data for the autocomplete textbox, so there won't be a roundtrip to the database on every keystroke?
If caching is prefered, can you guide me in the direction to how to implement caching for this purpose?
You have a couple things to ask yourself:
Is the data I'm pulling back dynamic?
If not, how often do I expect this call to occur?
If the answers are, 1- not really and 2 - call to happen frequently, you should cache it.
I don't know how your data access is setup, but I simply throw my data into cache objects like so:
public IQueryable<Category> FindAllCategories()
{
if (HttpContext.Current.Cache["AllCategories"] != null)
return (IQueryable<Category>)HttpContext.Current.Cache["AllCategories"];
else
{
IQueryable<Category> allCats = from c in db.Categories
orderby c.Name
select c;
// set cache
HttpContext.Current.Cache.Add("AllCategories", allCats, null, System.Web.Caching.Cache.NoAbsoluteExpiration, new TimeSpan(0, 0, 30, 0, 0), System.Web.Caching.CacheItemPriority.Default, null);
return allCats;
}
}
This is an example of one of my repository queries, based off of LINQ to SQL. It first checks the cache, if the entry exists in cache, it returns it. If not, it goes to the database, then caches it with a sliding expiration.
You sure can Cache your result, using the attribute like:
[OutputCache(Duration=60, VaryByParam="searchTerm")]
ASP.net will handle the rest.
I think caching in this case would require more work than simply storing every request. You'd want to focus more on the terms being searched than individual keys. You'd have to keep track of what terms are more popular and cache combinations of characters that make up those terms. I don't think simply caching every single request is going to get you any performance boost. You're just going to have stale data in your cache.
Well, how will caching in asp.net prevent server round trips? You'll still have server round trips, at best you will not have to look up the database if you cache. If you want to prevent server roundtrips then you need to cache at the client side.
While it's quite easily possible with Javascript (You need to store your data in a variable and check that variable for relevant data before looking up the server again) I don't know of a ready-tool which does this for you.
I do recommend you consider caching to prevent round-trips. In fact I have half a mind to implement javascript caching in one of my own websites reading this.
I'm needing to cache some data using System.Web.Caching.Cache. Not sure if it matters, but the data does not come from a database, but a plethora of custom objects.
The ASP.NET MVC is fairly new to me and I'm wondering where it makes sense for this caching to occur?
Model or Controller?
At some level this makes sense to cache at the Model level but I don't necessarily know the implications of doing this (if any). If caching were to be done at the Controller level, will that affect all requests, or just for the current HttpContext?
So... where should application data caching be done, and what's a good way of actually doing it?
Update
Thanks for the great answers! I'm still trying to gather where it makes most sense to cache given different scenarios. If one is caching the entire page, then keeping it in the view makes sense but where to draw the line when it's not the entire page?
I think it ultimately depends on what you are caching. If you want to cache the result of rendered pages, that is tightly coupled to the Http nature of the request, and would suggest a ActionFilter level caching mechanism.
If, on the other hand, you want to cache the data that drives the pages themselves, then you should consider model level caching. In this case, the controller doesn't care when the data was generated, it just performs the logic operations on the data and prepares it for viewing. Another argument for model level caching is if you have other dependencies on the model data that are not attached to your Http context.
For example, I have a web-app were most of my Model is abstracted into a completely different project. This is because there will be a second web-app that uses this same backing, AND there's a chance we might have a non-web based app using the same data as well. Much of my data comes from web-services, which can be performance killers, so I have model level caching that the controllers and views know absolutely nothing about.
I don't know the anwser to your question, but Jeff Atwood talks about how the SO team did caching using the MVC framework for stackoverflow.com on a recent hanselminutes show that might help you out:
http://www.hanselminutes.com/default.aspx?showID=152
Quick Answer
I would start with CONTROLLER caching, use the OutputCache attribute, and later add Model caching if required. It's quicker to implement and has instant results.
Detail Answer (cause i like the sound of my voice)
Here's an example.
[OutputCache(Duration=60, VaryByParam="None")]
public ActionResult CacheDemo() {
return View();
}
This means that if a user hits the site (for the cache requirements defined in the attribute), there's less work to get done. If there's only Model caching, then even though the logic (and most likely the DB hit) are cached, the web server still has to render the page. Why do that when the render result will always be the same?
So start with OutputCaching, then move onto Model caching as you performance test your site.
Output caching is also a lot simpler to start out with. You don't have to worry about web farm distributed caching probs (if you are part of a farm) and the caching provider for the model.
Advanced Caching Techniques
You can also apply donut caching -> cache only part of the UI page :) Check it out!
I would choose caching at the model level.
(In general, the advice seems to be to minimize business logic at the controller level
and move as much as possible into model classes.)
How about doing it like this:
I have some entries in the model represented by the class Entry
and a source of entries (from a database, or 'a plethora of custom objects').
In the model I make an interface for retrieving entries:
public interface IEntryHandler
{
IEnumerable<Entry> GetEntries();
}
In the model I have an actual implementation of IEntryHandler
where the entries are read from cache and written to cache.
public class EntryHandler : IEntryHandler
{
public IEnumerable<Entry> GetEntries()
{
// Check if the objects are in the cache:
List<Entry> entries = [Get entries from cache]
if (entries == null)
{
// There were no entries in the cache, so we read them from the source:
entries = [Get entries from database or 'plethora of custom objects']
[Save the retrieved entries to cache for later use]
}
return entries;
}
}
The controller would then call the IEntryHandler:
public class HomeController : Controller
{
private IEntryHandler _entryHandler;
// The default constructor, using cache and database/custom objects
public HomeController()
: this(new EntryHandler())
{
}
// This constructor allows us to unit test the controller
// by writing a test class that implements IEntryHandler
// but does not affect cache or entries in the database/custom objects
public HomeController(IEntryHandler entryHandler)
{
_entryHandler = entryHandler;
}
// This controller action returns a list of entries to the view:
public ActionResult Index()
{
return View(_entryHandler.GetEntries());
}
}
This way it is possible to unit test the controller without touching real cache/database/custom objects.
I think the caching should somehow be related to the model. I think the controller shouldn't care more about the data. The controller responsibility is to map the data - regardless where it come from - to the views.
Try also to think why you need to cache? do you want to save processing, data transmission or what? This will help you to know where exactly you need to have your caching layer.
It all depends on how expensive the operation is. If you have complicated queries then it might make sense to cache the data in the controller level so that the query is not executed again (until the cache expires).
Keep in mind that caching is a very complicated topic. There are many different places that you can store your cache:
Akamai / CDN caching
Browser caching
In-Memory application caching
.NET's Cache object
Page directive
Distributed cache (memcached)