We use NHibernate.Envers extension for audit purpose. Therefore audit entities are generated on the fly, although they are not part of our business model.
But the NHMetadataBuilder calls GetMappedClass method and makes no distinction between these audit entities and our business entities. However it crashes later on for these audit entities. The type returned by GetMappedClass is System.Collections.IDictionnary, which obviously does not make any sense to be added to metadata.
For now, we are just ignoring these entities by adding a safeguard in BuildMetadata method:
foreach (var meta in classMeta.Values)
{
if (!meta.EntityName.EndsWith("_AUD"))
{
AddClass(meta);
}
}
Can you think of a more generic way to do it ?
The pull request from lnu fixes the problem, and has been merged into the codebase. It's available now from github, and will be in the next Breeze release.
Thanks lnu.
Related
One of our contractors implemented a repository pattern with code first approach. We use Service Locator as DI pattern. what we do when we retrieve data from DB, we pass interface to GetQueryable function and get the data. However, I see serious performance issues on our application. I implemented MiniProfiler and MiniProfiler.EF to see where the bottleneck is.
We have a case table which has quite a few fields(around 25) and some of those fields are associated to other tables as one to one and one to many(only one field has many relation to other table). when I try to see the case detail, it runs around 400 SQL queries and SQL takes around 40 percent of the load time as far as the miniprofiler concerned. Here our GetQueryable and Find methods
public IQueryable<T> GetQueryable<T>(params string[] includes)
{
Type type = _impls.Value[typeof (T).Name].GetType();
DbSet dbSet = Db.Set(type);
foreach (var include in includes)
{
dbSet.Include(include);
}
return ((IQueryable<T>) dbSet);
}
I added included to this method to attach other related tables, but it did not make any difference. and here is the Find Method
public T Find<T>(long? id)
{
Type type = _impls.Value[typeof(T).Name].GetType();
return (T) Db.Set(type).Find(id);
}
I pretty much tried to apply all the performance improvements, but the number of the SQL queries has not gone down. I tried to disable lazy loading, but it caused many problems in other parts of the application.
Just some additional information, in case table, there are 70000 rows and in out dialogs table, there are 500000 rows. Case and Dialog are associates as one-to-many. and each case has 20-40 dialog entries.
My questions are;
Why does include not make any difference when I use?
Is there any other way to crop number of the queries run?
Do you think the implementation is the problem?
Thanks
Include returns a new IQueryable and does not modify the source query. In addition you can use the generic version of Set which simplifies the code a bit:
public IQueryable<T> GetQueryable<T>(params string[] includes)
{
IQueryable<T> query = Db.Set<T>();
foreach (var include in includes)
{
query = query.Include(include);
}
return query;
}
Step 1: Fire your contractor. Seriously. Like right now. That is some awful code. Not only did they miss something as simple and basic as using the generic version of Set, but they've successfully only made working with Entity Framework more complex, because all the repository does is proxy Entity Framework methods with its own unique and bastardized API.
That said, there's really not enough here to diagnose what your problem is. The use of Include may give you larger queries, but it should actually serve to reduce the overall number of queries issued. It's possible, you're just not using includes where you should be.
Now, the fact that you "tried to disable lazy loading, but it caused many problems in other parts of the application", means that you're relying too heavily on lazy-loading. Basically, you're loading in stuff you don't even know about, which is the antithesis of optimization. Ironically, you'd actually be best served by going ahead and disabling lazy-loading, and then tracking down where your code fails because of that. If you want to actually lazy-load that thing, you can use .Load (see: Explicit Loading). But, if you want to eager-load to reduce queries, then you know what includes you need to add.
Question:
What value does breeze provide when I need to implement my own POST/PUT/GET endpoints per entity in WebAPI?
Background:
This seems to be a common implementation of a serverside Breeze controller:
[BreezeController]
public class TodosController : ApiController {
readonly EFContextProvider<TodosContext> _contextProvider =
new EFContextProvider<TodosContext>();
// ~/breeze/todos/Metadata
[HttpGet]
public string Metadata() {
return _contextProvider.Metadata();
}
// ~/breeze/todos/Todos
// ~/breeze/todos/Todos?$filter=IsArchived eq false&$orderby=CreatedAt
[HttpGet]
public IQueryable<TodoItem> Todos() {
return _contextProvider.Context.Todos;
}
// ~/breeze/todos/SaveChanges
[HttpPost]
public SaveResult SaveChanges(JObject saveBundle) {
return _contextProvider.SaveChanges(saveBundle);
}
// other miscellaneous actions of no interest to us here
}
I'm in the middle of building a RESTish API that, up to this point, has endpoints like:
GET /api/todo/1
PUT /api/todo
POST /api/todo
It seems like Breeze requires the endpoints to be much simpler (for better or worse) - just a bunch of GETS and a SaveChanges POST endpoint.
This leads me to think that Breeze makes rapid development with a single web client, well, a breeze... but as soon as you have anonymous clients, you have to force them into whatever breeze interface conventions you've created in your client, which seems to defeat the purpose of RESTful API design. Is this the case?
Breeze is, first and formost, a client-side JavaScript framework. If you're not using Breeze on the client, the benefits of Breeze.WebApi are limited to
Enhanced OData query support ($select and $expand support, extended $orderby)
Save interception points (beforeSaveEntity and beforeSaveEntities events)
Save result handling (updated entity keys, concurrency columns)
Metadata extraction and serialization
As you've surmised, Breeze has a different philosophy from REST regarding CRUD operations.
Breeze is designed for clients who may want to C/U/D many resources, of different types, in a single transaction. This allows users to manipulate the data in complex ways without hitting the server, then saving their changes when they are ready. For example, one could create a new Order, move two OrderLineItems from one Order to another, delete a third OrderLineItem, modify the quantity on a fourth, and then SaveChanges(). Breeze even supports using localStorage to work completely disconnected from the server. Once reconnected, the changes can all be saved.
REST was designed to operate on one resource at a time. Each C/U/D operation must be performed against the server immediately so that the response code can be acted upon. It works well for applications with simple update needs, but not for data-entry applications. While transactions can be supported in REST, they are cumbersome at best.
Having said that, your server-side Breeze API is not limited to what you see in the Todos example. Breeze supports Named Saves, which allows you to have different endpoints for different operations. You can also use Save Interception to ensure that your save bundle only contains the types that it should. And naturally, there's nothing preventing you from exposing both APIs on your server, and having both fed by the same persistence layer.
If you have to decide between them, you should start with your users. Real users (not developers) don't care about REST, they care about what the application can do. Ultimately, REST gives your application all the semantics of HTTP, and Breeze gives it all the semantics of a relational or object database. Which one to expose to your users should depend upon the use cases you need to support.
Background:
I am using EF4 and ObjectContext. To optimze retrieval of complex object hierarchies, I manually execute database queries and then use ObjectContext.Translatey<T>(DataReader, entitySetName, mergeOptions.AppendOnly) to turn data rows into entities. I then attach the entities to the ObjectContext with Attach method. This also fixes relations between entities.
The problem:
After everything is loaded and set up I try to navigate from parent entity to a child entity (for example Parent.Childs.First()), but EF hits the database to load the kids, even though all the child entities are already present in the ObjectContext and EntitySet. It looks like the reason for this is that parent.Childs.IsLoaded is set to false which makes EF think that it still needs to load the relation.
Question:
How can I tell EF that EntitySet has already been loaded?
Is there a supported way to set RelatedEnd.IsLoaded to true. I wouldn't like to mess with the calling the RelatedEnd.SetIsLoaded internal method.
I found a smilar question here but it relates to DbContext and has no satifying answer ;-)
Matra
It looks like this was implemented in this change:
http://entityframework.codeplex.com/workitem/269
You can now iterate through your entities and tell them that their child collections are already loaded:
foreach (var entity in loadedEntities)
context.Entry(entity).Collection(a => a.SomeChildCollection).IsLoaded = true;
This will prevent entities in SomeChildCollection from being loaded when they are accessed from entity.
I'm not exactly sure what version of EF this appeared in but I'm guessing 6.0.0.
The only way to solve this is to turn off lazy loading. The question you have found on MSDN asks about DbContext but the answer mentions that there is no way to change the value in underlying libraries - ObjectContext API (= EF4 in your case) is the underlying library. Even in .NET 4.5 (EF5) setting IsLoaded is still not available on public API.
Linq To SQL's DataContext has an overload on SubmitChanges that allows for updates to continue when a Optimistic Concurrency Exception is thrown, and provides the developer with a mechanism to resolve the conflicts afterwards in a single Try Catch block.
Even the WCFDataServicesContext has a SaveChangedOptions.ContinueOnError parameter for its SaveChanges method that at least allows you to continue updating when an error has occurred and leaves conflicting updates unresolved so you can look into them later.
(1) Why then does the ObjectContext.SaveChanges method have no such option?
(2) Do any update patterns exist that will mimick the Linq To SQL behaviour? The examples I find on MSDN make it appear as if a single Try Catch block will see you home in the case of multiple updates. But this pattern does not allow you to investigate each conflicting update separately: it just alerts you to the first conflict and then gives you the option to "wipe the table clean in one sweep" to prevent any further optimistic concurrency exceptions from surfacing, without your knowing if any exist and what you would have liked to do about them.
Why then does the ObjectContext.SaveChanges method have no such option?
I think the simplest answer is because Linq-to-Sql, Entity Framework and WCF Data Services were all implemented by different teams and internal communication among these teams doesn't work as we would hope. I have described some interesting features missing in newer APIs in one of my former answers but I don't think this is a missing feature - I will explain it in the second part of my answer.
WCF Data Services have more interesting features which should be part of Entity framework as well. For example:
Change and Query interceptors
Batching multiple queries and SaveChanges operations to single call to server
Asynchronous operations - this will come to EF6 in form of async/await implementation
Do any update patterns exist that will mimick the Linq To SQL behaviour?
There is a pattern how to solve this but you will probably not like it. EF's SaveChanges works as Unit of work. It either saves all changes or none. If you have a scenario where your saving operation can result in case where only part of your changes is persisted than it should not be handled by single SaveChanges call. Each atomic set of changes should have its own SaveChanges call:
using (var scope = new TransactionScope(...)) {
foreach (var entity in someEntitiesToModify) {
try {
context.SomeEntities.Attach(entity);
context.ObjectStateManager.ChangeObjectState(entity, EntityState.Modified);
context.SaveChanges();
catch (OptimisticConcurrencyException e) {
// Do something here
context.Refresh(e.StateEntries[0].Entity, RefreshMode.ClientWins);
context.SaveChanges();
}
}
scope.Complete();
}
I think the reason why this feature doesn't exist is because it is not generic and as mentioned about it goes against unit of work pattern. Suppose this example:
You load an entity
You add a new dependent entity to navigation property of your loaded entity
You change something on the loaded entity
In the mean time somebody else concurrently delete your loaded entity
You trigger SaveChanges with relaxed conflict resolution
EF will try to save changes to the principal entity but it conflicts because there is no entity to update in the database
EF will continue because conflict resolution is relaxed
EF will try to insert dependent entity but it will fire SqlException because the principal entity doesn't exist in the database. This exception will break the persistence operation and you will not know why it is complaining about referential integrity because you have a principal entity. (It is possible that this insert will even not happen and EF fires another exception due to inconsistency of context's inner state but it depends on EF's internal implementation).
This immediately makes whole relaxing of conflict resolution much more complex feature. There are IMHO three ways to solve it:
Simply not support it. If you need conflict resolution per entity basis you can still use the example I showed above but for complex scenarios it may not work because complex scenarios are hard to solve.
Rebuild database change set each time the conflict occurs - it means to explore the remaining change set and exclude all entities related to conflicting entity and their relations an so on from the processed persistence. There is a problem: EF cannot exclude any changed entity from processing. That would break the meaning of unit of work and I repeat it one more: Relaxing conflict resolution can also break meaning of unit of work.
Let EF to proceed with dependencies even if the principal entity conflicted. This requires to handle the database exception and understand its content to know if the exception is fired due to conflicting principal or due to other error (which should fail whole persistence operation immediately). It can be quite difficult to understand database exceptions on the code level and moreover it is provider specific for every supported database.
It doesn't mean it may not be possible to make such functionality but it will need to cover all scenarios when it comes to relations and this can be pretty complex. I'm not sure if Linq-to-Sql handles this.
You can always make a suggestion on Data UserVoice or check out the code and try to implement it yourselves. Maybe I see this feature too complicated and it can be implemented easily.
Entity Framework 4, POCO objects and ASP.Net MVC2. I have a many to many relationship, lets say between BlogPost and Tag entities. This means that in my T4 generated POCO BlogPost class I have:
public virtual ICollection<Tag> Tags {
// getter and setter with the magic FixupCollection
}
private ICollection<Tag> _tags;
I ask for a BlogPost and the related Tags from an instance of the ObjectContext and send it to another layer (View in the MVC application). Later I get back the updated BlogPost with changed properties and changed relationships. For example it had tags "A" "B" and "C", and the new tags are "C" and "D". In my particular example there are no new Tags and the properties of the Tags never change, so the only thing which should be saved is the changed relationships. Now I need to save this in another ObjectContext. (Update: Now I tried to do in the same context instance and also failed.)
The problem: I can't make it save the relationships properly. I tried everything I found:
Controller.UpdateModel and Controller.TryUpdateModel don't work.
Getting the old BlogPost from the context then modifying the collection doesn't work. (with different methods from the next point)
This probably would work, but I hope this is just a workaround, not the solution :(.
Tried Attach/Add/ChangeObjectState functions for BlogPost and/or Tags in every possible combinations. Failed.
This looks like what I need, but it doesn't work (I tried to fix it, but can't for my problem).
Tried ChangeState/Add/Attach/... the relationship objects of the context. Failed.
"Doesn't work" means in most cases that I worked on the given "solution" until it produces no errors and saves at least the properties of BlogPost. What happens with the relationships varies: usually Tags are added again to the Tag table with new PKs and the saved BlogPost references those and not the original ones. Of course the returned Tags have PKs, and before the save/update methods I check the PKs and they are equal to the ones in the database so probably EF thinks that they are new objects and those PKs are the temp ones.
A problem I know about and might make it impossible to find an automated simple solution: When a POCO object's collection is changed, that should happen by the above mentioned virtual collection property, because then the FixupCollection trick will update the reverse references on the other end of the many-to-many relationship. However when a View "returns" an updated BlogPost object, that didn't happen. This means that maybe there is no simple solution to my problem, but that would make me very sad and I would hate the EF4-POCO-MVC triumph :(. Also that would mean that EF can't do this in the MVC environment whichever EF4 object types are used :(. I think the snapshot based change tracking should find out that the changed BlogPost has relationships to Tags with existing PKs.
Btw: I think the same problem happens with one-to-many relations (google and my colleague say so). I will give it a try at home, but even if that works that doesn't help me in my six many-to-many relationships in my app :(.
Let's try it this way:
Attach BlogPost to context. After attaching object to context the state of the object, all related objects and all relations is set to Unchanged.
Use context.ObjectStateManager.ChangeObjectState to set your BlogPost to Modified
Iterate through Tag collection
Use context.ObjectStateManager.ChangeRelationshipState to set state for relation between current Tag and BlogPost.
SaveChanges
Edit:
I guess one of my comments gave you false hope that EF will do the merge for you. I played a lot with this problem and my conclusion says EF will not do this for you. I think you have also found my question on MSDN. In reality there is plenty of such questions on the Internet. The problem is that it is not clearly stated how to deal with this scenario. So lets have a look on the problem:
Problem background
EF needs to track changes on entities so that persistance knows which records have to be updated, inserted or deleted. The problem is that it is ObjectContext responsibility to track changes. ObjectContext is able to track changes only for attached entities. Entities which are created outside the ObjectContext are not tracked at all.
Problem description
Based on above description we can clearly state that EF is more suitable for connected scenarios where entity is always attached to context - typical for WinForm application. Web applications requires disconnected scenario where context is closed after request processing and entity content is passed as HTTP response to the client. Next HTTP request provides modified content of the entity which has to be recreated, attached to new context and persisted. Recreation usually happends outside of the context scope (layered architecture with persistance ignorace).
Solution
So how to deal with such disconnected scenario? When using POCO classes we have 3 ways to deal with change tracking:
Snapshot - requires same context = useless for disconnected scenario
Dynamic tracking proxies - requires same context = useless for disconnected scenario
Manual synchronization.
Manual synchronization on single entity is easy task. You just need to attach entity and call AddObject for inserting, DeleteObject for deleting or set state in ObjectStateManager to Modified for updating. The real pain comes when you have to deal with object graph instead of single entity. This pain is even worse when you have to deal with independent associations (those that don't use Foreign Key property) and many to many relations. In that case you have to manually synchronize each entity in object graph but also each relation in object graph.
Manual synchronization is proposed as solution by MSDN documentation: Attaching and Detaching objects says:
Objects are attached to the object
context in an Unchanged state. If you
need to change the state of an object
or the relationship because you know
that your object was modified in
detached state, use one of the
following methods.
Mentioned methods are ChangeObjectState and ChangeRelationshipState of ObjectStateManager = manual change tracking. Similar proposal is in other MSDN documentation article: Defining and Managing Relationships says:
If you are working with disconnected
objects you must manually manage the
synchronization.
Moreover there is blog post related to EF v1 which criticise exactly this behavior of EF.
Reason for solution
EF has many "helpful" operations and settings like Refresh, Load, ApplyCurrentValues, ApplyOriginalValues, MergeOption etc. But by my investigation all these features work only for single entity and affects only scalar preperties (= not navigation properties and relations). I rather not test this methods with complex types nested in entity.
Other proposed solution
Instead of real Merge functionality EF team provides something called Self Tracking Entities (STE) which don't solve the problem. First of all STE works only if same instance is used for whole processing. In web application it is not the case unless you store instance in view state or session. Due to that I'm very unhappy from using EF and I'm going to check features of NHibernate. First observation says that NHibernate perhaps has such functionality.
Conclusion
I will end up this assumptions with single link to another related question on MSDN forum. Check Zeeshan Hirani's answer. He is author of Entity Framework 4.0 Recipes. If he says that automatic merge of object graphs is not supported, I believe him.
But still there is possibility that I'm completely wrong and some automatic merge functionality exists in EF.
Edit 2:
As you can see this was already added to MS Connect as suggestion in 2007. MS has closed it as something to be done in next version but actually nothing had been done to improve this gap except STE.
I have a solution to the problem that was described above by Ladislav. I have created an extension method for the DbContext which will automatically perform the add/update/delete's based on a diff of the provided graph and persisted graph.
At present using the Entity Framework you will need to perform the updates of the contacts manually, check if each contact is new and add, check if updated and edit, check if removed then delete it from the database. Once you have to do this for a few different aggregates in a large system you start to realize there must be a better, more generic way.
Please take a look and see if it can help http://refactorthis.wordpress.com/2012/12/11/introducing-graphdiff-for-entity-framework-code-first-allowing-automated-updates-of-a-graph-of-detached-entities/
You can go straight to the code here https://github.com/refactorthis/GraphDiff
I know it's late for the OP but since this is a very common issue I posted this in case it serves someone else.
I've been toying around with this issue and I think I got a fairly simple solution,
what I do is:
Save main object (Blogs for example) by setting its state to Modified.
Query the database for the updated object including the collections I need to update.
Query and convert .ToList() the entities I want my collection to include.
Update the main object's collection(s) to the List I got from step 3.
SaveChanges();
In the following example "dataobj" and "_categories" are the parameters received by my controller "dataobj" is my main object, and "_categories" is an IEnumerable containing the IDs of the categories the user selected in the view.
db.Entry(dataobj).State = EntityState.Modified;
db.SaveChanges();
dataobj = db.ServiceTypes.Include(x => x.Categories).Single(x => x.Id == dataobj.Id);
var it = _categories != null ? db.Categories.Where(x => _categories.Contains(x.Id)).ToList() : null;
dataobj.Categories = it;
db.SaveChanges();
It even works for multiple relations
The Entity Framework team is aware that this is a usability issue and plans to address it post-EF6.
From the Entity Framework team:
This is a usability issue that we are aware of and is something we have been thinking about and plan to do more work on post-EF6. I have created this work item to track the issue: http://entityframework.codeplex.com/workitem/864 The work item also contains a link to the user voice item for this--I encourage you to vote for it if you have not done so already.
If this impacts you, vote for the feature at
http://entityframework.codeplex.com/workitem/864
All of the answers were great to explain the problem, but none of them really solved the problem for me.
I found that if I didn't use the relationship in the parent entity but just added and removed the child entities everything worked just fine.
Sorry for the VB but that is what the project I am working in is written in.
The parent entity "Report" has a one to many relationship to "ReportRole" and has the property "ReportRoles". The new roles are passed in by a comma separated string from an Ajax call.
The first line will remove all the child entities, and if I used "report.ReportRoles.Remove(f)" instead of the "db.ReportRoles.Remove(f)" I would get the error.
report.ReportRoles.ToList.ForEach(Function(f) db.ReportRoles.Remove(f))
Dim newRoles = If(String.IsNullOrEmpty(model.RolesString), New String() {}, model.RolesString.Split(","))
newRoles.ToList.ForEach(Function(f) db.ReportRoles.Add(New ReportRole With {.ReportId = report.Id, .AspNetRoleId = f}))