Breeze seems to be reverting the fkey property value when the query expands the associated navigation property -- even when using the preserveChange merge strategy. We need a way to preserve all changes.
Steps: Modify an fkey property of an entity in the cache. Query from the server with expand for the associated nav prop. When the query completes (data merged), the fkey property value is the saved value (was reverted).
To demonstrate the problem I ran the following code fragment two times. The first time loads the cache and modifies entities[0].ProjectAreaRef from 1 to 48. The second time performs the same query to test merging. After the 2nd query orig is 48 and val is 1. The change in the cached entity was overwritten.
var query = new breeze.EntityQuery().from('Issue');
query = query.expand('oProjectArea');
entityManager.executeQuery(query).then(function (x) {
var entities = x.results;
var val = entities[0].ProjectAreaRef;
if (orig && val !== orig)
console.log('orig:' + orig + ' val:' + val);
entities[0].ProjectAreaRef = 48;
I guess one might say that expanding into a nav prop is asking to refresh the relationship. Otherwise, the parent entity retrieved (based on the fkey value on the server) might no longer be associated with the child entity. But, that’s fine for us. Let the parent entity be merged into the cache even though it’s no longer associated with the child entity.
Regardless of whether this is a breeze feature or a bug (IMO it's a bug), how can I get breeze to preserve changes to fkey properties? If it's a bug, do you know of a workaround? If it's a feature, is there another feature that allows me to use expand() while preserving changes to fkey properties?
Using latest version: 1.5.3 ... although same behavior in 1.4.11.
Ok, this was a bug and is now fixed in the latest GitHub repo. This fix will also go out in v 1.5.5 (sometime next month). Nice catch!
Related
I'm recently using this to add entity EF to my DB:
ctx.Entry(payment).State = payments.ID == 0 ? EntityState.Added : EntityState.Modified;
Instead of the common:
ctx.Payments.Add(payments);
The second is nice because I can choose if add or delete.
What's the real difference between the first and the second approch? And why there isn't a straightforward for update item? Such as:
ctx.Payments.Update(payments);
Which will "mirror" the EntityState.Modified one?
This is the code I'm using, setting a ViewModel from my MVC Action/Controller to my EF context:
IList<Payments> newPayments = activityViewModel.Payments.Select(p => mapper.Map<PaymentViewModel, Payments>(p)).ToList();
foreach (var payments in newPayments)
{
ctx.Entry(payments).State = payments.ID == 0 ? EntityState.Added : EntityState.Modified;
ctx.Payments.Add(payments);
}
ctx.SaveChanges();
There is no difference between adding a detached entity by setting its state vs. calling Add.
You can see in InternalEntityEntry.cs (wrapped by DbEntityEntry) that setting the state to Added just ends up calling InternalSet.Add. You can see in DbSet`.cs that calling Add also just ends up calling InternalSet.Add. They differ a bit in their preconditions, but in your case, the preconditions of both are met.
The nice part about the first approach, in your case, is that the same bit of code can either add an entity, or attach it and mark it as modified. As you noted, there is no single-method alternative for setting the state to Modified.
The second is nice because I can choose if add or delete.
You can also set the state to Deleted, so that is not an advantage of either over the other.
And why there isn't a straightforward for update item?
Likely because the only use case for it is when you're doing something strange, from EF perspective. Detaching, serialising, modifying, de-serialising and re-attaching entities may be the norm in your framework, but from EF perspective, those are advanced low-level operations with a somewhat enlarged risk of things going wrong. For instance, when your model contains any modifiable property also used for concurrency checks, the fact that your serialisation-deserialisation loses the property's original value means that the concurrency check is sure to fail.
EDIT 1
While I understand that for this particular scenario (and other alike) I could use the mapping editor alone to migrate my store correctly so that the values in the persistent store don't jump around, but that's not a solution to my current problem but only avoids addressing the root of the problem. I am keen on sticking with my custom migration policy as this will give me a lot of control through the migration process, especially for future scenarious where setting up a custom migration policy will work for me. This is for a long term solution and not just for this scenario.
I urge you to try and help me solve the current situation at hand rather than diverting me to lightweight migration or advising me to avoid using a migration policy. Thank you.
I really do look forward to sorting this out and your valuable input/ideas on what I could do to fix this problem.
What I have done:
I have a migration policy set up so that the source data can be copied into the destination data from version 1 of the core model to version 2.
This is the migration policy:
- (BOOL)createDestinationInstancesForSourceInstance:(NSManagedObject *)sInstance entityMapping:(NSEntityMapping *)mapping manager:(NSMigrationManager *)manager error:(NSError **)error {
// Create the product managed object
NSManagedObject *newObject = [NSEntityDescription insertNewObjectForEntityForName:[mapping destinationEntityName]
inManagedObjectContext:[manager destinationContext]];
NSString *productCode = [sInstance valueForKey:#"productCode"];
NSNumber *productPrice = [sInstance valueForKey:#"productPrice"];
[newObject setValue:productCode forKey:#"productCode"];
[newObject setValue:productPrice forKey:#"productPrice"];
//This is the field where the name has changed as well as the type.
[newObject setValue:[NSNumber numberWithBool:YES] forKey:#"productPriceNeedsUpdating"];
// Set up the association between the old source product and the new destination Product for the migration manager
[manager associateSourceInstance:sInstance withDestinationInstance:newObject forEntityMapping:mapping];
/*
A test statement to make sure the destination object contains the correct
values int he right properties:
Product description: <NSManagedObject: 0xb983780> (entity: Product; id: 0xb9837b0 <x-coredata:///Product/t97685A9D-09B4-475F-BDE3-BC9176454AEF6> ; data: {
productCode = 9999;
productPrice = "2.09";
productPriceNeedsUpdating = 1;
})
*/
// Set up the association between the old source product and the new destination Product for the migration manager
return YES;
}
So even though the tested properties show the correct values in runtime, the resultant values saved in the data model store is incorrect as seen in the snapshots.
Here is a comparison from version 1 to version 2 of the data store.
Version 1: Correct
to Version 2: Which is now storing the values incorrectly.
The expected output should have the Product price inserted into the productPrice field and not in the ProductPriceNeedsUpdating field which should actually only have boolean values.
Can anyone help me understand what I am doing wrong, or explain whats happening here?
UPDATE 1 - Here are my entity mappings:
Update 2 - 20/aug/2014 01:02 GMT
When I remove the attribute ProductPriceLastUpdated of type date from version 1, and remove the attribute ProductPriceNeedsUpdate of type boolean in version 2, leaving only the two attributes that both match in version 1 and 2, then everything works. Even though I can leave it here and move on, I cant ignore the users that are currently using version 1 of the database which does have that pointless ProductPriceLastUpdated attribute which I need the type converted to boolean and also have the name changed to ProductPriceNeedsUpdate. Thats when things start going weird, and the price values are shown in the ProductPriceNeedsUpdate field instead of the productPrice field.
I hope someone can address the original problem and tell me why it is that the entityMapping, or more so, property mapping is not being saved properly?
Update 3 - EntityMapping and properties:
Version 1
Version 2
Any ideas?
First, if you want to just use a lightweight migration (which you really should in this case) you can get rid of your custom migration policy. In this instance, it's not needed. And, as a matter of fact, you can get rid of your custom mapping model as well. All you need to do is in your Version 2 model, select the productPriceNeedsUpdating boolean flag, and in the Attribute Detail inspector on the right, set the default value to YES. This will achieve the goal you're try to get to with your custom migration policy.
However, if you really need to write this in code with your custom migration policy, I would still not use custom code. You can achieve this migration with only a mapping model. Simply select the ProductToProduct mapping, and in the value expression for productNeedsUpdating, enter YES, or 1.
EDIT
So, after a rather lengthy screen share, it was determined that the migration was using code from Marcus Zarra's Core Data book describing progressively migrating stores. When this was written, WAL mode was not the default mode with Core Data. When WAL mode is enabled, progressively migrating stores don't function well since there are two more files to deal with, the Write Ahead Log, and the Shared Memory file. When simply replacing the old store with a new one, without first removing those files, odd things happen, such as described in this post. The solution ultimately ended up being to disable WAL mode for the progressively migrating scenario so the extra files are not generated in the first place.
We have a breeze client solution in which we show parent entities with lists of their children. We do hard deletes on some child entities. Now when the user is the one doing the deletes, there is no problem, but when someone else does, there seems to be no way to invalidate the children already loaded in cache. We do a new query with the parent and expanding to children, but breeze attaches all the other children it has already heard of, even if the database did not return them.
My question: shouldn't breeze realize we are loading through expand and thus completely remove all children from cache before loading back the results from the db? How else can we accomplish this if that is not the case?
Thank you
Yup, that's a really good point.
Deletion is simply a horrible complication to every data management effort. This is true no matter whether you use Breeze or not. It just causes heartache up and down the line. Which is why I recommend soft deletes instead of hard deletes.
But you don't care what I think ... so I will continue.
Let me be straight about this. There is no easy way for you to implement a cache cleanup scheme properly. I'm going to describe how we might do it (with some details neglected I'm sure) and you'll see why it is difficult and, in perverse cases, fruitless.
Of course the most efficient, brute force approach is to blow away the cache before querying. You might as well not have caching if you do that but I thought I'd mention it.
The "Detached" entity problem
Before I continue, remember the technique I just mentioned and indeed all possible solutions are useless if your UI (or anything else) is holding references to the entities that you want to remove.
Oh, you'll remove them from cache alright. But whatever is holding references to them now will continue to have a reference to an entity object which is in a "Detached" state - a ghost. Making sure that doesn't happen is your responsibility; Breeze can't know and couldn't do anything about it if it did know.
Second attempt
A second, less blunt approach (suggested by Jay) is to
apply the query to the cache first
iterate over the results and for each one
detach every child entity along the "expand" paths.
detach that top level entity
Now when the query succeeds, you have a clear road for it to fill the cache.
Here is a simple example of the code as it relates to a query of TodoLists and their TodoItems:
var query = breeze.EntityQuery.from('TodoLists').expand('TodoItems');
var inCache = manager.executeQueryLocally(query);
inCache.slice().forEach(function(e) {
inCache = inCache.concat(e.TodoItems);
});
inCache.slice().forEach(function(e) {
manager.detachEntity(e);
});
There are at least four problems with this approach:
Every queried entity is a ghost. If your UI is displaying any of the queried entities, it will be displaying ghosts. This is true even when the entity was not touched on the server at all (99% of the time). Too bad. You have to repaint the entire page.
You may be able to do that. But in many respects this technique is almost as impractical as the first. It means that ever view is in a potentially invalid state after any query takes place anywhere.
Detaching an entity has side-effects. All other entities that depend on the one you detached are instantly (a) changed and (b) orphaned. There is no easy recovery from this, as explained in the "orphans" section below.
This technique wipes out all pending changes among the entities that you are querying. We'll see how to deal with that shortly.
If the query fails for some reason (lost connection?), you've got nothing to show. Unless you remember what you removed ... in which case you could put those entities back in cache if the query fails.
Why mention a technique that may have limited practical value? Because it is a step along the way to approach #3 that could work
Attempt #3 - this might actually work
The approach I'm about to describe is often referred to as "Mark and Sweep".
Run the query locally and calculate theinCache list of entities as just described. This time, do not remove those entities from cache. We WILL remove the entities that remain in this list after the query succeeds ... but not just yet.
If the query's MergeOption is "PreserveChanges" (which it is by default), remove every entity from the inCache list (not from the manager's cache!) that has pending changes. We do this because such entities must stay in cache no matter what the state of the entity on the server. That's what "PreserveChanges" means.
We could have done this in our second approach to avoid removing entities with unsaved changes.
Subscribe to the EntityManager.entityChanged event. In your handler, remove the "entity that changed" from the inCache list because the fact that this entity was returned by the query and merged into the cache tells you it still exists on the server. Here is some code for that:
var handlerId = manager.entityChanged.subscribe(trackQueryResults);
function trackQueryResults(changeArgs) {
var action = changeArgs.entityAction;
if (action === breeze.EntityAction.AttachOnQuery ||
action === breeze.EntityAction.MergeOnQuery) {
var ix = inCache.indexOf(changeArgs.entity);
if (ix > -1) {
inCache.splice(ix, 1);
}
}
}
If the query fails, forget all of this
If the query succeeds
unsubscribe: manager.entityChanged.unsubscribe(handlerId);
subscribe with orphan detection handler
var handlerId = manager.entityChanged.subscribe(orphanDetector);
function orphanDetector(changeArgs) {
var action = changeArgs.entityAction;
if (action === breeze.EntityAction.PropertyChange) {
var orphan = changeArgs.entity;
// do something about this orphan
}
}
detach every entity that remains in the inCache list.
inCache.slice().forEach(function(e) {
manager.detachEntity(e);
});
unsubscribe the orphan detection handler
Orphan Detector?
Detaching an entity can have side-effects. Suppose we have Products and every product has a Color. Some other user hates "red". She deletes some of the red products and changes the rest to "blue". Then she deletes the "red" Color.
You know nothing about this and innocently re-query the Colors. The "red" color is gone and your cleanup process detaches it from cache. Instantly every Product in cache is modified. Breeze doesn't know what the new Color should be so it sets the FK, Product.colorId, to zero for every formerly "red" product.
There is no Color with id=0 so all of these products are in an invalid state (violating referential integrity constraint). They have no Color parent. They are orphans.
Two questions: how do you know this happened to you and what do your do?
Detection
Breeze updates the affected products when you detach the "red" color.
You could listen for a PropertyChanged event raised during the detach process. That's what I did in my code sample. In theory (and I think "in fact"), the only thing that could trigger the PropertyChanged event during the detach process is the "orphan" side-effect.
What do you do?
leave the orphan in an invalid, modified state?
revert to the equally invalid former colorId for the deleted "red" color?
refresh the orphan to get its new color state (or discover that it was deleted)?
There is no good answer. You have your pick of evils with the first two options. I'd probably go with the second as it seems least disruptive. This would leave the products in "Unchanged" state, pointing to a non-existent Color.
It's not much worse then when you query for the latest products and one of them refers to a new Color ("banana") that you don't have in cache.
The "refresh" option seems technically the best. It is unwieldy. It could easily cascade into a long chain of asynchronous queries that could take a long time to finish.
The perfect solution escapes our grasp.
What about the ghosts?
Oh right ... your UI could still be displaying the (fewer) entities that you detached because you believe they were deleted on the server. You've got to remove these "ghosts" from the UI.
I'm sure you can figure out how to remove them. But you have to learn what they are first.
You could iterate over every entity that you are displaying and see if it is in a "Detached" state. YUCK!
Better I think if the cleanup mechanism published a (custom?) event with the list of entities you detached during cleanup ... and that list is inCache. Your subscriber(s) then know which entities have to be removed from the display ... and can respond appropriately.
Whew! I'm sure I've forgotten something. But now you understand the dimensions of the problem.
What about server notification?
That has real possibilities. If you can arrange for the server to notify the client when any entity has been deleted, that information can be shared across your UI and you can take steps to remove the deadwood.
It's a valid point but for now we don't ever remove entities from the local cache as a result of a query. But.. this is a reasonable request, so please add this to the breeze User Voice. https://breezejs.uservoice.com/forums/173093-breeze-feature-suggestions
In the meantime, you can always create a method that removes the related entities from the cache before the query executes and have the query (with expand) add them back.
In a previous question, it was stated that:
"On the client an unmapped property behaves in other respects like a mapped property"
"rejectChanges() reverts the property to that original value"
I'm experiencing the same issue described in that question: EntityManager.rejectChanges() doesn't revert unmapped properties to the original value, while EntityAspect.rejectChanges() does.
In the responses to that question, it was suggested that this was probably due to a coding error. I've made a plunker demonstrating the issue. Is there an error in my code that is causing this?
Edit - Updated Test Case:
test("reject changes reverts an unmapped property - only unmapped property changed", 1, function () {
var store = cloneModuleMetadataStore();
var originalTime = new Date(2013, 0, 1);
var Customer = function () {
this.lastTouched = originalTime;
};
store.registerEntityTypeCtor("Customer", Customer);
var manager = newEm(store);
// create a fake customer
var cust = manager.createEntity("Customer", { CompanyName: "Acme" },
EntityState.Unchanged);
var touched = cust.lastTouched();
// we change only the unmapped property (uncomment the next line and the test will pass)
//cust.CompanyName("Beta");
cust.lastTouched(new Date(touched.getTime() + 60000));
//cust.entityAspect.rejectChanges(); // roll back name change
manager.rejectChanges(); // would have same effect. Obviously less granular
ok(originalTime === cust.lastTouched(),
"'lastTouched' unmapped property should be rolled back. Started as {0}; now is {1}"
.format(originalTime, cust.lastTouched()));
});
you can see that in this environment, the test passes with entityAspect.rejectChanges(), but fails with manager.rejectChanges(). if a mapped property is changed along with the unmapped property, the test passes.
Updated answer 2/2/2014
Ok, what you have discovered is actually by design. And.. thanks for the test above, ( it makes understanding the issue much easier).
The issue here is that changes to unmapped properties do NOT change the EntityState of the entity. This decision was made because these changes do not actually ever need to be persisted to the server ( because there is nowhere to put them).
The second issue is that when calling EntityManager.rejectChanges we only process entities that have an Added, Modified or Deleted EntityState. Since an entity whose ONLY change is to an unmapped property does not fall into this category, the entity level rejectChanges call is never made.
There are several workarounds.
1) Call EntityAspect.setModified() after any change to an unmapped property. You can try this on the test above to see that it works. ( A slightly more complicated version of this is to use the EntityManager events to do this automatically).
2) Change any mapped property whenever you change an unmapped one.
3) Write your own EntityManager.rejectChanges that calls EntityAspect.rejectChanges on every entity in the EntityManager instead of just the 'changed' ones. This does have perf implications so I don't really recommend it unless you have a very small cache.
Please feel free to suggest an alternative that makes sense to you. We have considered adding settings to allow you to configure the treatment of unmapped properties. ( among these is whether an unmapped property change will change the entity state).
I can't repro this... and reviewing the code, the EntityManager.rejectChanges simply calls into the EntityAspect.rejectChanges for all entities within the manager.
So there are a couple of possibilities
1) The EntityAspect that you are NOT seeing rejectChanges work properly with is not actually "attached" to the EntityManager.
2) You are not actually comparing the behavior of "rejectChanges" on the SAME entity in both cases.
Take a look at the test cases within the DocCode sample in the Breeze zip. These tests require no UI and are typically very short. If you can paste a simple test here that fails in that environment, I will take a look. Having a UI involved often clouds the picture.
I'm learning about concurrency in conjunction with EF4.0 and have a question about the locking pattern used.
Say I configure a fixed concurrency mode on a version number property.
Now say I fetch a record (entity) from the database (context) and edit some property. Version gets incremented and when SaveChanges is called on its context. If the current database (context) version matches the version of the original record (entity) the save continues, otherwise an OptimisticConcurrencyException gets thrown by EF.
Now, my point of interest is the following: between the check of the versions there's always a small period of time, however small, it is there. So in theory someone else could've just updated the record between the comparison and the actual save, thus possibly corrupting the data.
How does this get solved? It feels as if the problem just gets pushed forward.
There is no period of time between checking versions and updating record because the database command looks like:
UPDATE SomeTable
SET SomeColumn = 'SomeValue'
WHERE Id = #Id AND Version = #OldVersion
SELECT ##ROWCOUNT
The check and update is one atomic operation. Rowcount will return 0 if no record with Id = #Id and Version = #OldVersion exists and that zero is translated to the exception.
This can (and probably is) solved using locking hints.
For SQL Server, EF can query (SELECT) from the database WITH UPDLOCK.
This tells the Database Engine that, you want to read a/several records, and nobody else can change those records until you perform an update thereafter.
If you want to see this for yourself, check out the Sql Server Profiler which will show you the queries in real-time.
Hope that helps.
CAVEAT: I can't say for sure that this is the way EF handles this scenario because I haven't checked myself but, certainly if you were going to do it yourself, this is one way to do it.