I'm using Wards excellent example of implementing many-to-many with breeze. (i'm going off the plunk in his post, cant link to it from here dont know why)
Everything is working great, changes are always saved to database correctly.
I am using 2 different (BreezeJS) EntityManagers: one for edits, and one as my "master". On saving the editor Em to database, it then imports all changes into the master Em so it all stays in sync. This has been working wonderfully for all my other functions.
However, when saving the many-to-many mapping, for some reason any deleted mappings are not removed from the master Em. (When I add mappings they correctly show up in the master Em right away).
Do I need to take another step to get my master Em to remove imported detached entities?
(FYI, everything is saving correctly to server, if I do hard page refresh, all my entities show up correctly).
My code for deleting entity on editor Em:myEntity.entityAspect.setDeleted();
Function below will export changed entities from editor Em:
function exportToMasterAfterSavingSuccess(saveResult){
if(saveResult.entities)
masterEm.importEntities(manager.exportEntities(entities, false));
}
And the corresponding import function on master em:
function importEntities(entities){
var imported = manager.importEntities(entities,{ mergeStrategy: breeze.MergeStrategy.OverwriteChanges});
}
Thanks for uncovering a bug. One shouldn't be able to export or import a detached entity. In future, Breeze will throw if you attempt to do either.
Now I'll discuss your issue, your aims, and what I recommend that you do.
Update master EntityManager after saving deleted entities
As I understand it you maintain a masterEm which has only the saved state of entities. You make and save your changes in a separate editEm. You import entities that you will change into the editEm, make changes, save them, and (if the save is successful), you export the saved entities from editEm and import them back into masterEm. This is a common "sandbox editing" pattern.
Trouble arises when you delete an entity in editEm. After save, that entity is "Detached" in the editEm but it's still in an "Unchanged" state back in the masterEm. How do you communicate the fact that the entity is deleted and remove it from masterEm?
This dilemma exists independent of the "many-to-many" scenario that inspired your question.
I can see why your practice of importing the now-Detached entity from editEm to masterEm seemed to work. Doing that in v.1.5.3 caused the corresponding entity in masterEm to change to the "Detached" state ... which is what you wanted. The bug, as you saw it, was that the importEntities method didn't handle update of navigation properties properly when the imported entity is in a "Detached" state. You proposed teaching importEntities to "do the right thing" in that scenario.
What actually happened here is that you discovered a bug. You should never have been able to export a "Detached" entity and you shouldn't have been able to import one either. Breeze should have thrown an exception when you tried to export or import a "Detached" entity.
There are all kinds of reasons why asking an EntityManager to export/import "Detached" entities is a bad idea. I leave explication of those reasons for another day.
Rather than "solve" the problem of importing related "Detached" entities, we will throw an error.
This means that your partial solution will cease to work, leaving you apparently worse of than you are today. Fortunately, I have an alternative approach for you. I've written this utility function and tested it in DocCode:
function updateMasterWithSaveResult(masterEm, sourceEm, saveResult) {
var imports = [];
var deletes = [];
saveResult.entities.forEach(function(entity) {
if (entity.entityAspect.entityState.isDetached()) {
deletes.push(entity);
} else {
imports.push(entity);
}
});
var exported = sourceEm.exportEntities(imports, {
includeMetadata: false,
asString: false // as JSON
});
masterEm.importEntities(exported);
deletes.forEach(function(detached) {
var entity = masterEm.getEntityByKey(detached.entityAspect.getKey());
entity && entity.entityAspect.setDetached();
});
}
Updated documentation
I just added this to our "Cool Breeze" documentation almost verbatim.
Please try this with the latest released version of breeze (1.5.3). A very similar bug was fixed there.
And just to be clear, and export can never include 'detached' entities ( only 'deleted' ones). Detached entities are, by definition, no longer attached to an EntityManager so the EntityManager no longer knows anything about them.
I'm having the exact same issue. I'm not doing many-to-many but still doing edits/adds/deletes in a separate entityManager.
I can confirm that the export DOES include Detached entities, and on import into the master entityManager with MergeStrategy.OverwriteChanges, those entities do become detached.
However, the newly detached entities are still associated with any related entities.
Reattaching and detaching seems to get everything back in sync as far as I can tell:
var result = manager.importEntities(imports);
result.entities
.filter(function (entity) { return entity.entityAspect.entityState.isDetached(); })
.forEach(function (entity) { manager.attachEntity(entity); manager.detachEntity(entity); });
Breeze 1.5.3
Github issue with pull request:
https://github.com/Breeze/breeze.js/issues/75
Related
This is a follow up to an earlier question: Core Data: change delete rule programmatically.
I'd like to rephrase my question, and will do that here.
Briefly, my app allows updating entries from a 3rd party database, but I'd like to keep user annotations. So my workflow is:
iterate over all entities
download external xml and parse it into a new entity
if user annotations, change their relationship from old entity to new entity
delete old entity
During the import, the old entity is in the main context, the new entity is in a temporary import context.
Number 3 gives me problems, if I just change the relationship, then they don't show if I update my UI. If I use the objectID to get the annotation and then change the relationship as follows:
NSManagedObjectID *objectId = oldAnnotation.objectID;
Annotation *newAnnotation = [importContext objectWithID: objectId];
[newEntry addAnnotationObject: newAnnotation];
It's still not working - it's not showing up.
EDIT: if I change the context in the second line to newEntry.managedObjectContext, I get an Illegal attempt to establish a relationship 'foo' between objects in different contexts error.
What am I missing?
UPDATE: After some late-night hair-pulling debugging, I found that I when I was fetching the newEntry, I was actually fetching the oldEntry, therefore none of the changes would show up. The answer below by #Mundi pointed me in the right direction.
Copying the old annotations worked using my code above, followed by copying the attributes. For some user input with relationships in itself, I had to do a "Deep Copy", which I found here: How can I duplicate, or copy a Core Data Managed Object?.
I think creating a new entity and deleting the old one is a problematic strategy. You should try to properly update the existing entities and only create new ones if they do not yet exist.
Whenever I need an object from a different context, I fetch it. That being said, your object id code should work. However, there could be all sorts of other glitches, that you should check:
Did you save the importContext?
Did you save its parent context, presumably the main context?
Was the modified object graph saved to the persistent store?
Are you checking the results after you have saved?
I'm using Breeze version 1.5.5 and Knockout 3.4.0 in a SPA. I have a function I created to remove an entity from an array of entities. I'm trying to implement an undo using rejectChanges() but the observableArray is not reverting. When I call manager.getEntities() I see that the EntityState is set to 'unchanged'. I'm simplified the function to an inane example:
var removeVehicle = function(data) {
data.entityAspect.setDeleted();
data.entityAspect.rejectChanges();
};
This function does not undelete the entity. However when I modify the function to just make a text change it undoes the change without issue.
var removeVehicle = function(data) {
data.model('test');
data.entityAspect.rejectChanges();
};
I'd like to be able to undo the deletion but I'm not sure where to start. I've also tried adding an entity beforehand and rejectChanges does indeed remove it.
EDIT: (2016-03-09) The 'parent' of this entity is an object loaded all at once when the page is activated (Durandal). I'm not sure if it's relevant but apparently Breeze has some trouble with complex objects.
EDIT: (2016-03-11) I forked the breeze.js.samples project and made similar changes to the Todo-Knockout-Require project therein (Breeze Fork for SO35852344.) I still have the same issue. The exact location of the code in question is the removeItem function in the viewModel.js file.
I use getEntityGraph extension and it works fine except in the following scenario:
add a new entity
don't save it and call setDeleted on the entity
call getEntityGraph by passing the entity and a np collection as parameters
When makePathSegmentFn is called, it crashes on this line :
grps.forEach(function(grp) {
vals = vals.concat(grp._entities.filter(function (en) {
return en.getProperty(fkName) === keyValue;
}));
});
en is null so it raises an exception. I've worked around the problem by checking if en is null and every seems to work fine. But perhaps that should be done in the original code if it's a bug ? Note that only one entity is null amongst all the entities in the np collection. I guess that's the one that was deleted, but can't tell for sure.
Update 29 April 2014
OK ... I get it now. You're talking about a deleted child entity, not a detached root entity.
Thanks for identifying this bug. I added a test for this scenario to DocCode, then fixed the bug. Both changes pushed to github. They will appear in the next official release. You can get the current getEntityGraph.js from github right now.
Original answer
I can't duplicate the particular failure you describe ... because getEntityGraph throws long before it gets to the makePathSegmentFn ... as it should do!
getEntityGraph is supposed to throw an exception (e.g., "'getEntityGraph' root[0] is a detached entity") when any of the root entities passed in are 'Detached'.
When you create a new entity and immediately delete it (without saving it first), its state changes from 'Added' to 'Detached'; it is no longer an entity in cache. That is expected behavior (see "EntityState transitions" in the "Inside the Entity" documentation topic). That's what happens when I follow your repro steps exactly.
Please provide a jsFiddle or plunker that demonstrates the error.
We have a breeze client solution in which we show parent entities with lists of their children. We do hard deletes on some child entities. Now when the user is the one doing the deletes, there is no problem, but when someone else does, there seems to be no way to invalidate the children already loaded in cache. We do a new query with the parent and expanding to children, but breeze attaches all the other children it has already heard of, even if the database did not return them.
My question: shouldn't breeze realize we are loading through expand and thus completely remove all children from cache before loading back the results from the db? How else can we accomplish this if that is not the case?
Thank you
Yup, that's a really good point.
Deletion is simply a horrible complication to every data management effort. This is true no matter whether you use Breeze or not. It just causes heartache up and down the line. Which is why I recommend soft deletes instead of hard deletes.
But you don't care what I think ... so I will continue.
Let me be straight about this. There is no easy way for you to implement a cache cleanup scheme properly. I'm going to describe how we might do it (with some details neglected I'm sure) and you'll see why it is difficult and, in perverse cases, fruitless.
Of course the most efficient, brute force approach is to blow away the cache before querying. You might as well not have caching if you do that but I thought I'd mention it.
The "Detached" entity problem
Before I continue, remember the technique I just mentioned and indeed all possible solutions are useless if your UI (or anything else) is holding references to the entities that you want to remove.
Oh, you'll remove them from cache alright. But whatever is holding references to them now will continue to have a reference to an entity object which is in a "Detached" state - a ghost. Making sure that doesn't happen is your responsibility; Breeze can't know and couldn't do anything about it if it did know.
Second attempt
A second, less blunt approach (suggested by Jay) is to
apply the query to the cache first
iterate over the results and for each one
detach every child entity along the "expand" paths.
detach that top level entity
Now when the query succeeds, you have a clear road for it to fill the cache.
Here is a simple example of the code as it relates to a query of TodoLists and their TodoItems:
var query = breeze.EntityQuery.from('TodoLists').expand('TodoItems');
var inCache = manager.executeQueryLocally(query);
inCache.slice().forEach(function(e) {
inCache = inCache.concat(e.TodoItems);
});
inCache.slice().forEach(function(e) {
manager.detachEntity(e);
});
There are at least four problems with this approach:
Every queried entity is a ghost. If your UI is displaying any of the queried entities, it will be displaying ghosts. This is true even when the entity was not touched on the server at all (99% of the time). Too bad. You have to repaint the entire page.
You may be able to do that. But in many respects this technique is almost as impractical as the first. It means that ever view is in a potentially invalid state after any query takes place anywhere.
Detaching an entity has side-effects. All other entities that depend on the one you detached are instantly (a) changed and (b) orphaned. There is no easy recovery from this, as explained in the "orphans" section below.
This technique wipes out all pending changes among the entities that you are querying. We'll see how to deal with that shortly.
If the query fails for some reason (lost connection?), you've got nothing to show. Unless you remember what you removed ... in which case you could put those entities back in cache if the query fails.
Why mention a technique that may have limited practical value? Because it is a step along the way to approach #3 that could work
Attempt #3 - this might actually work
The approach I'm about to describe is often referred to as "Mark and Sweep".
Run the query locally and calculate theinCache list of entities as just described. This time, do not remove those entities from cache. We WILL remove the entities that remain in this list after the query succeeds ... but not just yet.
If the query's MergeOption is "PreserveChanges" (which it is by default), remove every entity from the inCache list (not from the manager's cache!) that has pending changes. We do this because such entities must stay in cache no matter what the state of the entity on the server. That's what "PreserveChanges" means.
We could have done this in our second approach to avoid removing entities with unsaved changes.
Subscribe to the EntityManager.entityChanged event. In your handler, remove the "entity that changed" from the inCache list because the fact that this entity was returned by the query and merged into the cache tells you it still exists on the server. Here is some code for that:
var handlerId = manager.entityChanged.subscribe(trackQueryResults);
function trackQueryResults(changeArgs) {
var action = changeArgs.entityAction;
if (action === breeze.EntityAction.AttachOnQuery ||
action === breeze.EntityAction.MergeOnQuery) {
var ix = inCache.indexOf(changeArgs.entity);
if (ix > -1) {
inCache.splice(ix, 1);
}
}
}
If the query fails, forget all of this
If the query succeeds
unsubscribe: manager.entityChanged.unsubscribe(handlerId);
subscribe with orphan detection handler
var handlerId = manager.entityChanged.subscribe(orphanDetector);
function orphanDetector(changeArgs) {
var action = changeArgs.entityAction;
if (action === breeze.EntityAction.PropertyChange) {
var orphan = changeArgs.entity;
// do something about this orphan
}
}
detach every entity that remains in the inCache list.
inCache.slice().forEach(function(e) {
manager.detachEntity(e);
});
unsubscribe the orphan detection handler
Orphan Detector?
Detaching an entity can have side-effects. Suppose we have Products and every product has a Color. Some other user hates "red". She deletes some of the red products and changes the rest to "blue". Then she deletes the "red" Color.
You know nothing about this and innocently re-query the Colors. The "red" color is gone and your cleanup process detaches it from cache. Instantly every Product in cache is modified. Breeze doesn't know what the new Color should be so it sets the FK, Product.colorId, to zero for every formerly "red" product.
There is no Color with id=0 so all of these products are in an invalid state (violating referential integrity constraint). They have no Color parent. They are orphans.
Two questions: how do you know this happened to you and what do your do?
Detection
Breeze updates the affected products when you detach the "red" color.
You could listen for a PropertyChanged event raised during the detach process. That's what I did in my code sample. In theory (and I think "in fact"), the only thing that could trigger the PropertyChanged event during the detach process is the "orphan" side-effect.
What do you do?
leave the orphan in an invalid, modified state?
revert to the equally invalid former colorId for the deleted "red" color?
refresh the orphan to get its new color state (or discover that it was deleted)?
There is no good answer. You have your pick of evils with the first two options. I'd probably go with the second as it seems least disruptive. This would leave the products in "Unchanged" state, pointing to a non-existent Color.
It's not much worse then when you query for the latest products and one of them refers to a new Color ("banana") that you don't have in cache.
The "refresh" option seems technically the best. It is unwieldy. It could easily cascade into a long chain of asynchronous queries that could take a long time to finish.
The perfect solution escapes our grasp.
What about the ghosts?
Oh right ... your UI could still be displaying the (fewer) entities that you detached because you believe they were deleted on the server. You've got to remove these "ghosts" from the UI.
I'm sure you can figure out how to remove them. But you have to learn what they are first.
You could iterate over every entity that you are displaying and see if it is in a "Detached" state. YUCK!
Better I think if the cleanup mechanism published a (custom?) event with the list of entities you detached during cleanup ... and that list is inCache. Your subscriber(s) then know which entities have to be removed from the display ... and can respond appropriately.
Whew! I'm sure I've forgotten something. But now you understand the dimensions of the problem.
What about server notification?
That has real possibilities. If you can arrange for the server to notify the client when any entity has been deleted, that information can be shared across your UI and you can take steps to remove the deadwood.
It's a valid point but for now we don't ever remove entities from the local cache as a result of a query. But.. this is a reasonable request, so please add this to the breeze User Voice. https://breezejs.uservoice.com/forums/173093-breeze-feature-suggestions
In the meantime, you can always create a method that removes the related entities from the cache before the query executes and have the query (with expand) add them back.
In a previous question, it was stated that:
"On the client an unmapped property behaves in other respects like a mapped property"
"rejectChanges() reverts the property to that original value"
I'm experiencing the same issue described in that question: EntityManager.rejectChanges() doesn't revert unmapped properties to the original value, while EntityAspect.rejectChanges() does.
In the responses to that question, it was suggested that this was probably due to a coding error. I've made a plunker demonstrating the issue. Is there an error in my code that is causing this?
Edit - Updated Test Case:
test("reject changes reverts an unmapped property - only unmapped property changed", 1, function () {
var store = cloneModuleMetadataStore();
var originalTime = new Date(2013, 0, 1);
var Customer = function () {
this.lastTouched = originalTime;
};
store.registerEntityTypeCtor("Customer", Customer);
var manager = newEm(store);
// create a fake customer
var cust = manager.createEntity("Customer", { CompanyName: "Acme" },
EntityState.Unchanged);
var touched = cust.lastTouched();
// we change only the unmapped property (uncomment the next line and the test will pass)
//cust.CompanyName("Beta");
cust.lastTouched(new Date(touched.getTime() + 60000));
//cust.entityAspect.rejectChanges(); // roll back name change
manager.rejectChanges(); // would have same effect. Obviously less granular
ok(originalTime === cust.lastTouched(),
"'lastTouched' unmapped property should be rolled back. Started as {0}; now is {1}"
.format(originalTime, cust.lastTouched()));
});
you can see that in this environment, the test passes with entityAspect.rejectChanges(), but fails with manager.rejectChanges(). if a mapped property is changed along with the unmapped property, the test passes.
Updated answer 2/2/2014
Ok, what you have discovered is actually by design. And.. thanks for the test above, ( it makes understanding the issue much easier).
The issue here is that changes to unmapped properties do NOT change the EntityState of the entity. This decision was made because these changes do not actually ever need to be persisted to the server ( because there is nowhere to put them).
The second issue is that when calling EntityManager.rejectChanges we only process entities that have an Added, Modified or Deleted EntityState. Since an entity whose ONLY change is to an unmapped property does not fall into this category, the entity level rejectChanges call is never made.
There are several workarounds.
1) Call EntityAspect.setModified() after any change to an unmapped property. You can try this on the test above to see that it works. ( A slightly more complicated version of this is to use the EntityManager events to do this automatically).
2) Change any mapped property whenever you change an unmapped one.
3) Write your own EntityManager.rejectChanges that calls EntityAspect.rejectChanges on every entity in the EntityManager instead of just the 'changed' ones. This does have perf implications so I don't really recommend it unless you have a very small cache.
Please feel free to suggest an alternative that makes sense to you. We have considered adding settings to allow you to configure the treatment of unmapped properties. ( among these is whether an unmapped property change will change the entity state).
I can't repro this... and reviewing the code, the EntityManager.rejectChanges simply calls into the EntityAspect.rejectChanges for all entities within the manager.
So there are a couple of possibilities
1) The EntityAspect that you are NOT seeing rejectChanges work properly with is not actually "attached" to the EntityManager.
2) You are not actually comparing the behavior of "rejectChanges" on the SAME entity in both cases.
Take a look at the test cases within the DocCode sample in the Breeze zip. These tests require no UI and are typically very short. If you can paste a simple test here that fails in that environment, I will take a look. Having a UI involved often clouds the picture.