Edit 1
While I understand that for this particular scenario (and other alike) I could use the mapping editor alone to migrate my store correctly so that the values in the persistent store don't jump around, but that's not a solution to my current problem but only avoids addressing the root of the problem. I am keen on sticking with my custom migration policy as this will give me a lot of control through the migration process, especially for future scenarious where setting up a custom migration policy will work for me. This is for a long term solution and not just for this scenario.
I urge you to try and help me solve the current situation at hand rather than diverting me to lightweight migration or advising me to avoid using a migration policy. Thank you.
I really do look forward to sorting this out and your valuable input/ideas on what I could do to fix this problem.
What I have done:
I have a migration policy set up so that the source data can be copied into the new destination data from version 1 of the core model to version 2.
This is the migration policy:
- (BOOL)createDestinationInstancesForSourceInstance:(NSManagedObject *)sInstance entityMapping:(NSEntityMapping *)mapping manager:(NSMigrationManager *)manager error:(NSError **)error {
// Create the product managed object
Product *newProductInstance = [NSEntityDescription insertNewObjectForEntityForName:[mapping destinationEntityName]
inManagedObjectContext:[manager destinationContext]];
NSString *productCode = [sInstance valueForKey:#"productCode"];
NSNumber *productPrice = [sInstance valueForKey:#"productPrice"];
[newProductInstance setProductCode: productCode];
[newProductInstance setProductPrice:productPrice];
/**
The previous old product entries didnt have anything for the last attribute,
where as the new instances of product entity should have a default value of YES.
*/
[newProductInstance setProductPriceNeedsUpdating:[NSNumber numberWithBool:YES]];
/*
A test statement to make sure the destination object contains the correct
values int he right properties:
Product description: <NSManagedObject: 0xb983780> (entity: Product; id: 0xb9837b0 <x-coredata:///Product/t97685A9D-09B4-475F-BDE3-BC9176454AEF6> ; data: {
productCode = 9999;
productPrice = "2.09";
productPriceNeedsUpdating = 1;
})
*/
NSLog(#"Product description: %#", [newProductInstance description]);
// Set up the association between the old source product and the new destination Product for the migration manager
[manager associateSourceInstance:sInstance
withDestinationInstance:newProductInstance
forEntityMapping:mapping];
return YES;
}
So even though the tested properties show the correct values in runtime, the resultant values saved in the data model store is incorrect as seen in the snapshots.
Here is a comparison from version 1 to version 2 of the data store.
Version 1: Correct
to Version 2: Which is now storing the values incorrectly.
The expected output should have the Product price inserted into the productPrice field and not in the ProductPriceNeedsUpdating field which should actually only have boolean values.
Can anyone help me understand what I am doing wrong, or explain whats happening here?
UPDATE 1 - Here are my entity mappings:
Update 2 - 20/aug/2014 01:02 GMT
When I remove the attribute ProductPriceLastUpdated of type date from version 1, and remove the attribute ProductPriceNeedsUpdate of type boolean in version 2, leaving only the two attributes that both match in version 1 and 2, then everything works. Even though I can leave it here and move on, I cant ignore the users that are currently using version 1 of the database which does have that pointless ProductPriceLastUpdated attribute which I need the type converted to boolean and also have the name changed to ProductPriceNeedsUpdate. Thats when things start going weird, and the price values are shown in the ProductPriceNeedsUpdate field instead of the productPrice field.
I hope someone can address the original problem and tell me why it is that the entityMapping, or more so, property mapping is not being saved properly?
Update 3 - EntityMapping and properties:
Version 1
Version 2
For your scenario there is no need for a migration policy. You can just use the mapping model and do the following:
Map the non-changing entities to each other. In the mapping model editor it should say something like A productCode | $source.productCode. This is already filled in for you.
The attribute to be dropped (productPriceLastUpdated) should not appear at all because when you created the mapping model you specified the source and destination models. As the destination model does not contain this attribute, it won't show up.
Where the new boolean attribute productPriceNeedsUpdating is shown, enter '1' for the "Value Expression" (this is equivalent to #YES). It should look like this:
A productPriceNeedsUpdating | 1
Rather than define a policy, you can now just call
migrateStoreFromURL:type:options:withMappingModel:toDestinationURL:
destinationType:destinationOptions:error
You can do very complex operations by just using the mapping model editor.
Your screen shots are from the model editor. Make sure you get familiar with the mapping editor.
Related
One of my current core data entities - Entity1 - has a Boolean attribute called isSaved.
In the new core data model, I am planning to remove isSaved attribute and add a new Int attribute called type. And for all saved Entity1 objects, I'd like to set the value of type according to the value of isSaved in old core data model. (e.g. if isSaved is true, then type is 1, else type is 2).
I've read some articles about light weight core data migration, but none of them seems helpful.
Just wondering if there is any way that can make my planned migration work?
Lightweight migration can't do this. You'll have to create a mapping model and a subclass of NSEntityMigrationPolicy. It's not difficult but it's unfamiliar territory for most iOS developers. The steps run like this:
Create the mapping model. In Xcode, File --> New --> Mapping Model. When you click "Next", Xcode will ask for the source (old) and destination (new) model files for this mapping.
The model file will infer mappings where possible. Everything else will be blank. With your type and some other properties, it'll look something like the following. Entries like $source.timestamp mean to copy the existing value from before the migration.
Create a new subclass of NSEntityMigrationPolicy. Give the subclass an obvious name like ModelMigration1to2. This class will tell Core Data how to map the old boolean value to the new integer value.
Add a method to the subclass to convert the value. Something like the following. The method name doesn't matter but it's good if you choose something descriptive. You need to use ObjC types here-- e.g. NSNumber instead of Int and Bool.
#objc func typeFor(isSaved:NSNumber) -> NSNumber {
if isSaved.boolValue {
return NSNumber(integerLiteral: 1)
} else {
return NSNumber(integerLiteral: 2)
}
}
Go back to the mapping model and tell it to use your subclass as its custom mapping policy. That's in the inspector on the right under "custom policy". Be sure to include the module name and class name.
Tell the mapping model to use that function you created earlier to get values for the type property from the old isSaved property. The following says to call a function on the custom policy class named typeForIsSaved: (the : is important) with one argument, and that the argument should be the isSaved value on $source (the old managed object).
Migration should now work. You don't have to tell Core Data to use the mapping model-- it'll figure out that migration is needed and look for a model that matches the old and new model versions.
A couple of notes:
If you crash with an error that's something like Couldn't create mapping policy for class named... then you forgot the module name above in step 5 (or got it wrong).
If you get a crash with an unrecognized selector error then the method signature in step 4 doesn't match what you entered in step 6. This can also happen if you forget to include #objc in the function declaration.
Using Xcode 9.1 Beta with Swift 4, I find migration works but you have to be careful how you specify the transform method name, also it seems you need to mark your functions as #objc.
For example, my Value Expression:
FUNCTION($entityPolicy, "changeDataForData:" , $source.name)
My transformation policy method name:
class StudentTransformationPolicy: NSEntityMigrationPolicy {
#objc func changeData(forData: Data) -> String {
return String(data: forData, encoding: .utf8)!
}
}
Definitely tricky and took a lot of experimenting before I got it to trigger when launching my app after model changes. It might be easier to implement "createDestinationInstances" for your policy if all of this doesn't work, but we'll leave that for another day...
EDIT 1
While I understand that for this particular scenario (and other alike) I could use the mapping editor alone to migrate my store correctly so that the values in the persistent store don't jump around, but that's not a solution to my current problem but only avoids addressing the root of the problem. I am keen on sticking with my custom migration policy as this will give me a lot of control through the migration process, especially for future scenarious where setting up a custom migration policy will work for me. This is for a long term solution and not just for this scenario.
I urge you to try and help me solve the current situation at hand rather than diverting me to lightweight migration or advising me to avoid using a migration policy. Thank you.
I really do look forward to sorting this out and your valuable input/ideas on what I could do to fix this problem.
What I have done:
I have a migration policy set up so that the source data can be copied into the destination data from version 1 of the core model to version 2.
This is the migration policy:
- (BOOL)createDestinationInstancesForSourceInstance:(NSManagedObject *)sInstance entityMapping:(NSEntityMapping *)mapping manager:(NSMigrationManager *)manager error:(NSError **)error {
// Create the product managed object
NSManagedObject *newObject = [NSEntityDescription insertNewObjectForEntityForName:[mapping destinationEntityName]
inManagedObjectContext:[manager destinationContext]];
NSString *productCode = [sInstance valueForKey:#"productCode"];
NSNumber *productPrice = [sInstance valueForKey:#"productPrice"];
[newObject setValue:productCode forKey:#"productCode"];
[newObject setValue:productPrice forKey:#"productPrice"];
//This is the field where the name has changed as well as the type.
[newObject setValue:[NSNumber numberWithBool:YES] forKey:#"productPriceNeedsUpdating"];
// Set up the association between the old source product and the new destination Product for the migration manager
[manager associateSourceInstance:sInstance withDestinationInstance:newObject forEntityMapping:mapping];
/*
A test statement to make sure the destination object contains the correct
values int he right properties:
Product description: <NSManagedObject: 0xb983780> (entity: Product; id: 0xb9837b0 <x-coredata:///Product/t97685A9D-09B4-475F-BDE3-BC9176454AEF6> ; data: {
productCode = 9999;
productPrice = "2.09";
productPriceNeedsUpdating = 1;
})
*/
// Set up the association between the old source product and the new destination Product for the migration manager
return YES;
}
So even though the tested properties show the correct values in runtime, the resultant values saved in the data model store is incorrect as seen in the snapshots.
Here is a comparison from version 1 to version 2 of the data store.
Version 1: Correct
to Version 2: Which is now storing the values incorrectly.
The expected output should have the Product price inserted into the productPrice field and not in the ProductPriceNeedsUpdating field which should actually only have boolean values.
Can anyone help me understand what I am doing wrong, or explain whats happening here?
UPDATE 1 - Here are my entity mappings:
Update 2 - 20/aug/2014 01:02 GMT
When I remove the attribute ProductPriceLastUpdated of type date from version 1, and remove the attribute ProductPriceNeedsUpdate of type boolean in version 2, leaving only the two attributes that both match in version 1 and 2, then everything works. Even though I can leave it here and move on, I cant ignore the users that are currently using version 1 of the database which does have that pointless ProductPriceLastUpdated attribute which I need the type converted to boolean and also have the name changed to ProductPriceNeedsUpdate. Thats when things start going weird, and the price values are shown in the ProductPriceNeedsUpdate field instead of the productPrice field.
I hope someone can address the original problem and tell me why it is that the entityMapping, or more so, property mapping is not being saved properly?
Update 3 - EntityMapping and properties:
Version 1
Version 2
Any ideas?
First, if you want to just use a lightweight migration (which you really should in this case) you can get rid of your custom migration policy. In this instance, it's not needed. And, as a matter of fact, you can get rid of your custom mapping model as well. All you need to do is in your Version 2 model, select the productPriceNeedsUpdating boolean flag, and in the Attribute Detail inspector on the right, set the default value to YES. This will achieve the goal you're try to get to with your custom migration policy.
However, if you really need to write this in code with your custom migration policy, I would still not use custom code. You can achieve this migration with only a mapping model. Simply select the ProductToProduct mapping, and in the value expression for productNeedsUpdating, enter YES, or 1.
EDIT
So, after a rather lengthy screen share, it was determined that the migration was using code from Marcus Zarra's Core Data book describing progressively migrating stores. When this was written, WAL mode was not the default mode with Core Data. When WAL mode is enabled, progressively migrating stores don't function well since there are two more files to deal with, the Write Ahead Log, and the Shared Memory file. When simply replacing the old store with a new one, without first removing those files, odd things happen, such as described in this post. The solution ultimately ended up being to disable WAL mode for the progressively migrating scenario so the extra files are not generated in the first place.
So, my team has been having multiple issues while upgrading our existing app from Grails 1.3.7 to 2.1.0. The latest headache occurs when trying to save a domain class object that has a composite key based on two other domain objects.
We are hanging Grails on a legacy database which we cannot readily change, so all of the domain classes have custom mappings to hook up with it. Below is a quick, slimmed down version of the domain classes in question.
Class Product {
Short prodKey
String name
static hasMany = [groupProduct: GroupProduct]
//Also includes mapping to legacy db and simple constraints
}
Class Group {
Short groupKey
String name
static hasMany =[ groupProduct: GroupProduct]
//This domain class has several other mappings and variables, but they are not relevant
}
Class GroupProduct {
Group group
Product product
Character indicator
static belongsTo = [Product,Group]
static mapping = {
id composite: ["group", "product"]
group lazy:false, column:"GROUP_KEY", joinTable:"GROUP"
product lazy:false, column:"PROD_KEY", joinTable:"PRODUCT"
version false
}
//Only constraint is indicator is Y or N
}
In the app a user is able to select multiple products for a group to turn on or off via a checkbox list. The parameters contain the groupKey and a list of all checked products. The controller gets an instance of the specified group and then a list of all Products. The products are matched against the list in the parameters, every time a match is found a GroupProduct object is made with the indicator set to 'Y', otherwise a GroupProduct object is made with the indicator set to 'N'.
Class GroupProductController{
//allowedMethods and other actions...
def update = {
def groupInstance = Group.get(params.GroupId)
def groupProducts= []
def products= Products.list()
products.each{
def indicator = ...//code to get value of check box for this Product. Returns either Y or N, works as expected
def groupProduct= new GroupProduct(group:groupInstance ,
product:it,
indicator: indicator)
groupProducts.add(groupProduct)
}
groupInstance.discard()
groupProducts.each{
it.save(failOnError: true, flush:true)//This line throws a DB2 SQL error. SQLCODE=-407
}
}
}
Resulting error is:
org.hibernate.util.JDBCExceptionReporter|DB2 SQL Error: SQLCODE=-407, SQLSTATE=23502, SQLERRMC= , DRIVER=3.50.152
org.codehaus.groovy.grails.orm.hibernate.events.PatchedDefaultFlushEventListener|Could not synchronize database state with session
org.hibernate.exception.ConstraintViolationException: could not update: [GroupProduct#component[group,product]{product=Product#1, group=Group#938926168}]
at com.controllers.ProductGroupController$_closure2_closure8.doCall(ProductGroupController.groovy:86)
at com.nationwide.nas.beam.controllers.ProductGroupController$_closure2.doCall(ProductGroupController.groovy:79)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: com.ibm.db2.jcc.b.lm: DB2 SQL Error: SQLCODE=-407, SQLSTATE=23502, SQLERRMC= , DRIVER=3.50.152
at com.ibm.db2.jcc.b.wc.a(wc.java:575)
at com.ibm.db2.jcc.b.wc.a(wc.java:57)
at com.ibm.db2.jcc.b.wc.a(wc.java:126)
at com.ibm.db2.jcc.b.tk.b(tk.java:1593)
at com.ibm.db2.jcc.b.tk.c(tk.java:1576)
at com.ibm.db2.jcc.t4.db.k(db.java:353)
at com.ibm.db2.jcc.t4.db.a(db.java:59)
at com.ibm.db2.jcc.t4.t.a(t.java:50)
at com.ibm.db2.jcc.t4.tb.b(tb.java:200)
at com.ibm.db2.jcc.b.uk.Gb(uk.java:2355)
at com.ibm.db2.jcc.b.uk.e(uk.java:3129)
at com.ibm.db2.jcc.b.uk.zb(uk.java:568)
at com.ibm.db2.jcc.b.uk.executeUpdate(uk.java:551)
at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
... 5 more
The error occurs when trying to save the GroupProduct objects. According to IBM the error code -407 is caused by AN UPDATE, INSERT, OR SET VALUE IS NULL, BUT THE OBJECT COLUMN column-name CANNOT CONTAIN NULL VALUES. However, none of the variables for the GroupProducts are actually null. The Group and Product instances are pulled straight from the database, which means they have already been validated and shouldn't have any constraint violations, and I can see that the indicator field is being set correctly.
There is also no problem when running this code under the original 1.3.7 version of the project. If anyone could shed some light on this I'd be very grateful. Thanks
After much debugging and hunting through code, I managed to find the issue. All of our domain classes extend an abstract base domain, which has a createdTimestamp and updatedTimestamp field. Before doing an insert we set both fields, and before doing an update we update the updatedTimestamp.
The issue was that when we were saving, the new object had a null createdTimestamp field, which was throwing the error. Added in code to check if the GroupProduct object we were making already existed, and if so set the new objects createdTimestamp field to the existing object's before saving. Now everything works as expected.
Its strange that Grails 1.3.7 did not have any issues with this code, though. Only thing I can figure is that it automatically associated the new objects with existing ones in the DB. Probably the strangest behavior change I've found during the upgrade process. Hope this helps anyone running into a similar issue.
To save the primary animal in a zoo I do this:
-(void)makePrimary:(Animal*)animal
Zoo *currentZoo = [Zoo findFirstByAttribute:#"zooId" withValue:self.currentZooId];
currentZoo.primaryAnimalId = animal.animalId;
[[NSManagedObjectContext defaultContext] saveToPersistentStoreAndWait];
DLog(#"primaryAnimalId: %d", currentZoo.primaryAnimalId.intValue); //logs "3"
}
After doing this, the primaryAnimalId for currentZoo logs as 3. I am using the Firefox SQLite Manager to verify this. ZPRIMARYANIMALID is 3 there as well.
Now, I navigate to a different section of the app, where I need to display the primary animal. So I do this:
-(Animal*)getPrimaryAnimalForCurrentZoo
{
Zoo *currentZoo = [Zoo findFirstByAttribute:#"zooId" withValue:self.currentZooId];
DLog(#"primaryAnimalId: %d", currentZoo.primaryAnimalId.intValue); //logs 2
Animal *primaryAnimal = [Animal MR_findFirstByAttribute:#"animalId" withValue:currentZoo.primaryAnimalId];
return primaryAnimal;
}
Much to my chagrin, I get "2" for the primaryAnimalId, and thus the wrong animal is returned. "2" is the previous value of the animal before I change it via makePrimary:. currentZooId is 0 in both methods.
What I don't understand is how I can get the wrong value back from Core Data when I can clearly see the correct value currently in the database (in SQLite Manager). How is this possible and how do I fix it?
Did you update self.currentZooId to reflect your changes?
Update:
So, you're asking for the first record in your data set here. The thing is,you're not specifying an order. That is, findFirst just asks the data store for all, and then just returns the first in the list (have a look at the code for more specifics). Since that list can be in any order, you're going to get whatever the data store gives you in at particular instance. What you want to do is add a sort parameter to findFirst to make sure you always get the item you expect. Look at the headers to find the correct firstFirst method with a sort parameter.
I'm having trouble migrating a store entity attribute from String to Integer 16. Here are the steps I take:
Add Model Version...
In the new model, change Entity attribute from String to Int 16.
Select the new model in File Inspector > Versioned Core Data Model > Current Model
Create a mapping model for the old and new models.
Run
Here is the error:
Unresolved error Error Domain=NSCocoaErrorDomain Code=134140 "The
operation couldn’t be completed. (Cocoa error 134140.)"
UserInfo=0xbd5cd20 {reason=Can't find or automatically infer mapping
model for migration, destinationModel=...
The mapping model is there in the compiled .app:
and in the Project:
Migration works for attributes like Integer 16 > Integer 32, or when changing attribute names.
I tried creating a simple Core Data Project and migration worked automatically (with and without mapping model) from String to Integer 16 and back.
The strangest part is I tried looking programatically for all mapping models in the bundle and none are found for the current source/destination models.
This happens because Core Data is unable to automatically migrate your attribute. This is because it can't guarantee that a string will always fit in an int (even though you know your data does).
So what you need to do is use a mapping model. Here's how to do it:
In Xcode, create a new mapping model (File > New > New File), select Mapping Model in the Core Data section
Select the source and target models in the wizard
This basically puts you in the same place as the lightweight migration, everything is done automatically, except you have the option to override some mapping. Specifically, that one that is giving you troubles.
Create a new mapping policy class (Extend NSEntityMigrationPolicy)
Implement createDestinationInstancesForSourceInstance:entityMapping:manager:error: which will give you the source instance so you can convert that string into an int and store it in the new store.
Your code should look something like this:
- (BOOL)createDestinationInstancesForSourceInstance:(NSManagedObject *)sInstance entityMapping:(NSEntityMapping *)mapping manager:(NSMigrationManager *)manager error:(NSError **)error
{
NSManagedObject *newObject = [NSEntityDescription insertNewObjectForEntityForName:[mapping destinationEntityName] inManagedObjectContext:[manager destinationContext]];
// Copy all the values from sInstance into newObject, making sure to apply the conversion for the string to int when appropriate. So you should have one of these for each attribute:
[newObject setValue:[sInstance valueForKey:#"xyz"] forKey:#"xyz"];
[manager associateSourceInstance:sInstance withDestinationInstance:newObject forEntityMapping:mapping];
}
Then all you have to do is set that policy in the mapping model. Select the mapping model file, pick the appropriate Entity mapping and set the CustomPolicy on the right panel.
Be sure to change the migration settings to remove automatic type inference wherever you init Core Data
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithBool:YES], NSMigratePersistentStoresAutomaticallyOption, nil];
That should be it...
For those, who broked thousands of spears on "Can't find mapping model for migration" error, this might help:
Make sure, you created mapping file in proper folder/group (before pressing Cmd+N - select .xcdatamodeld file in project navigator).
Clean the project.
Rebuild the project and run.
In my case, app automagically found the mapping model after clean/rebuild =\