I got a NSFetchedResultsController that I set up using a NSManagedObjectContext. I perform a fetch using this context.
I have as well a NSBatchUpdateRequest that I set up using the same NSManagedObjectContext. I execute the request using the same NSManagedObjectContext.
When I perform the request with the NSBatchUpdateRequest, I can see that all my data have been updated.
If I restart the app, any fetch using NSFetchedResultsController is working as well.
THe problem is when I'm not restarting the app and that I do both operations one after one, I got a NSMergeConflict (0x17427a900) for NSManagedObject (0x1740d8d40) with objectID '0xd000000001b40000... error when I call the method save from my context.
I know that the problem comes from concurrent change on the same data but I don't know what is the solution? One might be to go through the NSMergePolicy class, but I doubt that's a clean way to solve my problem.
What should I do? Have two different contexts ? (how?)
Well it seems I might have found how to do it, but if you see anything wrong, please let me know.
When you do a batch update, you have the possibility to get as a result, whether nothing, the number of rows that were updated or a list of object IDs that were updated. You have to choose the last one.
Once you perform executeRequest from the context, you need to get the list of object IDs, loop through all of them to get every NSManagedObject into Faults thanks to the method objectWithID of the context object. If you don't know what Faults object are in Core Data, here is the explanation.
With every NSManagedObject you get, you need to refresh the context using its method refreshObject.
Once you've done that, you need to perform again the performFetch of your fetchedResultsController to come back to where you were before the batch update.
Tell me if I'm wrong somewhere.
Here is the code:
let batchUpdate = NSBatchUpdateRequest(entityName: "myEntity")
batchUpdate.propertiesToUpdate = ["myPropertieToUpdate" : currency.amountToCompute]
batchUpdate.affectedStores = managedContext.persistentStoreCoordinator?.persistentStores
batchUpdate.resultType = .UpdatedObjectIDsResultType
var batchError: NSError?
let batchResult = managedContext.executeRequest(batchUpdate, error: &batchError) as NSBatchUpdateResult?
if let result = batchResult {
println("Records updated \((result.result as [NSManagedObjectID]).count)")
// Extract Object IDs
let objectIDs = result.result as [NSManagedObjectID]
for objectID in objectIDs {
// Turn Managed Objects into Faults
let nsManagedObject: NSManagedObject = managedContext.objectWithID(objectID)
if let managedObject = nsManagedObject as NSManagedObject? {
managedContext.refreshObject(managedObject, mergeChanges: false)
}
}
// Perform Fetch
var error: NSError? = nil
if !fetchedResultsController.performFetch(&error) {
println("error: + \(error?.localizedDescription), \(error!.userInfo)")
}
} else {
println("Could not update \(batchError), \(batchError!.userInfo)")
}
EDIT:
Here are two links for more explanations:
http://code.tutsplus.com/tutorials/ios-8-core-data-and-batch-updates--cms-22164
http://www.bignerdranch.com/blog/new-in-core-data-and-ios-8-batch-updating/
Related
Suppose I have;
var itemArray = [Item]()
and Item is a NSManagedObject. Item has two attributes "Title":String and "Done":Boolean. Here is the picture of my data model.
// Item+CoreDataClass.swift
// This file was automatically generated and should not be edited.
//
import Foundation
import CoreData
public class Item: NSManagedObject {
}
When I change the value of Done and call context.save, it is automatically reflected to Persistent Container. However, when I remove an element from array by saying,
itemArray.remove(at: someindex)
and call context.save. The item is not deleted from Persistent Container. Only if I called,
context.delete(itemArray[someindex])
then the item is truly deleted from store.
So why only removing from itemArray and save context is not sufficient although changing an attribute' value and save context is sufficient for successful CRUD operation on Core Data?
When you change an attribute on an Item object then Core Data (actually the NSManagedObjectContext) detects that since the Item belongs to the NSManagedObjectContext and the item is marked as dirty. But your array has no connection to the NSManagedObjectContext in any way so any changes you make it to remains undetected by the NSManagedObjectContext and that is why you need to tell it explicitly that you want to delete the item you removed from the array.
Another way to look at it is that anything you create/define in your Core Data model is known by NSManagedObjectContext but anything created outside in swift code is unknown. If you start working with to-many relationships between entities you will see that then adding or removing objects from the to-many collection will be handled directly by the NSManagedObjectContext in a way you expected for your array.
The array var itemArray = [Item]() has no direct relation with the underlying database. Therefore removing items from that array doesn't affect the Core Data database.
To create, save or delete NSManagedObject instances in a Core Data database you need to call the related functions of a valid NSManagedObjectContext.
Any operation on CoreData should be done through NSManagedObjectContext as it is the scratchpad to access or update any entity in database. So in your case while deleting the Item entity, you should do that through context only to get reflected on database.
var itemArray = [Item]()
let context = //get your context
let fetchRequest = NSFetchRequest<NSFetchRequestResult>(entityName: "Item")
let predicate = NSPredicate(format: " (primaryKey == %#) ", "your_primary_key")
fetchRequest.predicate = predicate
itemArray = try! context.fetch(fetchRequest)
for i in 0 ..< itemArray.count where i < itemArray.count-1 {
context.delete(itemArray[i])
}
do {
try context.save()
} catch {
// print the error
}
I've recently been learning about Core Data and specifically how to do inserts with a large number of objects. After learning how to do this and solving a memory leak problem that I met, I wrote the Q&A Memory leak with large Core Data batch insert in Swift.
After changing NSManagedObjectContext from a class property to a local variable and also saving inserts in batches rather than one at a time, it worked a lot better. The memory problem cleared up and the speed improved.
The code I posted in my answer was
let batchSize = 1000
// do some sort of loop for each batch of data to insert
while (thereAreStillMoreObjectsToAdd) {
// get the Managed Object Context
let managedObjectContext = (UIApplication.sharedApplication().delegate as! AppDelegate).managedObjectContext
managedObjectContext.undoManager = nil // if you don't need to undo anything
// get the next 1000 or so data items that you want to insert
let array = nextBatch(batchSize) // your own implementation
// insert everything in this batch
for item in array {
// parse the array item or do whatever you need to get the entity attributes for the new object you are going to insert
// ...
// insert the new object
let newObject = NSEntityDescription.insertNewObjectForEntityForName("MyEntity", inManagedObjectContext: managedObjectContext) as! MyManagedObject
newObject.attribute1 = item.whatever
newObject.attribute2 = item.whoever
newObject.attribute3 = item.whenever
}
// save the context
do {
try managedObjectContext.save()
} catch {
print(error)
}
}
This method seems to be working well for me. The reason I am asking a question here, though, is two people (who know a lot more about iOS than I do) made comments that I don't understand.
#Mundi said:
It seems in your code you are using the same managed object context,
not a new one.
#MartinR also said:
... the "usual" implementation is a lazy property which creates the
context once for the lifetime of the app. In that case you are reusing
the same context as Mundi said.
Now I don't understand. Are they saying I am using the same managed object context or I should use the same managed object context? If I am using the same one, how is it that I create a new one on each while loop? Or if I should be using just one global context, how do I do it without causing memory leaks?
Previously, I had declared the context in my View Controller, initialized it in viewDidLoad, passed it as a parameter to my utility class doing the inserts, and just used it for everything. After discovering the big memory leak is when I started just creating the context locally.
One of the other reasons I started creating the contexts locally is because the documentation said:
First, you should typically create a separate managed object context
for the import, and set its undo manager to nil. (Contexts are not
particularly expensive to create, so if you cache your persistent
store coordinator you can use different contexts for different working
sets or distinct operations.)
What is the standard way to use NSManagedObjectContext?
Now I don't understand. Are they saying I am using the same managed
object context or I should use the same managed object context? If I
am using the same one, how is it that I create a new one on each while
loop? Or if I should be using just one global context, how do I do it
without causing memory leaks?
Let's look at the first part of your code...
while (thereAreStillMoreObjectsToAdd) {
let managedObjectContext = (UIApplication.sharedApplication().delegate as! AppDelegate).managedObjectContext
managedObjectContext.undoManager = nil
Now, since it appears you are keeping your MOC in the App Delegate, it's likely that you are using the template-generated Core Data access code. Even if you are not, it is highly unlikely that your managedObjectContext access method is returning a new MOC each time it is called.
Your managedObjectContext variable is merely a reference to the MOC that is living in the App Delegate. Thus, each time through the loop, you are merely making a copy of the reference. The object being referenced is the exact same object each time through the loop.
Thus, I think they are saying that you are not using separate contexts, and I think they are right. Instead, you are using a new reference to the same context each time through the loop.
Now, your next set of questions have to do with performance. Your other post references some good content. Go back and look at it again.
What they are saying is that if you want to do a big import, you should create a separate context, specifically for the import (Objective C since I have not yet made time to learn Swift).
NSManagedObjectContext moc = [[NSManagedObjectContext alloc]
initWithConcurrencyType:NSPrivateQueueConcurrencyType];
You would then attach that MOC to the Persistent Store Coordinator. Using performBlock you would then, in a separate thread, import your objects.
The batching concept is correct. You should keep that. However, you should wrap each batch in an auto release pool. I know you can do it in swift... I'm just not sure if this is the exact syntax, but I think it's close...
autoreleasepool {
for item in array {
let newObject = NSEntityDescription.insertNewObjectForEntityForName ...
newObject.attribute1 = item.whatever
newObject.attribute2 = item.whoever
newObject.attribute3 = item.whenever
}
}
In pseudo-code, it would all look something like this...
moc = createNewMOCWithPrivateQueueConcurrencyAndAttachDirectlyToPSC()
moc.performBlock {
while(true) {
autoreleasepool {
objects = getNextBatchOfObjects()
if (!objects) { break }
foreach (obj : objects) {
insertObjectIntoMoc(obj, moc)
}
}
moc.save()
moc.reset()
}
}
If someone wants to turn that pseudo-code into swift, it's fine by me.
The autorelease pool ensures that any objects autoreleased as a result of creating your new objects are released at the end of each batch. Once the objects are released, the MOC should have the only reference to objects in the MOC, and once the save happens, the MOC should be empty.
The trick is to make sure that all object created as part of the batch (including those representing the imported data and the managed objects themselves) are all created inside the autorelease pool.
If you do other stuff, like fetching to check for duplicates, or have complex relationships, then it is possible that the MOC may not be entirely empty.
Thus, you may want to add the swift equivalent of [moc reset] after the save to ensure that the MOC is indeed empty.
This is a supplemental answer to #JodyHagins' answer. I am providing a Swift implementation of the pseudocode that was provided there.
let managedObjectContext = NSManagedObjectContext(concurrencyType: NSManagedObjectContextConcurrencyType.PrivateQueueConcurrencyType)
managedObjectContext.persistentStoreCoordinator = (UIApplication.sharedApplication().delegate as! AppDelegate).persistentStoreCoordinator // or wherever your coordinator is
managedObjectContext.performBlock { // runs asynchronously
while(true) { // loop through each batch of inserts
autoreleasepool {
let array: Array<MyManagedObject>? = getNextBatchOfObjects()
if array == nil { break }
for item in array! {
let newEntityObject = NSEntityDescription.insertNewObjectForEntityForName("MyEntity", inManagedObjectContext: managedObjectContext) as! MyManagedObject
newObject.attribute1 = item.whatever
newObject.attribute2 = item.whoever
newObject.attribute3 = item.whenever
}
}
// only save once per batch insert
do {
try managedObjectContext.save()
} catch {
print(error)
}
managedObjectContext.reset()
}
}
These are some more resources that helped me to further understand how the Core Data stack works:
Core Data Stack in Swift – Demystified
My Core Data Stack
I am inserting tens of thousands of objects into my Core Data entity. I have a single NSManagedObjectContext and I am calling save() on the managed object context every time I add an object. It works but while it is running, the memory keeps increasing from about 27M to 400M. And it stays at 400M even after the import is finished.
There are a number of SO questions about batch insert and everyone says to read Efficiently Importing Data, but it's in Objective-C and I am having trouble finding real examples in Swift that solve this problem.
There are a few things you should change:
Create a separate NSPrivateQueueConcurrencyType managed object context and do your inserts asynchronously in it.
Don't save after inserting every single entity object. Insert your objects in batches and then save each batch. A batch size might be something like 1000 objects.
Use autoreleasepool and reset to empty the objects in memory after each batch insert and save.
Here is how this might work:
let managedObjectContext = NSManagedObjectContext(concurrencyType: NSManagedObjectContextConcurrencyType.PrivateQueueConcurrencyType)
managedObjectContext.persistentStoreCoordinator = (UIApplication.sharedApplication().delegate as! AppDelegate).persistentStoreCoordinator // or wherever your coordinator is
managedObjectContext.performBlock { // runs asynchronously
while(true) { // loop through each batch of inserts
autoreleasepool {
let array: Array<MyManagedObject>? = getNextBatchOfObjects()
if array == nil { break }
for item in array! {
let newObject = NSEntityDescription.insertNewObjectForEntityForName("MyEntity", inManagedObjectContext: managedObjectContext) as! MyManagedObject
newObject.attribute1 = item.whatever
newObject.attribute2 = item.whoever
newObject.attribute3 = item.whenever
}
}
// only save once per batch insert
do {
try managedObjectContext.save()
} catch {
print(error)
}
managedObjectContext.reset()
}
}
Applying these principles kept my memory usage low and also made the mass insert faster.
Further reading
Efficiently Importing Data (old Apple docs link is broken. If you can find it, please help me add it.)
Core Data Performance
Core Data (General Assembly post)
Update
The above answer is completely rewritten. Thanks to #Mundi and #MartinR in the comments for pointing out a mistake in my original answer. And thanks to #JodyHagins in this answer for helping me understand and solve the problem.
I've tried Googling and searching SO, thinking it would be quite easy to find but I'm surprised there were little to no examples of my problem in Swift.
I have several threads and from what I've read, it's best to have a separate managed object context for each thread. If I want to access an object from another context, I should pass around the objectID.
My question is, how should I pass the objectID to the context in a new thread?
Below is how the construction of my situation is:
func doSomething(context: NSManagedObjectContext) {
let person = Persons(context: context) // NSManagedObject
dispatch_async(dispatch_get_global_queue(Int(QOS_CLASS_USER_INITIATED.value), 0)) {
let background = (UIApplication.sharedApplication().delegate as! AppDelegate).cdh.backgroundContext!
Birthdays(context: background, person) // NSManagedObject
saveContext(background)
}
}
Birthdays has a one to one relationship with Persons. If I execute this code it will give me an error saying:
Illegal attempt to establish a relationship 'person1' between objects in different contexts
Obviously because they are in separated contexts. So I tried to get the objectID of person1 by let personsObjectID = person1.objectID, but I don't know how I should be using it to pass it to the other thread. Any help is appreciated.
Seems like the only thing I needed to do was to add:
let person = background.objectWithID(orders.objectID as! Persons
If I've understood it correctly, objectWithID fetches the object from the store and puts it in the context, returning the managed object. It would be nice if someone could verify this if it's true.
func doSomething(context: NSManagedObjectContext) {
let person = Persons(context: context) // NSManagedObject
dispatch_async(dispatch_get_global_queue(Int(QOS_CLASS_USER_INITIATED.value), 0)) {
let background = (UIApplication.sharedApplication().delegate as! AppDelegate).cdh.backgroundContext!
let person = background.objectWithID(orders.objectID as! Persons
Birthdays(context: background, person) // NSManagedObject
saveContext(background)
}
}
Hendrik, you are correct, you should use objectWithID to instantiate the NSManagedObject on the required context.
Before getting into my issue, please have a look at this image.
Here is the actual data model:
I retrieve a set of Records from a web API, create objects out of them, save them in core data and display them in the Today view. By default these records are returned for the current date.
The user can tap on Past button to go to a separate view where he can choose a past or future date from a date picker view and view Records for that selected date. This means I have to call the API again passing the selected date, retrieve the data and save that data in core data and display them. When the user leaves this view, this data should be discarded.
This is the important part. Even though I get a new set of data, the old original data for the current date in the Today view must not go away. So if/when the user returns to the Today view, that data should be readily available as he left it without the app having to call the API and get the data for the current date again.
I thought of creating a separate NSManagedObjectContext to hold these temporary data.
I have a separate class called DatabaseManager to handle core data related tasks. This class initializes with an instance of `NSManagedObjectContext. It creates the managed object classes in the given context.
import CoreData
import Foundation
import MagicalRecord
import SwiftyJSON
public class DatabaseManager {
private let context: NSManagedObjectContext!
init(context: NSManagedObjectContext) {
self.context = context
}
public func insertRecords(data: AnyObject, success: () -> Void, failure: (error: NSError?) -> Void) {
let json = JSON(data)
if let records = json.array {
for recordObj in records {
let record = Record.MR_createInContext(context) as Record
record.id = recordObj["Id"].int
record.name = recordObj["Name"].string!
record.date = NSDate(string: recordObj["Date"].string!)
}
context.MR_saveToPersistentStoreAndWait()
success()
}
}
}
So in the Today view I pass NSManagedObjectContext.MR_defaultContext() to insertRecords() method. I also have a method to fetch Records from the given context.
func fetchRecords(context: NSManagedObjectContext) -> [Record]? {
return Record.MR_findAllSortedBy("name", ascending: true, inContext: context) as? [Record]
}
The data is retrieved from the API, saved in core data and gets displayed successfully. All good so far.
In the Past View, I have to do basically the same thing. But since I don't want the original data to change. I tried to do this a few ways which MagicalRecord provides.
Attempt #1 - NSManagedObjectContext.MR_context()
I create a new context with NSManagedObjectContext.MR_context(). I change the date in Past view, the data for that selected date gets retrieved and saved in the database successfully. But here's the issue. When I fetch the objects from core data, I get that old data as well. For example, each day has only 10 records. In Today view I display 10 records. When the fetch objects in the Past view, I get 20 objects! I assume it's the old 10 objects plus the new ones. Also when I try to display them in the tableview, it crashes with a EXC_BAD_ACCESS error in the cellForRowAtIndexPath method.
override func tableView(tableView: UITableView, cellForRowAtIndexPath indexPath: NSIndexPath) -> UITableViewCell {
let cell = tableView.dequeueReusableCellWithIdentifier("Cell", forIndexPath: indexPath) as UITableViewCell
let record = records[indexPath.row]
cell.textLabel?.text = record.name // EXC_BAD_ACCESS
cell.detailTextLabel?.text = record.date.toString()
return cell
}
Attempt #2 - NSManagedObjectContext.MR_newMainQueueContext()
The app crashes when I change the date with the following error.
'+entityForName: nil is not a legal NSPersistentStoreCoordinator for searching for entity name 'Record''
Attempt #3 - NSManagedObjectContext.MR_contextWithParent(NSManagedObjectContext.MR_defaultContext())
Same result as Attempt #1.
Attempt #4 - From Hal's Answer I learned that even though I create two MOCs, they both refer to the same NSPersistentStore. So I created another new store to hold the temporary data in my AppDelegate.
MagicalRecord.setupCoreDataStackWithStoreNamed("Records")
MagicalRecord.setupCoreDataStackWithStoreNamed("Records-Temp")
Then when I change the date to get the new data, I set that temporary store as the default store like this.
func getDate(date: NSDate) {
let url = NSPersistentStore.MR_urlForStoreName("Records-Temp")
let store = NSPersistentStore(persistentStoreCoordinator: NSPersistentStoreCoordinator.MR_defaultStoreCoordinator(), configurationName: nil, URL: url, options: nil)
NSPersistentStore.MR_setDefaultPersistentStore(store)
let context = NSManagedObjectContext.MR_defaultContext()
viewModel.populateDatabase(date, context: context)
}
Note that I'm using the default context. I get the data but it's the same result as Attempt 1 and 3. I get 20 records. They include data from both the old date and the new date. If I use NSManagedObjectContext.MR_context(), it would simply crash like in Attempt 1.
I also discovered something else. After creating the stores in App Delegate, I printed out the default store name println(MagicalRecord.defaultStoreName()) in the Today's view. Strangely it didn't print the name I gave the store which is Records. Instead it showed Reports.sqlite. Reports being the project's name. Weird.
Why do I get the old data as well? Am I doing something with when initializing a new context?
Sorry if my question is a little confusing so I uploaded a demo project to my Dropbox. Hopefully that will help.
Any help is appreciated.
Thank you.
Thread Safety
First of all I want to mention the Golden Rule of Core Data. NSManagedObject's are not thread safe, hence, "Thou shalt not cross the streams" (WWDC). What this means is that you should always access a Managed Object in its context and never pass it outside of its context. This is why your importer class worries me, you are inserting a bunch of objects into a context without guaranteeing that you are running the insert inside the Context.
One simple code change would fix this:
public func insertRecords(data: AnyObject, success: () -> Void, failure: (error: NSError?) -> Void) {
let json = JSON(data)
context.performBlock { () -> Void in
//now we are thread safe :)
if let records = json.array {
for recordObj in records {
let record = Record.MR_createInContext(context) as Record
record.id = recordObj["Id"].int
record.name = recordObj["Name"].string!
record.date = NSDate(string: recordObj["Date"].string!)
}
context.MR_saveToPersistentStoreAndWait()
success()
}
}
}
The only time you don't need to worry about this is when you are using the Main Queue Context and accessing objects on the main thread, like in tableview's etc.
Don't forget that MagicalRecord also has convenient save utilities that create context's ripe for saving :
MagicalRecord.saveWithBlock { (context) -> Void in
//save me baby
}
Displaying Old Records
Now to your problem, the following paragraph in your post concerns me:
The user can tap on Past button to go to a separate view where he can
choose a past or future date from a date picker view and view Records
for that selected date. This means I have to call the API again
passing the selected date, retrieve the data and save that data in
core data and display them. When the user leaves this view, this data
should be discarded.
I don't like the idea that you are discarding the information the user has requested once they leave that view. As a user I would expect to be able to navigate back to the old list and see the results I just queried without another unecessary network request. It might make more sense to maybe have a deletion utility that prunes your old objects on startup rather than while the user is accessing them.
Anyways, I cannot illustrate how important it is that you familiarize yourself with NSFetchedResultsController
This class is intended to efficiently manage the results returned from
a Core Data fetch request.
You configure an instance of this class using a fetch request that
specifies the entity, optionally a filter predicate, and an array
containing at least one sort ordering. When you execute the fetch, the
instance efficiently collects information about the results without
the need to bring all the result objects into memory at the same time.
As you access the results, objects are automatically faulted into
memory in batches to match likely access patterns, and objects from
previous accessed disposed of. This behavior further serves to keep
memory requirements low, so even if you traverse a collection
containing tens of thousands of objects, you should never have more
than tens of them in memory at the same time.
Taken from Apple
It literally does everything for you and should be your go-to for any list that shows objects from Core Data.
When I fetch the objects from core data, I get that old data as well
Thats to be expected, you haven't specified anywhere that your fetch should include the reports in a certain date range. Here's a sample fetch:
let fetch = Record.MR_createFetchRequest()
let maxDateForThisController = NSDate()//get your date
fetch.predicate = NSPredicate(format: "date < %#", argumentArray: [maxDateForThisController])
fetch.fetchBatchSize = 10// or an arbitrary number
let dateSortDescriptor = NSSortDescriptor(key: "date", ascending: false)
let nameSortDescriptor = NSSortDescriptor(key: "name", ascending: true)
fetch.sortDescriptors = [dateSortDescriptor,nameSortDescriptor]//the order in which they are placed in the array matters
let controller = NSFetchedResultsController(fetchRequest: fetch,
managedObjectContext: NSManagedObjectContext.MR_defaultContext(),
sectionNameKeyPath: nil, cacheName: nil)
Importing Discardable Records
Finally, you say that you want to see old reports and use a separate context that won't save to the persistent store. Thats also simple, your importer takes a context so all you would need to do is make sure that your importer can support imports without saving to the persistent store. That way you can discard the context and the objects will go with it. So your method signature could look like this:
public func insertRecords(data: AnyObject, canSaveToPersistentStore: Bool = true,success: () -> Void, failure: (error: NSError?) -> Void) {
/**
Import some stuff
*/
if canSaveToPersistentStore {
context.MR_saveToPersistentStoreWithCompletion({ (complete, error) -> Void in
if complete {
success()
} else {
error
}
})
} else {
success()
}
}
The old data that was in your persistent store, and addressed with the original MOC, is still there, and will be retrieved when the second MOC does a fetch. They're both looking at the same persistent store. It's just that the second MOC also has new data fetched from your API.
A synchronous network operation saving to Core Data will hang your app, and (for a large enough set of records) cause the system to kill your app, appearing to the user as a crash. Your client is wrong on that point, and needs to be educated.
Break apart your logic for fetching, saving, and viewing. Your view that shows a particular date's records should just do that--which it can do, if it accepts a date and uses a predicate.
Your 'cellForRowAtIndexPath' crash smells like a problem with a missing or misspelled identifier. What happens if you hard code a string instead of using 'record.name'?