I'm using a batch insert request like this:
let insertRequest = NSBatchInsertRequest(entity: EntryObject.entity(), managedObjectHandler: { object in
guard index < total else { return true }
if let entryObject = object as? EntryObject {
let entry = entries[index]
entryObject.setValues(entry: entry, shouldAttemptFetchExistingTags: true)
}
index += 1
return false
})
try context.execute(insertRequest)
Each EntryObject needs to be related to several other entities. However, NSBatchInsertRequest doesn't allow creating relationships. Each object is saved directly to the store without loading it into a context.
So my plan is to fetch each of these objects afterwards and apply the necessary relationships.
I'm using a batch insert request because EntryObject can contain several movies that are expensive as crap to create, and I'm continually getting super hard to trace crashes.
However when I go back to fetch these objects.. they don't exist.
I tried storing the objectIDs in a temporary array. That threw the same error.
Then I tried manually merging the changes into the context I performed the batch insert on, in addition to the viewContext.
insertRequest.resultType = NSBatchInsertRequestResultType.objectIDs
let result = try context.execute(insertRequest) as! NSBatchInsertResult
if let objectIDs = result.result as? [NSManagedObjectID], !objectIDs.isEmpty {
let save = [NSInsertedObjectsKey: objectIDs]
NSManagedObjectContext.mergeChanges(fromRemoteContextSave: save, into: [PersistenceController.viewContext, context])
}
And when that didn't work, I watch the 2018 best practices for CoreData and got this going..
try context.performAndWait {
let request = NSPersistentHistoryChangeRequest ()
request.resultType = .transactionsAndChanges
let result = try context.execute (request) as! NSPersistentHistoryResult
let transactions = result.result as! Array<NSPersistentHistoryTransaction>
for transaction in transactions {
context.mergeChanges(fromContextDidSave: transaction.objectIDNotification())
}
}
try context.save()
It's supposed to merge all the changes to the store into the main context.
However when I go back to fetch these objects, on either context.. they don't exist. Not until I kill the app and start over.
I've been trying to figure out how to completely reinit a store.. but have had no success.
How do I fetch these objects right after I create them using an NSBatchInsertRequest?
Related
Currently, I perform multiple update operations via the following code.
func updateOrders(_ updates : [(objectID: NSManagedObjectID, order: Int64)]) {
if updates.isEmpty {
return
}
let coreDataStack = CoreDataStack.INSTANCE
let backgroundContext = coreDataStack.backgroundContext
backgroundContext.perform {
for update in updates {
let objectID = update.objectID
let order = update.order
let nsPlainNote = try! backgroundContext.existingObject(with: objectID) as! NSPlainNote
nsPlainNote.order = order
}
RepositoryUtils.saveContextIfPossible(backgroundContext)
}
}
Since I would like to
Make the update operations run faster
Avoid delegate of NSFetchedResultController from being notified
I would like to utilise NSBatchUpdateRequest for performing such update operation.
However, I don't find a way, how I can apply array of NSManagedObjectID and array of Int64 value, to NSBatchUpdateRequest.
Given an array of NSManagedObjectID and Int64, is it possible to use NSBatchUpdateRequest to perform updated on CoreData?
You must use NSPredicate to set object id
func updateOrders(_ updates : [(objectID: NSManagedObjectID, order: Int64)]) {
updates.forEach {
let request = NSBatchUpdateRequest(entityName: "NSPlainNote")
request.propertiesToUpdate = ["order": $0.order]
request.predicate = NSPredicate(format: "objectID == %#", $0.objectID)
let result = try? context.execute(request)
}
}
NSBatchUpdateRequest is not suitable for your task since using it makes sense for large amount of records with a common attribute's value so that you can filter all by your criteria and update all fields with your values at once.
The fact is that the NSBatchDeleteRequest is an NSPersistentStoreRequest which operates at the SQL level in the persistent store itself and it doesn't update your in-memory objects after execution thats why it works so fast and Core Data translates your native requests to a SQL ones where you can not use dynamically code to get and insert needed data from dictionary etc. but you can update the current value of a filed e.g.:
let batchRequest = NSBatchUpdateRequest(entityName: "Note")
batchRequest.predicate = predicate
// Increase `order` value
batchRequest.propertiesToUpdate = ["order" : NSExpression(format: "order + 1")]
do {
try context.execute(batchRequest)
}
catch {
fatalError(error.localizedDescription)
}
I have two entities, User and Post. A User contains many Post, and each Post has only one User.
I am trying to batch insert more than 1000 posts.
private func newBatchInsertRequest(with posts: [PostData]) -> NSBatchInsertRequest {
var index = 0
let total = posts.count
let batchInsert = NSBatchInsertRequest(entity: Post.entity()) { (managedObject: NSManagedObject) -> Bool in
guard index < total else { return true }
if let post = managedObject as? Post {
let data = posts[index]
post.createdData = data.createdDate
post.identifier = data.identifier
post.text = data.text
}
index += 1
return false
}
PersistenceController.shared.container.performBackgroundTask { context in
try? context.execute(batchInsert)
try? context.save()
}
}
It inserts all the posts that I want to insert. However, I can not configure how to set their User.
I tried to use the following code, but it did not work.
let updateRequest = NSBatchUpdateRequest(entity: Post.entity())
updateRequest.resultType = .updatedObjectIDsResultType
updateRequest.propertiesToUpdate = ["user": user]
try? context.execute(updateRequest)
I get the following error.
Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'Invalid relationship ((<NSRelationshipDescription: 0x2801045a0>), name item, isOptional 1, isTransient 0, entity Post, renamingIdentifier item
I can set their user one by one, but it is inefficient due to long processing time.
How to update the user property of Posts in a more efficient way?
Update
All of these posts belongs to one User which does not exist yet. I need to create it before or after executing the NSBatchInsertRequest.
User has three properties
1. createdDate: Date
2. identifier: UUID
3. name: String
My goal is to insert Post that belongs to one Use either using NSBatchInsertRequest or in a private context so that it does not block the main thread.
Here's what I would try:
Create the User before your batch operation. Yep, it's going to be on a different context — just make sure you grab its NSManagedObjectID (it's a property on NSManagedObject) and save it on a private property.
Inside your PersistenceController.shared.container.performBackgroundTask you have a reference to your private context. Use the managed object ID from before to fetch/register the same User in your private context:
let user = context.object(with: userManagedObjectID)
Then pass this user into the method that creates your batch insert request to set the user property on your Post objects.
I believe if you set up your inverse relationship up in the data model editor you don't need to populate User.posts. That should happen automatically for you.
I solved this problem in different way.
In my entity which needs to have relationship added placeholder attributed tmpID.
When creating batch inserts setting tmpID to particular string. As example parent objectID parent.objectID.
Executing batch insert.
Then fetching all object from CoreData with tmpID.
Then going throw everyone and setting:
Relationship
tmpID to nil, the do not waist a memory.
Saving managed object context.
To merge I am using function:
/// Executes the given `NSBatchDeleteRequest` and directly merges the changes to bring the given managed object context up to date.
///
/// - Parameter batchDeleteRequest: The `NSBatchDeleteRequest` to execute.
/// - Throws: An error if anything went wrong executing the batch deletion.
public func executeAndMergeChanges(using batchDeleteRequest: NSBatchDeleteRequest) throws {
batchDeleteRequest.resultType = .resultTypeObjectIDs
let result = try execute(batchDeleteRequest) as? NSBatchDeleteResult
let changes: [AnyHashable: Any] = [NSDeletedObjectsKey: result?.result as? [NSManagedObjectID] ?? []]
NSManagedObjectContext.mergeChanges(fromRemoteContextSave: changes, into: [self])
}
// Use case:
do {
try container.viewContext.executeAndMergeChanges(using: deleteReqest)
try container.viewContext.save()
} catch {
Logger.app.e("Can not destroy/save clean table: \(name) -> \(error)")
}
I've used core data to store 10000-20000 records. if i try to save and fetch 10000 records memory and cpu consumption was huge due to that app is getting crash in iphone 6 plus and earlier devices.
Here is the saving methods:----
//InboxCoredata Saving Method i..(calling saving method ii)
func InboxSaveInCoreDataWith(array: [[String: AnyObject]])
{
_ = array.map{self.InboxCreateCollectionEntityFrom(dictionary: $0)}
do
{
try CoreDataStack.sharedInstance.persistentContainer.viewContext.save()
print("Inbox Data saved Sucessfully in Coredata ")
}
catch let error
{
print(error)
}
}
// Inbox Coredata saving method ii
func InboxCreateCollectionEntityFrom(dictionary: [String: AnyObject]) -> NSManagedObject?
{
let context = CoreDataStack.sharedInstance.persistentContainer.viewContext
if let inboxEntity = NSEntityDescription.insertNewObject(forEntityName: "InboxData", into: context) as? InboxData {
inboxEntity.fileType = dictionary["FileType"] as? String
inboxEntity.sender = dictionary["Sender"] as? String
inboxEntity.mailPath = dictionary["MailPath"] as? String
inboxEntity.fullMail = dictionary["FullMail"] as? NSObject
inboxEntity.attachmentName = dictionary["AttachmentName"] as? String
inboxEntity.size = dictionary["Size"] as! Int32
inboxEntity.date = dictionary["Date"] as? NSDate
inboxEntity.dateForSearch = dictionary["DateForSearch"] as? String
inboxEntity.inboxMail = dictionary["InboxMail"] as? String
return inboxEntity
}
return nil
}
And Here is the method for fetching:----
strongSelf.inboxDataFromCoreData(fetchLimit: 0) // Methods calling in viewdidload
//MARK: - Fetching inboxData from Coredata
func inboxDataFromCoreData(fetchLimit :Int)
{
var inboxCoredataFetch = [[String : AnyObject]]()
let context = CoreDataStack.sharedInstance.persistentContainer.viewContext
let fetchRequest = NSFetchRequest<NSFetchRequestResult>(entityName: String(describing: InboxData.self))
do {
fetchRequest.sortDescriptors = [NSSortDescriptor(key: "date", ascending: false)]
fetchRequest.fetchLimit = fetchLimit
let results = try context.fetch(fetchRequest) as! [InboxData]
print("inbox_coredata:\(results .count)")
for data in results
{
let sender = data.sender
let mailPath = data.mailPath
let fileType = data.fileType
let fullMail = data.fullMail
let attachmentName = data.attachmentName
let size = data.size
let date = data.date
let dateForsearch = data.dateForSearch
let inboxMail = data.inboxMail
inboxCoredataFetch.append(["Sender" : sender as AnyObject,"MailPath": mailPath! as AnyObject, "FileType":fileType as AnyObject, "FullMail":fullMail as AnyObject, "AttachmentName": attachmentName as AnyObject, "Size":size as AnyObject,"Date": date as AnyObject,"DateForSearch" :dateForsearch as AnyObject,"InboxMail":inboxMail as AnyObject])
}
}
catch let err as NSError {
print(err.debugDescription)
}
var sortingdata = inboxCoredataFetch as Array
let mailBoxSortDescriptor = NSSortDescriptor(key: "Date", ascending:false, selector: nil)
let dateDescriptor = NSSortDescriptor(key: "AttachmentName", ascending: true, selector: #selector(NSString.caseInsensitiveCompare))
sortingdata = ((sortingdata as NSArray).sortedArray(using: [ mailBoxSortDescriptor,dateDescriptor]) ) as! [[String : AnyObject]]
inboxTotalMailData = sortingdata
if appDataLoadingFirst == true
{
appDataLoadingFirst = false
displayTotalData = sortingdata
DispatchQueue.main.async {
self.hideActivityIndicator()
self.collectionView.reloadData()
}
}
}
Core data structure is like this:::---
i've too much confusion questions on these.
Core data is not good for storing 20000 records??
Do i need to refreshObject(object,mergeChnage:bool) for huge data
everytime i've to refresh object as like that.
Instruments indicates memory leak in fetching results at let results =
try context.fetch(fetchrequest.....line why??
Do i need to save and fetch data in batches will that increases app
performence and memory reduce??
Why CPU indicates 100% sometimes ??
if i display 10000 records in collection view(data loading from arrray)
causes any issue?? if yes what kind of issue??
Need your valuable suggesstion and help to make me perfect????
You're using a lot of memory and CPU time because:
When you're creating new InboxData entries, you go create one for every single entry in the array before you save changes. Your code means that all of those must be in memory at the same time. If you have thousands of entries, that's a lot of memory.
When you fetch entries, you fetch every single entry every time. Again, your code means that you must have all of them in memory at the same time. And again, lots of entries mean lots of memory. One of the main reasons to use Core Data is that it's almost always possible to load only a subset of your data into memory, instead of everything. You're loading everything.
You copy a bunch of attributes from managed objects into dictionaries-- so now you have two copies of the data in memory instead of one.
Your code that fetches data sorts the data in memory to create sortingdata. You already have a ton of objects loaded into memory; now you're doing a ton of work to sort them all. This will be at least part of the reason you peg the CPU at 100%.
After sorting the data you assign the result to inboxTotalMailData and to displayTotalData, which are defined somewhere outside this function. This means that all of the data in sortingdata remains in memory even after this function finishes.
Some things that would help:
When saving data, save at regular intervals-- every 50 entries, or every 100, or whatever number gives good results. Do not attempt to create many thousands of objects and keep them all in memory.
Have the fetch request sort the data instead of fetching and then sorting. Fetch requests can have sort descriptors, and the sorting is done by SQLite. This will be far more efficient in both CPU time and memory use.
Try to avoid fetching everything at once, because with thousands of records that means a lot of memory, no matter what else you do.
Don't copy the data unless you have some strongly compelling reason to do so. You can use managed objects directly, it's almost never appropriate to duplicate the data like this.
Since you appear to be using a collection view, consider using NSFetchedResultsController. It's designed to work with table and collection views and will help keep memory and CPU use down.
I'm fairly new to CoreData, and I'm trying to make a game. I have a couple of questions I was hoping you guys could help me out with some guidance:
- does GameKit already have some sort of CoreData integrated in it? I am not sure if I am overthinking this CoreData stuff if there's already something that replaces it in GameKit.
. . .
Anyways, assuming the answer to the above question is "no. GameKit has nothing to save your game". I will proceed with my current "Save game" code which is the following:
func saveCurrentMatch()
{
/* CORE DATA STUFF:
FIRST NEED TO VERIFY IF THIS GAME HAS BEEN PREVIOUSLY SAVED, IF SO THEN UPDATE, ELSE JUST SAVE
Entity: MatchData
Attributes: scoreArray (String), playerArray (String), myScore (Int), matchID (Int), isWaiting (Bool), isRealTime (Bool), gameLog (String)
*/
let context = myAppDelegate.persistentContainer.viewContext
let request = NSFetchRequest<NSFetchRequestResult>(entityName: "MatchData")
request.returnsObjectsAsFaults = false
do
{
let gamesInProgress = try context.fetch(request)
print (gamesInProgress.count)
if gamesInProgress.count > 0 //HERE CHANGE THIS TO LOOK FOR THE MATCH ID OF THIS GAME!!
{
gameExistsinCD = true
}
else
{
gameExistsinCD = false
}
}
catch
{
print ("Error Reading Data: \(error.localizedDescription)")
}
if gameExistsinCD
{
//CODE TO UPDATE MATCH INSTEAD OF SAVING NEW ONE
}
else
{
// CODE TO SAVE A NEW MATCH FOR THE FIRST TIME
let matchData = NSEntityDescription.insertNewObject(forEntityName: "MatchData", into: context)
matchData.setValue(isRealTime, forKey: "isRealTime")
matchData.setValue(currentScore?[0], forKey: "myScore")
matchData.setValue(currentScore?.map{String($0)}.joined(separator: "\t"), forKey: "scoreArray") // IS THIS CODE CORRECT? I'M TRYING TO SAVE AN ARRAY OF INTS INTO A SINGLE STRING
matchData.setValue(currentPlayers?.joined(separator: "\t"), forKey: "playerArray")
matchData.setValue(true, forKey: "isWaiting") //will change later to update accordingly.
matchData.setValue(matchID, forKey: "matchID")
matchData.setValue(gameLog, forKey: "gameLog")
do
{
try context.save()
print ("CoreData: Game Saved!")
}
catch
{
print ("Error Saving Data: \(error.localizedDescription)")
}
}
}
My main concern is on the fetch request, how do I check all the core-data if this match has already been saved? and if so, whats the code for updating an Entity instead of inserting a new one?
Any guidance is appreciated, thanks!
Don't let Core Data scare you. It can be a fine way to save local data and despite some comments, it is not slow when done right. In fact, Core Data can be quite fast.
You can simplify your code a lot by using your Object class in a more normal fashion instead of using setValue calls. Your create code can be changed to this:
// CODE TO SAVE A NEW MATCH FOR THE FIRST TIME
if let matchData = NSEntityDescription.insertNewObject(forEntityName: "MatchData", into: context) as? MatchData {
matchData.isRealTime = isRealTime
matchData.myScore = currentScore?[0]
matchData.scoreArray = currentScore?.map{String($0)}.joined(separator: "\t") // IS THIS CODE CORRECT? I'M TRYING TO SAVE AN ARRAY OF INTS INTO A SINGLE STRING
// You can certainly save it this way and code it in and out. A better alternative is to have a child relationship to another managed object class that has the scores.
matchData.playerArray = currentPlayers?.joined(separator: "\t")
matchData.isWaiting = true
matchData.matchID = matchID
matchData.gameLog = gameLog
}
This is a much more readable and normal way to set your object properties. Any time you change a property on a core data managed object then it will get saved the next time you save the context.
As far as finding a current record that matches the ID, I like to add classes like that to my Managed Object class itself:
class func findByID(_ matchID: String) -> MatchData? {
let myAppDelegate = UIApplication.shared.delegate as! AppDelegate
let context = myAppDelegate.persistentContainer.viewContext
let request = NSFetchRequest<NSFetchRequestResult>(entityName: "MatchData")
let idPredicate = NSPredicate(format: "matchID = \(matchID)", argumentArray: nil)
request.predicate = idPredicate
var result: [AnyObject]?
var matchData: MatchData? = nil
do {
result = try context.fetch(request)
} catch let error as NSError {
NSLog("Error getting match: \(error)")
result = nil
}
if result != nil {
for resultItem : AnyObject in result! {
matchData = resultItem as? MatchData
}
}
return matchData
}
Then any place you need the match data by ID you can call the class function:
if let matchData = MatchData.findByID("SomeMatchID") {
// Match exists
}
Core data is basically wrapper around SQL database. It is very efficient when you are working with high volume of data that need to be stored. So please consider either you had such requirements, otherwise perhaps it can be wise to store data in user defaults, or settings.
If it is, there is few things you need to know.
It is very useful to create you own model classes. Open core data model file, open "Editor/Create NSManagedObject subclass". It will allow you to refer direct properties, instead of KVC(setValue:forKey:).
Alway mind what thread you are working in. It is unsafe to work with objects, created in other threads.
Your gamesInProgress contains array of objects you fetched from your database.
So basically instead of
if gameExistsinCD
{
//CODE TO UPDATE MATCH INSTEAD OF SAVING NEW ONE
}
else
{
// CODE TO SAVE A NEW MATCH FOR THE FIRST TIME
let matchData = NSEntityDescription.insertNewObject(forEntityName: "MatchData", into: context)
matchData.setValue(isRealTime, forKey: "isRealTime")
<...>
you can do
let matchData = (gamesInProgress.first ??
NSEntityDescription.insertNewObject(forEntityName: "MatchData", into: context)) as! <YouEntityClass>
matchData.isRealTime = isRealTime
<...>
PS: https://www.raywenderlich.com/173972/getting-started-with-core-data-tutorial-2
I am having an issue with core data's predicate method, in that it seems to be somewhat inconsistent in the results it is returning. I have a "Users" entity that is populated from a web API, rather than deleting all the entries then downloading and adding the users, I prefer to set a flag for all users "isModified" to false, then download the users over the API and update or add users depending on whether they are in the store or not, setting the "isModified" flag when they are added/updated. Finally the code deletes any users that did not have their "isModified" flag set. I do it this way because in the event that the web API is not available I don't lose all my users and the app can continue to work.
The problem I am having is that the method that deletes users that haven't had their "isModified" flag set is deleting users that HAVE been updated!
Here's the method:
func deleteUsersNotUpdated() -> Bool {
// default result
var result = false
// setup the fetch request
let fetchRequest = NSFetchRequest(entityName: "Users")
// set a predicate that filters only records with updated set to false
fetchRequest.predicate = NSPredicate(format: "isModified == %#", false)
do {
let fetchResults = try self.managedObjectContext.executeFetchRequest(fetchRequest) as! [NSManagedObject]
for user in fetchResults {
print("Deleting \(user)")
self.managedObjectContext.deleteObject(user)
try self.managedObjectContext.save()
}
result = true
} catch let error as NSError {
print("\(error)")
}
return result
}
The method mostly works, but intermittently will delete a user for no good reason, i.e. even though it has the "isModified" flag set, e.g. here is the output of the line: print("Deleting (user)")
Deleting <NSManagedObject: 0x7b6d6ec0> (entity: Users; id: 0x7b64abd0 <x-coredata://1A7606EB-3539-4E85-BE1C-15C722AD7125/Users/p14> ; data: {
created = "2016-01-17 16:54:21 +0000";
familyName = Doe;
givenName = John;
isModified = 1;
pin = 3932;
type = STAFF;
})
As you can see, the "isModified" flag is very definitely set, yet the predicate is supposed to select only records that have the flag reset (false).
The method above is part of a class I have created, it's basically a CRUD class for the Users entity.
Scratching my head here!
I can supply the rest of the code if required.
I think your code looks perfectly fine (although you should save outside the loop). Swift booleans are automatically converted to NSNumber and back in most situations. There are many valid ways to write this predicate.
The only possible explanation that comes to mind is that another class is using the same managed object context to modify the fetched objects. If you have a single context app this a conceivable scenario.
The standard way to avoid this is to make the changes on a background context.
Use an boolean literal rather than a object placeholder %# for a boolean value
NSPredicate(format: "isModified == FALSE")
and don't call context.save() in each iteration of the repeat loop, call it once after the repeat loop.