I am using CoreData and am having issues when I try to saveContext in the performBackgroundTask function of the container.
I am calling this from multiple places at the same time. My understanding is that each time the thread will be different leading to issues when trying to save to the persistentStore.
I thought of using childContext's and other approaches until I came across the following article:
https://blog.five.agency/how-to-import-a-large-data-set-using-core-data-6c248a503148
In the article the coreData stack is as follows:
final class DataCoordinator {
//MARK: - singleton
static let sharedInstance = DataCoordinator()
//MARK: - init
public var container : NSPersistentContainer
private init() {
container = NSPersistentContainer(name: "Model")
container.loadPersistentStores(completionHandler: { (_, error) in
if let error = error {
fatalError("Unresolved error \(error)")
}
})
}
//MARK: - perform methods
static func performBackgroundTask(_ block: #escaping (NSManagedObjectContext) -> Void) {
DataCoordinator.sharedInstance.container.performBackgroundTask(block)
}
static func performViewTask(_ block: #escaping (NSManagedObjectContext) -> Void) {
block(DataCoordinator.sharedInstance.container.viewContext)
}
}
My question is using this approach, will the static func performBackgroundTask overcome this problem and always be giving me the same background context on the same queue or is this no different than what I was previously doing with container.performBackgroundTask everytime?
Wondering how I can save from multiple places at the same time on the same queue?
Hi first thing is its not about queue, NSManagedObjectContext run on different queues, like viewContext run on main queue. So in your case I think you want to use same NSManagedObjectContext to save from different places ? If this is the case you can get one newBackgroundContext and save the reference of it and use this context to save your task from all the places. Use the below method of NSPersistentContainer to get a background context.
func newBackgroundContext() -> NSManagedObjectContext
If my understanding about your problem is wrong please let me know more so I can give you other solution.
Again the below method of the tutorial u mentioned will not work for you
static func performBackgroundTask(_ block: #escaping (NSManagedObjectContext) -> Void) {
DataCoordinator.sharedInstance().container.performBackgroundTask(block)
}
because performBackgroundTask always create a new context. Apple doc says:
Each time this method is invoked, the persistent container creates a new NSManagedObjectContext with the concurrencyType set to NSManagedObjectContextConcurrencyType.privateQueueConcurrencyType. The persistent container then executes the passed in block against that newly created context on the context’s private queue.
Related
I would like to know if just saving a background context like in the below example is enough or I should also save the main context and why.
extension NSManagedObjectContext {
private func saveOrRollback(context: NSManagedObjectContext) -> Bool {
do {
if context.hasChanges {
try context.save()
return true
}
} catch {
log(error, message: "Couldn't save. Rolling back.", tag: .coreData)
context.rollback()
return false
}
return false
}
func performChanges(block: #escaping () -> Void) {
let privateContext = NSManagedObjectContext(concurrencyType: .privateQueueConcurrencyType)
privateContext.parent = self
privateContext.perform {
block()
_ = self.saveOrRollback(context: privateContext)
}
}
}
I would like to know if just saving a background context like in the below example is enough or I should also save the main context and why.
A Core Data managed object context is a workspace; changes that you make to the data exist only in that context until you save, at which point the changed objects are written back to the data store. If you made changes in the "background" context that you want to persist, you need to save that context. If you made changes in the "main" context that you want to persist, you need to save that context.
If there's a parent/child relationship between the two contexts, if you want to permanently save changes in the child context you'll need to first save in the child context and then save in the parent context. For example, if your "main" context is the parent store for your "background" context, saving in the background context will push the changes up to the main context, and you'll then need to save in the main context. (And if the main context is a child of some other context, you'll need to save all the way up the chain until you reach the root context.)
If you wanna save data on disk, you should also save it on main context.
Because if CoreData context have parent context, then save data just move changes to parent context.
Alternative:
I recommended use new API. You can init CoreData via: NSPersistentContainer.
It's more easy to use.
It's create separated content for UI:
persistentContainer.viewContext
It's can automatically merge changes to viewcontext.
persistentContainer.viewContext.automaticallyMergesChangesFromParent = true
And perform actions on background queue via much easy:
persistentContainer.performBackgroundTask { context in
...
do {
try context.save()
}
catch {
print(error.localizedDescription)
}
...
}
I use coreData as persistent store.
To read data, I use (only essential parts are shown):
func fetchShoppingItems(completion: #escaping (Set<ShoppingItem>?, Error?) -> Void) {
persistentContainer.performBackgroundTask { (backgroundManagedContext) in
let fetchRequest: NSFetchRequest<CDShoppingItem> = CDShoppingItem.fetchRequest()
var shoppingItems: Set<ShoppingItem> = []
do {
let cdShoppingItems: [CDShoppingItem] = try backgroundManagedContext.fetch(fetchRequest)
for nextCdShoppingItem in cdShoppingItems {
nextCdShoppingItem.managedObjectContext!.performAndWait {
Thread.sleep(forTimeInterval: 0.1) // This seems to be required
let nextShoppingItem = ShoppingItem.init(name: nextCdShoppingItem.name!)
shoppingItems.insert(nextShoppingItem)
} // performAndWait
} // for all cdShoppingItems
completion(shoppingItems, nil)
return
} catch let error as NSError {
completion(nil, error)
return
} // fetch error
} // performBackgroundTask
} // fetchShoppingItems
To test the coreData implementation, I wrote a unit test that creates multiple threads that write to and read from coreData concurrently.
This test runs only successfully, if the instruction
Thread.sleep(forTimeInterval: 0.1) // This seems to be required
is inserted in the performAndWait closure.
If it is commented out, nextCdShoppingItem is often read back with nil attributes, and the function crashes due to the forced unwrap.
I am not sure, if nextCdShoppingItem.managedObjectContext!.performAndWait is correct or if I had to use backgroundManagedContext.performAndWait, but with backgroundManagedContext the effect is the same.
I do not understand why inserting a small delay before accessing an attribute of a managed object is necessary to avoid the problem.
Any hint is welcome!
EDIT:
I investigated the issue further, and found the following:
Every time nextCdShoppingItem is read back by the background thread (called read thread below) as nil, there is also another background thread that tries to save its own managedContext after all records in its managedContext have been deleted (called write thread below).
Apparently the read thread tries to fetch a record that has just been deleted by the write thread.
So the problem is definitively a multithreading issue, and I found a solution (see my answer below).
performAndWait will add the block to the queue and schedule it to run, just like perform, but performAndWait will not return until the block is complete. Since you are inside a loop of cdShoppingItems, the loop does not stop and wait for the block to return. By adding the thread sleep, you are essentially slowing down the loop and giving core data enough time to complete its fetch. The forced unwrap crash is probably an indication that it's lost its nextCdShoppingItem reference.
I would consider refactoring where you do not need to query core data inside a loop. If it's possible, add the name attribute to CDShoppingItem so you don't have to fetch it to build a ShoppingItem object.
Edit: took a stab at a refactor although I don't know your exact use case:
func fetchShoppingItems(completion: #escaping (Set<ShoppingItem>?, Error?) -> Void) {
persistentContainer.performBackgroundTask { (backgroundManagedContext) in
let fetchRequest: NSFetchRequest<CDShoppingItem> = CDShoppingItem.fetchRequest()
do {
var shoppingItems: Set<ShoppingItem> = []
let cdShoppingItems: [CDShoppingItem] = try backgroundManagedContext.fetch(fetchRequest)
for nextCdShoppingItem in cdShoppingItems {
if let name = nextCdShoppingItem.name {
let nextShoppingItem = ShoppingItem.init(name: name)
shoppingItems.insert(nextShoppingItem)
}
}
completion(shoppingItems, nil)
} catch let error as NSError {
print("Error fetching CDShoppingItem: \(error)")
completion(nil, error)
} // fetch error
return
} // performBackgroundTask
} // fetchShoppingItems
To prevent the multithreading issue, I tried 2 things:
1) Since iOS10, a persistentStore of SQL type maintains a connection pool for concurrent access to the pool, and it is possible to set a maximum pool size, see the WWDC video. I did so using
private lazy var persistentContainer: NSPersistentContainer = {
let container = NSPersistentContainer(name: modelName)
container.loadPersistentStores(completionHandler: { (storeDescription, error) in
if let error = error as NSError? {
// ...
} else {
storeDescription.setOption(NSNumber(1), forKey: NSPersistentStoreConnectionPoolMaxSizeKey)
}
})
return container
}()
to prevent concurrent access to the persistentStore. Unfortunately, this did not solve the problem for an unknown reason.
2) I then tried to serialize read and write operations by setting up a serial queue:
private let coreDataSerialQueue = DispatchQueue(label: "com.xxx.ShopEasy.coreDataManager") // serial by default
It is used for the read and write accesses in the following way:
coreDataSerialQueue.async {
let backgroundManagedContext = self.persistentContainer.newBackgroundContext()
backgroundManagedContext.performAndWait {
// …
} // performAndWait
} // coreDataSerialQueue.async
This did solve the problem.
Please note that it would be wrong to use
coreDataSerialQueue.async {
self.persistentContainer.performBackgroundTask { (backgroundManagedContext) in
// …
} // performBackgroundTask
} // coreDataSerialQueue.async
because performBackgroundTask would fork another asynchronous thread and thus break the serialization.
We are running an App through Citrix Secure Hub, it seems that sometimes there is a rollback with loosing some Data in CoreData.
As i understand, CoreData is having something like an working copy of all the objects, and sometimes its tries to persist that on the filesystem.
Well tried to simulate the behavior but without any success, we could not find out any data loss or rollbacked data in our test environment.
So is there a way to force iOS to write the current "working copy" on the disk to prevent any data loss when using too much memory (and maybe crash)? We call our save function after
As we already found out:
We were NOT using:
func applicationWillResignActive(_ application: UIApplication) {
print("applicationWillResignActive")
}
to save the context, could this be a problem (we are already saving the context after every created object) ?
At the Moment we dont really handle problems when the context could not be saved, are there any recommendations how to handle that in a productive environment? And is it a good thing to maybe crash to app to prevent the user from struggeling with data loss?
Edit: this is the used Core Data Handler:
import Foundation
import CoreData
let context = CoreDataManager.shared.managedObjectContext
func saveContext(_ completion: (() -> Void)? = nil) {
CoreDataManager.shared.save(completion)
}
func saveContextSync() {
CoreDataManager.shared.saveSync()
}
class CoreDataManager: NSObject {
static let shared = CoreDataManager()
lazy var managedObjectContext: NSManagedObjectContext = {
var managedObjectContext = NSManagedObjectContext(concurrencyType: .mainQueueConcurrencyType)
managedObjectContext.persistentStoreCoordinator = persistentStoreCoordinator
return managedObjectContext
}()
And our save functionality:
#objc func save(_ completion: (() -> Void)?) {
saveAsync(completion)
}
func saveAsync(_ completion: (() -> Void)?) {
func save() {
context.perform {
do { try context.save() }
catch {
// HERE WE NEED TO HANDLE IT FOR A PRODUCTIVE ENVIRONMENT
}
completion?()
}
}
if Thread.isMainThread {
save()
} else {
DispatchQueue.main.async {
save()
}
}
}
func saveSync() {
func save() {
context.performAndWait {
do { try context.save() }
catch { print(error)
// TRY TO REPRODUCE MEMORY LOSS APP TO SEE WHAT HAPPENS
abort()
}
}
}
if Thread.isMainThread {
save()
} else {
DispatchQueue.main.sync {
save()
}
}
}
Edit 2: This question in Objective C should be very similar:
Core Data reverts to previous state without apparent reason
Edit 3: It seems that there is no crash, some users telling me that they are adding data, then just press the home button and after a couple of hours the data from the last "task" is lost.
There are three possible causes.
Write Conflicts
Core data generally wants writes to be done in a single synchronous way. If you write in multiple ways at the same time to the same object (even if they are touching different properties and don't strictly conflict), it will be a merge conflict. You can set a merge policy (by default the value is 'error' - meaning don't apply the changes) but that is really a bad solution because you are tell core-data to lose information silently. see NSPersistentContainer concurrency for saving to core data for a setup to prevent merge conflicts.
Closing the app with unsaved data
If you setup your core-data correctly this shouldn't happen. The correct way to setup core data to only read from the 'viewContext' and write in a single synchronous way. Each writing is done in a single atomic block and the UI is only updated after it is saved. If you are displaying information from a context that is not saved to disk this can be a problem. For example it appears that your app only uses a single main-thread context for both reading and writing. Making changes to that context and not calling save will leave the app in a state where there are major changes that are only in memory and not on disk.
There is an error saving to disk
This is by far the rarest event, but there are users that have really really full disks. If this happens there is NOTHING that you can do to save. There is physically no room left on the disk. Generally the correct thing to do is to tell the user and leave it at that.
Without knowing more about your particular setup it hard to say for certain what your problem is. I would recommend the following setup:
use NSPersistentContainer
only read from the viewContext and never write to it.
make an operation queue for writing to core data
for every operation, create a context, make changes to it, and then save. Do not pass any managed object into or out of these blocks.
This should deal with all but the third problem.
I have a set of NSOperations which make network requests. Imagine a case where we have a User object in Realm and I want to make some changes and then send a PATCH request to the server:
let operation = UpdateUserOperation(updatedUser)
Then the operation runs on a different thread so it has to resolve a thread safe reference:
class UpdateUserOperation : Operation {
var userReference : ThreadSafeReference<User>
init(_ user: User) {
userReference = ThreadSafeReference(to: user)
}
func main() {
// We're probably on a different thread so resolve the reference
let user = try! Realm().resolve(userReference)
// That invalidated `userReference` so we need to create a new
// one to use below...
userReference = ThreadSafeReference(to: user)
sendUserPatchRequest(user) { response in
// We might be in _another_ thread again :(
let realm = try! Realm()
let user = realm.resolve(userReference)
try! realm.write {
user.updateFromResponse(response)
}
}
}
}
This feels like a really unclean way to do this – re-fetching the user so many times to do a pretty simple task. It feels especially onerous because we need to re-up the thread-safe reference – they aren't re-usable. In Core Data, we'd be able to choose a single NSManagedObjectContext to do our work in and ensure thread safety by using managedObjectContext.perform { /* ... */ }, but that sort of functionality is unavailable in Realm.
Am I missing anything? Is there a better way to do this, or am I stuck re-fetching the object each time I need to use it?
There is no documentation or sample code explaining if we can share the viewContext with the app extension or not.
AFAK, the app and the extension run in different processes and we should NOT share moc with another process/thread. I should not share the viewContext the containing app is using with the app extension.
So should we create another viewContext to use in app extension(? but NSPersistentContainer only provides one viewContext) or use a background context in app extension(???)
While an extension is running, it communicates directly only with the host app. There is no direct communication between a running extension and its containing app; typically, the containing app isn’t even running while its extension is running. In addition, the containing app and the host app don’t communicate at all.
So since they all run in different processes, so maybe (???) I can come to the conclusion that when the app extension ask for the viewContext, and when the containing app ask for the viewContext, the 2 viewContext(s) are actually distinct instances?
class Master {
static let shared: Master = Master()
lazy var persistentContainer: CCPersistentContainer = {
let container = CCPersistentContainer(name: "xt")
container.loadPersistentStores(completionHandler: { (storeDescription, error) in
if let error = error as NSError? {
fatalError("Unresolved error \(error), \(error.userInfo)")
}
})
return container
}()
private var _backgroundContext: NSManagedObjectContext!
var backgroundContext: NSManagedObjectContext {
if _backgroundContext == nil {
_backgroundContext = persistentContainer.newBackgroundContext()
}
return _backgroundContext
}
var viewContext: NSManagedObjectContext {
return persistentContainer.viewContext
}
func saveContext() {
let context = persistentContainer.viewContext
if context.hasChanges {
do {
try context.save()
} catch {
let nserror = error as NSError
fatalError("Unresolved error \(nserror), \(nserror.userInfo)")
}
}
}
}
---- More On How To Sync Data Across Processes ----
1. WWDC 15: 224_hd_app_extension_best_practices
This WWDC session talks about how to post notification x processes.
2. NSMergePolicy
A policy object that you use to resolve conflicts between the persistent store and in-memory versions of managed objects.
3. WWDC 17: 210_hd_whats_new_in_core_data
4. UserDefaults(suiteName: AppGroups.primary)!
You can't share viewContext between the app and an extension. I don't mean you shouldn't, I mean it's actually, literally impossible even if you wanted to do so. An app and its extension are two separate processes. The viewContext object is created at run time by a process. There's no way for an app to hand off a variable in memory to a different process on iOS, so there's no possibility of using the same instance in both. You also have two different persistent container objects, again because they're created when the app or extension runs.
These two containers or view contexts might well use the same persistent store file. That's not unusual, and it allows the app and extension to access the same data.