Can I have a single Private Managed Object context that is being accessed by Multiple NSOperation ?
I have 2 two options :
Have a managed object context per NSOperation.
i.e if there are 100 NSoperation 100 context will be created.
Have a single context and multiple NSOperation.
i.e Single Context and 100 NSOperations accessing it.
Which can be a better option.
The correction solution is option 1. Create a queue with a concurrency count of 1 and do all your writing with the queue. This will avoid any write conflict which can lead to losing information. If you need to access information for the main thread you should use a global main thread context (in NSPersistentContainer it is call viewContext).
If this is going to slow then you should investigate the work that you are doing. Generally each operation should be pretty quick, so if you are finding that they are not you might be doing something wrong (a common issue is doing a fetch for each imported object - instead of one large fetch). Another solution is to split up large tasks into several smaller task (importing large amount of data). You can also set different priority - giving higher priority to actions that the user initiated.
You shouldn't be afraid of creating contexts. They are not that expensive.
According to what you said in the comments, that you didn't actually write lots of data in the single operation and just did a fetch for an object, I suggest using single MOC.
Usual reason to have multiple MOC is to read/update/save lots of data independently of Main context and of any other contexts. In such flow you would be able to save that objects concurrently from different contexts.
But it's not your case, If I understood correctly.
For the single fetch there would be enough just one Private context, however I believe there wouldn't be lots overhead in creating many contexts. But why to do extra work?
1.So, you create private MOC
let privateContext = NSManagedObjectContext(concurrencyType:.privateQueueConcurrencyType)
2.Create each operation and pass MOC
let operation = MyOperation(context: privateContext)
3.In the operation perform sync call to private MOC with function.
In such way you should avoid any concurrent problem with single MOC
func performAndWait(_ block: #escaping () -> Swift.Void)
for example
let myObject: Object?
privateContext.performAndWait {
myObject = privateContext.fetch(...)
}
// do what you need with myObject
Related
To test my Core Data implementation I have enabled the launch argument com.apple.CoreData.ConcurrencyDebug 1. I am getting breakpoints triggered whenever I access a managed object from within a Task using Swifts new async APIs.
I use a single context (viewContext) for fetching and ephemeral background contexts to perform write operations. See some snippets of the important parts at the bottom.
My app functions perfectly and I get no breakpoints triggered except for the scenarios where I access a Core Data managed object from within a Task.
See an example here
func performReloadForecasts() {
Task {
await reloadForecasts()
}
}
The method reloadForecasts is too long to post here but it references the users favourite location (Hence the locations) and uses it to fetch the forecast from an API for that location.
Referencing the users locations in views or view models functions fine.
But from what I can tell, by using a Task to perform an asynchronous operation, I am performing the task in a whole different thread that is chosen (from a pool?) at run time.
Usually the thread that the Task is run in is com.apple.root.user-initiated-qos.cooperative (serial) which makes sense I suppose.
How can I refactor or change either my asynchronous functions (such as reloadForecast) or my core data stack (detailed below) to perform operations in a manor that abides by the concurrency rules for Core Data?
Can I force a Task to run on a particular thread? Since my main context is the viewContext it would have to be the main thread, which sort of defeats the purpose of the async Task.
Can I refactor my core data stack to create some sort of thread safe reference to my managed objects? I have seen suggestions to pass object ID's in and refetch the object inside the target thread but surely there is a more elegant way to abide by the concurrency rules.
Core Data Implementation
Context Setup
container = NSPersistentCloudKitContainer(name: "Model")
context = container.viewContext
context.automaticallyMergesChangesFromParent = true
context.mergePolicy = NSMergeByPropertyStoreTrumpMergePolicy
Fetching
#Published private var locations: [LocationModel] = []
private func reloadData() {
context.perform { [context] in
do {
self.locations = try context.fetch(LocationModel.fetchRequest())
} catch {
Logger.error("Failed to reload persistence layer.")
Logger.error(error.localizedDescription)
}
}
}
Performing Write Operations
func perform(_ block: #escaping (NSManagedObjectContext) -> Void) {
do {
let context = container.newBackgroundContext()
try context.performAndWait {
block(context)
try context.save()
}
} catch {
Logger.error(error.localizedDescription)
}
}
I fixed some of these issues by adding the #MainActor flag to the Task
Task { #MainActor in
await reloadForecasts()
}
However I was still getting breakpoints for certain issues, especially higher order functions like maps or sorts. I ended up adding the #MainActor wrapper to all my view models.
This fixed all of the weird crashes regarding the higher order functions accessing the core data objects, but I faced new problems. My second core data context used for saving objects was now the cause of concurrency breakpoints being triggered.
This made more sense and was much more debug-able. I had strong references to objects fetched in the main context, used to construct another object in the background context.
Model A <---> Model B
I had Model A that had relationship to another Model B. To setup the relationship to Model B, I used a reference to a Model B object that had been fetched on the main context. But I was creating the Model A object in the background thread.
To solve this I used the suggested methods of refetching the required objects by ObjectID in the correct context. (Using a bunch of nice helper methods to make things easier)
Here's a forum post asking a related question about ensuring asynchronous tasks are run on the main thread
https://forums.swift.org/t/best-way-to-run-an-anonymous-function-on-the-main-actor/50083
My understanding of the new swift concurrency models is that when you await on an async function, the task is run on another thread chosen from a pool and when the task is complete, execution returns to the point (and thread) you used await.
In this case I have forced the Task to start on the main thread (By using #MainActor), execute its task on a thread from the available pool, and return back to the main thread once its completed.
The swift concurrency explains some of this in detail: https://docs.swift.org/swift-book/LanguageGuide/Concurrency.html
I use NSPersistentContainer as a dependency in my classes. I find this approach quite useful, but there is a dilemma: I don't know in which thread my methods will be called. I found a very simple solution for this
extension NSPersistentContainer {
func getContext() -> NSManagedObjectContext {
if Thread.isMainThread {
return viewContext
} else {
return newBackgroundContext()
}
}
}
Looks wonderful but I still have a doubt is there any pitfalls? If it properly works, why on earth Core Data confuses us with its contexts?
It's OK as long as you can live with its inherent limitations, i.e.
When you're on the main queue, you always want the viewContext, never any other one.
When you're not on the main queue, you always want to create a new, independent context.
Some drawbacks that come to mind:
If you call a method that has an async completion handler, that handler might be called on a different queue. If you use this method, you might get a different context than when you made the call. Is that OK? It depends what you're doing in the handler.
Changes on one background context are not automatically available in other background contexts, so you run the risk of having multiple contexts with conflicting changes.
The method suggests a potential carelessness about which context you're using. It's important to be aware of which context you're using, because managed objects fetched on one can't be used with another. If your code just says, hey give me some context, but doesn't track the contexts properly, you increase the chance of crashes from accidentally crossing contexts.
If your non-main-queue requirements match the above, you're probably better off using the performBackgroundTask(_:) method on NSPersistentContainer. You're not adding anything to that method here.
[W]hy on earth Core Data confuses us with its contexts?
Managed object contexts are a fundamental part of how Core Data works. Keeping track of them is therefore a fundamental part of having an app that doesn't corrupt its data or crash.
I'm using MagicalRecord to manage my core data. My app often receives chunks of data that needs iterated over then individually added to the store as records, and saved. I put this code into an NSOperation. It looks something like this:
class AddIncomingMessageOperation: NSOperation {
override func main() {
let context = NSManagedObjectContext.MR_context()
self.message = Message.createMessage(
json: self.messageJSON,
context: context)
if self.message != nil {
context.MR_saveToPersistentStoreAndWait()
}
}
}
Running this on NSOperationQueue.mainQueue seems to work without issue, other than holding up the UI. But my next move is to run all of these operations on their own background operation queue. Adding them to this NSOperationQueue and running them then results in some mixed up data.
What I mean by this - my createMessage function checks for an existing Conversation object, and adds it to the conversation if it exists, or creates one if it doesn't. So it requires knowing what already exists in the store. What seems to be happening how, with them running on a background queue, is that they're creating conversations that have just been created in another operation.
I can solve all of this by setting operationQueue.maxConcurrentOperationCount = 1, but that obviously slows the whole thing down.
You cannot use the main thread context on a background thread. Not in Core Data and not in Magical Record.
Use the MR methods designed for background operations, such as saveWithBlock. Create background contexts with MR_newContext.
If you make use of these simple APIs, you might be able to dispense with the cumbersome NSOperation subclasses.
What I'm trying to do:
perform background sync with a web API without freezing the UI. I'm using MagicalRecord but it's not really specific to it.
make sure I'm using contexts & such correctly
What my question really is: is my understanding correct? Plus a couple questions at the end.
So, the contexts made available by MagicalRecord are:
MR_rootSavingContext of PrivateQueueConcurrencyType which is used to persist data to the store, which is a slow process
MR_defaultContext of MainQueueConcurrencyType
and for background you would want to work with a context generated by MR_context(), which is a child of MR_defaultContext and is of PrivateQueueConcurrencyType
Now, for saving in an asynchronous way, we have two options:
MR_saveToPersistentStoreWithCompletion() which will save all the way up to MR_rootSavingContext and write to disk
MR_saveOnlySelfWithCompletion() which will save only up to the parent context (i?e. MR_defaultContext for a context created with MR_context)
From there, I thought that I could do the following (let's call it Attempt#1) without freezing the UI:
let context = NSManagedObjectContext.MR_context()
for i in 1...1_000 {
let user = User.MR_createInContext(context) as User
context.MR_saveOnlySelfWithCompletion(nil)
}
// I would normally call MR_saveOnlySelfWithCompletion here, but calling it inside the loop makes any UI block easier to spot
But, my assumption was wrong. I looked into MR_saveOnlySelfWithCompletion and saw that it relies on
[self performBlock:saveBlock];
which according to Apple Docs
Asynchronously performs a given block on the receiver’s queue.
So I was a bit puzzled, since I would expect it not to block the UI because of that.
Then I tried (let's call it Attempt#2)
let context = NSManagedObjectContext.MR_context()
for i in 1...1_000 {
let user = User.MR_createInContext(context) as User
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)) { () -> Void in
context.MR_saveOnlySelfWithCompletion(nil)
}
}
And this does the job, but it doesn't feel right.
Then I found something in the release notes of iOS 5.0
When sending messages to a context created with a queue
association, you must use the performBlock: or performBlockAndWait:
method if your code is not already executing on that queue (for the
main queue type) or within the scope of a performBlock... invocation
(for the private queue type). Within the blocks passed to those
methods, you can use the methods of NSManagedObjectContext freely.
So, I'm assuming that:
Attempt#1 freezes the UI because I'm actually calling it from the main queue and not within the scope of a performBlock
Attempt#2 works, but I'm creating yet another thread while the context already has its own background thread
So of course what I should do is use saveWithBlock:
MagicalRecord.saveWithBlock { (localContext) -> Void in
for i in 1...1_000 {
User.MR_createInContext(context)
}
}
This performs the operation on a direct child of MR_rootSavingContext which is of PrivateQueueConcurrencyType.
Thanks to rootContextChanged, any change that goes up to MR_rootSavingContext will be available to MR_defaultContext.
So it seems that:
MR_defaultContext is the perfect context when it comes to displaying data
edits are preferably done in an MR_context (child of MR_defaultContext)
long running tasks such as a server sync are preferably done using saveWithBlock
What it still don't get is how to work with MR_save[…]WithCompletion(). I would use it on MR_context but since it blocked the main thread in my test cases I don't see when it becomes relevant (or what I missed…).
Thanks for your time :)
Ok, I am rarely using magical records but since you said you question is more general I will attempt an answer.
Some theory: When creating a context you pass an indicator as to whether you want it to be bound on the main or a background thread
let context = NSManagedObjectContext(concurrencyType: NSManagedObjectContextConcurrencyType.PrivateQueueConcurrencyType)
By "bound" we mean that a thread is referenced by the context internally. In the example above a new thread is created and owned by the context. This thread is not used automatically but must be called explicitly as:
context.performBlock({ () -> Void in
context.save(nil)
return
});
So your code with 'dispatch_async' is wrong because the thread the context is bound to can only be referenced from the context itself (it is a private thread).
What you have to infer from the above is that if the context is bound to the main thread, calling performBlock from the main thread will not do anything different that calling context methods straight.
To comment on your bullet points at the end:
MR_defaultContext is the perfect context when it comes to displaying data: An NSManagedObject must be accessed from the context it is
created so it is actually the only context that you can feed the
UI from.
edits are preferably done in an MR_context (child of MR_defaultContext): Edits are not expensive and you should follow
the rule above. If you are calling a function that edits an NSManagedObject's properties from the main thread (like at the tap of a button)
you should update the main context. Saves on the other hand are
expensive and this is why your main context should not be linked to a
persistent store directly but just push its edits down to a root
context with background concurrency owning a persistent store.
long running tasks such as a server sync are preferably done using saveWithBlock Yes.
Now, In attempt 1
for i in 1...1_000 {
let user = User.MR_createInContext(context) as User
}
context.MR_saveOnlySelfWithCompletion(nil)
There is no need to save for every object creation. Even if the UI was not blocked it is wasteful.
About MR_context. In the documentation for magical records I cannot see a 'MR_context' so I am wondering if it is a quick method to access the main context. If it is so, it will block.
I am creating a NSManagedObject (subclass) with certain attributes. At the same time, I am executing some code/a block that does some network operation given the attributes of my NSManagedObject. Now, some times that network operation might fail or take too long, so I want to add the ability to cancel the execution of that code/block.
I was thinking of making the code/block an NSThread, and then I have the ability to call [theThread cancel]. However, how do I associate the NSThread with my NSManagedObject, given that I cannot add properties to NSManagedObject Categories? Is it OK to just add the property to the definition of the NSManagedObject itself? Seems legal, but subsequent changes to the Core Data model would overwrite my code, I guess.
But maybe there is an entirely different and better way to accomplish what I am trying to do? Any ideas?
First, new code really should prefer GCD or NSOperationQueue over NSThread. If you find yourself using NSThread it's time to slow down and revisit your design and implementation requirements.
Second, using NSManagedObject across threads is really, really bad. If you do anything but exceedingly trivial things, it can get very difficult to do right as well.
Finally, no matter how you do your threaded network access, you should prefer to grab the data from the managed object, and pass that instead of the managed object itself
If you must access the managed object, make sure your managed object context is of either NSMainQueueConcurrencyType or NSPrivateQueueConcurrencyType and access the managed object like by invoking performBlock or performBlockAndWait using the managedObjectContext property of the managed object.
EDIT
Ok, let me check this with you. What I a currently doing is spawning a
backgroundContext, create a new NSManagedObject using performBlock,
then save that background Context, switch to the parent context (using
performblock), obtain the newly created object in that context using
existingObjectWithId:. Then, I create a NSOperation subclass, tie the
NSManagedObject (from the parent context) to that NSOperation subclass
(it's a property on the subclass) and put that operation in a
NSOperationQueue. Within that NSOperation, the NSManagedObject gets
changed. It seems to work fine, does that look ok? – user1013725
Um... maybe??? I didn't follow that. Could you please post the code? That would be much more precise and more easy to understand.
#JodyHagins So I am not using performBlock, but maybe that's ok
because the managedObjectContext is the main context? – user1013725
No.
If the main context is created with either alloc] init] or alloc] initWithConcurrencyType:NSConfinementConcurrencyType then you must use it only when you know you are running on the main thread.
If it is created with alloc] initWithConcurrencyType:NSMainThreadConcurrencyType then you must use it only when you know you are running on the main thread or within one of the performBlock methods.