What I'm trying to do:
perform background sync with a web API without freezing the UI. I'm using MagicalRecord but it's not really specific to it.
make sure I'm using contexts & such correctly
What my question really is: is my understanding correct? Plus a couple questions at the end.
So, the contexts made available by MagicalRecord are:
MR_rootSavingContext of PrivateQueueConcurrencyType which is used to persist data to the store, which is a slow process
MR_defaultContext of MainQueueConcurrencyType
and for background you would want to work with a context generated by MR_context(), which is a child of MR_defaultContext and is of PrivateQueueConcurrencyType
Now, for saving in an asynchronous way, we have two options:
MR_saveToPersistentStoreWithCompletion() which will save all the way up to MR_rootSavingContext and write to disk
MR_saveOnlySelfWithCompletion() which will save only up to the parent context (i?e. MR_defaultContext for a context created with MR_context)
From there, I thought that I could do the following (let's call it Attempt#1) without freezing the UI:
let context = NSManagedObjectContext.MR_context()
for i in 1...1_000 {
let user = User.MR_createInContext(context) as User
context.MR_saveOnlySelfWithCompletion(nil)
}
// I would normally call MR_saveOnlySelfWithCompletion here, but calling it inside the loop makes any UI block easier to spot
But, my assumption was wrong. I looked into MR_saveOnlySelfWithCompletion and saw that it relies on
[self performBlock:saveBlock];
which according to Apple Docs
Asynchronously performs a given block on the receiver’s queue.
So I was a bit puzzled, since I would expect it not to block the UI because of that.
Then I tried (let's call it Attempt#2)
let context = NSManagedObjectContext.MR_context()
for i in 1...1_000 {
let user = User.MR_createInContext(context) as User
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)) { () -> Void in
context.MR_saveOnlySelfWithCompletion(nil)
}
}
And this does the job, but it doesn't feel right.
Then I found something in the release notes of iOS 5.0
When sending messages to a context created with a queue
association, you must use the performBlock: or performBlockAndWait:
method if your code is not already executing on that queue (for the
main queue type) or within the scope of a performBlock... invocation
(for the private queue type). Within the blocks passed to those
methods, you can use the methods of NSManagedObjectContext freely.
So, I'm assuming that:
Attempt#1 freezes the UI because I'm actually calling it from the main queue and not within the scope of a performBlock
Attempt#2 works, but I'm creating yet another thread while the context already has its own background thread
So of course what I should do is use saveWithBlock:
MagicalRecord.saveWithBlock { (localContext) -> Void in
for i in 1...1_000 {
User.MR_createInContext(context)
}
}
This performs the operation on a direct child of MR_rootSavingContext which is of PrivateQueueConcurrencyType.
Thanks to rootContextChanged, any change that goes up to MR_rootSavingContext will be available to MR_defaultContext.
So it seems that:
MR_defaultContext is the perfect context when it comes to displaying data
edits are preferably done in an MR_context (child of MR_defaultContext)
long running tasks such as a server sync are preferably done using saveWithBlock
What it still don't get is how to work with MR_save[…]WithCompletion(). I would use it on MR_context but since it blocked the main thread in my test cases I don't see when it becomes relevant (or what I missed…).
Thanks for your time :)
Ok, I am rarely using magical records but since you said you question is more general I will attempt an answer.
Some theory: When creating a context you pass an indicator as to whether you want it to be bound on the main or a background thread
let context = NSManagedObjectContext(concurrencyType: NSManagedObjectContextConcurrencyType.PrivateQueueConcurrencyType)
By "bound" we mean that a thread is referenced by the context internally. In the example above a new thread is created and owned by the context. This thread is not used automatically but must be called explicitly as:
context.performBlock({ () -> Void in
context.save(nil)
return
});
So your code with 'dispatch_async' is wrong because the thread the context is bound to can only be referenced from the context itself (it is a private thread).
What you have to infer from the above is that if the context is bound to the main thread, calling performBlock from the main thread will not do anything different that calling context methods straight.
To comment on your bullet points at the end:
MR_defaultContext is the perfect context when it comes to displaying data: An NSManagedObject must be accessed from the context it is
created so it is actually the only context that you can feed the
UI from.
edits are preferably done in an MR_context (child of MR_defaultContext): Edits are not expensive and you should follow
the rule above. If you are calling a function that edits an NSManagedObject's properties from the main thread (like at the tap of a button)
you should update the main context. Saves on the other hand are
expensive and this is why your main context should not be linked to a
persistent store directly but just push its edits down to a root
context with background concurrency owning a persistent store.
long running tasks such as a server sync are preferably done using saveWithBlock Yes.
Now, In attempt 1
for i in 1...1_000 {
let user = User.MR_createInContext(context) as User
}
context.MR_saveOnlySelfWithCompletion(nil)
There is no need to save for every object creation. Even if the UI was not blocked it is wasteful.
About MR_context. In the documentation for magical records I cannot see a 'MR_context' so I am wondering if it is a quick method to access the main context. If it is so, it will block.
Related
To test my Core Data implementation I have enabled the launch argument com.apple.CoreData.ConcurrencyDebug 1. I am getting breakpoints triggered whenever I access a managed object from within a Task using Swifts new async APIs.
I use a single context (viewContext) for fetching and ephemeral background contexts to perform write operations. See some snippets of the important parts at the bottom.
My app functions perfectly and I get no breakpoints triggered except for the scenarios where I access a Core Data managed object from within a Task.
See an example here
func performReloadForecasts() {
Task {
await reloadForecasts()
}
}
The method reloadForecasts is too long to post here but it references the users favourite location (Hence the locations) and uses it to fetch the forecast from an API for that location.
Referencing the users locations in views or view models functions fine.
But from what I can tell, by using a Task to perform an asynchronous operation, I am performing the task in a whole different thread that is chosen (from a pool?) at run time.
Usually the thread that the Task is run in is com.apple.root.user-initiated-qos.cooperative (serial) which makes sense I suppose.
How can I refactor or change either my asynchronous functions (such as reloadForecast) or my core data stack (detailed below) to perform operations in a manor that abides by the concurrency rules for Core Data?
Can I force a Task to run on a particular thread? Since my main context is the viewContext it would have to be the main thread, which sort of defeats the purpose of the async Task.
Can I refactor my core data stack to create some sort of thread safe reference to my managed objects? I have seen suggestions to pass object ID's in and refetch the object inside the target thread but surely there is a more elegant way to abide by the concurrency rules.
Core Data Implementation
Context Setup
container = NSPersistentCloudKitContainer(name: "Model")
context = container.viewContext
context.automaticallyMergesChangesFromParent = true
context.mergePolicy = NSMergeByPropertyStoreTrumpMergePolicy
Fetching
#Published private var locations: [LocationModel] = []
private func reloadData() {
context.perform { [context] in
do {
self.locations = try context.fetch(LocationModel.fetchRequest())
} catch {
Logger.error("Failed to reload persistence layer.")
Logger.error(error.localizedDescription)
}
}
}
Performing Write Operations
func perform(_ block: #escaping (NSManagedObjectContext) -> Void) {
do {
let context = container.newBackgroundContext()
try context.performAndWait {
block(context)
try context.save()
}
} catch {
Logger.error(error.localizedDescription)
}
}
I fixed some of these issues by adding the #MainActor flag to the Task
Task { #MainActor in
await reloadForecasts()
}
However I was still getting breakpoints for certain issues, especially higher order functions like maps or sorts. I ended up adding the #MainActor wrapper to all my view models.
This fixed all of the weird crashes regarding the higher order functions accessing the core data objects, but I faced new problems. My second core data context used for saving objects was now the cause of concurrency breakpoints being triggered.
This made more sense and was much more debug-able. I had strong references to objects fetched in the main context, used to construct another object in the background context.
Model A <---> Model B
I had Model A that had relationship to another Model B. To setup the relationship to Model B, I used a reference to a Model B object that had been fetched on the main context. But I was creating the Model A object in the background thread.
To solve this I used the suggested methods of refetching the required objects by ObjectID in the correct context. (Using a bunch of nice helper methods to make things easier)
Here's a forum post asking a related question about ensuring asynchronous tasks are run on the main thread
https://forums.swift.org/t/best-way-to-run-an-anonymous-function-on-the-main-actor/50083
My understanding of the new swift concurrency models is that when you await on an async function, the task is run on another thread chosen from a pool and when the task is complete, execution returns to the point (and thread) you used await.
In this case I have forced the Task to start on the main thread (By using #MainActor), execute its task on a thread from the available pool, and return back to the main thread once its completed.
The swift concurrency explains some of this in detail: https://docs.swift.org/swift-book/LanguageGuide/Concurrency.html
I use NSPersistentContainer as a dependency in my classes. I find this approach quite useful, but there is a dilemma: I don't know in which thread my methods will be called. I found a very simple solution for this
extension NSPersistentContainer {
func getContext() -> NSManagedObjectContext {
if Thread.isMainThread {
return viewContext
} else {
return newBackgroundContext()
}
}
}
Looks wonderful but I still have a doubt is there any pitfalls? If it properly works, why on earth Core Data confuses us with its contexts?
It's OK as long as you can live with its inherent limitations, i.e.
When you're on the main queue, you always want the viewContext, never any other one.
When you're not on the main queue, you always want to create a new, independent context.
Some drawbacks that come to mind:
If you call a method that has an async completion handler, that handler might be called on a different queue. If you use this method, you might get a different context than when you made the call. Is that OK? It depends what you're doing in the handler.
Changes on one background context are not automatically available in other background contexts, so you run the risk of having multiple contexts with conflicting changes.
The method suggests a potential carelessness about which context you're using. It's important to be aware of which context you're using, because managed objects fetched on one can't be used with another. If your code just says, hey give me some context, but doesn't track the contexts properly, you increase the chance of crashes from accidentally crossing contexts.
If your non-main-queue requirements match the above, you're probably better off using the performBackgroundTask(_:) method on NSPersistentContainer. You're not adding anything to that method here.
[W]hy on earth Core Data confuses us with its contexts?
Managed object contexts are a fundamental part of how Core Data works. Keeping track of them is therefore a fundamental part of having an app that doesn't corrupt its data or crash.
I'm using MagicalRecord to manage my core data. My app often receives chunks of data that needs iterated over then individually added to the store as records, and saved. I put this code into an NSOperation. It looks something like this:
class AddIncomingMessageOperation: NSOperation {
override func main() {
let context = NSManagedObjectContext.MR_context()
self.message = Message.createMessage(
json: self.messageJSON,
context: context)
if self.message != nil {
context.MR_saveToPersistentStoreAndWait()
}
}
}
Running this on NSOperationQueue.mainQueue seems to work without issue, other than holding up the UI. But my next move is to run all of these operations on their own background operation queue. Adding them to this NSOperationQueue and running them then results in some mixed up data.
What I mean by this - my createMessage function checks for an existing Conversation object, and adds it to the conversation if it exists, or creates one if it doesn't. So it requires knowing what already exists in the store. What seems to be happening how, with them running on a background queue, is that they're creating conversations that have just been created in another operation.
I can solve all of this by setting operationQueue.maxConcurrentOperationCount = 1, but that obviously slows the whole thing down.
You cannot use the main thread context on a background thread. Not in Core Data and not in Magical Record.
Use the MR methods designed for background operations, such as saveWithBlock. Create background contexts with MR_newContext.
If you make use of these simple APIs, you might be able to dispense with the cumbersome NSOperation subclasses.
I have recently developed an iOS application with Swift which handles lots of background HTTP tasks and updates not only UI, but also static data for current session (lots of arrays, variables, etc) according to response data. I may be counted as new at iOS Developing and there are some points in which I am confused:
Updating UI from a background task is handled via GCD API. I have always handled these updates using:
dispatch_async(dispatch_get_main_queue, {
// Update UI
})
Let me give a scenario and clarify my point:
I have a view controller with a UITableView subview. This table view will display the list of something (lets say user names). I prepared and resumed an NSURLSessionDataTask:
let request = NSMutableURLRequest(URL: someURL)
let task = NSURLSession.sharedSession().dataTaskWithRequest(request) {
data, response, error in
// Handle error case
// Parse data, and receive a user list
// AppData.userList = parsed list
// Update table view
}
Some of my testers have faced some crashes related with dispatch calls and run loop in which I could not find the reason behind. I think this is related with my dispatch calls. Now I am reconsidering my design for this and here are my questions:
What is the difference of updating static data (array, dictionary, etc.) inside and outside of a dispatch_async call on main queue in completion handler of a http task (UI will be updated inside a dispatch call anyway, after the updates on my data)?. How can I ensure thread safety for background threads while reading from, inserting into or removing from an array?
Does making a dispatch_async call inside a closure (for task completion handler) may cause any problem?
Any clear comment or guiding would be very helpful! Thanks a lot already now
I'll try to give an aswer even if I didn't have a clear view about the answer.
You must update your UI from the main thread because UIKit objects (there is some exception if you want to draw on offscreen bitmap context) are not thread safe.
Here is what apple say about it:
Note: For the most part, UIKit classes should be used only from an
application’s main thread. This is particularly true for classes
derived from UIResponder or that involve manipulating your
application’s user interface in any way.
All the rendering routine should run on the main thread, most probably due to GPU acceleration and events management.
In contrast Foundation objects (except for some mutable one are thread safe), so can manage/manipulate and use them on a different thread.
Thread safety means that you can easily share you objects between threads.
If you use Foundation object on a background thread there is no problem at all, if you are using mutable once just inside that thread everything should work, the problem with mutable objects arise when you want to add objects to an array (for instance) from more threads.
If you provide your own classes you should provide thread safety by yourself.
First things first:
let request = NSMutableURLRequest(URL: someURL)
let task = NSURLSession.sharedSession().dataTaskWithRequest(request) { [weak self]
(data, response, error) in
if let weakself = self {
// ...
dispatch_async(dispatch_get_main_queue()) {
// update UI
}
}
}
Whenever you're making asynchronous calls, you need to make sure there are no self references to prevent any possible circular reference (memory leak).
What is the difference of updating static data (array, dictionary, etc.) inside and outside of a dispatch_async call on main queue in completion handler of a http task (UI will be updated inside a dispatch call anyway, after the updates on my data)?. How can I ensure thread safety for background threads while reading from, inserting into or removing from an array?
There is no difference between updating data inside and outside of a dispatch_async. You just need to make sure you're not mutating the array or dictionary while iterating through them. You can either do this by locking the data structure or creating a temporary shallow copy.
For example, if you're reading an Array that might be altered by another thread:
var integers = Array<Int>()
// The following creates an IMMUTABLE shallow copy of mutable array
let ints = integers
for value in ints {
// use value
}
// OR use locking
objc_sync_enter(integers)
for value in integers {
// use value
}
objc_sync_exit(integers)
// in another thread - lock before mutating
objc_sync_enter(integers)
integers.append(someIntValue)
objc_sync_exit(integers)
Of course, you can use other locking mechanisms. But the point is, you just need to make sure data is accessed in thread safe manner.
Does making a dispatch_async call inside a closure (for task completion handler) may cause any problem?
The answer is NO. As long as you make sure no reference to self exists within those closures and data accessible by competing threads are accessed/altered in a thread safe manner.
I have a method that runs in a background thread using a copy of NSManagedObjectContext which is specially generated when the background thread starts as per Apples recommendations.
In this method it makes a call to shared instance of a class, this shared instance is used for managing property values.
The shared instance that managed properties uses a NSManagedObjectContext on the main thread, now even though the background thread method should not use the NSManagedObjectContext on the main thread, it shouldn't really matter if the shared property manager class does or does not use the such a context as it only returns scalar values back to the background thread (at least that's my understanding).
So, why does the shared property class hang when retrieving values via the main threads context when called from the background thread? It doesn't need to pass an NSManagedObject or even update one so I cannot see what difference it would make.
I can appreciate that my approach is probably wrong but I want to understand at a base level why this is. At the moment I cannot understand this whole system enough to be able to think beyond Apples recommended methods of implementation and that's just a black magic approach which I don't like.
Any help is greatly appreciated.
Does using:
[theContext performBlock:^{
// do stuff on the context's queue, launch asynchronously
}];
-- or --
[theContext performBlockAndWait:^{
// do stuff on the context's queue, run synchronously
}];
-- just work for you? If so, you're done.
If not, take a long, hard look at how your contexts are setup, being passed around, and used. If they all share a root context, you should be able to "move" data between them easily, so long as you lookup any objectIDs always on your current context.
Contexts are bound to threads/queues, basically, so always use a given context as a a reference for where to do work. performBlock: is one way to do this.