Updating Data vs Updating UI Dilemma - ios

I have recently developed an iOS application with Swift which handles lots of background HTTP tasks and updates not only UI, but also static data for current session (lots of arrays, variables, etc) according to response data. I may be counted as new at iOS Developing and there are some points in which I am confused:
Updating UI from a background task is handled via GCD API. I have always handled these updates using:
dispatch_async(dispatch_get_main_queue, {
// Update UI
})
Let me give a scenario and clarify my point:
I have a view controller with a UITableView subview. This table view will display the list of something (lets say user names). I prepared and resumed an NSURLSessionDataTask:
let request = NSMutableURLRequest(URL: someURL)
let task = NSURLSession.sharedSession().dataTaskWithRequest(request) {
data, response, error in
// Handle error case
// Parse data, and receive a user list
// AppData.userList = parsed list
// Update table view
}
Some of my testers have faced some crashes related with dispatch calls and run loop in which I could not find the reason behind. I think this is related with my dispatch calls. Now I am reconsidering my design for this and here are my questions:
What is the difference of updating static data (array, dictionary, etc.) inside and outside of a dispatch_async call on main queue in completion handler of a http task (UI will be updated inside a dispatch call anyway, after the updates on my data)?. How can I ensure thread safety for background threads while reading from, inserting into or removing from an array?
Does making a dispatch_async call inside a closure (for task completion handler) may cause any problem?
Any clear comment or guiding would be very helpful! Thanks a lot already now

I'll try to give an aswer even if I didn't have a clear view about the answer.
You must update your UI from the main thread because UIKit objects (there is some exception if you want to draw on offscreen bitmap context) are not thread safe.
Here is what apple say about it:
Note: For the most part, UIKit classes should be used only from an
application’s main thread. This is particularly true for classes
derived from UIResponder or that involve manipulating your
application’s user interface in any way.
All the rendering routine should run on the main thread, most probably due to GPU acceleration and events management.
In contrast Foundation objects (except for some mutable one are thread safe), so can manage/manipulate and use them on a different thread.
Thread safety means that you can easily share you objects between threads.
If you use Foundation object on a background thread there is no problem at all, if you are using mutable once just inside that thread everything should work, the problem with mutable objects arise when you want to add objects to an array (for instance) from more threads.
If you provide your own classes you should provide thread safety by yourself.

First things first:
let request = NSMutableURLRequest(URL: someURL)
let task = NSURLSession.sharedSession().dataTaskWithRequest(request) { [weak self]
(data, response, error) in
if let weakself = self {
// ...
dispatch_async(dispatch_get_main_queue()) {
// update UI
}
}
}
Whenever you're making asynchronous calls, you need to make sure there are no self references to prevent any possible circular reference (memory leak).
What is the difference of updating static data (array, dictionary, etc.) inside and outside of a dispatch_async call on main queue in completion handler of a http task (UI will be updated inside a dispatch call anyway, after the updates on my data)?. How can I ensure thread safety for background threads while reading from, inserting into or removing from an array?
There is no difference between updating data inside and outside of a dispatch_async. You just need to make sure you're not mutating the array or dictionary while iterating through them. You can either do this by locking the data structure or creating a temporary shallow copy.
For example, if you're reading an Array that might be altered by another thread:
var integers = Array<Int>()
// The following creates an IMMUTABLE shallow copy of mutable array
let ints = integers
for value in ints {
// use value
}
// OR use locking
objc_sync_enter(integers)
for value in integers {
// use value
}
objc_sync_exit(integers)
// in another thread - lock before mutating
objc_sync_enter(integers)
integers.append(someIntValue)
objc_sync_exit(integers)
Of course, you can use other locking mechanisms. But the point is, you just need to make sure data is accessed in thread safe manner.
Does making a dispatch_async call inside a closure (for task completion handler) may cause any problem?
The answer is NO. As long as you make sure no reference to self exists within those closures and data accessible by competing threads are accessed/altered in a thread safe manner.

Related

Correctly accessing Core Data from another thread using the new Asynchronous Task in Swift

To test my Core Data implementation I have enabled the launch argument com.apple.CoreData.ConcurrencyDebug 1. I am getting breakpoints triggered whenever I access a managed object from within a Task using Swifts new async APIs.
I use a single context (viewContext) for fetching and ephemeral background contexts to perform write operations. See some snippets of the important parts at the bottom.
My app functions perfectly and I get no breakpoints triggered except for the scenarios where I access a Core Data managed object from within a Task.
See an example here
func performReloadForecasts() {
Task {
await reloadForecasts()
}
}
The method reloadForecasts is too long to post here but it references the users favourite location (Hence the locations) and uses it to fetch the forecast from an API for that location.
Referencing the users locations in views or view models functions fine.
But from what I can tell, by using a Task to perform an asynchronous operation, I am performing the task in a whole different thread that is chosen (from a pool?) at run time.
Usually the thread that the Task is run in is com.apple.root.user-initiated-qos.cooperative (serial) which makes sense I suppose.
How can I refactor or change either my asynchronous functions (such as reloadForecast) or my core data stack (detailed below) to perform operations in a manor that abides by the concurrency rules for Core Data?
Can I force a Task to run on a particular thread? Since my main context is the viewContext it would have to be the main thread, which sort of defeats the purpose of the async Task.
Can I refactor my core data stack to create some sort of thread safe reference to my managed objects? I have seen suggestions to pass object ID's in and refetch the object inside the target thread but surely there is a more elegant way to abide by the concurrency rules.
Core Data Implementation
Context Setup
container = NSPersistentCloudKitContainer(name: "Model")
context = container.viewContext
context.automaticallyMergesChangesFromParent = true
context.mergePolicy = NSMergeByPropertyStoreTrumpMergePolicy
Fetching
#Published private var locations: [LocationModel] = []
private func reloadData() {
context.perform { [context] in
do {
self.locations = try context.fetch(LocationModel.fetchRequest())
} catch {
Logger.error("Failed to reload persistence layer.")
Logger.error(error.localizedDescription)
}
}
}
Performing Write Operations
func perform(_ block: #escaping (NSManagedObjectContext) -> Void) {
do {
let context = container.newBackgroundContext()
try context.performAndWait {
block(context)
try context.save()
}
} catch {
Logger.error(error.localizedDescription)
}
}
I fixed some of these issues by adding the #MainActor flag to the Task
Task { #MainActor in
await reloadForecasts()
}
However I was still getting breakpoints for certain issues, especially higher order functions like maps or sorts. I ended up adding the #MainActor wrapper to all my view models.
This fixed all of the weird crashes regarding the higher order functions accessing the core data objects, but I faced new problems. My second core data context used for saving objects was now the cause of concurrency breakpoints being triggered.
This made more sense and was much more debug-able. I had strong references to objects fetched in the main context, used to construct another object in the background context.
Model A <---> Model B
I had Model A that had relationship to another Model B. To setup the relationship to Model B, I used a reference to a Model B object that had been fetched on the main context. But I was creating the Model A object in the background thread.
To solve this I used the suggested methods of refetching the required objects by ObjectID in the correct context. (Using a bunch of nice helper methods to make things easier)
Here's a forum post asking a related question about ensuring asynchronous tasks are run on the main thread
https://forums.swift.org/t/best-way-to-run-an-anonymous-function-on-the-main-actor/50083
My understanding of the new swift concurrency models is that when you await on an async function, the task is run on another thread chosen from a pool and when the task is complete, execution returns to the point (and thread) you used await.
In this case I have forced the Task to start on the main thread (By using #MainActor), execute its task on a thread from the available pool, and return back to the main thread once its completed.
The swift concurrency explains some of this in detail: https://docs.swift.org/swift-book/LanguageGuide/Concurrency.html

core data with multiple NSOperation

Can I have a single Private Managed Object context that is being accessed by Multiple NSOperation ?
I have 2 two options :
Have a managed object context per NSOperation.
i.e if there are 100 NSoperation 100 context will be created.
Have a single context and multiple NSOperation.
i.e Single Context and 100 NSOperations accessing it.
Which can be a better option.
The correction solution is option 1. Create a queue with a concurrency count of 1 and do all your writing with the queue. This will avoid any write conflict which can lead to losing information. If you need to access information for the main thread you should use a global main thread context (in NSPersistentContainer it is call viewContext).
If this is going to slow then you should investigate the work that you are doing. Generally each operation should be pretty quick, so if you are finding that they are not you might be doing something wrong (a common issue is doing a fetch for each imported object - instead of one large fetch). Another solution is to split up large tasks into several smaller task (importing large amount of data). You can also set different priority - giving higher priority to actions that the user initiated.
You shouldn't be afraid of creating contexts. They are not that expensive.
According to what you said in the comments, that you didn't actually write lots of data in the single operation and just did a fetch for an object, I suggest using single MOC.
Usual reason to have multiple MOC is to read/update/save lots of data independently of Main context and of any other contexts. In such flow you would be able to save that objects concurrently from different contexts.
But it's not your case, If I understood correctly.
For the single fetch there would be enough just one Private context, however I believe there wouldn't be lots overhead in creating many contexts. But why to do extra work?
1.So, you create private MOC
let privateContext = NSManagedObjectContext(concurrencyType:.privateQueueConcurrencyType)
2.Create each operation and pass MOC
let operation = MyOperation(context: privateContext)
3.In the operation perform sync call to private MOC with function.
In such way you should avoid any concurrent problem with single MOC
func performAndWait(_ block: #escaping () -> Swift.Void)
for example
let myObject: Object?
privateContext.performAndWait {
myObject = privateContext.fetch(...)
}
// do what you need with myObject

-allKeys on background thread results in error: __NSDictionaryM was mutated while being enumerated

I've come across an interesting issue using mutable dictionaries on background threads.
Currently, I am downloading data in chunks on one thread, adding it to a data set, and processing it on another background thread. The overall design works for the most part aside from one issue: On occasion, a function call to an inner dictionary within the main data set causes the following crash:
*** Collection <__NSDictionaryM: 0x13000a190> was mutated while being enumerated.
I know this is a fairly common crash to have, but the strange part is that it's not crashing in a loop on this collection. Instead, the exception breakpoint in Xcode is stopping on the following line:
NSArray *tempKeys = [temp allKeys];
This leads me to believe that one thread is adding items to this collection while the NSMutableDictionary's internal function call to -allKeys is enumerating over the keys in order to return the array on another thread.
My question is: Is this what's happening? If so, what would be the best way to avoid this?
Here's the gist of what I'm doing:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^(void) {
for (NSString *key in [[queue allKeys] reverseObjectEnumerator]) { //To prevent crashes
NEXActivityMap *temp = queue[key];
NSArray *tempKeys = [temp allKeys]; //<= CRASHES HERE
if (tempKeys.count > 0) {
//Do other stuff
}
}
});
You can use #synchronize. And it will work. But this is mixing up two different ideas:
Threads have been around for many years. A new thread opens a new control flow. Code in different threads are running potentially concurrently causing conflicts as you had. To prevent this conflicts you have to use locks like #synchronized do.
GCD is the more modern concept. GCD runs "on top of threads" that means, it uses threads, but this is transparent for you. You do not have to care about this. Code running in different queues are running potentially concurrently causing conflicts. To prevent this conflicts you have to use one queue for shared resources.
You are already using GCD, what is a good idea:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^(void) {
The same code with threads would look like this:
[[NSThread mainThread] performSelector:…];
So, using GCD, you should use GCD to prevent the conflicts. What you are doing is to use GCD wrongly and then "repair" that with locks.
Simply put all accesses to the shared resource (in your case the mutable dictionary referred by temp) into on serial queue.
Create a queue at the beginning for the accesses. This is a one-timer.
You can use one of the existing queues as you do in your code, but you have to use a serial one! But this potentially leads to long queues with waiting tasks (in your example blocks). Different tasks in a serial queue are executed one after each other, even there are cpu cores idle. So it is no good idea to put too many tasks into one queue. Create a queue for any shared resource or "subsystem":
dispatch_queue_t tempQueue;
tempQueue = dispatch_queue_create("tempQueue", NULL);
When code wants to access the mutable dictionary, put it in a queue:
It looks like this:
dispatch_sync( tempQueue, // or async, if it is possible
^{
[tempQueue setObject:… forKey:…]; // Or what you want to do.
}
You have to put every code accessing the shared resource in the queue as you have to put every code accessing the shared resource inn locks when using threads.
From Apple documentation "Thread safety summary":
Mutable objects are generally not thread-safe. To use mutable objects
in a threaded application, the application must synchronize access to
them using locks. (For more information, see Atomic Operations). In
general, the collection classes (for example, NSMutableArray,
NSMutableDictionary) are not thread-safe when mutations are concerned.
That is, if one or more threads are changing the same array, problems
can occur. You must lock around spots where reads and writes occur to
assure thread safety.
In your case, following scenario happens. From one thread, you add elements into dictionary. In another thread, you accessing allKeys method. While this methods copies all keys into array, other methods adds new key. This causes exception.
To avoid that, you have several options.
Because you are using dispatch queues, preferred way is to put all code, that access same mutable dictionary instance, into private serial dispatch queue.
Second option is passing immutable dictionary copy to other thread. In this case, no matter what happen in first thread with original dictionary, data still will be consistent. Note that you will probably need deep copy, cause you use dictionary/arrays hierarchy.
Alternatively you can wrap all points, where you access collections, with locks. Using #synchronized also implicitly create recursive lock for you.
How about wrapping where you get the keys AND where you set the keys, with #synchronize?
Example:
- (void)myMethod:(id)anObj
{
#synchronized(anObj)
{
// Everything between the braces is protected by the #synchronized directive.
}
}

MagicalRecord with NSOperation causing persistence issues

I'm using MagicalRecord to manage my core data. My app often receives chunks of data that needs iterated over then individually added to the store as records, and saved. I put this code into an NSOperation. It looks something like this:
class AddIncomingMessageOperation: NSOperation {
override func main() {
let context = NSManagedObjectContext.MR_context()
self.message = Message.createMessage(
json: self.messageJSON,
context: context)
if self.message != nil {
context.MR_saveToPersistentStoreAndWait()
}
}
}
Running this on NSOperationQueue.mainQueue seems to work without issue, other than holding up the UI. But my next move is to run all of these operations on their own background operation queue. Adding them to this NSOperationQueue and running them then results in some mixed up data.
What I mean by this - my createMessage function checks for an existing Conversation object, and adds it to the conversation if it exists, or creates one if it doesn't. So it requires knowing what already exists in the store. What seems to be happening how, with them running on a background queue, is that they're creating conversations that have just been created in another operation.
I can solve all of this by setting operationQueue.maxConcurrentOperationCount = 1, but that obviously slows the whole thing down.
You cannot use the main thread context on a background thread. Not in Core Data and not in Magical Record.
Use the MR methods designed for background operations, such as saveWithBlock. Create background contexts with MR_newContext.
If you make use of these simple APIs, you might be able to dispense with the cumbersome NSOperation subclasses.

Saving NSManagedObjectContext without hitting the main thread

What I'm trying to do:
perform background sync with a web API without freezing the UI. I'm using MagicalRecord but it's not really specific to it.
make sure I'm using contexts & such correctly
What my question really is: is my understanding correct? Plus a couple questions at the end.
So, the contexts made available by MagicalRecord are:
MR_rootSavingContext of PrivateQueueConcurrencyType which is used to persist data to the store, which is a slow process
MR_defaultContext of MainQueueConcurrencyType
and for background you would want to work with a context generated by MR_context(), which is a child of MR_defaultContext and is of PrivateQueueConcurrencyType
Now, for saving in an asynchronous way, we have two options:
MR_saveToPersistentStoreWithCompletion() which will save all the way up to MR_rootSavingContext and write to disk
MR_saveOnlySelfWithCompletion() which will save only up to the parent context (i?e. MR_defaultContext for a context created with MR_context)
From there, I thought that I could do the following (let's call it Attempt#1) without freezing the UI:
let context = NSManagedObjectContext.MR_context()
for i in 1...1_000 {
let user = User.MR_createInContext(context) as User
context.MR_saveOnlySelfWithCompletion(nil)
}
// I would normally call MR_saveOnlySelfWithCompletion here, but calling it inside the loop makes any UI block easier to spot
But, my assumption was wrong. I looked into MR_saveOnlySelfWithCompletion and saw that it relies on
[self performBlock:saveBlock];
which according to Apple Docs
Asynchronously performs a given block on the receiver’s queue.
So I was a bit puzzled, since I would expect it not to block the UI because of that.
Then I tried (let's call it Attempt#2)
let context = NSManagedObjectContext.MR_context()
for i in 1...1_000 {
let user = User.MR_createInContext(context) as User
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)) { () -> Void in
context.MR_saveOnlySelfWithCompletion(nil)
}
}
And this does the job, but it doesn't feel right.
Then I found something in the release notes of iOS 5.0
When sending messages to a context created with a queue
association, you must use the performBlock: or performBlockAndWait:
method if your code is not already executing on that queue (for the
main queue type) or within the scope of a performBlock... invocation
(for the private queue type). Within the blocks passed to those
methods, you can use the methods of NSManagedObjectContext freely.
So, I'm assuming that:
Attempt#1 freezes the UI because I'm actually calling it from the main queue and not within the scope of a performBlock
Attempt#2 works, but I'm creating yet another thread while the context already has its own background thread
So of course what I should do is use saveWithBlock:
MagicalRecord.saveWithBlock { (localContext) -> Void in
for i in 1...1_000 {
User.MR_createInContext(context)
}
}
This performs the operation on a direct child of MR_rootSavingContext which is of PrivateQueueConcurrencyType.
Thanks to rootContextChanged, any change that goes up to MR_rootSavingContext will be available to MR_defaultContext.
So it seems that:
MR_defaultContext is the perfect context when it comes to displaying data
edits are preferably done in an MR_context (child of MR_defaultContext)
long running tasks such as a server sync are preferably done using saveWithBlock
What it still don't get is how to work with MR_save[…]WithCompletion(). I would use it on MR_context but since it blocked the main thread in my test cases I don't see when it becomes relevant (or what I missed…).
Thanks for your time :)
Ok, I am rarely using magical records but since you said you question is more general I will attempt an answer.
Some theory: When creating a context you pass an indicator as to whether you want it to be bound on the main or a background thread
let context = NSManagedObjectContext(concurrencyType: NSManagedObjectContextConcurrencyType.PrivateQueueConcurrencyType)
By "bound" we mean that a thread is referenced by the context internally. In the example above a new thread is created and owned by the context. This thread is not used automatically but must be called explicitly as:
context.performBlock({ () -> Void in
context.save(nil)
return
});
So your code with 'dispatch_async' is wrong because the thread the context is bound to can only be referenced from the context itself (it is a private thread).
What you have to infer from the above is that if the context is bound to the main thread, calling performBlock from the main thread will not do anything different that calling context methods straight.
To comment on your bullet points at the end:
MR_defaultContext is the perfect context when it comes to displaying data: An NSManagedObject must be accessed from the context it is
created so it is actually the only context that you can feed the
UI from.
edits are preferably done in an MR_context (child of MR_defaultContext): Edits are not expensive and you should follow
the rule above. If you are calling a function that edits an NSManagedObject's properties from the main thread (like at the tap of a button)
you should update the main context. Saves on the other hand are
expensive and this is why your main context should not be linked to a
persistent store directly but just push its edits down to a root
context with background concurrency owning a persistent store.
long running tasks such as a server sync are preferably done using saveWithBlock Yes.
Now, In attempt 1
for i in 1...1_000 {
let user = User.MR_createInContext(context) as User
}
context.MR_saveOnlySelfWithCompletion(nil)
There is no need to save for every object creation. Even if the UI was not blocked it is wasteful.
About MR_context. In the documentation for magical records I cannot see a 'MR_context' so I am wondering if it is a quick method to access the main context. If it is so, it will block.

Resources