I am working on an application which initially fetches some data from the backend service and store in the app sqlite using core data. The managed objects undergo many changes during the app usage. In a timely manner the data should be synchronised with the backend service.
I tried to use two managed object context, one for user interface interaction and one for synchronisation with backend so that I can merge the context. But the context used by the UI is not saved to the persistent storage every time and hence when i fetch using the second context there is mismatch in the managed objects.
Secondly I tried using the parent child context. Parent context for UI interaction and child for timely data synchronisation. But sometime I see the UI gets blocked forever. I have made sure that actions on the managed objects are executed in a block on respective managed object context.
My question is, how can I handle the new data received from the service and update the managed object without blocking the user interface.
Here's a post that describes a few possible setups. I'll summarize some of the main points and solutions.
Your second attempt failed because your UI (parent) context was still doing store I/O on the main thread, which is why it was blocking.
Solution 1 - Keep your contexts separate like you tried first.
Change propagation between the contexts is achieved by subscribing to the NSManagedObjectContextDidSaveNotification and calling mergeChangesFromContextDidSaveNotification: on the other context.
So you subscribe to the notification from the private service context; and then call -mergeChanges... on the UI context, which keeps them synchronized and is very fast.
Solution 2 - make your UI context a child of a third Private Queue context. In other words, your stack is Persistent Store <- New Private Context <- UI (Main Queue) Context <- Private synchronization context. This moves your disk I/O to the background which should keep your UI responsive. Note however, that now you also have to call save on the UI context and the new context to persist changes from the background; plus this approach is slower than Solution 1
Related
I'm in the middle of development of an iOS application after working quite some time with Core Data and Magical Record and having an error:
error: NULL _cd_rawData but the object is not being turned into a fault
I didn't know Core Data before this project and as it turns out I was very naive to think I can just work with Magical Record without worrying about concurrency, as I haven't dedicated any thoughts/work about managed contexts for the main thread and background threads.
After A LOT of reading about Core Data Managed Object Contexts and Magical Record, I understand that:
NSManagedObjects are not thread safe.
NSManagedObjectId IS thread safe.
I can use: Entity *localEntity = [entity MR_inContext:localContext] of Magical Record to work with an entity in a background thread's context.
I should use Magical Record's saveWithBlock:completion: and saveWithBlockAndWait: methods to get a managed context to use for background threads.
A little information regarding my application:
I'm using the latest version of Magical Record which is 2.2.
I have a backend server which my application talks to a lot.
Their communication is similar to Whatsapp, as it uses background threads for communicating with the server and updating managed objects upon successful responses.
I'm wrapping the model with DataModel objects that hold the managed objects in arrays for quick referencing of UI/background use.
Now - my questions are:
Should I fetch only from the UI thread? Is it okay that I'm holding the managed objects in DataModel objects?
How can I create a new entity from a background thread and use the newly created entity in the DataModel objects?
Is there a best design scenario that I should use? specifically when sending a request to the server and getting a response, should I create a new managed context and use it throughout the thread's activity?
Let me know if everything is clear. If not, I'll try and add clarity.
Any help or guidelines would be appreciated.
Thanks!
I'm not working with MagicalRecord, but these questions are more related to CoreData than to MagicalRecord, so I'll try to answer them :).
1) Fetching from main(UI) thread
There are many ways how to design app model, so two important things I've learned using CoreData for few years:
when dealing with UI, always fetch objects on main thread. As You correctly stated NSManagedObjects are not thread safe so you can't (well, can, but shouldn't) access their data from different thread. NSFetchedResultsController is Your best friend when you need to display long lists (eg. for messages – but watch out for request's batchSize).
you should design your storage and fetches to be fast and responsive. Use indexes, fetch only needed properties, prefetch relationships etc.
on the other hand if you need to fetch from large amount of data, you can use context on different thread and transfer NSManagedObjectIDs only. Let's say your user has huge amount of messages and you want to show him latest 10 from specific contact. You can create background context (private concurrency), fetch these 10 message IDs (NSManagedObjectIDResultType), store them in array (or any other suitable format for you), return them to your main thread and fetch those IDs only. Note that this approach speed things up if fetch takes long because of predicate/sortDescriptor not if the "problem" is in the process of turning faults into objects (eg. large UIImage stored in transformable attribute :) )
2) Creating entity in background
You can create the object in background context, store it's NSManagedObjectID after you save context (object has only temporary ID before save) and send it back to your main thread, where you can perform fetch by ID and get the object in your main context.
3) Working with background context
I don't know if it's exactly the best, but I'm pretty much satisfied with NSManagedObjectContext observation and merging from notifications. Checkout:
mergeChangesFromContextDidSaveNotification:
So, you create background context, add main context as observer for changes (NSManagedObjectContextObjectsDidChangeNotification) and background context automatically sends you notifications (every time you perform save) about all of it's changes – inserted/updated/deleted objects (no worries, you can just merge it by calling mergeChangesFromContextDidSaveNotification:). This has many advantages, such as:
everything is updated automatically (every object you have fetched in "observing context" gets updated/deleted)
every merge runs in memory (no fetches, no persisting on main thread)
if you implement NSFetchedResultsController's delegate method, everything updates automatically (not exactly everything – see below)
On the other side:
take care about merging policy (NSMangedObjectContext mergePolicy)
watchout for referencing managed objects that got deleted from background (or just another context)
NSFetchedResultsController updates only on changes to "direct" attributes (checkout this SO question)
Well, I hope it answers your questions. If everything comes up, don't hesitate to ask :)
Side note about child contexts
Also take a peek to child contexts. They can be powerful too. Basically every child context sends it's changes to parent context on save (in case of "base" context (no parent context), it sends it's changes to persistent coordinator).
For example when you are creating edit/add controller you can create child context from your main context and perform all changes in it. When user decides to cancel the operation, you just destroy (remove reference) the child context and no changes will be stored. If user decides to accept changes he/she made, save the child context and destroy it. By saving child context all changes are propagated to it's parent store (in this example your main context). Just be sure to also save the parent context (at some point) to persist these changes (save: method doesn't do the bubbling). Checkout documentation of managing parent store.
Happy coding!
So I've been playing with this "issue" for a long time here and I keep getting into issues:
Here is more or less what I would like for my dataflow
User Segues to Controller
Controller pings FTP site (if up continue)
Loop through each parent record
Parse each record (this is a slow process) turning it and all sub data into a giant JSON structure which is stored in a temp directory
Update the Screen to the user with the progress of the parsing task
Upload the JSON Structure via FTP
Once successfully uploaded update the "isUploaded" field on the record
Now where I've go into all sort of issues is with keeping the UI updated. I'm designing a somewhat simple UI where we have a status bar that shows the process of each JSON file parsing task. I can make things work but once I' try to get a nice responsive UI i've run into all sorts of issues.
I understand I'm supposed to do CoreData on the main thread, but, in doing so my UI doesn't update and becomes unresponsive.
I segue a NSManagedObjectContext over to this controller and I understand from that objectContext I can likely reference the NSPersistantStoreCoordiantor and create a second NSManagedObjectContext for a second threat and somehow I can likely synchronize these threads.
I've gotten things into a mess with performSelectorOnMainThread and performSelectorInBackground calls all over the place etc.
Either I get things working the way I'd like or I get some sort of error with my managedObjectContext's and I'm thinking perhaps its time for a little rewrite instead of trying to salvage what I have.
Can somebody point me to some ideas about what i should use? I can't seem to exactly wrap my head correctly around the multithreaded core-data concepts as to whether I'm supposed to use NSOperationQueues GCD or some other way of doing things.
CoreData has the ability to operate in a multithreaded environment, as long as you keep managed objects and contexts bound to their proper thread/queue.
To scrape the surface see HERE
You can create a background context by initialising it as a "private queue" bound context (see HERE for more details).
Basically, this would mean that in order to use this context properly you have to execute code on it wrapped in the contexts performBlock: or performBlockAndWait: methods.
you can use these methods to queue "tasks" for the context to execute (serial execution queue).
this code will be executed in the background.
You could also create a context that is confined to a specific thread (NSConfinementConcurrencyType) and then only access it in the thread it was created in without the use of the "perform block" methods.
merging the changes in this context to your main context is done by either setting the main context as a parent for the context after initialisation, or by registering to its "did save" notification and merging the changes to the main context (the first method is easier to implement).
One solution to your flow would be:
Create a main queue context (to be passes around to all view controllers)
Create an operation queue
observe a status entity (fetched in the main context) in your view (directly or by using a FRC)
each operation would create a "private queue" context with the main context as parent
when updates are ready to be committed (including status updates), save the private context and the main context using performBlockAndWait: (just to keep the operation coherent)
This will still affect your main thread if there are many updates as all saves will go through the main context.
Try to segment your saves to avoid locking the main context for too long.
I am developing an application that uses Core Data for internal storage. This application has the following functionalities :
Synchronize data with a server by downloading and parsing a large XML file then save the entries with core data.
Allow user to make fetches (large data fetches) and CRUD operations.
I have read through a lot and a lot of documentation that there are several patterns to follow in order to apply multithreading with Core Data :
Nested contexts : this patterns seems to have many performance issues (children contexts block ancestor when making fetches).
Use one main thread context and background worker contexts.
Use a single context (main thread context) and apply multithreading with GCD.
I tried the 3 mentioned approaches and i realized that 2 last ones work fine. However i am not sure if these approaches are correct when talking about performance.
Is there please a well known performant pattern to apply in order to make a robust application that implements the mentioned functionalities ?
rokridi,
In my Twitter iOS apps, Retweever and #chat, I use a simple two MOC model. All database insertions and deletions take place on a private concurrent insertionMOC. The main MOC merges through -save: notifications from the insertionMOC and during merge processing emits a custom UI update notification. This lets me work in a staged fashion. All tweets come in to the app are processed on the background and are presented to the UI when everything is done.
If you download the apps, #chat's engine has been modernized and is more efficient and more isolated from the main thread than Retweever's engine.
Anon,
Andrew
Apple recommends using separate context for each thread.
The pattern recommended for concurrent programming with Core Data is
thread confinement: each thread must have its own entirely private
managed object context. There are two possible ways to adopt the
pattern: Create a separate managed object context for each thread
and share a single persistent store coordinator. This is the
typically-recommended approach. Create a separate managed object
context and persistent store coordinator for each thread. This
approach provides for greater concurrency at the expense of greater
complexity (particularly if you need to communicate changes between
different contexts) and increased memory usage.
See the apple Documentation
As per apple documentation use Thread Confinement to Support Concurrency
Creating one managed object context per thread. It will make your life easy. This is for when you are parsing large data in background and also fetching data on main thread to display in UI.
About the merging issue there are some best ways to do them.
Never pass objects between thread, but pass object ids to other thread if necessary and access them from that thread, for example when you are saving data by parsing xml you should save them on current thread moc and get the ids of them, and pass to UI thread, In UI thread re fetch them.
You can also register for notification and when one moc will change you will get notified by the user info dictionary which will have updated objects, you can pass them to merge context method call.
Scenario:
I am on IOS using Magical record configured to operate against a SQLite database. By default, MR configures coredata to serialize all writes back to the parent context on the main thread.
The pattern I use is that when I am not on the main thread I create a separate NSManagedObjectContext for coredata operations using something like MagicalRecord:MR_saveWithBlockAndWait. Magical record creates the context, hooks it up to the parent context, performs whatever operations you specify in the callback block and finally saves. Importantly, the save is supposed to be committed before the operation finishes.
When I am done working on the background thread I usually notify the UI that something has happened; e.g: something is downloaded/uploaded/changed.
On the UI thread I then create a new fetch request using the default context on the main thread. The problem is that occasionally coredata doesn't find the new object I just committed previously. The problem manifests itself in subtle race conditions where if the UI thread is slightly slow due to animations or whatever everything works fine - but sometimes it doesn't find the new object.
From what I have read fetch requests are always supposed to go to the disk. There is also a staleness property on the MOC but it sounds like that is only regarding the cache and is bypassed if you do a fetch request.
Has anyone encountered similar issues and have any insights? thanks.
Sure. If you save changes on a background managed object context, but your UI context has already loaded that object, the UI context may just give you data from its cache instead of from the store file.
The usual approach to using multiple contexts is:
Observe NSManagedObjectContextDidSaveNotification so that you'll know when the background context saves changes.
In your handler for this notification, call mergeChangesFromContextDidSaveNotification: on the UI context, so that it will update itself with changes from the other context.
You probably want to set the mergePolicy on your UI context, because the default is to just give up if there are any conflicting changes.
This applies to any multiple context scenario where each context needs to be updated with changes saved by a different context.
I'm running into some trouble working with CoreData in a multithreaded app using NSOperations. I am using nested ManagedObjectContexts through MagicalRecord (2.0.3) as follows:
Root Context (saves to disk)
|
Main Thread Context (for populating the UI)
|
Sub-Context(s) (used to add/edit/remove data)
I have a single NSOperationQueue to handle all data processing.
For the most part, things work right, I can asynchronously download data, then feed it to an NSOperation which then writes it to one of the sub-contexts. Saving at the end of the operation pushes the changes to the main context and the UI updates. Great!
The problem is that if a sub-context deletes an entity and saves (pushing it to the main context), a sibling sub-context will still think that it exists. So then if a sibling tries to fault the entity and pull it from it's parent (the main context) I get a crash.
I have 2 questions:
Should I use MOC notifications to merge the changes pushed to the main MOC back to it's other children? I this and was getting another crash...
Should I even have mutltiple sub-contexts? MOC's are supposed to be associated with a single thread (MagicalRecord helps automate this for me), and I have a single NSOperationQueue for saving data, so shouldn't I only have 1 sub-context? I've verified that sometimes my saves are performed by different contexts.
I'd appreciate any advice. Thanks.
You can, and should, have multiple sub contexts. However, I'm not sure if the classic "thread isolation mode" of contexts is the model you should have anymore. That is what you're doing when you say that a context should belong to a particular thread. MagicalRecord 2.0x uses private queue contexts now, and as such will behave a tad differently. There are no rules saying that sibling contexts need to stay in sync. You'd have to do that yourself. A very simple solution would be to listen to "did save" notifications on the context you're saving, and either reset, or create a new context on the other thread. You can do this with notifications, or a completion black that MagicalRecord provides.
Hope this helps