I am downloading json data from server, parsing it and storing it in to the database using CoreData. So next time when the view is loaded, it fetch the data using Core Data and along with that parsing and db insertion also happens. Before insertion, I am deleting the existent data also. Following is the flow that I am using:
Checks for data in database.
If available, fetch and render data in UI. And downloading data from server is also do along with fetching and rendering.
Delete the existing data and insert again.
The above steps crashes the application some time. What is the best approach to handle this case?
What I felt, sometimes at the time of fetching, download completes and deletes data. It may be depends on the speed of fetching.
That's why I got the question in mind that NSPersistentStore is thread safe?
Can anyone help to understand what approach to follow in this case?
Sree.
Related
I'm building an app that gets posts from a WordPress blogging site and display on a tableView. Each table view cell displays the post image, title and excerpt text. That's a course project, and use of Core Data is required. So my question is, for a better user experience, should I display the image on the cell straight after downloaded and then save to the Store or should I save to the Store, fetch, and then display?
Some considerations:
when the app launches it will check the internet connectivity, if the connection is establish, the store will be cleaned and the latest post will be downloaded.
It will download 5 posts at a time
Scrolling up will perform the download of older posts.
This is a opinion oriented question. The best I can do is to let you know how I design my apps to deal with this case in the past.
My solution :
Use NSFetchedResultsController to read the data from core data and use the NSFetchedResultsController's fetchedObjects array as a data source to your UITableViewController or UICollectionViewController
Implement the delegates of NSFetchedResultsController which will get triggered when data in Core data changes. This way u can efficiently update your CollectionView and TableView and show the data changes on UI ASAP.
Use Background Contexts to modify the data that way your main thread will be free and application remains responsive.
In order to create Background Context, I prefer parent child context architecture rather than traditional multi context architecture. parent child context architecture is easy to understand, keeps the code clean.
Never save image into core data. Instead save the image downloaded to document directory and save the relative link to the downloaded files in CoreData.
Remember I said relative path to deleted file not the absolute file. Because your application folder/sandbox path changes on killing and relaunching the app. Saving the absolute path to the file in core data is a perfect recipe to screw up the logic.
Don't worry about the delay in saving data to core data. The delay might be in fraction of second which you won't even notice. Saving the data in a array and then fetching the data from core data and updating array is a complete no.
Personally, using an array to save data instead of NSFetchedResultsController's fetchedObjects is a complete NO. why ? Simple, Array is not thread safe, because you will background thread to make web service call and parse data you might have multiple threads accessing Array simultaneously. Because Array isn't thread safe, you might easily get into state of data inconsistency and might lead to crash as well.
Finally use libraries like SDWebImage to efficiently load the image. SDWebImage will not only loads the images but also caches them at various levels (In RAM and HardDisk as well) there by loading images fast and swift.
If you are planning to use pagination to fetch data, use scrollView's delegate of scrollViewDidScroll to figure out when user scrolls to bottom of the table/collectionView and make web service call and fetch data in background thread update core data. As soon as you update mainObjectContext NSFetchedResults controller's delegate will be triggered and you should be able to update the UI immediately.
I am confused when to use
1) Asynchronous fetch
2) multiple managed object context
in core data while fetching large number of records, please help.
You want to use Aysnch fetch when you are remotely pulling data. Since that data pull will take time (if on main thread) your app will freeze until it has got the requested data. The user will think that the app froze and nothing is happening when in reality it is fetching data to display.
You would use multiple managed object context when you have to modify data in your core data data base. Refer this answer for more.
You do the first one for your network requests and second one when modifying data in your database managed by core data.
My question surrounds around one single point - data management in mobile application. I have created a mobile application where data comes from server. The data includes both text and images. Following are the steps I am doing for this :
First launch :
1. Get server data.
2. Save server data in Sqlite database.
3. Show Sqlite data.
Next launches :
1. Show Sqlite data.
2. Get server data in background.
3. Delete previous Sqlite data.
4. Save new server data in Sqlite database.
5. Show Sqlite data.
I have couple of questions on these steps :
1. Is this the right approach ? Other way could be showing data every time from server but that would not display the data on screen immediately (depending on internet speed).
2. I also thought of comparing the Sqlite data with the new server data. But faced a big challenge. The new server data might have new records or deleted records. Also, I could not find an appropriate approach to compare each database field with JSON data.
So what is the best approach to compare local Sqlite data with new server data ?
3. Each time I delete the Sqlite data and insert new data and then refresh the screen (which has a UITableView), it blinks for a second which is obvious. How to avoid this issue if steps 3, 4, 5 are followed ?
4. How should I proceed with data update in case I come back on the screen each time or when the application becomes active ? I am very aware of NSOperationQueues or using GCD for that matter. But what if I am crazy and go back and forth to screen again and again. There will be a number of NSOperations in the queue.
It's a challenge to synchronise server data, I've done that before, and if you can spend time on it I'd say it's the best solution.
You may need creation and modification dates on both server and local objects, to compare them - this will let you decide which objects to add, update and delete.
If the server sends you only the recently updated objects you can save a lot of traffic and improve performance (but deleted objects will be harder to detect).
If the data is only changed in the server it's easier, when the app can change the data too it becomes more complicated (but it seems that it's not your case). It also depends on how complex the database is, of course.
If you don't want to invest some time in doing this, just fetching all data everytime works too, even if it is not ideal! Instead of showing the old data and blinking it, you can just make the user wait 2-3 seconds when entering, while you get the new data. Or instead you can fetch the data only when starting the app, and so when you get to that view controller it will be ready already.
It's a complex problem that everyone faces at some point, so I'm curious to see what other people will suggest :)
This is a good question.
I personally think downloading data, store locally and later try to sync is a dangerous scenario. Easy to introduce bugs, master <-> slave issues (what data should be master, if multiple devices would be used etc.)
I think something like this could be a working approach:
1. I would try to look at possibilities to lazy load the data from the server on-demand. That is when a user have a View that should display data, load that specific data with the creation of that specific View. This ensures the data is allways in sync.
2. Tackling the need to reload data from server from every view, could be done by simply storing the downloaded data as objects in memory (not using SqlLite). The view will try to load the needed data trough your cache manager, and it would serve it from memory, if available. If not in memory simply get the data from your server and add it to your memory cache.
The memory cache could be a home made data manager wrapping a Dictionary stored on you AppDelegate, or some global "Singelton" to wrap the cache management/storing and data loading.
3. With lazy loaded data and memory cache you would need to make sure any updates (changes, new records, deleted records) updates your memory data model, as well as pushing these changes to the server as soon as possible. Depending on data size etc. you could force the user to wait, or do it directly as background process.
4. To ensure the data is in sync, you should make sure that you periodically invalidate (delete) the local memory records in the cache and thereby force data updates from the server. Best approach would probably be to have a last updated timestamp for each record in the memory cache. So the periodical invalidator would only delete "old records" from the memory cache (once again not from the server).
To save server from unnecessary data load, the data should still load on demand when the user needs it in a view, and not as part of "cache invalidation".
5. Depending on the data size you might need to look at "cache invalidation". Could be as simple as when xx records are stored, start deleting old objects from memory cache (not server, only locally on device).
6. If data sync is absolutely critical you might want to look at refreshing your memory cache for a record, just before you allow the user to change data. E.g. when user taps "Edit" or similar, you grab the latest data from server for that record. This is just to make sure the user is not going to update a record using outdated data and thereby accidentally overriding any changes made remote, or on another device etc.
--
My take on it. I do not believe there is a "perfect right way" to do this. But this would be what I would try to do.
Hope this will help with some ideas and inspiration.
How about this:
If data exists in SqlLite, load into "in-memory" copy and show it
In background load new server data
delete old sqlite data if it exists (note that the in-memory copy remains)
save new server data to sqlite
load new sqlite data into "in-memory" copy and show it.
If no data was found in step 1, display a "loading" screen to the user during step 2.
I'm making the assumption that the data from SqlLite is small enough to keep a copy in memory to show in your UITable view (The UITable view would always show data from in-memory).
It may be possible to combine steps 4 and 5 if the data is small enough to hold two copies in memory at the same time (you would create a new in-memory copy and swap with the visible copy when complete).
Note:
I don't talk about error handling here, but I would suggest that you don't delete the sqlite data until you have new data to replace it with.
This approach also eliminates the need to determine if this is the first launch or not. The logic always remains the same which should make it a little easier to implement.
Hope this is useful.
You can do same things more efficiently by MultiVersion Concurrency Control (MVCC), which uses a counter (sort of a very simple "time stamp") for every data record, which is updated whenever the record is changed means you need to get those data which is Updated after last sync call that reduces lots of redundant data and bandwidth.
Source: MultiVersion Concurrency Control
I have a problem with RestKit 0.22 and Core Data for saving. To be sure that the rest of my project doesn't cause the problem, I reproduce it with RKTwitterCoreData in RestKit examples. I simply remove seeding part to start with an empty sqlite database. My app is based on the same concept.
My problem happens the first time I run the app. The app receives the data from twitter and displays it. But if I look at the sqlite database itself, it's empty. When I start the app the second time, the database is filling.
Is it possible for RestKit to save the data at the moment it receives it?
Is it possible that RestKit cache all data before saving them. I though that my problem happened because managedObjectCache in RKManagedObjectStore is set with RKInMemoryManagedObjectCache but I also tried with RKFetchRequestManagedObjectCache and I got the same result.
I also tried to save manually in the success bloc with [[[RKManagedObjectStore defaultStore] persistentStoreManagedObjectContext] save:nil] and that didn't work.
Thank you
RestKit does save the mapping result automatically, and before calling the success completion block.
If you want to check what is in the store, ask the MOC (main thread or persistent store MOC) and check the registeredObjects / run a fetch request.
You should not look at the SQLite file(s). The table structure is a private implementation detail, and if you load the file at the command line with the wrong options you won't necessarily see what you expect.
I'm developing an app with Core Data that periodically downloads all the data from a webservice. Since the download can fail or be cancelled by the user, I want to be able to roll back to the previous state. I tried undoing the NSManagedObjectContext, but that seemed a bit slow (I have tens of thousands of entities). What I'm doing right now is making a backup of the persistent store file, download the data, and, if the download fails, replace the store file with the backup. This seems to work correctly, except there seems to be a delay after I can fetch entities from the store: if after the download I go immediately to a UITableView that uses an NSFetchedResultsController, I find it empty. If I wait some seconds, everything is ok.
So my question is: has anyone had this kind of delays too? Is there something that can be done to avoid this problem, something that forces everything to be ready, even if it blocks the thread?
I haven't used this setup but I think the delay you are seeing is probably caused by Core Data having to clear all it's caching. Core Data uses If you use a cache with the fetched results controller it will have to test and then delete it's existing cache.
I think the best thing to do is to tear down you Core Data stack and reboot it from scratch. That includes recreating a fresh fetched results controller.