I have what I would presume is a very common situation, but as I'm new to iOS programming, I'm not sure of the optimal way to code it.
Synopsis:
I have data on a server which can be retrieved by the iPhone app via a REST service. On the server side, the data is objects with a foreign key (an integer id number).
I'm storing the data retrieved via REST in Core Data. The managed objects have an "objId" attribute so that I can uniquely identify the managed objects in the rest of my code.
My app must always reflect the server data.
On subsequent requests made to the server:
some objects may not be returned, they have been deleted on the server - in which case I need to delete the corresponding objects from Core Data - so that I'm reflecting the state of the server correctly.
some objects have attributes which have changed, therefore the corresponding managed objects need updating with the new data.
my solution - and question to you
To get things going in my app, I made the easiest solution of deleting all objects in Core Data, then adding all new objects in, created with the latest server side data.
I don't think this is the best way to approach it :) As I progress on with my app, I now want to link up my tableview with NSFetchedResultsController, and have realised that my approach of deleting everything and re-adding is not going to work any more.
What is the tried and trusted way of syncing Core Data with server side data?
Do I need to make a fetch request for each object id I get back from the server, and then update the object with the new data?
And then go through all of the objects in core data and see which ones have not been updated, and delete those?
Is that the best way to do it? It just seems a little expensive to do a fetch for each object in Core Data, that's all.
Pseudo code is fine for any answers :)
thanks in advance!
Well, consider your download. First, you should be doing this in a background thread (if not, there are lots of SO posts that talk about how to do that).
I would suggest that you implement what makes sense first, and then, after you can get valid performance data from running Instruments, consider performance optimization. Of course, use some common sense on "easy" performance stuff (your design can take care of the big ones easily enough).
Anyway, get your data from the online resource, and then, for each object fetched, use the "unique object id" to fetch the object from core data. You know there is only one object with that ID, so you can set fetchLimit to 1 on your fetch request. You can also configure your "object id" attribute to be an INDEX in the database. This way, you get the fastest search from the underlying database, and it knows to stop looking once it finds your one object. This should be pretty snappy.
Now you have your object. Change any attributes necessary. Save, rinse, and repeat.
Furthermore, for several reasons, you may want to know when objects were last updated. I'd suggest adding a timestamp to each object that gets changed with the current time every time an object is changed. This will also help in deleting objects. Since your online database does not tell you which objects are deleted, you must have some way to know that an item is "old and no longer needed."
An easy way to do this is to remember the time you started your update. After processing all objects from the download, you now have a way to find all the objects that were deleted from the online database. Basically, any object with a "last update" timestamp before the time you began the update should be removed (since they were not added or modified in the last update). You can also index the database on this field, which will make finding those objects faster - unless your database is huge, I'd wait to see what Instruments has to say about this one though.
Related
I'm in the middle of adding an "offline mode" feature to an app I'm currently working on. Basically the idea is that users should able to make changes to the data, for example, edit the description of an item, without being connected to the internet, and the changes should survive between app launches.
Each change would normally result in an API request when working online but situation is different in offline mode.
Right now this is implemented by storing all data coming from the API in a Core Data database that acts as a cache. Entities that can be edited by user in addition to normal attributes have the following ones:
locallyCreated - whether the object was created offline
locallyDeleted - object was deleted offline
locallyUpdated - updated
This makes it possible to look for new/deleted/updated objects and send corresponding API requests when doing sync.
This worked well for creating and deleting objects, however, one disadvantage I found with this approach is when new data is retrieved from the API all local changes (i.e. attributes of objects marked as locally updated) are lost, which means that they have to be stored separately somehow.
What would be the best way to approach this problem?
Since you have your locallyUpdated key, the obvious answer is to modify your code that imports server changes, so that it doesn't overwrite changes to any object marked as changed. One way or another you need to avoid overwriting those changes, and you're already keeping a record of which objects have changes, so you already have the tools for a basic solution.
But you'll soon run into the complexity of syncing data. What if the local object has changes on one key, but the incoming data from the server has changes on a different key? You can't resolve that just by knowing that the local copy has changed somehow. Maybe you decide that the server always wins, or that the local copy always wins. Those are easy, if they make sense for your app. If you need to merge changes though, you have some work ahead of you. You would need to record not only a Boolean value indicating that changes were made, but also a list of which keys had changed. This can get complicated, but it's the nature of data syncing.
I am working on an application where I have a connection to a database. The database contains from 300MB to 4GB worth of data as each customer has their own database. My issue that I am having is in gathering the data, because of the potential database size, just downloading and storing the information locally isn't possible. The data can get quite complex and can vary. For an example:
A customer has a Job and they want to search for that job from the app.
I then fetch a list of jobs matching the search criteria.
The customer sees the job they want to view and I start the gathering process.
This job can potentially touch many tables, sometimes repeatedly..
There is the jobs table, a relational table to map to a person. Then there is another table that contains non-customer relational information, then there are calendar events associated to the job, which in tun can associate different people. Then there are emails attached to the job, which in turn can bring in additional people and events.
So I have a working model that gathers all of this information. The problem I have is that I cannot figure out a great method of signaling to my view that the data is completely downloaded. My initial thought was to use the NotificationCenter to message when the certain parts of the task were finished, allowing the core Job object to notify the view when everything was complete.
I know this is a pretty generalized question, but I'm honestly stumped as to how to take an unknown number of table results and translate that into a notice that my app can actually use.
My initial recommendation would be Core Data. It's designed for this kind of problem. No, I'm not saying to download the entire database into Core Data. I'm saying to use Core Data to manage your object model, because that's what it's good at.
As you receive data from the server, compose it into NSManagedObjects and stick them in the data store. On the UI side, create an NSFetchedResultsController to keep you informed as the data updates asynchronously. You don't necessarily need to persist this store. You could just keep it in memory and throw it away whenever you're done with the query, but keeping it on disk could be a nice caching solution. Again, don't think of Core Data as "a local database." Think of it as a model persistence engine that you can query for objects.
One advantage of this model is that you can provide the best available data to the user as it becomes available. But say you really don't want to get the information until it's all available. That's fine, too. Just let the network side keep updating its context, and then only save it when everything's complete. That way NSFetchedResultsController gets a single atomic update. The nice things with Core Data is that it has these concepts built in, so you can adjust your update strategy without requiring massive redesign.
The Notification Center will work great for this.
Post the notification at logical points in your data load to trigger a UI update for your users.
Design question:
My app talks to a server. Json data being sent/received.
Data on server is always changing, and I want users to see most current data, not stored/cached data. So I require a user to be logged in order to use the app, and care not to persist data in the app.
Should I still use CoreData and map it to Json's.?
Or can I just create custom model classes and map Json's to it's properties, and have nsarray properties, which point to its child objects, etc. ?
Which is better?
Thanks
If you dont want to persist data, I personally think core data would be overkill for this application
Core Data is really for local persistance. If the data was not changing so often and you didnt want them to have to get an updated data everytime the user visited the page, then you would load the JSON and store it locally using CoreData.
Use plain old objective-c objects for now. It's not hard to switch to Core Data in future, but once you've done so it gets a lot harder to change your schema.
That depends on what your needs are.
If you need the app to work offline, you need to store your information somehow in the client.
In order to save on network usage, you could store locally, then query the server to see if it had an updated answer -- you could do this by sending a time stamp to the server and return a 304 Not Modified if the entity hasn't changed.
Generally, it depends on how much time you have to put into the app and what your specific requirements are, but as a general rule I would optimise for as low bandwidth usage as possible, as that not only reduces potential data costs, but also means the answers will be more quickly available to your users (when online and they have not changed) and also available offline.
If you do not wish to store data locally at all,
I use Core Data to store objects in my app. Basically I store objects valuable to user, for example things that user liked or places where he wants to go. There is a hidden relationship between every of these objects and user himself. Everything is perfectly logical. But now I want to store (cache) some data that is not directly linked to user, for example result of search requests. These objects mostly are used only in one particular place and could have some expire time like cache or something like that. So I need to store these objects but be able to remove them from storage in future. The thing is that these objects are of the same entity as my valuable objects. Another thing is that there could be a relationship between these non-valuable objects and some valuable objects.
What is the best way to store such non-valuable objects and, most important, to clean Core Data Storage from them?
It depends on the use you need for these items.
basically, these are items that have an expiry date (temporary objects), you could use a timestamp on each object, or you could create an new entity relating to the items (one-to-one relationship) that you can query and delete by under conditions you specify.
When you open the store coordinator, you could have a cleanup rule (that you will need to implement) it will run in the background and remove items that are no longer relevant (make sure not to display them to the user, or access them when you perform the cleanup).
try to keep your cleanup in the background and perform it at times which will not disturbe the user.
in my own opinion, it is better to mark the items as hidden, then deleting them immediately as deletion is a heavy operation that might block the UI. also try and keep your deletions small (not all objects at once, but small batches of objects)
I have records that are added, updated. Then sync them with server.
According to server response, if one of them fail to update, I would like to have that NSManagedObject to previous value. As I research, UndoManager works as stack, so I can't find any record with Id and undo that record, am I right?
And finally, what would you suggest for this issue?
You could track your objects by introducing your own ID attribute and syncing that with the server. I think this is a solid and robust design - I have used it many times without problems.
Apple does provide an objectID with each managed object, but this is really meant to ensure consistency of data across different managed object contexts. I would not recommend "abusing" this ID for external systems.
Your server could provide the old values (along with the message that it was not updated) and you could write that back into your Core Data store, finding the record using your ID attribute. For more granular change and update management, you could even use a time stamp attribute.