The problem is that I want to get all of the objectIDs and I want to minimize data transfer in-between the server and iOS app. So, let's say I have a dozen of thousands of PFObjects on PARSE, they are updated and deleted and my app needs to update its knowledge about what objects are present on PARSE without using push-notifications (I am handling such a case when user disallows them). I can not just load all of the PFObjects every time my UIViewController presents data, retrieve PFObjects' ids and start checking whether my local store (not PARSE Local Datastore, different one) has such ids, since PFObjects themselves are large and there is a plenty of them, but it is ok for me to just load all of the objectIds. So, how to do this, and is it possible at all?
Some method of PFQuery like getAllObjectIds would be very helpful, but there seems to be no such methods.
You can solve the Add and Update situation but for the Delete its easier t use the straightforward solution and periodically request all object.
Here is a solution for the Update/Add object case:
and save the most recentUpdated Date
In the first request to parse set order object by updatedAt:
[query orderByDescending:#"updatedAt"];//orderByDescending
For any futur query set greaterThan:mostRecentUpdatedAt to get only updated and added objects:
if ([[NSUserDefaults standardUserDefaults] objectForKey:#"mostRecentUpdatedAt"]){
NSDate* mostRecentUpdatedAt = [[NSUserDefaults standardUserDefaults] objectForKey:#"mostRecentUpdatedAt"];
[query whereKey:#"updatedAt" greaterThan:mostRecentUpdatedAt];
}
save mostRecentUpdatedAt for futur queries:
if (results.count) {
PFObject* firtObj = [results firstObject];
NSDate* mostRecentUpdatedAt = firtObj.updatedAt;
[[NSUserDefaults standardUserDefaults] setObject:mostRecentUpdatedAt forKey:#"mostRecentUpdatedAt"];
[[NSUserDefaults standardUserDefaults] synchronize];
}
For the Delete case you need a request to count the number or objects and compared to your count. this should be done immediately after checking for add/update since in this case if count mismatch then there is a delete situation. then get all objects again (no magic to be done to identify the deleted object since its already deleted!!).
You can not query the Parse DB for just the objectId's. You would have to pull the entire PFObject and then loop through and store just the id's and discard the rest.
Alternatively, I think you could create a PFRelation from the objects that you are interested in and the user to track just those objects.
Related
I'm currently using coredata for my project. But when the api returns 54000 objects that the app need to update, the user has to wait almost 2 hours.
It's the major problem for the current project and I am thinking to use sqlite and not using coredata anymore to update thousands of objects.
Is it a right decision to use Sqlite or is there any suggestion for CoreData? I can't decide. Any help will be great. Thank you.
Here is what I am doing:
NSManagedObjectContext *privateObjectContext = [AppDelegate appDelegate].privateManagedObjectContext;
[privateObjectContext performBlock:^{
int i = 1;
for (NSDictionary *item in itemlist) {
i++;
[fetchRequest setPredicate:[NSPredicate predicateWithFormat:
#"itemID == %#",[item objectForKey:#"item_id"]
]];
NSError *error;
NSMutableArray *inventories = [[NSMutableArray alloc]initWithArray:
[privateObjectContext executeFetchRequest:fetchRequest
error:&error]];
ItemManagedObject *itemMO;
if(inventories.count){
itemMO = inventories.firstObject;
}else{
itemMO = [NSEntityDescription insertNewObjectForEntityForName:#"ItemObject"
inManagedObjectContext:privateObjectContext];
}
[itemMO prepareWithDictionary:item];
}
NSError *error;
if (![privateObjectContext save:&error]) {
completionHandler(NO);
}
}
Core Data provides NSBatchUpdateRequest which allows you to make updates directly on the persistent store without involving instantiating and processing managed objects in memory.
You should run this code using the core data performance instrument as well. If itemList contains 54,000 objects then you are performing 54,000 fetches to the persistent store to check a single ID each time. It would be far faster to fetch all of the IDs up front and then check the results in memory than to perform repeated fetch requests - that code will be almost as slow in raw SQL as it is in Core Data.
This code also looks wrong:
ItemManagedObject *itemMO;
if(itemMO.count){
It's never going to pass that if test, unless you've missed a line somewhere.
2 hours is very long. That's weird.
Yet you can massage your code by having core data do less work. Much less work.
Perform a single fetch request instead of 54K fetch requests
Don't call a managed object property setter when a property value does not change, so that no object is unnecessarily flagged as dirty, and Core Data does not have to perform a costly but useless update of the object when the "save" method is invoked.
This will dramatically reduce the amount of work performed by Core Data, and the performance of your application.
The second point is easy, but very verbose: compare each individual property values with dictionary values before calling setters.
The first point requires an algorithm change:
Perform a single fetch request, sorted by id (with [NSFetchRequest setSortDescriptors:])
Sort dictionaries by id (with [NSArray sortedArray...])
Synchronize the two sorted lists (it is paramount that both lists are sorted):
NSEnumerator *itemMOEnum = [itemMOs objectEnumerator];
NSEnumerator *dicEnum = [dictionaries objectEnumerator];
ItemManagedObject *itemMO = [itemMOEnum nextObject];
NSDictionary *itemDic = [dicEnum nextObject];
while (itemDic) {
NSComparisonResult comparison = itemMO ? [itemDic[#"item_id"] compare:itemMO.itemID] : NSOrderedAscending;
switch (comparison) {
case NSOrderedSame:
// id present in both lists: update
[itemMO prepareWithDictionary:itemDic];
itemMO = [itemMOEnum nextObject];
itemDic = [dicEnum nextObject];
break;
case NSOrderedAscending: {
// id present only in dictionaries: create
itemMO = [NSEntityDescription insertNewObjectForEntityForName:#"ItemObject"
inManagedObjectContext:privateObjectContext];
[itemMO prepareWithDictionary:itemDic];
itemDic = [dicEnum nextObject];
} break;
case NSOrderedDescending:
// id present only in managed object: delete or do nothing
itemMO = [itemMOEnum nextObject];
break;
}
}
while (itemMO) {
// id present only in managed object: delete or do nothing
itemMO = [itemMOEnum nextObject];
}
And save.
Finally, maybe SQLite will be faster (see https://github.com/groue/GRDB.swift/wiki/Performance for an attempt at comparing the performance of Core Data with SQLite libraries).
But SQLite won't turn a slow algorithm into a fast one.
I've never redone a core data project in sqlite or visa versa. So I cannot tell you whether there is a performance difference or not/
However the 54k = 2 hours thing sounds very strange. You talk about an API which makes me suspect a server is involved, your question is about databases. Certainly 2 hours sounds way too long and makes me wonder whether you have issues with the core design of your database. For example, lack of indexes. Depending on your queries and database, a single update could be triggering all sorts of heavy duty processing.
Another though is why are you processing this column of data on a device. It's a lot to handle and I wonder if there are ways to reduce the volume down, selectively do updates or perhaps even better - move it to a server.
I think you need to rethink your question. Provide more context about the database, exactly what you are doing with it and why.
CoreData is not a database manager but a object graph and persistent manager. CoreData can store its objects in a sqlite database but also in XML files or binary file (the developer chooses the option best suited to its needs).
The main difference between CoreData and a database manager is that to access an object with CoreData, CoreData need to instantiate the objective-C/Swift corresponding object.
Sqlite can access part of data without having to extract the full record containing the data.
And then, CoreData need to maintain the relational graph between objects (the relationships between 2 CoreData classes, and in general, in both ways).
So, when updating 54k objects, you ask CoreData to instantiate 54k objects (in memory) and to eventually update their relationships.
That is very heavy work for CoreData on mobile.
Perhaps your CoreData model is not correctly optimized.
Perhaps you should save the CoreData context regularly and flush CoreData scratchpad (the part of memory containing actually read or updated objects).
But in my experience, CoreData is not suited to heavy data work.
Re-implementing your needs with sqlite can be quite some work if you want to be able to re-instantiate your classe objects from sqlite records and manage quite automatic relationship, but it is doable. I did it on some projects. This add the benefit to have a model object that is more shareable with other platform as Android for instance, as sqlite is available on many platforms.
One more thing: sqlite is more suited to be used from multiple threads. CoreData is more touchy about this, and need one context by thread, and eventually, some contexts synchronization.
I have a logic problem in my app using Parse, Regarding which path to choose to save in traffic, and if someone has already faced a similar problem, I will really appreciate the help. Also, you can end up helping other developers facing the same problem
I have a social app where there is a feed with objects, and users can bookmark ("favorite") these objects
I studied the Parse documents and concluded that among the pointers, relations and arrays, the best way to store favorite would be a objectId's array stored in the class of users. Each time an object is bookmarked by the user, the ObjectID of this object is stored in a objectID's array belonging to that user. The reason for the choice is:
It is easy to create the bookmark's view and show them to the user, since I just have to search for the user's ObjectID's array and finding those present in the class of objects
Saving only the objectID and not the entire object, I will save in traffic and I keep the app and traffic clean
But my logic problem is as follows. If user1 has created an object, and user2 bookmarked it, and then user1 decided to delete the object, I would have to search the objectID of this deleted object in each favorite array of each user!
So my question is, what would be less expensive for traffic of my App? Store the entire object when a user bookmark it, automating removal when a user deletes the object? Or just store the ObjectID, and perform the search on each array for each user when this object is deleted?
You can create Parse class for your feed objects. After that create an array column to store objectIds of users who added your feed item to favorit. When you want to find all the objects specific user bookmarked do something like
PFQuery *query = [PFQuery queryWithClassName:#"feedObjects"];
[query whereKey:#"favoritesArray" equalTo:#"YOUR USER OBJECT ID"];
[query findObjectsInBackground];
To remove object simply do
PFObject *feedItem = [PFQuery getObjectOfClass:#"feedObjects" objectId:#"ITEM TO REMOVE OBJECTID"];
[feedItem deleteInBackground];
Hope that helps :)
When I'm trying to save a PFObject without network connection it is successfully saved locally, whenever I start the app again with network connection, it seems that the object is going to be saved to the server, but all parameters are empty.
I'm doing the following procedure, first I create a PFObject with different values and call saveEventually. During these steps I do not have an internet connection (airplane mode is on), therefore it can't be saved to the server and was saved locally.
PFObject *contact = [PFObject objectWithClassName:#"Contact"];
[contact setObject:[PFUser currentUser] forKey:kRelatedToUserKey];
[contact setObject:self.firstname forKey:kFirstnameKey];
[contact setObject:self.lastname forKey:kLastnameKey];
[contact saveEventually];
Now, I'm closing the app and start it again, I'm loading my contacts like that. I'm getting back the correct object with all the correct value for firstname, lastname, etc.
PFQuery *postQuery = [PFQuery queryWithClassName:#"Contact"];
[postQuery whereKey:#"related_to_user" equalTo:[PFUser currentUser]];
[postQuery fromLocalDatastore];
[postQuery findObjectsInBackgroundWithBlock:^(NSArray *objects, NSError *error) {
if (!error) {
// here I get back the correct object from local storage, with all values
}
}];
Now, when I'm switching airplane mode off, it tries to save the object to the server, but all I can see in the backend is an objectId without any values.
Whereas the locally saved object has all the correct values.
So what are you trying to do exactly? Here's how it reads:
Your querying 'Contact' class and once you query it and find an object by it's ID (an object you don't do anything with [PFObject *contact]) then you create a PFObject for a completely different Class?? You could circumvent all that if you simply just want to post a PFObject, but maybe you omitted other code that wasn't relevant to the question? But ok. To answer your question saveEventually works hand-in-hand with local data store, so you shouldn't have any issues, as you can see, it get's called, but your values aren't being saved like the objectID is. The objectID is created autonomously, that's why it gets saved and nothing else. I have literally tried to duplicate your error in every way possible but can't, it's your values, they are returning nil. I even used macros (it looks like thats what your setting your keys as), emulated airplane mode etc. To verify, query your pinned object and see what it returns. Additionally, it's best when you do a callback to try to include an if statement or switch case that explicity defines it respectively for better practice :
{
if (succeeded) {
debt.parseID = newDebt.objectId;
}];
}];
Furthermore, be careful about putting essential tasks in the success block because an important element of saveEventually is that if it doesn't complete before the app is terminated and if the object is still in memory, it will try again, but if the object is no longer in memory it will try again the next run-time, but WITHOUT the success block.
Troubleshoot your property values (self.contact | self.amount | self.incomingDebt) how are you defining these
We've come a long ways from the original post, so in an effort to try and bring it back, the real and only issue here is saveEventually.
Saves this object to the server at some unspecified time in the future, even if Parse is currently inaccessible.
The main intent of saveEventually :
Use this when you may not have a solid network connection, and don’t need to know when the save completes
If more than 10MB of data is waiting to be sent, subsequent calls to will cause old saves to be silently discarded until the connection can be re-established, and the queued objects can be saved.
You have no control over when this gets called. Additionally, saving eventually caches the data on the local disk until it can be successfully uploaded, so pinning the same object is redundant. Saving eventually, if you think about it, is a local datastore of it's own, it's stores them on the local disk until internet is available (local datastore)
You have two ways that could circumvent this. Local datastore is a core-data-esque feature enabling users to forego NSFRC with a simple one-liner code pin/pinInBackground:. You could simply pin the objects and when you know there is internet again unpin and save to your backend. Alternatively you could do it the other way, call reachability immediately, and if there is no internet pin: the object, else saveInBackground: first. Or simply take advantage of their caching policies.
REFERENCES :
saveEventually : API Class Reference
Caching Policies : Documentation
It was a bug on the sdk.(1.6.2)
Submitted here: https://developers.facebook.com/bugs/1554492594835537/
I had a similar problem, I actually found that removing or not calling [Parse enableLocalDatastore]; resulted in saveEventually performing as expected (using Parse 1.6.2). I had assumed [Parse enableLocalDatastore]; would be required for this to work.
This bug is now fixed in Parse version 1.6.3!
Parse.com has a saveEventually option that stores data to disc and waits for a network connection to proceed and save it to server.
However, sometimes when a network connection is available and the user quickly switches screens, the new data is not yet saved to the server, causing views to show old data.
For parse.com users, this happens when I run a new query and the saveEventually call is not yet complete.
I would like to implement my own cache system.
I would like to call saveEventually with callback upon completion, so then I would delete the cached data.
This way, I could check first if there's cached data before making network connection.
I think I can do it using NSCache and NSDiscardableContent, I already read the docs, but it would really help me if I could see some simple sample code for creating these objects and storing them on user's device.
Any suggestions?
UPDATE:
OK, this an idea of workaround solution.
Every object that I call saveEventually on I will add to a NSArray and store it in documentsDirectory, then, upon saveEventually completion I will delete the array in documentsDirectory.
If the app closes before completion, I wouldn't get a callback but it wouldn't be a problem because I will perform the query on the server and get the stored NSArray in device as well, I would then create a unique array giving preference to the objects in the documentsDirectory.
I think this could work. My concern is if it will make my app slower.
Any thoughts?
I haven't tried this, but how about setting your query's cache policy, like:
query.cachePolicy = kPFCachePolicyCacheElseNetwork;
If saveEventually is using the regular parse cache, then it should do just what you want.
EDIT - Since the above is apparently not going to work, I can think of several schemes to employ your own cache, but they're all stuck with the fundamental race condition problem: say you save eventually, then query again. Say you've fixed your query code to grab the local object that's being eventually saved.
What if you modify that cached object and save it eventually (again)? Do we know that parse will correctly serialize the two saves?
If you're not worried about this (maybe the second query never generates a save, or you're willing to roll the dice on the race condition), then you have a lot of choices. Let's take the straight-forward one:
Create a singleton with an NSCache property. Give these methods which are in it's public interface...
// call this instead of saveEventually
- (void)cacheThenSaveEventually:(NSArray *)pfObjects {
for (PFObject *object in pfObjects) {
[self.cache setObject:object forKey:[object objectId]];
[object saveEventually:^(BOOL success) {
if (success) [self.cache removeObjectForKey:[object objectId]];
}];
}
}
// call this with the result of a query that might have been in a race with saveEventually
- (NSArray *)freshenUpResults:(NSArray *)pfObjects {
NSMutableArray *fresherObjects = [NSMutableArray array];
for (PFObject *object in pfObjects) {
PFObject *fresher = [self.cache objectForKey:[object objectId]];
[fresherObjects addObject:(fresher)? fresher : object];
}
return [NSArray arrayWithArray:fresherObjects];
}
What kind of database do you suggest? I want to store user email, username, password, and a couple other random pieces of information. It doesn't have to be fancy. Just a simple database. Are there any free options?
The user information needs to be stored in the keychain to keep it secure.
Any other information could be stored in any one of:
User defaults NSUserDefaults
File on disk (maybe a plist)
Database Core Data (technically just a file on disk)
Which you choose depends on what the data is, how much there is and what kind of access you need to it.
If your data is small and chosen by the user as some kind of setting then user defaults makes sense and is the lowest cost for you to implement.
To use a database, check out Core Data intro.
Wain is right but I think as you want to store small amount of data for further use, the most efficient ways is to use NSUserDefault.
NSUserDefault stores data in NSDictionary type things.
I think this is the step you have to take:
1- check if data exists. I mean if user selected the number if the last run of your app. So in viewDidLoad method:
NSMutableDictionary *userDefaultDataDictionary = [[[NSUserDefaults standardUserDefaults] dictionaryForKey:ALL_DATA_KEY] mutableCopy];
if (userDefaultDataDictionary) {
// so the dictionary exists, which means user has entered the number in previous app run
// and you can read it from the NSDictionaty:
if(userDefaultDataDictionary[LABLE_KEY]){
//and store it
}
}
2 - you can implement some method like syncronize to store data in NSUserDefault every time something has been changed.
- (void) synchronize
{
NSMutableDictionary *dictionaryForUserDefault = [[[NSUserDefaults standardUserDefaults] dictionaryForKey:ALL_DATA_KEY] mutableCopy];
if(!dictionaryForUserDefault)
dictionaryForUserDefault = [[NSMutableDictionary alloc] init];
dictionaryForUserDefault[LABLE_KEY] = //data you want to store
[[NSUserDefaults standardUserDefaults] setObject:dictionaryForUserDefault forKey:ALL_DATA_KEY];
[[NSUserDefaults standardUserDefaults] synchronize];
}
P.S. and don't forget to #define your keys for your dictionary:
#define LABLE_KEY #"Lables"
#define ALL_DATA_KEY #"AllData"
Store it in a plist. If you're talking about data pertaining to one or a few users, that's probably the easy thing. here is a simple example.
Since you say database, store in Sqlite. There's some provided stuff for it already in xcode.
The entire database is contained in one file, which can be moved around if you need to.
Here is some more information on how to use one in your app.