Saving PFFile Eventually - ios

A PFObject can be saveEventually to be sync on Parse when network is reachable, while keeping it locally meanwhile.
If your file contains a PFFile, the file must be savedInBackground before your PFObject can be save.
How to saveEventually a PFFile, for it to be send now, or later when network is reachable ?

As you might know, that feature isn't available within ParseSDK, so after seeing a few posts vaguely explaining how to bypass this, I wrote a sample working XCode project
That's only a working PoC with limitations such as only working for a single Parse class to associate saved PFFile on.
It requires Reachability pod 'Reachability', '~> 3.2'
How to use it ? Well, I guess the sample projects describes it well, but here is a piece of code to understand how it works :
(Remember to run pod install to resolve dependencies before running example)
/*
This example uses an UIImage, but this works with any file writable as NSData
We begin by writing this image in our tmp directory with an uuid as name.
*/
UIImage *nyancat = [UIImage imageNamed:#"nyancat.jpg"];
NSData *imageData = UIImageJPEGRepresentation(nyancat, 0.5);
NSString *filename = [[NSUUID UUID] UUIDString];
NSURL *fileUrl = [PFFileEventuallySaver fileURLInTmpWithName:filename];
[imageData writeToURL:fileUrl atomically:YES];
/*
We create a PFObject (you can pass an array to below function if you need your file to be saved on several objects). If upload works on first time, do what you want with your file, like linking it on your PFobject.
If saving fails, it'll be retried as soon as network is available, on this session or nexts launches of app.
In that case, the pointer at key kPFFILE_MANAGER_OBJECT_FILE_KEY of your PFFObject will be set with the PFFile, then saved eventually within PFFileEventuallySaver
*/
PFObject *object = [PFObject objectWithClassName:kPFFILE_CONTAINER_OBJECT_CLASSNAME];
[[PFFileEventuallySaver getInstance] trySaveobjectAtURL:fileUrl associatedObjects:#[object] withBlock:^(PFFile *file, NSError *error) {
if(!error)
{
NSLog(#"[First try, network is fine] File saved, saving PFObject");
object[kPFFILE_MANAGER_OBJECT_FILE_KEY] = file;
[object saveEventually];
NSLog(#"Try again disabling your network connection");
}
else
{
NSLog(#"No network, connect back your wifi, or relaunch app. Your file will be sent");
}
} progressBlock:^(int percentDone) {
NSLog(#"[First try, network is fine] Sending file %d/100%%", percentDone);
}];
This could be greatly improved, but I thought you guys might found that useful, as I would've wanted to find a similar working example.

Related

Save eventually on PFObject with PFFile (Parse Local Datastore)?

Goal
I am trying to save a PFObject that has a PFFile as an attribute. I am using the new Local Datastore for iOS, so I would like to save this PFObject with the saveEventually() method.
The Problem
The problem I am encountering is that the saveEventually() method doesn't seem to like saving the PFFiles. I tried to saveEventually() my object without any PFFile attached, and that worked fine. As soon as my PFFile was reattached, Xcode threw a couple of breakpoint notices (errors?) but did not terminate the app, and it appears as though all went well - however a check on the Parse Data Browser confirms that the save did not go through. Prior to the Local Datastore feature I don't believe this save would have been possible - it would have thrown the "Unable to saveEventually a PFObject with a relation to a new, unsaved PFFile." error. It seems as though the Local Datastore feature has fixed this, as it states in the iOS Local Datastore docs:
"Pinning a PFObject is recursive, just like saving, so any objects
that are pointed to by the one you are pinning will also be pinned.
When an object is pinned, every time you update it by fetching or
saving new data, the copy in the local datastore will be updated
automatically. You don't need to worry about it at all."
I have updated the SDK to the latest version (v1.6.2). Any ideas?
PFFiles still don't support saveEventually see here
That page was last updated : 2015-01-23
You could pinInBackgroundWithBlock and if successful save the PFFile to a temporary folder in you app bundle and delete it when necessary or unpinned
I just released a class which allows to saveEventually a PFFile.
You can find it here :
/*
This example uses an UIImage, but this works with any file writable as NSData
We begin by writing this image in our tmp directory with an uuid as name.
*/
UIImage *nyancat = [UIImage imageNamed:#"nyancat.jpg"];
NSData *imageData = UIImageJPEGRepresentation(nyancat, 0.5);
NSString *filename = [[NSUUID UUID] UUIDString];
NSURL *fileUrl = [PFFileEventuallySaver fileURLInTmpWithName:filename];
[imageData writeToURL:fileUrl atomically:YES];
/*
We create a PFObject (you can pass an array to below function if you need your file to be saved on several objects). If upload works on first time, do what you want with your file, like linking it on your PFobject.
If saving fails, it'll be retried as soon as network is available, on this session or nexts launches of app.
In that case, the pointer at key kPFFILE_MANAGER_OBJECT_FILE_KEY of your PFFObject will be set with the PFFile, then saved eventually within PFFileEventuallySaver
*/
PFObject *object = [PFObject objectWithClassName:kPFFILE_CONTAINER_OBJECT_CLASSNAME];
[[PFFileEventuallySaver getInstance] trySaveobjectAtURL:fileUrl associatedObjects:#[object] withBlock:^(PFFile *file, NSError *error) {
if(!error)
{
NSLog(#"[First try, network is fine] File saved, saving PFObject");
object[kPFFILE_MANAGER_OBJECT_FILE_KEY] = file;
[object saveEventually];
NSLog(#"Try again disabling your network connection");
}
else
{
NSLog(#"No network, connect back your wifi, or relaunch app. Your file will be sent");
}
} progressBlock:^(int percentDone) {
NSLog(#"[First try, network is fine] Sending file %d/100%%", percentDone);
}];

Parse backend new Error for uploading files - Error: File name must be a string - PFFile

I'm building an IOS app and using parse.com as a backend. All of a sudden, today I am getting this error - Error: File name must be a string without any other explanation. I'm certain the file name that I am using is a string. I haven't changed any of the Native IOS code so it must be a new Parse issue. The only difference - today I updated my account to the new pricing model on Parse, wondering if that's related?
// Code snippet
NSString *fileName = #"test.mov"; // Also tried just #"test"
PFFile *file = [PFFile fileWithName:fileName data:fileData]; // NSData - 286553 bytes
[file saveInBackgroundWithBlock:^(BOOL succeeded, NSError *error) {
// Break point right here - this is where the error occurs.
NEWS:
ALL should go to: https://developers.facebook.com/bugs/622479264497355/?comment_id=1426426400945705
And post your account email to Hector Ramos.
Update2:
This has been fixed and is preparing for a deploy.
The problem is: Basically, if you are currently using LESS than 1GB of data storage (for example, I'm using 679MB), instead of correctly calculating that I'm using (679 / (1024MB * 20)) = 3.3% of the quota, its calculating it as 679/20, which gets the result of 3395%. Parse, if you see this, PLEASE... its really just a few lines of code.. please resolve this ASAP!
This was an internal error and has been resolved. Parse posted an incident report here.
EDIT: As several people have pointed out, the error is still present even though Parse claims it has been resolved. Hrmmmmmm...

NSFileWrapper fails when from iCloud and works from local directory

I have a problem syncing NSFileWrapper documents with iCloud. I am able to create my wrapper and save it to my ubiquitous container.
When I try to read it from the device that created it, it works. When I try to read form another device that got it from iCloud, it crashes.
Some code:
This function to add a wrapper container with a NSString
- (void) addNSString:(NSString*)_string toFileWrapper:(NSFileWrapper*)_wrapper forKey:(NSString*)_key {
NSData *data = [NSKeyedArchiver archivedDataWithRootObject:_string];
if(data) {
[_wrapper addRegularFileWithContents:data preferredFilename:_key];
}
}
And then here is how I decode it:
- (id) unarchiveObjectFromWrappers:(NSDictionary*)_wrappers withKey:(NSString*)_key {
id value = nil;
NSFileWrapper *wrapper = [_wrappers valueForKey:_key];
if(wrapper) {
NSData *data = [wrapper regularFileContents];
if(data) {
value = [NSKeyedUnarchiver unarchiveObjectWithData:data];
}
}
return value;
}
The decoding part works on one device and not on the others (EXC_BAD_ACCESS when the NSKeyedUnarchiver tries to unarchive from the NSData. The NSData seems good, it has the proper length and everything but when I try to log its datas for example it crashes).
My guess is that the NSFileWrapper doesn't download its full content, only its structure and that I have to do something to make it available. But I don't know what.
Any ideas?
========
Edit:
NSURLUbiquitousItemIsDownloadedKey says that the file is downloaded BUT if I try to copy it to the sandbox it fails with this error: "The operation couldn’t be completed. Bad file descriptor"
So the file is either not uploaded properly to iCloud or not downloaded properly...
It drove me crazy too. The solution is rather simple, yet totally undocumented by Apple. You must download the file specifically. Only the file wrapper is downloaded automatically, but not its contents. That's why the check says the file exists.
Before copying the file over, call something like this:
[[NSFileManager defaultManager]startDownloadingUbiquitousItemAtURL:cloudURL error:nil];
Related: Cannot sync simple text file with iCloud (bad file descriptor)

IOS: Using stringWithContentsOfURL when network is unavailable

In this code poll from within my app for a reachable network
("http://soxxx9.cafe24.com/event.php")
NSString * szURL =[NSString stringWithFormat:#"http://soxxx9.cafe24.com/event.php"];
NSURL *url = [NSURL URLWithString:[szURL stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding ]];
NSString *strData;
while(1)
{
NSError *error = nil;
strData = [NSString stringWithContentsOfURL:url
encoding:NSUTF8StringEncoding
error:&error];
if(!error)
break;
//String data is not owned by me, no need to release
}
If you have a better way, please teach me.
This code seems to be heavily power consuming when network is out : you'll try million times to download something that is unreachable...
Have a look at the Reachability class, provided by Apple (http://developer.apple.com/library/ios/#samplecode/Reachability/Introduction/Intro.html). You'll find ARCified versions on gitHub (https://github.com/tonymillion/Reachability for example).
The idea is to register for notifications about the network reachability.
So, in your code :
Check network resource availability before retrieving the string you want.
If this is available, use your code WITHOUT the while(TRUE)
Check your string for any error while retrieving it client side = in your code
If the network is not available, you'll have to inform the user that network is unreachable, and register for reachability notifications to retrieve your string as soon as it is reachable again for example.
You should a class to handle the connection for you. This way you have more control of what's going on with it. MKNetworkKit is a solution, you can check it here.

What is the fastest way to load a large CSV file into core data

Conclusion
Problem closed, I think.
Looks like the problem had nothing to do with the methodology, but that the XCode did not clean the project correctly in between builds.
It looks like after all those tests, the sqlite file that was being used was still the very first one that wasn't indexed......
Beware of XCode 4.3.2, I have nothing but problems with Clean not cleaning, or adding files to project not automatically being added to the bundle resources...
Thanks for the different answers..
Update 3
Since I invite anybody to just try the same steps to see if they get the same results, let me detail what I did:
I start with blank project
I defined a datamodel with one Entity, 3 attributes (2 strings, 1 float)
The first string is indexed
In did finishLaunchingWithOptions, I am calling:
[self performSelectorInBackground:#selector(populateDB) withObject:nil];
The code for populateDb is below:
-(void)populateDB{
NSLog(#"start");
NSPersistentStoreCoordinator *coordinator = [self persistentStoreCoordinator];
NSManagedObjectContext *context;
if (coordinator != nil) {
context = [[NSManagedObjectContext alloc] init];
[context setPersistentStoreCoordinator:coordinator];
}
NSString *filePath = [[NSBundle mainBundle] pathForResource:#"input" ofType:#"txt"];
if (filePath) {
NSString * myText = [[NSString alloc]
initWithContentsOfFile:filePath
encoding:NSUTF8StringEncoding
error:nil];
if (myText) {
__block int count = 0;
[myText enumerateLinesUsingBlock:^(NSString * line, BOOL * stop) {
line=[line stringByReplacingOccurrencesOfString:#"\t" withString:#" "];
NSArray *lineComponents=[line componentsSeparatedByString:#" "];
if(lineComponents){
if([lineComponents count]==3){
float f=[[lineComponents objectAtIndex:0] floatValue];
NSNumber *number=[NSNumber numberWithFloat:f];
NSString *string1=[lineComponents objectAtIndex:1];
NSString *string2=[lineComponents objectAtIndex:2];
NSManagedObject *object=[NSEntityDescription insertNewObjectForEntityForName:#"Bigram" inManagedObjectContext:context];
[object setValue:number forKey:#"number"];
[object setValue:string1 forKey:#"string1"];
[object setValue:string2 forKey:#"string2"];
NSError *error;
count++;
if(count>=1000){
if (![context save:&error]) {
NSLog(#"Whoops, couldn't save: %#", [error localizedDescription]);
}
count=0;
}
}
}
}];
NSLog(#"done importing");
NSError *error;
if (![context save:&error]) {
NSLog(#"Whoops, couldn't save: %#", [error localizedDescription]);
}
}
}
NSLog(#"end");
}
Everything else is default core data code, nothing added.
I run that in the simulator.
I go to ~/Library/Application Support/iPhone Simulator/5.1/Applications//Documents
There is the sqlite file that is generated
I take that and I copy it in my bundle
I comment out the call to populateDb
I edit persistentStoreCoordinator to copy the sqlite file from bundle to documents at first run
- (NSPersistentStoreCoordinator *)persistentStoreCoordinator
{
#synchronized (self)
{
if (__persistentStoreCoordinator != nil)
return __persistentStoreCoordinator;
NSString *defaultStorePath = [[NSBundle mainBundle] pathForResource:#"myProject" ofType:#"sqlite"];
NSString *storePath = [[[self applicationDocumentsDirectory] path] stringByAppendingPathComponent: #"myProject.sqlite"];
NSError *error;
if (![[NSFileManager defaultManager] fileExistsAtPath:storePath])
{
if ([[NSFileManager defaultManager] copyItemAtPath:defaultStorePath toPath:storePath error:&error])
NSLog(#"Copied starting data to %#", storePath);
else
NSLog(#"Error copying default DB to %# (%#)", storePath, error);
}
NSURL *storeURL = [NSURL fileURLWithPath:storePath];
__persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:[self managedObjectModel]];
NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], NSMigratePersistentStoresAutomaticallyOption,
[NSNumber numberWithBool:YES], NSInferMappingModelAutomaticallyOption, nil];
if (![__persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:storeURL options:options error:&error])
{
NSLog(#"Unresolved error %#, %#", error, [error userInfo]);
abort();
}
return __persistentStoreCoordinator;
}
}
I remove the app from the simulator, I check that ~/Library/Application Support/iPhone Simulator/5.1/Applications/ is now removedI rebuild and launch again
As expected, the sqlite file is copied over to ~/Library/Application Support/iPhone Simulator/5.1/Applications//Documents
However the size of the file is smaller than in the bundle, significantly!
Also, doing a simple query with a predicate like this predicate = [NSPredicate predicateWithFormat:#"string1 == %#", string1]; clearly shows that string1 is not indexed anymore
Following that, I create a new version of the datamodel, with a meaningless update, just to force a lightweight migration
If run on the simulator, the migration takes a few seconds, the database doubles in size and the same query now takes less than a second to return instead of minutes.
This would solve my problem, force a migration, but that same migration takes 3 minutes on the iPad and happens in the foreground.
So hat's where I am at right now, the best solution for me would still be to prevent the indexes to be removed, any other importing solution at launch time just takes too much time.
Let me know if you need more clarifications...
Update 2
So the best result I have had so far is to seed the core data database with the sqlite file produced from a quick tool with similar data model, but without the indexes set when producing the sqlite file. Then, I import this sqlite file in the core data app with the indexes set, and allowing for a lightweight migration. For 2 millions record on the new iPad, this migration stills take 3 minutes. The final app should have 5 times this number of records, so we're still looking at a long long processing time.
If I go that route, the new question would be: can a lightweight migration be performed in the background?
Update
My question is NOT how to create a tool to populate a Core Data database, and then import the sqlite file into my app. I know how to do this, I have done it countless times. But until now, I had not realized that such method could have some side effect: in my case, an indexed attribute in the resulting database clearly got 'unindexed' when importing the sqlite file that way.
If you were able to verify that any indexed data is still indexed after such transfer, I am interested to know how you proceed, or otherwise what would be the best strategy to seed such database efficiently.
Original
I have a large CSV file (millions of lines) with 4 columns, strings and floats.
This is for an iOS app.
I need this to be loaded into core data the first time the app is loaded.
The app is pretty much non functional until the data is available, so loading time matters, as a first time user obviously does not want the app to take 20 minutes to load before being able to run it.
Right now, my current code takes 20 min on the new iPad to process a 2 millions line csv file.
I am using a background context to not lock the UI, and save the context every 1,000 records
The first idea I had was to generate the database on the simulator, then to copy/paste it in the document folder at first launch, as this is the common non official way of seeding a large database. Unfortunately, the indexes don't seem to survive such a transfer, and although the database was available after just a few seconds, performance is terrible because my indexes were lost. I posted a question about the indexes already, but there doesn't seem to be a good answer to that.
So what I am looking for, either:
a way to improve performance on loading millions of records in core data
if the database is pre-loaded and moved at first startup, a way to keep my indexes
best practices for handling this kind of scenario. I don't remember using any app that requires me to wait for x minutes before first use (but maybe The Daily, and that was a terrible experience).
Any creative way to make the user wait without him realizing it: background import while going through tutorial, etc...
Not Using Core Data?
...
Pre-generate your database using an offline application (say, a command-line utility) written in Cocoa, that runs on OS X, and uses the same Core Data framework that iOS uses. You don't need to worry about "indexes surviving" or anything -- the output is a Core Data-generated .sqlite database file, directly and immediately usable by an iOS app.
As long as you can do the DB generation off-line, it's the best solution by far. I have successfully used this technique to pre-generated databases for iOS deployment myself. Check my previous questions/answers for a bit more detail.
I'm just starting out with SQLite and I need to integrate a DB into one of my apps that will have a lot of indexed data in a SQLite database. I was hoping I could do some method where I could bulk insert my information into a SQLite file and add that file to my project. After discovering and reading through your question, the provided answer and the numerous comments, I decided to check out the SQLite source to see if I could make heads or tails of this issue.
My initial thought was that the iOS implementation of SQLite is, in fact, throwing out your indices. The reason is because you initially create your DB index on x86/x64 system. The iOS is an ARM processor, and numbers are handled differently. If you want your indexes to be fast, you should generate them in such a way that they are optimized for the processor in which they will be searched.
Since SQLite is for multiple platforms, it would make since to drop any indices that have been created in another architecture and rebuild them. However, since no one wants to wait for an index to rebuild the first time it is accessed, the SQLite devs most likely decided to just drop the index.
After digging into the SQLite code, I've come to the conclusion that this is most likely happening. If not for the processor architecture reason, I did find code (see analyze.c and other meta-information in sqliteint.h) where indices were being deleted if they were generated under an unexpected context. My hunch is that the context that drives this process is how the underlying b-tree data structure was constructed for the existing key. If the current instance of SQLite can't consume the key, it deletes it.
It is worth mentioning that the iOS Simulator is just that-- a simulator. It is not an emulator of the, hardware. As such, your app is running in a pseudo-iOS device, running on an x86/x64 processor.
When your app and SQLite DB are loaded to your iOS device, an ARM-compiled variant is loaded, which also links to the ARM compiled libraries within iOS. I couldn't find ARM specific code associated with SQLite, so I imagine Apple had to modify it to their suit. The could also be part of the problem. This may not be an issue with the root-SQLite code, it could be an issue with the Apple/ARM compiled variant.
The only reasonable solution that I can come up with is that you can create a generator application that you run on your iOS machine. Run the application, build the keys, and then rip the SQLite file from the device. I'd imagine such a file would work across all devices, since all ARM processors used by iOS are 32-bit.
Again, this answer is a bit of an educated guess. I'm going to re-tag your question as SQLite. Hopefully a guru may find this and be able to weigh in on this issue. I'd really like to know the truth for my own benefit.

Resources