How do I save thousands of objects to Parse.com? [closed] - ios

Question

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question appears to be off-topic because it lacks sufficient information to diagnose the problem. Describe your problem in more detail or include a minimal example in the question itself.
Closed 8 years ago.
Improve this question
In my iOS app that uses Parse, some users will need to save thousand of objects in one action. I've tried iterating through the array of data and creating/saving the objects one by one, but this results in the objects saving to my data browser pretty slowly. Each object only needs to contain a few strings, so I don't understand why its taking so long to save these objects.
Is there a faster way to save thousands of objects to Parse?

Answer 1

Edited:
I've always used [PFObject saveAllInBackground:array block:^(BOOL succeeded, NSError *error) {}]; but....another method I have just attempted semi-successfully, was to upload a Json string as a PFFile(no 128k limit), and then use cloud code to parse it and create the necessary PFObjects. I was able to get this to work with small quantities, but unfortunately the cloud code timed out when using a large quantity. I instead opted to utilize a background job to perform the parsing. This takes a considerable amount of time before the data is completely available, but can handle the large amount of data. The upload time itself was much quicker. When using 1000 objects with 3 strings each upload was roughly .8 seconds, vs 23 seconds doing a save all in background, 5000 objects 3 strings each was only 2.5 seconds upload time. In addition to the quicker time you also get progress updates. Depending on the use-case, utilizing this alternative may work best if immediate and quick upload is important, vs making the data immediately available.
IOS Code:
NSMutableArray *array = [NSMutableArray array];
for (int i = 0; i<5; i++) {
//subclass of PFObject
Employee *employee = [Employee object];
employee.firstName = #"FName";
employee.lastName = #"LName";
employee.employeeID = #"fid54";
[array addObject:[employee dictionaryWithValuesForKeys:employee.allKeys]];
}
//Seperate class only to store the PFFiles
PFObject *testObject = [PFObject objectWithClassName:#"fileTestSave"];
testObject[#"testFile"] = [PFFile fileWithData:[NSJSONSerialization dataWithJSONObject:array options:0 error:nil]];
NSLog(#"started");
//**notice I am only saving the test object with the NSData from the JSONString**
[testObject saveInBackgroundWithBlock:^(BOOL succeeded, NSError *error) {
if (!error && succeeded) NSLog(#"succeeded");
else NSLog(#"error");
}];
Edited: Instead of saving in the beforeSave or afterSave Cloud Code which can cause timeout issues, the background job below can be run anytime. It grab all rows in the "fileTestSave" table, parses the JSON strings in those rows, and adds them to the "Person" table. Once completed it will rows from the table. All asynchronously!
var _ = require('underscore.js');
Parse.Cloud.job("userMigration", function(request, status)
{
// Set up to modify user data
Parse.Cloud.useMasterKey();
//Table called fileTestSave stores a PFFile called "testFile" which we will use an HTTPRequest to get the data. Is there a better way to get the data?
//This PFFile stores a json string which contains relavent data to add to the "Person" table
var testFileSave = Parse.Object.extend("fileTestSave");
var query = new Parse.Query(testFileSave);
query.find().then(function(results)
{
//Generate an array of promises
var promises = [];
_.each(results, function(testFileSaveInstance){
//add promise to array
promises.push(saveJsonPerson(testFileSaveInstance));
});
//only continue when all promises are complete
return Parse.Promise.when(promises);
}).then(function()
{
// Set the job's success status
console.log("Migration Completed NOW");
status.success("Migration completed");
}, function(error) {
// Set the job's error status
status.error("Uh oh, something went wrong.");
});
});
function saveJsonPerson(fileTestSave)
{
//Get the pffile testfile
var testFile = fileTestSave.get("testFile");
//get the fileURL from the PFFile to generate the http request
var fileURL = testFile["url"]();
//return the promise from the httpRequest
return Parse.Cloud.httpRequest({
method:"GET",
url: fileURL
}).then(function(httpResponse){
//return the promise from the parsing
return parsehttpResponse(httpResponse,fileTestSave);
},
function(error){
console.log("http response error");
}
);
}
function parsehttpResponse(httpResponse,fileTestSave)
{
var jsonArray = eval( '(' + httpResponse.text + ')' );
var saveArray =[];
//parse each person in the json string, and add them to the saveArray for bulk saving later.
for (i in jsonArray)
{
var personExtend = Parse.Object.extend("Person");
var person = new personExtend();
person.set("classDiscriminator",jsonArray[i]["classDiscriminator"]);
person.set("lastName",jsonArray[i]["lastName"]);
person.set("firstName",jsonArray[i]["firstName"]);
person.set("employeeID",jsonArray[i]["employeeID"]);
saveArray.push(person);
};
//return the promise from the saveAll(bulk save)
return Parse.Object.saveAll(
saveArray
).then(function(){
//return the promise from the destory
return fileTestSave.destroy(
).then(function(){
},function(error){
console.log("error destroying");
}
);
},function(error){
console.log("Error Saving");
}
);
}
Old Cloud Code that timed out as reference:
Parse.Cloud.afterSave("fileTestSave", function(request) {
//When accessing PFFiles you don't get the actual data, there may be an easier way, but I just utitlized an HTTPRequest to get the data, and then continued parsing.
var file = request.object.get("testFile");
var fileURL = file["url"]();
console.log("URL:"+fileURL);
Parse.Cloud.httpRequest({
method:"GET",
url: fileURL,
success: function(httpResponse)
{
var jsonArray = eval( '(' + httpResponse.text + ')' );
var saveArray =[];
for (i in jsonArray)
{
var personExtend = Parse.Object.extend("Person");
var person = new personExtend();
//May be a better way to parse JSON by using each key automatically, but I'm still new to JS, and Parse so I set each individually.
person.set("classDiscriminator",array[i]["classDiscriminator"]);
person.set("lastName",array[i]["lastName"]);
person.set("firstName",array[i]["firstName"]);
person.set("employeeID",array[i]["employeeID"]);
saveArray.push(person);
};
Parse.Object.saveAll(saveArray,
{
success: function(list) {
// All the objects were saved.
},
error: function(error) {
// An error occurred while saving one of the objects.
},
});
},
error: function(httpResponse) {
console.log("http response error");
}
});
});

Answer 2

Another method for uploading thousands of objects in the background, again this takes some time but can be sized to avoid timing out, as arrays are saved in chunks recursively. I've had no problem saving 10k+ items. Implemented as a category, just enter how many objects at a time you want save at a time, it will save them in the background serially, and recursively until all objects are saved, it also features progress updating via a separate block.
// PFObject+addOns.h
#import <Parse/Parse.h>
#interface PFObject (addOns)
+(void)saveAllInBackground:(NSArray *)array chunkSize:(int)chunkSize block:(PFBooleanResultBlock)block progressBlock:(PFProgressBlock)progressBlock;
#end
#import "PFObject+addOns.h"
#interface PFObject (addOns_internal)
+(void)saveAllInBackground:(NSArray *)array chunkSize:(int)chunkSize block:(PFBooleanResultBlock)block trigger:(void(^)())trigger;
#end
#implementation PFObject (addOns)
+(void)saveAllInBackground:(NSArray *)array chunkSize:(int)chunkSize block:(PFBooleanResultBlock)block progressBlock:(PFProgressBlock)progressBlock
{
unsigned long numberOfCyclesRequired = array.count/chunkSize;
__block unsigned long count = 0;
[PFObject saveAllInBackground:array chunkSize:chunkSize block:block trigger:^() {
count++;
progressBlock((int)(100.0*count/numberOfCyclesRequired));
}];
}
+(void)saveAllInBackground:(NSArray *)array chunkSize:(int)chunkSize block:(PFBooleanResultBlock)block trigger:(void(^)())trigger
{
NSRange range = NSMakeRange(0, array.count <= chunkSize ? array.count:chunkSize);
NSArray *saveArray = [array subarrayWithRange:range];
NSArray *nextArray = nil;
if (range.length<array.count) nextArray = [array subarrayWithRange:NSMakeRange(range.length, array.count-range.length)];
[PFObject saveAllInBackground:saveArray block:^(BOOL succeeded, NSError *error) {
if(!error && succeeded && nextArray){
trigger(true);
[PFObject saveAllInBackground:nextArray chunkSize:chunkSize block:block trigger:trigger];
}
else
{
trigger(true);
block(succeeded,error);
}
}];
}
#end

Answer 3

I think you should be able to do this with sending the save process in quantities of five into the background, so to speak fork it, "thread" it, as apple would refer to it.
here is the link to apples ios threading guides.
I have not used it yet, but I will soon need it as well, as I'm working on a massive database app.
here's the link
https://developer.apple.com/library/mac/Documentation/Cocoa/Conceptual/Multithreading/AboutThreads/AboutThreads.html

Answer 4

If you have an array of objects, you can use saveAllInBackgroundWithBlock. This method takes an array of PFObjects as its argument:
https://parse.com/docs/ios/api/Classes/PFObject.html#//api/name/saveAllInBackground:block:

Answer 5

For the faster processing, you can use the parse cloud code, whihc is simply a javascript. You can create a function which takes the array of the data as argument and then in function, you can save the objects.
Parse cloud code has better processing speed than the native one.
For its use, you can refer :
https://parse.com/docs/cloud_code_guide
https://parse.com/docs/js_guide