We are developing an application to collect beacon information and sync new/updated data to the server. We are using core data to collect beacon information, but occasionally we've found that the core data framework gives incorrect results when syncing to the server. Sometimes predicates are failing after the first iteration while fetching data from the core data to sync. So, we are seeking advice about any issues you have had with prolonged use of core data and persistent storage of the database.
I am using following steps to sync core data records to server one by one
-(void)syncDataToServer{
if( !isSycInProgress ){
//predicate fetch records which newly created and which are synced to server yet
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"isSynced == %#",[NSNumber numberWithBool:NO]];
//fetch only one record at one time to utilize memory
NSArray *beaconRecordArray = [[ISCoreDataManager sharedManager] fetchObjectList:#"BeaconRecords" predicate:predicate attributeName:#"endTime" batchSize:1 offset:0 isAscending:YES inContext:mainContext];
BeaconRecords *beacon = nil;
if(beaconRecordArray.count){
isSycInProgress = YES;
beacon = [beaconRecordArray firstObject];
{
//create request parameter dictionary
NSDictionary *dict = #{
#"uid": [beacon.uID description],
#"title":beacon.title.length?beacon.title:#"No title",
#"url": beacon.urlString?beacon.urlString:#"",
};
//call webservice to upload records on server
__block NSManagedObjectID *objId = beacon.objectID;
[[WebServiceHelper sharedInstance] callPostDataWithMethod:beconUrlPostAddBecon withParameters:dict withHud:YES success:^(NSDictionary *response)
{
if([response isKindOfClass:[NSDictionary class]] && [response[#"status"] intValue] == 200){
NSString *responseStr =response[#"response_crypt_data"];
NSError *error;
//Decrypt data to fetch parameters in response
NSData *decryptedData = [RNDecryptor decryptData:[[NSData alloc] initWithBase64EncodedString:responseStr options:0] withPassword:secretKey error:&error];
DLog(#"decrypted data %#", [[NSString alloc] initWithData:decryptedData encoding:NSUTF8StringEncoding]);
//parse data
//convert data into collection format (for parsing)
NSError *parseJsonError = nil;
NSDictionary *decryptedResponse = [NSJSONSerialization JSONObjectWithData:decryptedData options:NSJSONReadingMutableContainers error:&parseJsonError];
//store Server id in local db for future reference to update record
BeaconRecords *beaconRec = [self.privateContext existingObjectWithID:objId error:nil];
[self.privateContext performBlock:^{
if(!error && !parseJsonError){
beaconRec.serverID = [NSString stringWithFormat:#"%#",decryptedResponse[#"id"]];
beaconRec.isSynced = #(YES);
[[ISCoreDataManager sharedManager] dbSaveInContext:self.privateContext];
}
//update isSynced flag in local db
isSycInProgressBeaconRecords = NO;
dispatch_async( dispatch_get_main_queue(), ^{
[self syncDataToServer]; // call this method recursively to upload all records which are pending to sync
});
}];
}
else
{
isSycInProgress = NO;
dispatch_async(dispatch_get_main_queue(), ^{
[self syncDataToServer]; // call this method recursively to upload all records which are pending to sync
});
}
} errorBlock:^(id error) {
isSycInProgress = NO;
dispatch_async(dispatch_get_main_queue(), ^{
[self syncDataToServer]; // call this method recursively to upload all records which are pending to sync
});
}];
}
return;
}
isSycInProgress = NO;
}
}
After successfully syncing 1st record to server, when I call same sync function recursively to upload 2nd record, that predicate is giving me same record which I uploaded before. And strange thing is isSynched property of fetched object is (yes) and my predicate says it should be (_NO).
Related
I have two UIViewControllers in a Tab Bar
In one of the TabBar I am making an api call using AFNetworking and this api call is saving data in CoreData.
Here is my code
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
for (int i = 0; i < cartList.count; i++)
{
NSDictionary *dict = [cartList objectAtIndex:i];
NSFetchRequest *request = [Orders fetchRequest];
request.predicate = [NSPredicate predicateWithFormat:#"orderId = %#", [dict objectForKey:kiD]];
NSError *error = nil;
NSArray *itemsList = context executeFetchRequest:request error:&error];
if (itemsList.count == 0)
{
Orders *order = [NSEntityDescription insertNewObjectForEntityForName:#"Orders" inManagedObjectContext:appDel.persistentContainer.viewContext];
[order updateWithDictionary:dict];
order.isNew = NO;
}
else
{
Orders *order = [itemsList objectAtIndex:0];
[order updateWithDictionary:dict];
order.isNew = NO;
}
}
dispatch_async(dispatch_get_main_queue(), ^{
[appDel saveContext];
[self refreshValues:NO];
});
});
In second VIewController I am doing something like that. If I switch the tab controllers very fast the app crashes at
[appDel saveContext];
most probably because the last time viewContext was used by other UIviewController in Background thread.
What is the work around I can adopt to fix this problem
If this is correctly implemented
[appDel.persistentContainer performBackgroundTask:^(NSManagedObjectContext * _Nonnull context)
{
NSFetchRequest *request = [Categories fetchRequest];
NSBatchDeleteRequest *deleteReq = [[NSBatchDeleteRequest alloc] initWithFetchRequest:request];
NSError *deleteError = nil;
[appDel.persistentContainer.viewContext executeRequest:deleteReq error:&deleteError];
for (int i = 0; i < dataArr.count; i++)
{
Categories *category = [NSEntityDescription insertNewObjectForEntityForName:#"Categories" inManagedObjectContext:appDel.persistentContainer.viewContext];
[category updateWithDictionary:[dataArr objectAtIndex:i]];
}
#try {
NSError *error = nil;
[context save:(&error)];
} #catch (NSException *exception)
{
}
[self getCategoryItems];
}];
Core-data is not thread-safe, neither for reading for for writing. If you violate this ever core-data can fail in unexpected ways. So even if it appears to work you can find core-data suddenly crashing for no apparent reasons. In other words, accessing core-data from the wrong thread is undefined.
There are a few possible solutions:
1) only use the main thread for reading and writing to core-data. This is an OK solution for simple apps that don't do a lot of data import or export and have relatively small data sets.
2) Wrap NSPersistentContainer's performBackgroundTask in an operation queue and only write to core-data through that method and never write to the viewContext. When you use performBackgroundTask the method gives you a context. You should use the context to fetch any objects that you need, modify them, save the context and then discard the context and the objects.
If you try to write using both performBackgroundTask and writing directly to the viewContext you can get write conflicts and lose data.
Create a child NSManagedObjectContext object with NSPrivateQueueConcurrencyType your data processing in background queue.
Read Concurrency guide for more info.
I am struggling to come up with an elegant solution to a problem involving multiple network requests. If you need further information than what has been provided then please don't hesitate to ask.
I need to download data from a server, create core data objects from it and then use information from those objects to download the next set of data. So I am transversing a hierarchy.
So for example:
I make my first request to the server and pull down the Regions which is made up of 4 objects (North, South, East, West). I am saving these to core data.
Once that is done (not sure best way to track this) I then need to do a fetch request on the region entity to get back those 4 objects. Each region contains a number of counties which I need to request from the server. So I loop through the regions and make a network request for each region.
I loop through each returned dictionary (one for each region) to create each county.
Here is my code to download the regions and county:
+ (void)downloadRegions
{
NSString *search = #"organisation";
NetworkHandler *networkHandler = [[NetworkHandler alloc] init];
[networkHandler downloadData:search];
}
+ (void)downloadCounty
{
NSManagedObjectContext *context = [DatabaseHandler sharedHandler].managedObjectContext;
NSFetchRequest *request = [[NSFetchRequest alloc] init];
NSEntityDescription *entity = [NSEntityDescription entityForName:#"Region" inManagedObjectContext:context];
[request setEntity:entity];
//NSPredicate *searchFilter = [NSPredicate predicateWithFormat:#"attribute = %#", searchingFor];
NSError *error = nil;
NSArray *results = [context executeFetchRequest:request error:&error];
NSLog(#"%#", results);
NSString *search = #"organisation?code=";
for (Region *region in results) {
NSString *s = [search stringByAppendingString:[NSString stringWithFormat:#"%#", region.code]];
NetworkHandler *networkHandler = [[NetworkHandler alloc] init];
[networkHandler downloadData:s];
}
}
Both of the above methods call:
- (void)downloadData:(NSString *)searchUrl
{
NSString *apiURL = [kBaseURL stringByAppendingPathComponent:#"api"];
NSString *finalURL = [apiURL stringByAppendingPathComponent:searchUrl];
self.dataTask = [self.session dataTaskWithURL:[NSURL URLWithString:finalURL]
completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
NSDictionary *json = [NSJSONSerialization JSONObjectWithData:data options:0 error:nil];
//self.jsonData[searchUrl] = json;
[self createObjectInDatabase:json andSearchURL:searchUrl];
NSLog(#"This has been reached");
}];
[self.dataTask resume];
}
- (void)createObjectInDatabase:(id)data andSearchURL:(NSString *)searchURL
{
if ([searchURL isEqual:#"organisation"]) {
[Region createNewRegionWithData:data inManagedObjectContext:[DatabaseHandler sharedHandler].managedObjectContext];
}
else {
[LAT createNewLATWithData:data inManagedObjectContext:[DatabaseHandler sharedHandler].managedObjectContext];
}
}
I am not sure if I am doing this the best way. In regards to making the request, creating the object and then making the next request.
My biggest issue is knowing when to make the next request. i.e - knowing when the download has completed and all the core data objects have been created successfully before making a request for those objects and using them in the next request. I am currently making the second request manually but need it to be done automatically.
I hope that is clear. I am finding it hard to explain :-). Thanks in advance.
What if you passed a string into downloadData: that the completion handler then used to post a notification as it was finishing? When you receive each notification, you know which step of the process to go to next.
I have method called collectData in my app which is the most important part of my View Controller. In that method I do a couple of signicant things (downloading, parsing, saving to persistent store), so it would be easier for you to take a look:
-(void)collectData
{
// Downloading all groups and saving them to Core Data
[[AFHTTPRequestOperationManager manager] GET:ALL_GROUPS parameters:nil success:^(AFHTTPRequestOperation *operation, id responseObject) {
NSMutableDictionary* groups = [NSMutableDictionary new];
NSMutableArray* newIds = [NSMutableArray new];
NSError *error;
// Saving everything from response to MOC
for (id group in responseObject) {
Group *groupEntity = [NSEntityDescription insertNewObjectForEntityForName:#"Group" inManagedObjectContext:self.moc];
groupEntity.name = [group valueForKey:#"name"];
groupEntity.cashID = [group valueForKey:#"id"];
groupEntity.caseInsensitiveName = [[group valueForKey:#"name"] lowercaseString];
groupEntity.selected = #NO;
// Filling up helping variables
groups[groupEntity.cashID] = groupEntity;
[newIds addObject:groupEntity.cashID];
}
// Fetching existing groups from Persistant store
NSFetchRequest* r = [NSFetchRequest fetchRequestWithEntityName:#"Group"];
[r setIncludesPendingChanges:NO];
r.predicate = [NSPredicate predicateWithFormat:#"cashID IN %#",newIds];
NSArray *existingGroups = [self.moc executeFetchRequest:r error:&error];
// Deleting groups which already are in database
for (Group* g in existingGroups) {
Group* newGroup = groups[g.cashID];
g.name = [newGroup valueForKey:#"name"];
g.cashID = [newGroup valueForKey:#"cashID"];
g.caseInsensitiveName = [[newGroup valueForKey:#"name"] lowercaseString];
[self.moc deleteObject:newGroup];
}
// Saving Entity modification date and setting it to pull to refresh
[self saveModificationDate:[NSDate date] forEntityNamed:#"Group"];
[self.pullToRefreshView.contentView setLastUpdatedAt:[self getModificationDateForEntityNamed:#"Group"]
withPullToRefreshView:self.pullToRefreshView];
// Save groups to presistant store
if (![self.moc save:&error]) {
NSLog(#"Couldn't save: %#", [error localizedDescription]);
}
[[self fetchedResultsController] performFetch:&error];
[self.pullToRefreshView finishLoading];
[self.tableView reloadData];
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
// Show alert with info about internet connection
[self.pullToRefreshView finishLoading];
UIAlertView *internetAlert = [[UIAlertView alloc] initWithTitle:#"Ups!" message:#"Wygląda na to, że nie masz połączenia z internetem" delegate:self cancelButtonTitle:#"OK" otherButtonTitles:nil, nil];
[internetAlert show];
}];
}
So when I start collecting data (first run or push to refresh) this method is blocking UI.
I want to avoid this but when I put the success block into another dispatch_async and get back to main queue only for [self.tableView reloadData] I face problem with saving to persistent store or something with bad indexes.
How can I do this whole thing in background and leave UI responsive to the user?
Just an idea, give it a try using dispatch_sync. Have a look at this explanation here where log result something similar to your need. Put [yourTableView reloadData] after synchronous block.
Hope it helps!
It seems AFNetwork call is not async so just try to call your method via performselector.
I am currently developing an app that that will be used by association members that are going to a large annual conference.
The app will pull data from a database that is created by the app and populate it via a web service. The web service is split into 8 pages (this will likely go up). Each page represents a table in the database. The app will have several table views that will be populated by data in one or more of the tables in the database.
What I need is a the best method for going through the list of tables, connecting to their respective web service pages and then populating the respective database tables. This updating needs to take place in the background so the UI doesn't become unresponsive and/or show a downloading/updating/waiting kind of status.
So far I have a static array of the table names and have a loop that goes through the array and appends a URL string with the names, for example:
-(void)startUpdate
{
NSArray* tableNames = #[#"speaker", #"exhibitor", #"workshop", #"workshopspeakers", #"schedule", #"location", #"feedback", #"note", #"usage", #"user"];
NSUInteger loopCount = tableNames.count;
for (int i = 0; i < loopCount; ++i){
NSString *tableName = [tableNames objectAtIndex:i];
[self fetchObjectsWithTableName:[tableName mutableCopy] completion:^(NSArray* objects, NSError*error){
if (error) {
} else {
}
}];
}
}
fetchObjectsWithTableName method then has the connections and retrieves the data:
-(void)fetchData:(NSString *)tableName
withCompletion:(completion_t)completionHandler
{
NSString *currentURL = [NSString stringWithFormat:#"https://testapi.someURL.com/api/congress/%#", tableName];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:currentURL]];
[request addValue:#"application/json" forHTTPHeaderField:(#"Accept")];
[NSURLConnection sendAsynchronousRequest:request
queue:[[NSOperationQueue alloc] init]
completionHandler:^(NSURLResponse *response, NSData *data, NSError *error)
{
NSError* err = error;
NSArray* objects; // final result array as a representation of JSON Array
if (response) {
NSHTTPURLResponse *newResp = (NSHTTPURLResponse*)response;
if (newResp.statusCode == 200) {
NSLog(#"FetchData - Status code = %li", (long)newResp.statusCode);
if ([data length] >0 && error == nil)
{
NSError* localError;
objects = [NSJSONSerialization JSONObjectWithData:data options:kNilOptions error:&error];
if (objects) {
if (completionHandler) {
completionHandler(objects, nil);
}
//NSLog(#"Objects in current table - %# = %#", tableName, objects);
[self.tables addObject:objects];
// NSLog(#"Tables now = %#", self.tables);
NSLog(#"FetchData - Objects in current table - %# = %lu", tableName, (unsigned long)objects.count);
return;
} else {
err = localError;
}
} else {
NSLog(#"FetchData - objects is empty");
return;
// err = ...
}
}
NSLog(#"FetchData - Response code not 200#");
}
if (objects == nil) {
NSLog(#"FetchData - Nothing found in table: %#", tableName);
//assert(err);
if (completionHandler) {
completionHandler(nil, err);
}
}
}];
}
This currently goes through the array of table names, makes a connection based on each one and pulls back JSON data and stores it in a temporary array 'objects'. I think what I need now is that in each iteration of this 'objects' array is copied to the relevant table in the database, i.e. 'speaker' table name makes a connection: https://testapi.someURL.com/api/congress/speaker and the JSON is entered into the database under the table 'speaker'. How and where do I do that? Will I need to add a completion handler to startUpdate? If so, how? I don't understand completion handlers despite looking at several examples. Thanks.
No, do it in the NSURLConnection completion block after you have updated your temporary storage.
But, change your approach overall.
If you're only willing to change a bit, start using NSOperationQueue to limit the number of connections that you're trying to make at the same time. Preferably also use Core Data.
If you're willing to make a bigger change, definitely move to Core Data and look at using a framework like RestKit to do all of the download, mapping and storage for you.
(note, in both cases you need to set the max concurrent operation limit to prevent the app from flooding the network with requests - a limit of 5 should be good).
I am building an app for displaying assets (PDF, Video, Etc).
It starts by downloading a JSON and parsing it into Core Data Objects <-- This part works fine.
These objects are a hierarchical set of Nodes that have a relationships set up in my model. each node can either be a FILE or a FOLDER. <-- no problems.
Then I have instance methods built into my NSManagedObject Subclasses that will download the file associated with that object (ie. a PDF). Then it sets
self.isAvailable = [NSNumber numberWithBool:YES];
Meanwhile, I have a UITableView that displays a list of assets. eventually it will update in real-time, but for now this is where I am having issue. I first had the view controller keep a pointer to the CoreData object that represents the folder it displays, but it appears that If the context gets updated, the pointer becomes invalid (ie. fails to fault).
Core data is not being very specific on what the problem is, or even where its happening, but it seems to crash when I set isAvailable with
*** Terminating app due to uncaught exception 'NSObjectInaccessibleException', reason: 'CoreData could not fulfill a fault for '0x1d5f9e50 <x-coredata://EDE66B97-B142-4E87-B445-76CAB965B676/Node/p58>''
I feel like the problem is that I shouldn't just keep a strong reference to a core data object as my model. Is there a better (less crashy) way to do this?
I have started playing with NSFetchedResultsController and using objectID's instead, but I haven't gotten anywhere yet.
- (void)populateChildren {
NSString * urlString = [NSString stringWithFormat:#"%#/%#", [CMPConstants hostURLString], self.SBUCode];
NSURL *url = [NSURL URLWithString:urlString];
NSURLRequest * request = [NSURLRequest requestWithURL:url];
[NSURLConnection sendAsynchronousRequest:request queue:self.downloadQueue completionHandler:^(NSURLResponse * response, NSData * data, NSError * error) {
if (data) {
NSDictionary * dict = [NSJSONSerialization JSONObjectWithData:data options:0 error:nil];
[self processParsedObject:dict];
} else {
NSLog(#"%#", urlString);
}
}];
}
#pragma mark - Parse JSON into NSManagedObjects
- (void)processParsedObject:(id)object {
[self processParsedObject:object depth:0 parent:nil key:nil];
[[NSManagedObjectContext MR_contextForCurrentThread] MR_saveToPersistentStoreAndWait];
}
- (void)processParsedObject:(id)object depth:(int)depth parent:(Node *)parent key:(NSString*)key {
if ([object isKindOfClass:[NSDictionary class]]) {
if (depth == 0) {
// Grab content node if depth is 0;
object = [object valueForKey:#"content"];
}
// FIXME: Change this to a real primary key once we get one.
static NSString * primaryKey = #"name";
// Look for existing object
Node * testNode = [Node MR_findFirstByAttribute:primaryKey withValue:[object valueForKey:primaryKey]];
// Create new node pointer
Node * newNode;
if (testNode) {
// Update existing Node
newNode = testNode;
} else {
// Build a new Node Object
newNode = [Node MR_createEntity];
newNode.isAvailable = [NSNumber numberWithBool:NO];
}
// Get keys
NSArray * keys = #[#"name",
#"type",
#"index",
#"size",
#"videoDemensions",
#"videoId",
#"fileName",
#"fileType",
#"path"];
if ([[object valueForKey:#"type"] isEqual:[NSNull null]]) {
NSLog(#"%#", object);
}
// Loop to set value for keys.
for (NSString * key in keys) {
id value = [object valueForKey:key];
if (![[object valueForKey:key] isKindOfClass:[NSNull class]]) {
[newNode setValue:value forKey:key];
}
}
// Set calculated properties.
[newNode setSbu:[self SBUCode]];
[newNode setParent:parent];
[[NSManagedObjectContext MR_contextForCurrentThread] MR_saveToPersistentStoreAndWait];
// Sync local file.
if (!newNode.isAvailable.boolValue) {
[newNode aquireFileInQueue:self.downloadQueue];
}
// Process children
for(NSString * newKey in [object allKeys]) {
id child = [object objectForKey:newKey];
[self processParsedObject:child depth:depth+1 parent:newNode key:newKey];
}
} else if ([object isKindOfClass:[NSArray class]]) {
for(id child in object) {
[self processParsedObject:child depth:depth+1 parent:parent key:nil];
}
} else {
// Nothing here, this processes each field.
}
}
This Method is an instance method of the Node class.
- (void)aquireFileInQueue:(NSOperationQueue *)queue {
if ([self.type isEqualToString:#"VIDEO"]) {
// Videos are available, but not downloaded.
self.isAvailableValue = YES;
return;
}
if (self.path == nil || self.fileName == nil) {
NSLog(#"Path or Filename for %# was nil", self.name);
return;
}
// Build the download URL !! MAKE SURE TO ADD PERCENT ESCAPES, this will protect against spaces in the file name
// Also make sure to slash-separate the path and fileName
NSURL * downloadURL = [NSURL URLWithString:[NSString stringWithFormat:#"%#/%#",
[self.path stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding],
[self.fileName stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding]]];
// Build the download request
NSURLRequest * downloadRequest = [NSURLRequest requestWithURL:downloadURL];
// FIXME: Authentication Code for JSON service
// Show network activity indicator
[[UIApplication sharedApplication] setNetworkActivityIndicatorVisible:YES];
// Send Asynchronus Request for fileData
[NSURLConnection sendAsynchronousRequest:(NSURLRequest *)downloadRequest queue:queue completionHandler:^(NSURLResponse * response, NSData * data, NSError * error) {
// Hide network activity indicatior
[[UIApplication sharedApplication] setNetworkActivityIndicatorVisible:NO];
// Cast URL Response to HTTPURLResponse
NSHTTPURLResponse * httpResponse = (NSHTTPURLResponse *)response;
// If statusCode is 200 (successful) and data is not nil, save data
if (httpResponse.statusCode == 200 && data) {
[data writeToURL:[self fileURL] atomically:NO];
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
[self setIsAvailable:[NSNumber numberWithBool:YES]];
}];
}
}];
}
- (void)prepareForDeletion {
// Remove file from Filesystem
[[NSFileManager defaultManager] removeItemAtURL:[self fileURL] error:nil];
}
- (NSURL *)fileURL {
// Return local file URL
return [NSURL fileURLWithPath:[NSString stringWithFormat:#"%#/%#", [Node applicationDocumentsDirectory], self.fileName]];
}
I am not familiar with MagicalRecords
A 'Could not fullfil fault' error occur when a context is holding an un-faulted object (an object stub), but the actual object in the database does not exist (deleted or was never saved).
My first advice:
If you work in a multithreaded environment, try to hold faulted objects.
use -existingObjectWithId:error: and fetch requests with:
[fetchRequest setReturnsObjectsAsFaults:NO];
[fetchRequest setIncludesPropertyValues:YES];
[fetchRequest setRelationshipKeyPathsForPrefetching:/*relationships you can afford to prefetch*/];
My second advice (to debug your issue):
Print you deletedObjects set before each save you make to the store to see which context caused the fault.
My third advice:
merge changes to the main context (my guess is that MagicalRecords does that for you).
note 1: deletes may be implied (you don't explicitly use deleteObject: by setting a relationship in cascade/deny mode for example)
note 2: you can not avoid this exception in a multithreaded environment (AFAIK), unless you pass all your saves through the main context (using parentContext) or by always using prefetched objects (not using relationships directly).