I am currently having an issue with Simperium on iOS with CoreData. Upon launching the app for the first time, Simperium tries to sync with CoreData.
Sometimes it will work 100% correctly, and other times it will set some of the relationships to "nil" despite the Simperium data on the server NOT saying that. This is then NEVER fixed unless I re-install the entire application. And then I roll the dice again to see if the relationships are linked properly on startup.
I cannot find a pattern in this. The relationships that are nil are completely random. Sometimes this entity over here will have nil relationships, and then the next time I try it a different entity will have nil relationships.
All relationships are optional and there is nothing wrong with my CoreData file.
Has anybody had an issue like this? I found the exact same issue in a bug from 2014, but it's been forever since then.
Exact same issue I am having can be found here: https://github.com/Simperium/simperium-ios/issues/250
Side Note: If you read that issue, he also clarifies that he has a problem with editing the CD file WHILE Simperium is syncing, causing nil relationships. Has anybody confirmed this happening? If so this might be my problem.
It's highly likely that this glitch is caused by the scenario you've described (editing the CD file while Simperium is Sync'ing).
Core Data deals, internally, with locks to maintain data integrity. Accessing the sql storage directly might result in data corruption (i'm assuming you're editing the file via either a Firefox plugin, in the simulator, or accessing the file via a Filesystem API).
Please, try to reproduce the issue without accessing the Core Data's sqlite file directly (always go thru NSPersistentStoreCoordinator / NSManagedObjectContext).
If you do succeed, and there's effectively a bug, we'd love to get it fixed! (In which case, please, open an issue the main repository, including as many details as possible).
Thanks for your interest in Simperium!
Related
I've spent the better part of a workday trying to solve this.
Background
I have a simple core data model, with books and reading sessions. The books have covers (images) that are stored as binary data with "Allows External Storage".
On iOS 11.4 and below, everything works fine all the time. When I save a new session everything gets updated properly.
Problem
Since iOS 12, when I create a new reading session and link it to the book, about every second time, core data generates a SQL statement that also updates the book cover field, sometimes resulting in a bad reference (to file on disk) which often results in the cover being nil when restarting the app, and almost always creates duplicate copy of the cover on disk (as can be seen in Simulator's _EXTERNAL_DATA folder).
In-memory context and objects remain correct though (and everything in the UI is therefore OK), until the app is restarted, then the cover is often nil.
iOS 12 specific
On iOS 12, I can deterministically reproduce the error in the simulator, on physical devices, and users have reported the error as well. I cannot reproduce the error on iOS 11.4, and no users reported the error previous to iOS 12.
Steps taken
I've enabled "-com.apple.CoreData.ConcurrencyDebug 1", so it shouldn't be that I'm accessing anything from the wrong queue. I've also enabled "-com.apple.CoreData.SQLDebug 3" so that I can see exactly what gets written.
I've made sure the Book instance (and therefore the cover) is not modified by my code before the association with the new Session by checking hasChanges, just before I do newSession.book = book and context.save().
To be 100% sure I'm not touching the cover property on any thread I've short-circuited my getters and setters for that property. No improvement.
I've tried using objectID to request an instance of the book just before the association and save. No improvement.
I've even tried the option where the context keeps strong references to all objects, just to make sure it was not some kind of memory management issue. No improvement.
Question
Any ideas for next steps?
Status update
This is a defect in iOS 12. See accepted answer below for a detailed description of a resonable workaround.
Update: The underlying Core Data issue appears to be resolved in iOS 12.1 (verified in beta 4). We will keep the workaround described below in our app, and won't be recommending using the External Storage option any time soon.
After talking to Apple engineers and filing the Radar mentioned above, we couldn’t wait around for a fix, so we took the hit and switched to storing files on the filesystem and managing it directly ourselves.
Another alternative that we considered was migrating our model not to allow External Storage for BLOBs, but I don't know what impact that would have had on performance and I was also worried about a model migration at a time when this part of iOS seems to be unstable, especially after reading stories like this in the past: Core Data: don’t store large files as binary data – Alexander Edge – Medium
It wasn't too much of a pain to implement local storage ourselves. You just need to have a unique identifier for each record that you can use to create a filename so you can map files to records. We added an extension to our Managed Object subclass with methods for reading, writing and deleting the files. Now, instead of calling e.g. article.photo = image.pngData(), we now need to call something like article.savePhoto(image.pngData()) and then we do similar when we want to retrieve the image. You can also add some code to these methods to support backwards compatibility with any images that are currently stored in Core Data.
Deletion was a little more tricky because our objects are deleted from multiple places in the code, including cascading deletes. In the end I opted to do it in the managed object's prepareForDeletion method but it is not ideal. There is plenty of discussion of how best to implement this here: cocoa - How to handle cleanup of external data when deleting unsaved Core Data objects? - Stack Overflow
Finally, to prevent our app crashing when a non-Optional binary attribute has disappeared because of this bug, I override awakeFromFetch in my Managed Object subclass to ensure that any required attributes are not nil, and if they are, I set them to a placeholder image so that they can be saved without the validation failing.
When my iOS app is started for the first time, it initializes a few entities with default data. The same process is completed for every device of the same user when the app is first installed on the device. This leads to a problem with Simperium, because, even if I have a constraint on one of the entity's attributes, it create duplicates. How can avoid that from happening? Is there a way to make Simperium skip objects locally initialized ? Or to prevent it from inserting duplicates?
Would setting the simperiumKey of the entity to a custom constant value which is the same for all the devices of the user fix the issue? I mean, would that prevent Simperium from importing the very same entity from different devices that would produce duplicates and sync conflicts ?
Because I tried that option and it looks like the buckets on Simperium Server are ok, but when I dispose the view and I open it again the entities are not there anymore. When I restart the app they are back again... it looks like they desappear to reappear after resetting the app.. very strange.
UPDATE
The problem is that I get duplicated entities if I try to insert the same entity from different device. For example. When I setup my app for the first time, I have a function that initializes CURRENCY entities with codes and other things. The same operation would be done from a different device if the user decides to install the app on a new one. In this case, because the new device will initialize again the CURRENCY entities, those info will generate duplicates and conflict errors. I need a way to make Simperium understand that the entities locally inizialized in the devices must not be duplicated. I would remove the inheritance from SPManagedObject in order to stop Simperium from syncing the entities, but in their turn they have relations with other entities and that would definitively create problems with Simperium, because it will try to sync entities which have relations with objects not inheriting from SPManagedObject. Hope you now have a more clear idea.
After reading various posts on this subject I've understood that not signing out (that means: not calling signOutAndRemoveLocalData) and setting custom simperiumKey-s would prevent data duplication. I tested this solution and it apparently looks good. My app needs Simperium to sync data with other devices of the same user, but it mainly works with the local CoreData database. I hope this solution is good for this scenario.
I have a rather big CoreData Model which suddenly behaves weirdly. There are three relevant entities:
User
Story
Group
User has a to-many relationship named stories to Story objects and a to-many-relationship groups to Group objects. Both also have an appropriate inverse. I use mogenerator to generate convenience accessors for all the properties and relationships. Both are ordered to-many relationships.
Now it sometimes happens that when I ask the user object for its groups (like user.groups) that I actually get an NSOrdered set of XNGStory objects. In the few cases that I have observed this they were the same objects that are returned by user.stories. The two sets weren't the same (different pointers), but their contents were (I checked by calling array] valueForKeyPath:#"objectID.URIRepresentation"] on them and got the same URIs in the same order both cases).
It feels like there is some way in my app that stores stories in the groups relationship but I checked the code and there are only Groups-related classes touching groups relationship and only story-related classes touching the story-relationship. Also there is no interaction or relationship between the two they are used in completely separate parts of the app, so I have no idea how this kind of data corruption can happen.
Is there any reason why such a data corruption might happen? I would have expected CoreData to complain when I store objects in the wrong kind of relationship accidentally.
One more thing: So far we were only able to reproduce it on iOS 10.
Things we tried:
We suspected a memory issue first (because an object of the wrong type turns up where it's not supposed to be), but running with Address Sanitizer and NSZombies enabled didn't reveal any issues. Also, once the issue occurs it persists over app restarts and using the debugger to poke around in the objects also shows a consistent(ly wrong) data model so it's not just one pointer that's wrong.
Deactivated WAL (issue still exists)
Looked at sqlite file (both with and without WAL) after crash with Core Data Editor: It displays Group-objects for the groups relationship, so all seems to be good on the sqlite-database-level -> Somehow the relationships get mixed up when loaded from the file.
As best I can tell, this seems to be an iOS 10 bug.
We have a similar problem with an app that uses Core Data - a User object has a relationship Addresses (to Address objects) and Friends (to other User objects).
For some reason under iOS 10 Core Data occasionally chooses to return the Address objects in the Friends relationship.
We've regression tested back to iOS 9 and this issue doesn't occur.
There's an open bug on Open Radar for the same issue - 26826183. Interestingly we've only started seeing this issue under 10.0.2 but that Radar report was based on an early iOS 10 Beta.
We have tested this extensively and also conclude that it is an iOS 10 bug. It also does not seem to be related to any of our code, just the data model. We were able to recreate the issue in an example project using our data model, Xcode 8's code generation and iOS 10's new NSPersistentContainer (before it broke using mogenerator and the "old way of setting up a CoreData stack").
To reproduce the issue we do the following:
At startup, if there is no User-object, create a User-object and add 10 Group objects to it and save.
Then add a Story to it, still on the main-context, and save.
Kill the app (via iOS swipe-up or simply via pressing "stop" in Xcode)
Start the app again. On startup, fetch the User object, add a Story object to to the stories relationship and save.
Check the user.groups relationship on a background or main context. It contains XNGStory objects now, although it should still have XNGGroup objects.
I filed a Radar with the example project. OpenRadar is here (without the example project, as it contains our data model).
The issue seems to be highly dependent on the data-model setup. It occurs when all-but-on relationships on our User entity are ordered. If all are ordered, it doesn't occurs, neither does it occur if two or more are unordered.
I m making an app with CoreData and iCloud integration. I went through the Apple Documentation to adapt my already working app to integrate iCloud synchronization.
Inserting objects and deleting seems to be working fine. When I delete the app and reinstall it on my phone all synchronized data are correctly restored.
My issue is when I modify an NSManagedObject. The context which I used to query CoreData to fetch my object does not detect any changes on my objects when I modify a field. Therefore the context is not saved. If I try to force saving context even when no changes are detected, nothing is saved.
I went through stack oververflow and found that the context must have a stalenessInterval set to 0.0. This did nothing to my app. Do you have any idea on what could be wrong ?
I noticed that the context carried by the NSManagedObject seems different than the one I used to fetch data. If I call save method on this context, nothing happened either.
I am completely lost, since I thought it would ba as easy as inserting and deleting objects.
Thanks for your help !
(ps: I code with Swift but even Objective C code is acceptable as answer :) )
I was actually trying to use two databases whereas I should have used configurations to separate entities saved to the cloud and those saved locally.
For those who wants more information on CoreData with iCloud i suggest to go through this video from Apple WWDC 2012 which helps a lot getting into this subject.
Current Setup:
MVC-1, onViewDidLoad, creates a Managed Object, fetches data from the Managed Object and updates various UI elements.
MVC-2: Same exact setup. I copied and pasted the same code from MVC-1 into MVC-2.
Issues I am seeing:
After launch and opening MVC-1, all the code is executed without any errors. The ManagedObject is created, the fetch requests on the ManageObjectContext work and the UI is properly updated. However, when switching to MVC-2 it seems that none of the data, that MVC-2 is suppose to fetch, is actually being updated. All the UI Elements in MVC-2 have the same data from the last time the application was launched.
Furthermore, if I launch the application and open MVC-2 first I get the same results, only MVC-1 does not appear to be updated this time around.
What is it that I am over looking? I have explicit saves to the database being made. I am (at least I think I am) creating two different ManagedObjects. Do I need to somehow close one before creating the other? Any advice would be very helpful. Thanks.
Figured it out.
I was saving the MOC, however I was not writing the NSManagedDocument to disk (I am using a ManagedDocument as a container for the Core Data database).
So naturally, when I switched to the MVC-2 I would not have the updated data.
I now save the MOC, write the MD to disk and close the MD at viewWillDisappear.
Thanks for the help gusy.