I am building app in iOS that saves data in Dropbox. Multiple device can use the same data. While doing this, sometime two device may overwrite same file. To avoid this situation is there any like lock file for writing.
Any alternative workaround solutions are also welcome.
While I don't know the Dropbox API, I would always be careful with a locking mechanism. I know from some systems, that the locks lead to a problem, if for example the app crashes or quits and the lock does not get released.
A very simple approach though would be to store the modification date when you have read the file. Then, before saving changes, compare your stored value with the most current one. If they are different, the file was modified. Next ask your users how to proceed and either commit the changes, cancel or create a new file with the same name and some appendix. That is how some sync clients I use are dealing with this problem.
Related
Let me get this out of the way right now: yes, it was almost certainly a mistake to not use Core Data. However, I was new to iOS development when I made these decisions, and I had no idea I'd be hamstrung like this. Moreover, the app is intended to also run on Android (eventually), so I avoided platform-specific APIs wherever possible.
I have an iOS app that stores data in a local SQLite database file. The data stored in the file is provided by the user, so it's important that it be kept safe. I had plans to "do this later", and later is now here. I am quickly coming to the realization that it won't be as straightforward as I had hoped...
I now understand that it won't be possible to seamlessly synchronize data across devices, and I'm willing to accept that limitation until I manage to migrate to Core Data. However, in the meantime I'd at least like the SQLite database to be backed up periodically so users can feel safe using the app on a single device. I was thinking I would do this:
periodically (e.g. once a week) copy the SQLite file from local storage into cloud storage, thus ensuring it is backed up
when the app starts, if the local store is missing or corrupted but the file exists in the cloud storage, ask the user if they would like to copy it over
The biggest problem with this approach is that the user could run the app on multiple devices and therefore the data stored in iCloud could be from any one of those devices, but only one. To combat that, I thought I could just use a per-device, unique name for the file in cloud storage. I would generate this using UIDevice.identifierForVendor.
So my startup logic would be:
Determine the unique name for the cloud file.
Is the local file missing or corrupted, and if so, does the cloud file exist?
2.1. Ask the user if they would like to restore from the cloud file. Make it really hard for them to say no because doing so will lose all their data.
2.2. If they say yes, copy the cloud file to the local file storage.
Open the local database file.
And running in the background I would occasionally copy the database file from local to cloud storage.
I would like to know whether this a sensible approach until I do Core Data integration. Also, are there any hidden "gotchas" that I'm perhaps missing?
UPDATE: as #TomHarrington pointed out in a comment, it turns out my database file is already sitting in /Documents, which is backed up to iTunes and any iCloud account. So my question morphs into this:
Should I simply ensure my database has a device-specific name so that it is not clobbered by the app running on another device connected to the same iCloud account?
I'm going to answer my question, since I ended up going down this path and finding a MASSIVE blocker. There is a bug in the UIDevice.identifierForVendor API that causes it to regenerate every time a new version of the app is installed! See here. This of course rules out using it as a device identifier. sigh
I think I'm SOL with that approach. Instead, I might generate a GUID on first execution and use that as my identifier. Problem is, I need to store that somewhere that isn't backed up to iCloud.
Ugh, I may just give up here and say my app can't be run on multiple devices until Core Data integration is done.
UPDATE: I ended up generating an identifier on first run and storing it in the keychain (as a local entry only so it isn't backed up to iCloud).
I have a CoreData app (using https://github.com/lhunath/UbiquityStoreManager), backed by iCloud. In one use case a user with a local store enables iCloud (where data already exists). I want to prompt the user to make a decision of whether to migrate the local data to iCloud or just use the iCloud version. As part of this, I'd like to display the device name and last sync date of the version in iCloud.
I've been tinkering around with my NSPersistentStore's metadata, but that doesn't appear to get synced to iCloud.
Any suggestions?
You could use iCloud's key-value store to store the device name & date of the last sync.
My no doubt unpopular suggestion is "don't". Trying to determine what is in iCloud at any given time puts you on pretty shaky ground. You may be able to get it to work most of the time, but there will always be circumstances where it breaks down.
If you really must import some data when first enabling iCloud, I suggest just always importing the data, and then deduping later as the iCloud data comes in. As ugly as it sounds, that's the only approach really guaranteed to work with Apple's approach.
It is worth taking a look at other Core Data sync frameworks like TICDS and Ensembles. They take a more sane approach to data identity, which means you can avoid the whole deduping step. (Disclosure: I develop Ensembles)
do a metadata query on the iCloud files and check the most recent transaction log file in iCloud. See the link below for a sample app that uses this approach to check whether the app is properly synchronised with iCloud.
http://ossh.com.au/design-and-technology/software-development/sample-library-style-ios-core-data-app-with-icloud-integration/
EDIT:
I just realised I don't get the actual device name, but once you have found the most recent log file then use this to get the device. Just be aware this call may be expensive.
NSFileVersion *openedVersion = [NSFileVersion currentVersionOfItemAtURL:fileURL];
return openedVersion.localizedNameOfSavingComputer;
When a file is passed into an iOS application by the document interaction system, a copy of the file is stored in the application bundle's Documents/Inbox folder. After the application has processed the file, it obviously needs to remove the file from Documents/Inbox, otherwise the folder will continue to grow and waste storage on the device.
I am uncomfortable with this simple solution (A), however, because my app needs to interact with the user before it can finish processing and removing the file. If the user suspends the app during this interaction period, and the app then gets killed while it is in the background, the stale file will not be removed when the app starts up the next time. Of course I can improve my app to cover this scenario, but I suspect that there will always be another border case that will leave me with an "unclean" Documents/Inbox folder.
A preferrable solution (B) therefore would be to remove the Documents/Inbox folder at an appropriate time (e.g. when the app launches normally, i.e. not via document interaction). I am still uncomfortable with this because I would be accessing a filesystem path whose location is not officially documented anywhere. For instance, my app would break if in a future version of iOS the document interaction system no longer places files in Document/Inbox.
So my questions are:
Would you recommend solution A or B?
Do you use a different approach and can you maybe give an outline of how your app manages Document/Inbox?
Last but not least: Do you know a piece of official Apple documentation that covers the topic and that I have overlooked?
Since I have asked this question, I have implemented the following solution:
I have redesigned my app so that it now immediately processes a file handed over to it via documentation interaction, without involving the user at all. Unless the app crashes, or is suspended and killed, in the middle of processing the file, this should always leave me with a clean Documents/Inbox.
To cover the (rare) case of a crash or suspend/kill, my app removes the Documents/Inbox folder when it is launched normallly (i.e. without the purpose of document interaction). To achieve this the Documents/Inbox folder path is necessarily hardcoded.
Here are the thoughts that went into the solution:
Redesigning the app
Initially it seemed like a good idea to offer the user a choice how she wanted a file to be processed - after all, offering a choice would make the app more flexible and provide the user with more freedom, right?
I then realized that I was trying to shift the responsibility for deciding how document interaction should be handled to the user. So I bit the bullet, made the hard decisions up-front, and then went happily on to implement a simple and straightforward document interaction system in my app.
As it turns out, no user interaction also means that the app is easier to use, so here's a win-win situation, both for me as a developer and for the users of my app.
Removing Documents/Inbox folder during app launch
Removing the Documents/Inbox folder during app launch is just an afterthought, not an essential part of how my app handles document interaction
Therefore I am quite willing to take the risk that Apple might change the filesystem path of the inbox folder at some point in the future. The worst thing that can happen is that my app will start to accumulate a few files that are leftovers from crash or suspend/kill scenarios.
And finally, a few thoughts for further development:
If it ever turns out that there really should be more than one way how the app can handle document interaction, I would add a user preference so that the user has to make a decision up-front, and the app does not need to stop its processing to ask the user for a choice.
If it ever turns out that user interaction is absolutely unavoidable in the middle of the document interaction handling process, I would look at this approach: 1) Before the user is allowed to interact, move the file from Documents/Inbox to some sort of "staging" folder; 2) Let user interaction take place; 3) Process the file in the "staging" folder, in whatever way the user chose. The important thing here is that the "staging" folder is in a known location and is completely managed by the app. Should the user suspend and then kill the app in the middle of the user interaction step, an appropriate action can simply be taken when the app is launched for the next time.
EDIT
In iOS 7 it is no longer possible to remove Documents/Inbox once it has been created. The NSFileManager method removeItemAtPath:error: returns Cocoa error 513 which resolves to NSFileWriteNoPermissionError (cf. this list of Foundation constants). The error does not seem to be related to POSIX permissions, however, it rather looks as if the system itself interferes with the attempt at deletion (possibly protection of the app bundle structure?).
Also noteworthy is that nowadays Apple explicitly names Documents/Inbox in the documentation of the UIApplicationDelegate method application:openURL:sourceApplication:annotation:. They also say that
[...] Your app has permission to read and delete files in this directory but does not have permission to write to them. If you want to modify a file, you must move it to a different directory first.
There is more stuff about possible encryption of the files in the docs, but you should read up on that yourself.
This problem has become much more complicated with the introduction of the Files app, and the "Open in place" feature.
If you do not have "Supports opening documents in place" switched on, in your info.plist, then things are pretty much the same, and files opened from any other app still appear in the Documents/Inbox directory. But files opened from the Files app appear in another inbox, currently at tmp/<bundle ID of app>-inbox. It is still recommended to delete the file once you are done with it, but there is less need to occasionally clean the directory, because the tmp directory gets cleaned by iOS once in a while anyways.
If you do have "Supports opening documents in place" switched on, then things change drastically. Files opened from the Files app and some other apps are no longer copied into an inbox, but they are passed to you at their original location. That is typically some location inside the Files app itself, inside another app referenced from the Files app, or even some general iCloud location. If you expose the files in your Documents folder, then it could even be one of your own app's files.
Needless to say, if you get such a file, you must not delete it. But if the file comes in an inbox, which will still happen a lot as well, then you must delete it. In order to determine this, the options of the application:openURL:options: call contains an UIApplicationOpenURLOptionsOpenInPlaceKey key. If that has value (NSNumber) NO, then the file is in an inbox, and it should be deleted. It it has value YES, then it is opened in-place, and must not be deleted.
Note that for files that are opened in place, you also need to gain permission to use them first. You do that but surrounding the access to the file by startAccessingSecurityScopedResource and stopAccessingSecurityScopedResource calls. See Apple documentation for details.
I am just now facing this same problem. Like you I don't have a good solution but in response to your questions I'm leaning towards option A over option B because I don't like the idea of potentially having an issue with future releases of the OS. Since, in my case, there is no user interaction once I display the preview of the document, I can go ahead and delete it when when I receive the –documentInteractionControllerDidEndPreview: delegate callback. This is theoretical in that I haven't coded this yet and won't get to it for a while as it is a low priority item. If it doesn't work or there are other issues, I will report back here. The Google search I entered in order to find documentation from Apple pointed to this StackOverflow post. I haven't seen any other useful information from Apple or anyone else on the subject.
I've been having trouble getting an app submitted to the App Store. This is due to the fact that that database, which is updatable, is too large for the iCloud backup limitations. Most of the data in the db is static, but one table records the user's schedule for reviewing words (this is a vocabulary quiz).
As far as I can tell, I have two or three realistic options. The first is to put the whole database into the Library/Cache directory. This should be accepted, because it's not backed up to iCloud. However, there's no guarantee that it will be maintained during app updates, per this entry in "Make App Backups More Efficient" at this url:
http://developer.apple.com/library/IOs/#documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/PerformanceTuning/PerformanceTuning.html
Files Saved During App Updates
When a user downloads an app update, iTunes installs the update in a new app directory. It then moves the user’s data files from the old installation over to the new app directory before deleting the old installation. Files in the following directories are guaranteed to be preserved during the update process:
<Application_Home>/Documents
<Application_Home>/Library
Although files in other user directories may also be moved over, you should not rely on them being present after an update.
The second option is to put the data into the NSDocuments or NSLibrary directory, as mark it with the skipBackupFlag. However, one problem is this flag doesn't work for iOS 5.0 and previous per this entry in "How do I prevent files from being backed up to iCloud and iTunes?" at
https://developer.apple.com/library/ios/#qa/qa1719/_index.html
Important The new "do not back up" attribute will only be used by iOS 5.0.1 or later. On iOS 5.0 and earlier, applications will need to store their data in <Application_Home>/Library/Caches to avoid having it backed up. Since this attribute is ignored on older systems, you will need to insure your app complies with the iOS Data Storage Guidelines on all versions of iOS that your application supports
This means that even if I use the "skipBackupFlag", I'll still have the problem that the database is getting backed up to the cloud, I think.
So, the third option, which is pretty much of an ugly hack, is to split the database into two. Put the updatable part into the NSLibrary or NSDocuments directory, and leave the rest in application resources. This would have the small, updatable part stored on the cloud, and leave the rest in the app resources directory. The problem is that this splits the db for no good reason, and introduces possible performance issues with having two databases open at once.
So, my question is, is my interpretation of the rules correct? Am I going to have to go with option 3?
p.s. I noticed in my last post cited urls were edited to links without the url showing. How do I do this?
Have you considered using external file references as described in https://developer.apple.com/library/IOS/#releasenotes/DataManagement/RN-CoreData/_index.html . Specifically, refer to "setAllowsExternalBinaryDataStorage:" https://developer.apple.com/library/IOS/documentation/Cocoa/Reference/CoreDataFramework/Classes/NSAttributeDescription_Class/reference.html#//apple_ref/occ/instm/NSAttributeDescription/setAllowsExternalBinaryDataStorage: . Pushing out large data into a separate file can help reduce database size .
I store some data in my iOS app directly in a local .sqlite file. I chose to do this instead of CoreData because the data will need to be compatible with non-Apple platforms.
Now, I'm trying to come up with the best way to sync this file over iCloud. I know you can't sync it directly, for many reasons. I know CoreData is able to sync its DBs, but even ignoring that using CD would essentially lock this file into Apple platforms (I think? I've only looked into CD a bit), I need the iCloud syncing of this file to work across ALL of iCloud's supported platforms - which is supposed to include Windows. I have to assume that there won't be any compatibility for the CoreData files in the Windows API. Planning out the best way to accomplish this would be a lot easier if Apple would tell us any more than "There will be a Windows API [eventually?]"
In addition, I'll eventually need to implement at least one more sync service to support platforms that iCloud does not. It would be helpful, though not required, if the method I use for iCloud can be mostly reused for future services.
For these reasons, I don't think CoreData can help me with this. Am I correct in thinking this?
Moving on from there, I need to devise an algorithm for this, or find an existing one or an existing 3rd party solution. I haven't stumbled across anything yet. However, I have been mulling over a couple possible methods I could implement:
Method 1:
Do something similar to how CoreData syncs sqlite DBs: send "transaction logs" to iCloud instead and build each local sqlite file off of those.
I'm thinking each device would send a (uniquely named) text file listing all the sql commands that that device executed, with timestamps. The device would store how far along in each list of commands it has executed, and continue from that point each time the file is updated. If it received updates to multiple log files at once, it would execute each command in timestamp order.
Things could get 'interesting' efficiency-wise once these files get large, but it seems like a solvable problem.
Method 2:
Periodically sync a copy of the working database to iCloud. Have a modification timestamp field in every record. When an updated copy of the DB comes through, query all the records with newer timestamps than some reference time and update the record in the local DB from the new data.
I see many potential problems with this method:
-Have to implement something further to recognize record deletion.
-The DB file could get conflicts. It might be possible to deal with them by handling each conflict version in timestamp order.
-Determining the date to check each update from could be tricky, as it depends on which device the update is coming from.
There are a lot of potential problems with method 2, but method 1 seems doable to me...
Does anyone have any suggestions as to what might be the best course of action? Any better ideas than my "Method 1" (or reasons why it wouldn't work)?
Try those two solutions from Ray Wenderlich:
Exporting/Importing data through mail:
http://www.raywenderlich.com/1980/how-to-import-and-export-app-data-via-email-in-your-ios-app
File Sharing with iTunes:
http://www.raywenderlich.com/1948/how-integrate-itunes-file-sharing-with-your-ios-app
I found it quite complex but helped me a lot.
Both method 1 and method 2 seem doable. Perhaps a combination of the two in fact - use iCloud to send a separate database file that is a subset of data - i.e. just changed items. Or maybe another file format instead of sqlite db - XML/JSON/CSV etc.
Another alternative is to do it outside of iCloud - i.e. a simple custom web service for syncing. So each change gets submitted to a central server via JSON/XML over HTTP, and then other devices pull updates from that.
Obviously it depends how much data and how many devices you want to sync across, and whether you have access to an appropriate server and/or budget to cover running such a server. iCloud will do that for "free" but all it really does is transfer files. A custom solution allows you to define your syncing model as you wish, but you have to develop and manage it and pay for it.
I've considered the possibility of transferring a database file through iCloud but I think that I would run into classic problems of timing - slow start for the user - and corrupted databases if the app is run on multiple devices simultaneously. (iPad/iPhone for example).
Sooo. I've had to use the transaction logs method. It really is difficult to implement, but once in place, seems ok.
I am using Apple's SharedCoreData sample as the base for this work. This link requires an Apple Developer Account.
I did find a much much better solution from Tim Roadley however this only works for IOS and I needed both IOS and MacOS.
rant> iCloud development really has to get easier and more stable! /rant