I have been trying to retrofit a large app to have it store its files in iCloud. The app is not really "Document-based" but the files may be thought of as configuration or preference files.
To be honest I haven't followed all the Apple iCloud guidelines. Some really don't fit in with the flow of the app. When the app starts I read the files from the ubiquity directory (using normal file reads) and when I write them I use a normal file write to the ubiquity directory. When the app start, I also call:
[fileManager startDownloadingUbiquitousItemAtURL:url error:&error];
I do have an NSFilePresenter watching for changes in the ubiquity directory. It notifies me of file changes but there are never any conflict notifications.
My problem is that often when I upload a file to iCloud, it will create a separate file with a number appended. E.g.
MyFile.skyset
MyFile 2 .skyset
These seem to show up when more than one app has been writing MyFile.skyset to the ubiquity directory.
They don't seem to be conflicted file versions. If I use NSFileVersion to look for conflicts and other versions, I only see the one version of MyFile.skyset and it is not in conflict.
I can't find any documentation that explains what this "versioned" file is and what to do about it. Any thoughts as to what is going on here?
Well, I don't fully understand the "versions" but the problem was solved by changing how I updated the files in the ubiquity directory.
I changed from:
[fileMgr removeItemAtPath:dstPath error:nil];
[fileMgr copyItemAtPath:srcPath toPath:dstPath error:&error];
To:
NSData *data = [NSData dataWithContentsOfFile:srcPath];
[data writeToFile:dstPath atomically:NO];
I looks like the operation of deleting the file before copying the updated information to the ubiquity directory was confusing the system. After switching to the second form I'm no longer seeing this problem.
Hope this answer helps someone else.
Related
I'm developing an IOS app which download some video and save it, but after update app or reinstall it the directory of app change, and app can not access to previous downloaded files.
Is there any way to save downloaded file to location out of application domain directory?
the code for generating path to save file is.
NSURL *documentsDirectoryURL = [[NSFileManager defaultManager] URLForDirectory:NSDocumentDirectory inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil];
return [documentsDirectoryURL URLByAppendingPathComponent:[response suggestedFilename]];
I cannot verify the update part of your question.
I use the application's Documents all the time and after an update that is untouched, so I can still use any files added to it in a previous version.
For the reinstalling (i.e. first delete the app, then re-download it from the AppStore and run it) there is obviously no such way, unless you consider what ravi.p suggested in his comment (adding it to PhotosAlbum). That would beat the entire purpose of a reinstall. If a user deletes the app they want to delete all its data. That's why the warning dialog specifically makes that clear. If you could circumvent this would beat the intention, wouldn't it?
The previous Documents directory isn't changed in this case, it is deleted and on the reinstall a new one is created.
I didn't check it recently, but I believe even the full path of the Documents directory doesn't change after an update, including the hashed part that is created by the sandboxing mechanism. My guess is that you're either confusing something with the update process or simply get a different file name in the newer app version.
Edit:
Okay, so apparently the application's folders can change on an update, at least according to what you further explained in your comment. :)
Then I am wondering how you can find your sqlite file again, but in ay way there is an easy solution to your problem:
Do not save the full path in your database. That is bad practice anyways, since the method you already used to get it in the first place (after downloading the file) is meant to be used for accessing the folder. So you only save
[response suggestedFilename]
in your database (going from your posted code).
In places where you need to access the file you then simply rebuild your path in just the same way you do in the first line of code you posted.
You can even write a convenience method for this in some place (a model class or the class taking care about DB access).
And by the way: Again, updating and reinstalling are two different things, especially the process in which Xcode copies your development build onto your development device is a different one from how users ultimately update your app from the store.
If I save a file to the documents directory in a folder, how long does it stay on the disk? Forever (in that case I would have to delete it manually if I wish to do so) or it is deleted every time the app closes?
Files in the documents directory will persist until either the user deletes the app, resets their device or you remove the files in code. Also note however that the contents of the documents directory are backed up by iTunes so could persist longer than you would otherwise expect.
On the other hand, the tmp directory is not so persistent and its contents won't necessarily survive an app relaunch (they're also not backed up by iTunes) so you could use this if you don't want persistant storage. /Library/Caches/ is similar in that it isn't backed up but it is persistent. From the docs regarding tmp:
The system will periodically purge these files [in the tmp folder] when your app is not running; therefore, you cannot rely on these files persisting after your app terminates.
Basically, if you want to store something for a short period and it doesn't matter if it's deleted by the OS, use tmp. If you want something that will persist app launches and whatnot but still won't be backed up via iTunes, use the cache folder in Library. It you want something that persists and is backed up, use documents.
Extra info based on comment
If the images should disappear when the app is relaunched then you should use tmp. But take note that the tmp directory is not guaranteed to be cleared on an app relaunch. It's certainly possible that the images will still be there the next time you load the app. It's a bit of a gamble really. If it is vital to kill the images then wipe them manually somewhere suitable:
NSArray* tmpDirectory = [[NSFileManager defaultManager] contentsOfDirectoryAtPath:NSTemporaryDirectory() error:NULL];
for (NSString *file in tmpDirectory) {
[[NSFileManager defaultManager] removeItemAtPath:[NSString stringWithFormat:#"%#%#", NSTemporaryDirectory(), file] error:NULL];
}
I'm storing some files in the Library directory in an iOS app, using the following methods to construct it. In the end, I can call [MyClass dataDirectory] to do my file handling and all is well. I've recently discovered, however, that some files seem to be mysteriously disappearing out of this directory. According to the documentation, this should not be the case. Is this a safe place to store persistent files?
The console output of this directory is: ~/var/mobile/Containers/Data/Application/{id}/Library/Data
+ (NSString*)libraryDirectory
{
return [NSSearchPathForDirectoriesInDomains(NSLibraryDirectory, NSUserDomainMask, YES) lastObject];
}
+ (NSString*)dataDirectory
{
NSString* dir = [[self libraryDirectory] stringByAppendingPathComponent:#"Data"];
BOOL isDir=NO;
NSError * error = nil;
NSFileManager *fileManager = [NSFileManager new];
if (![fileManager fileExistsAtPath:dir isDirectory:&isDir] && isDir)
{
[[NSFileManager defaultManager] createDirectoryAtPath:dir
withIntermediateDirectories:YES
attributes:nil
error:&error];
}
[self addSkipBackupAttributeToItemAtURL:[NSURL fileURLWithPath:dir isDirectory:YES]];
if (error != nil) {
DDLogError(#"Fatal error creating ~/Library/Data directory: %#", error);
}
return dir;
}
And the skip method:
+ (BOOL)addSkipBackupAttributeToItemAtURL:(NSURL *)URL
{
if ([[NSFileManager defaultManager] fileExistsAtPath:[URL path]])
{
assert([[NSFileManager defaultManager] fileExistsAtPath: [URL path]]);
NSError *error = nil;
BOOL success = [URL setResourceValue: [NSNumber numberWithBool: YES]
forKey: NSURLIsExcludedFromBackupKey error: &error];
if(!success){
DDLogError(#"Error excluding %# from backup %#", [URL lastPathComponent], error);
}
return success;
}
return YES;
}
In the code you posted, the first problem is here:
if (![fileManager fileExistsAtPath:dir isDirectory:&isDir] && isDir)
At the point where this is evaluated, isDir will default to NO, and will be set to NO if the file does not exist or is not a directory. This will prevent the directory from being created. Remove && isDir or change to || !isDir to get the logic you want.
Now on to your original question:
Is this (a subdirectory of NSLibraryDirectory) a safe place to store persistent files?
Yes. NSLibraryDirectory is backed up by default. To comply with the iOS Data Storage Guidelines an application should not store user-created data in that location, but it is a safe place to store application data. NSApplicationSupportDirectory is a directory that is generally within the NSLibraryDirectory, and is the preferred place to store this kind of data. Data within that location will be backed up, and will be migrated during application and OS updates.
The iOS Data Storage Guidelines, File System Programming Guide, and App Programming Guide for iOS all provide guidance on where to put files, and how they will be backed up from standard file system locations.
Unless those files have had their NSURLIsExcludedFromBackupKey/kCFURLIsExcludedFromBackupKey resource metadata value altered. Then it gets much more complicated.
Files 'Excluded From Backup'
Generally, if a file outside of a Documents directory can be backed up, the system assumes it can also purge it under low space or other conditions. This is why setting NSURLIsExcludedFromBackupKey to YES on a file allows the file to persist even in low storage conditions. If your application sets NSURLIsExcludedFromBackupKey to YES for a file, your application assumes responsibility for the life of that file.
The catch here is that the backup process and the purge process do not follow the same logic. Apple's documentation indicates that for the purposes of controlling the backup behavior, it is possible to set NSURLIsExcludedFromBackupKey on a directory. The children of that directory will effectively inherit that resource value (in practice, this may not be accurate). The purge process, however, does not seem to have the same behavior. It may not check the backup exclusions of the parent directories and apply it to children, and as a result if a file does not have NSURLIsExcludedFromBackupKey explictly set it may be purged.
This gets even more complicated. If you were to read the documentation for the constant NSURLIsExcludedFromBackupKey you would see:
Some operations commonly made to user documents cause this property to be reset to false; consequently, do not use this property on user documents.
This actually applies to much more than user documents. For example, if you were to perform an atomic write on a file such as:
[thing writeToURL:URL atomically:YES encoding:NSUTF8StringEncoding error:&error]
If the file at URL had NSURLIsExcludedFromBackupKey set to YES before the write, it would now appear to be set to NO. An atomic write like this will first create a temporary file, write to that, and replace the original with the new file. In doing so, file and URL resource flags are not preserved. The original file had the NSURLIsExcludedFromBackupKey resource value set, the newly created file at the same location now does not. This is just one example; many Foundation APIs perform atomic writes like this implictly.
There are scenarios where this gets even more complex. When an application is updated it is installed into a new location with a new application container path. Data inside the old application container is migrated. There are few guarantees regarding what may or may not be migrated as part of the update process. It may be everything, it may be only some things. In particular there are is no guidance concerning how files or directories marked with the NSURLIsExcludedFromBackupKey resource attribute will be treated. In practice it seems that these are often the least likely files to be migrated, and when they are migrated the NSURLIsExcludedFromBackupKey attribute is rarely preserved.
OS updates are also an issue. Historically Over-The-Air updates have been problematic and have caused the NSURLIsExcludedFromBackupKey resource attribute to be effectively cleared or ignored. A "major" OS update will clear the device and restore from a backup - which is equivalent to migrating to new hardware. Files marked with the NSURLIsExcludedFromBackupKey resource attribute will not be migrated, and the application will have to re-create them.
Update scenarios are described in TechNote 2285: Testing iOS App Updates
Because of this, when using NSURLIsExcludedFromBackupKey it is generally best to set the value on every access, and as always should be done through the File Coordination APIs (unless you are writing to a shared group container, which is an entirely different set of issues). If the NSURLIsExcludedFromBackupKey resource attribute value is lost files can be purged at any time. Ideally an application should not depend on the NSURLIsExcludedFromBackupKey or how the OS may (or may not!) handle it, but instead be designed such that the data could be recreated on demand. That may not always be possible.
It's clear from your question and the code that you posted that you are somewhat dependant on NSURLIsExcludedFromBackupKey ensuring that your file(s) have an application-controlled lifetime. As you can see from the above, that may not always be the case: there are many, many common scenarios where that resource attribute value can disappear, and with it your files.
It is also worth noting that NSFileProtection attributes work the same way, and can disappear in the same scenarios (and a few more).
TL;DR; What should I do?
Based on your question, code, and the description of the behavior you are seeing:
Setting the NSURLIsExcludedFromBackupKey value on the directory containing the file(s) you are interested in preserving may not be enough to prevent them from being purged. It would be wise to set NSURLIsExcludedFromBackupKey on every access to the actual files, rather than just a parent directory. Also attempt to ensure this resource value is set after any write to the file, especially through a high level API that may be doing atomic writes, etc.
All NSFileManager and file reading/writing operations should use file coordination. Even in an application that is single threaded there will be other processes interacting with "your" files. Processes like the daemons that run backups or purge files during low space conditions. Between your -fileExistsAtPath: and the -setResourceValue:forKey:error: another process could alter, delete, or move your file and its attributes. -setResourceValue:forKey:error: will actually return YES and no error in many cases where it did nothing, like the file not existing.
Files and directories marked with NSURLIsExcludedFromBackupKey are the responsibility of the application to manage. The application should still purge those files or their contents at some appropriate time, or set limits on their growth. If you look at the per-application disk usage information on a device, you can probably guess the names of some applications that do not do this correctly.
Test update scenarios as described in TechNote 2285: Testing iOS App Updates. often. Ideally the iOS Simulator would have a "Simulate Low Disk Space" capability similar to simulating memory warnings, but at this time it does not.
If at all possible, alter application logic to recreate these files if they go missing.
In the documentation you linked it is stated that
Critical data should be stored in the /Documents directory. Critical data is any data that cannot be recreated by your app, such as user documents and other user-generated content.
It is also mentioned that
Cached data should be stored in the /Library/Caches directory. Examples of files you should put in the Caches directory include (but are not limited to) database cache files and downloadable content, such as that used by magazine, newspaper, and map apps. Your app should be able to gracefully handle situations where cached data is deleted by the system to free up disk space.
The directory you are using is not explicitly mentioned for storing user data, it is used by the system and is not save for your data. It's guarantied to be untouched by an update of your app, but that's it
To find the documents folder you could do something like
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsFolderPath = [paths firstObject];
After reading Apple documentation, I used URLsForDirectory to obtain the Library destination within my app, and my objective is to download content from a hosted service & cache it in the Library folder, so that User cannot interact with the contents through iTunes.
NSArray *docPaths= [[NSFileManager defaultManager] URLsForDirectory:NSLibraryDirectory inDomains:NSUserDomainMask];
NSLog(#"%#",[docPaths description]);
NSString *docPath=[(NSURL *)[docPaths objectAtIndex:0] path];
docPath = [docPath stringByAppendingPathComponent:#"audios"];
Running & executing the code several times(various simulators, and iOS 8.0 device) I realized that somehow the content being fetched seems to be no longer accessible, so I logged the library destination path, and after running app every time the destination path seems to have changed:
/var/mobile/Containers/Data/Application/83725F33-C7EA-4F89-B69F-0AECF26FA77A/Library/"
/var/mobile/Containers/Data/Application/4627FC86-C3A4-4A1A-9721-AF73D808433E/Library/"
/var/mobile/Containers/Data/Application/709CCA84-936A-4596-933A-D6779758FF85/Library/
Has anybody faced a similar issue? If so how did it got corrected? And is there anything I've missed out here?
I had the same issue. I think the variable part changes only when the code is recompiled (ie not if you just rerun without making changes), so should not affect a live app. But I decided in the end not to save the path - just to use the same code (as you use above) both when saving and retrieving the data. It seems to work, in spite of the fact that the path actually changes between runs (so the simulator must copy the files across, or rename the folder).
I'm looking for a better way to download a bunch of files from my iCloud container to my sandbox. This is what I currently use:
for (FileAttachments *fileLink in items) {
NSURL *cloudFileURL = [fileLink urlForCloudAttachmentWithName];
NSURL *fileURL = [backupCloudLocalDirectoryURL URLByAppendingPathComponent: fileLink.fileName];
NSFileCoordinator *fileCoordinator = [[NSFileCoordinator alloc] initWithFilePresenter:nil];
[fileCoordinator coordinateReadingItemAtURL:fileURL options:NSFileCoordinatorReadingWithoutChanges error:&error
byAccessor:^(NSURL *newURL) {
NSError *blockError = nil;
[fileManager copyItemAtURL:cloudFileURL toURL:fileURL error:&blockError];
}];
}
}
Is there any problem with making a copy of the iCloud item this way? In production, I have some users complaining that all their files weren't downloaded. Is it better to use NSFileManager's startDownloadingUbiquitousItemAtURL:error instead? If so, why?
Thanks.
It's still not completely clear to me from comments how you're discovering the URLs, however some important details that affect your situation are:
Using coordinateReadingItemAtURL:block: has nothing to do with downloading files from iCloud. The purpose of NSFileCoordinator is to coordinate among readers and writers of a file so that, for example, you don't get two threads trying to write to the same file at the same time. You use file coordinators with iCloud because the iCloud system needs to read and write files and so does your app. Using a coordinator avoids corrupting the file, but again, has nothing to do with downloading the file.
To download a file from iCloud you need to use startDownloadingUbiquitousItemAtURL:error: and then wait until the file downloads. The normal flow here is:
a. Use NSMetadataQuery to find files that exist in the iCloud account
b. Use startDownloadingUbiquitousItemAtURL:error: to make sure they're available on the local device.
The reason you need to use this call is simply because that's how iCloud works on iOS. Files don't download until you ask them to download, and this is how you ask. [On OS X it's different, everything automatically downloads.]
You cannot simply copy from an iCloud URL to another location unless you already know that the file has been downloaded. The file-copy operation does not download the file, so if the file isn't available locally, the copy will fail.
You must use a metadata query to identify the files and their download status then, if they have not been downloaded initiate the download, and using the metadata query determine when the download is complete and then copy the file from the ubiquity container to the apps sandbox directory using a file coordinator. If you try copying the file while it's partially downloaded you may get some strange results.
I had the same problem with you.
My case is that: When network disconnects, iCloud service copy file from iCloud container to sandbox. When this line executes, it can not go into the block to copy file. This is the reason why this file can not be copied.
fileCoordinator coordinateReadingItemAtURL:fileURL options:NSFileCoordinatorReadingWithoutChanges error:&error byAccessor:^(NSURL *newURL)
My solution is: Before copying file from iCLoud container to sandbox, you must check network. If it is not available, don't execute this code (return method). If network connects, execute fileCoordinator and copy.
More info: When copy file from ICloud container to sandbox, method fileManager copyItemAtURL:toURL:error: is OK because I implement this method and it's good.
Hope this works.