If I save a file to the documents directory in a folder, how long does it stay on the disk? Forever (in that case I would have to delete it manually if I wish to do so) or it is deleted every time the app closes?
Files in the documents directory will persist until either the user deletes the app, resets their device or you remove the files in code. Also note however that the contents of the documents directory are backed up by iTunes so could persist longer than you would otherwise expect.
On the other hand, the tmp directory is not so persistent and its contents won't necessarily survive an app relaunch (they're also not backed up by iTunes) so you could use this if you don't want persistant storage. /Library/Caches/ is similar in that it isn't backed up but it is persistent. From the docs regarding tmp:
The system will periodically purge these files [in the tmp folder] when your app is not running; therefore, you cannot rely on these files persisting after your app terminates.
Basically, if you want to store something for a short period and it doesn't matter if it's deleted by the OS, use tmp. If you want something that will persist app launches and whatnot but still won't be backed up via iTunes, use the cache folder in Library. It you want something that persists and is backed up, use documents.
Extra info based on comment
If the images should disappear when the app is relaunched then you should use tmp. But take note that the tmp directory is not guaranteed to be cleared on an app relaunch. It's certainly possible that the images will still be there the next time you load the app. It's a bit of a gamble really. If it is vital to kill the images then wipe them manually somewhere suitable:
NSArray* tmpDirectory = [[NSFileManager defaultManager] contentsOfDirectoryAtPath:NSTemporaryDirectory() error:NULL];
for (NSString *file in tmpDirectory) {
[[NSFileManager defaultManager] removeItemAtPath:[NSString stringWithFormat:#"%#%#", NSTemporaryDirectory(), file] error:NULL];
}
Related
I am implementing a resumable upload protocol that uploads in the background on iOS, which means I have to use an NSURLSessionUploadTask with a file. Since it's a resumable upload protocol, the file needs to be truncated based on the data that has already been received by the server, so I need to save a new temporary file to disk that has only the bytes to be uploaded within it.
If I can create that temporary upload file in the tmp/ or /Library/Caches/, can I trust that it will be kept as long as the NSURLSession is running?
EDIT: When an upload fails, the server will be saving the bytes it has already received and communicating that to the client. The client then should only send part of the file, which is why I need to create a smaller temporary file that must not be deleted mid-upload.
Huh? You provide the entire file, and the system takes care of managing the partial upload/download, and notifies you once the transfer is complete. In the case of a download, t hands you a temporary file once the download is complete and you have to save it to a permanent location.
You should not be mucking around with partial files at all.
EDIT:
You don't have access to tmp or /Library/Caches/, except through the sandbox. You can get access to the caches directory with the call
[NSSearchPathForDirectoriesInDomains(
NSCachesDirectory,
NSUserDomainMask, YES) lastObject];
It's my understanding that the caches directory only gets purged on restart, or if the device gets critically low on space, but I seem to remember that the docs are vague on when, exactly, the caches directory gets cleared.
You would probably be better off saving your file to the documents directory, then deleting it once you're done with it.
The answer to your question is no. NSURLSessionUploadTask's description appears to support keeping the source file around but it's misleading:
"In iOS, when you create an upload task for a file in a background session, the system copies that file to a temporary location and streams data from there"
But it says nothing about whether it will keep the original source file in the tmp directory. Specifically for your case where your server supports uploading partial files and you need to restart them after failures. Or in the more common situation where you need to manually restart an entire failed upload, for example from a retry-able server error, or if user killed your app and then restarted it (iOS doesn't continue uploads for user killed apps).
In these cases you can't count on the file still being around if you create it in the apps tmp directory. The file system programming guide tells us this.
https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html
"Use this directory to write temporary files that do not need to persist between launches of your app. Your app should remove files from this directory when they are no longer needed; however, the system may purge this directory when your app is not running. The contents of this directory are not backed up by iTunes or iCloud."
So any tmp directory files can be deleted by iOS when your app stops running, and I can confirm I've seen this in production releases of our app. If you think you may need the source file for the upload again, you must store it in your own app directory, and manage deleting it yourself when done with it. Sorry, extra work, but I don't know of any way around it.
I have been trying to retrofit a large app to have it store its files in iCloud. The app is not really "Document-based" but the files may be thought of as configuration or preference files.
To be honest I haven't followed all the Apple iCloud guidelines. Some really don't fit in with the flow of the app. When the app starts I read the files from the ubiquity directory (using normal file reads) and when I write them I use a normal file write to the ubiquity directory. When the app start, I also call:
[fileManager startDownloadingUbiquitousItemAtURL:url error:&error];
I do have an NSFilePresenter watching for changes in the ubiquity directory. It notifies me of file changes but there are never any conflict notifications.
My problem is that often when I upload a file to iCloud, it will create a separate file with a number appended. E.g.
MyFile.skyset
MyFile 2 .skyset
These seem to show up when more than one app has been writing MyFile.skyset to the ubiquity directory.
They don't seem to be conflicted file versions. If I use NSFileVersion to look for conflicts and other versions, I only see the one version of MyFile.skyset and it is not in conflict.
I can't find any documentation that explains what this "versioned" file is and what to do about it. Any thoughts as to what is going on here?
Well, I don't fully understand the "versions" but the problem was solved by changing how I updated the files in the ubiquity directory.
I changed from:
[fileMgr removeItemAtPath:dstPath error:nil];
[fileMgr copyItemAtPath:srcPath toPath:dstPath error:&error];
To:
NSData *data = [NSData dataWithContentsOfFile:srcPath];
[data writeToFile:dstPath atomically:NO];
I looks like the operation of deleting the file before copying the updated information to the ubiquity directory was confusing the system. After switching to the second form I'm no longer seeing this problem.
Hope this answer helps someone else.
I'm storing some files in the Library directory in an iOS app, using the following methods to construct it. In the end, I can call [MyClass dataDirectory] to do my file handling and all is well. I've recently discovered, however, that some files seem to be mysteriously disappearing out of this directory. According to the documentation, this should not be the case. Is this a safe place to store persistent files?
The console output of this directory is: ~/var/mobile/Containers/Data/Application/{id}/Library/Data
+ (NSString*)libraryDirectory
{
return [NSSearchPathForDirectoriesInDomains(NSLibraryDirectory, NSUserDomainMask, YES) lastObject];
}
+ (NSString*)dataDirectory
{
NSString* dir = [[self libraryDirectory] stringByAppendingPathComponent:#"Data"];
BOOL isDir=NO;
NSError * error = nil;
NSFileManager *fileManager = [NSFileManager new];
if (![fileManager fileExistsAtPath:dir isDirectory:&isDir] && isDir)
{
[[NSFileManager defaultManager] createDirectoryAtPath:dir
withIntermediateDirectories:YES
attributes:nil
error:&error];
}
[self addSkipBackupAttributeToItemAtURL:[NSURL fileURLWithPath:dir isDirectory:YES]];
if (error != nil) {
DDLogError(#"Fatal error creating ~/Library/Data directory: %#", error);
}
return dir;
}
And the skip method:
+ (BOOL)addSkipBackupAttributeToItemAtURL:(NSURL *)URL
{
if ([[NSFileManager defaultManager] fileExistsAtPath:[URL path]])
{
assert([[NSFileManager defaultManager] fileExistsAtPath: [URL path]]);
NSError *error = nil;
BOOL success = [URL setResourceValue: [NSNumber numberWithBool: YES]
forKey: NSURLIsExcludedFromBackupKey error: &error];
if(!success){
DDLogError(#"Error excluding %# from backup %#", [URL lastPathComponent], error);
}
return success;
}
return YES;
}
In the code you posted, the first problem is here:
if (![fileManager fileExistsAtPath:dir isDirectory:&isDir] && isDir)
At the point where this is evaluated, isDir will default to NO, and will be set to NO if the file does not exist or is not a directory. This will prevent the directory from being created. Remove && isDir or change to || !isDir to get the logic you want.
Now on to your original question:
Is this (a subdirectory of NSLibraryDirectory) a safe place to store persistent files?
Yes. NSLibraryDirectory is backed up by default. To comply with the iOS Data Storage Guidelines an application should not store user-created data in that location, but it is a safe place to store application data. NSApplicationSupportDirectory is a directory that is generally within the NSLibraryDirectory, and is the preferred place to store this kind of data. Data within that location will be backed up, and will be migrated during application and OS updates.
The iOS Data Storage Guidelines, File System Programming Guide, and App Programming Guide for iOS all provide guidance on where to put files, and how they will be backed up from standard file system locations.
Unless those files have had their NSURLIsExcludedFromBackupKey/kCFURLIsExcludedFromBackupKey resource metadata value altered. Then it gets much more complicated.
Files 'Excluded From Backup'
Generally, if a file outside of a Documents directory can be backed up, the system assumes it can also purge it under low space or other conditions. This is why setting NSURLIsExcludedFromBackupKey to YES on a file allows the file to persist even in low storage conditions. If your application sets NSURLIsExcludedFromBackupKey to YES for a file, your application assumes responsibility for the life of that file.
The catch here is that the backup process and the purge process do not follow the same logic. Apple's documentation indicates that for the purposes of controlling the backup behavior, it is possible to set NSURLIsExcludedFromBackupKey on a directory. The children of that directory will effectively inherit that resource value (in practice, this may not be accurate). The purge process, however, does not seem to have the same behavior. It may not check the backup exclusions of the parent directories and apply it to children, and as a result if a file does not have NSURLIsExcludedFromBackupKey explictly set it may be purged.
This gets even more complicated. If you were to read the documentation for the constant NSURLIsExcludedFromBackupKey you would see:
Some operations commonly made to user documents cause this property to be reset to false; consequently, do not use this property on user documents.
This actually applies to much more than user documents. For example, if you were to perform an atomic write on a file such as:
[thing writeToURL:URL atomically:YES encoding:NSUTF8StringEncoding error:&error]
If the file at URL had NSURLIsExcludedFromBackupKey set to YES before the write, it would now appear to be set to NO. An atomic write like this will first create a temporary file, write to that, and replace the original with the new file. In doing so, file and URL resource flags are not preserved. The original file had the NSURLIsExcludedFromBackupKey resource value set, the newly created file at the same location now does not. This is just one example; many Foundation APIs perform atomic writes like this implictly.
There are scenarios where this gets even more complex. When an application is updated it is installed into a new location with a new application container path. Data inside the old application container is migrated. There are few guarantees regarding what may or may not be migrated as part of the update process. It may be everything, it may be only some things. In particular there are is no guidance concerning how files or directories marked with the NSURLIsExcludedFromBackupKey resource attribute will be treated. In practice it seems that these are often the least likely files to be migrated, and when they are migrated the NSURLIsExcludedFromBackupKey attribute is rarely preserved.
OS updates are also an issue. Historically Over-The-Air updates have been problematic and have caused the NSURLIsExcludedFromBackupKey resource attribute to be effectively cleared or ignored. A "major" OS update will clear the device and restore from a backup - which is equivalent to migrating to new hardware. Files marked with the NSURLIsExcludedFromBackupKey resource attribute will not be migrated, and the application will have to re-create them.
Update scenarios are described in TechNote 2285: Testing iOS App Updates
Because of this, when using NSURLIsExcludedFromBackupKey it is generally best to set the value on every access, and as always should be done through the File Coordination APIs (unless you are writing to a shared group container, which is an entirely different set of issues). If the NSURLIsExcludedFromBackupKey resource attribute value is lost files can be purged at any time. Ideally an application should not depend on the NSURLIsExcludedFromBackupKey or how the OS may (or may not!) handle it, but instead be designed such that the data could be recreated on demand. That may not always be possible.
It's clear from your question and the code that you posted that you are somewhat dependant on NSURLIsExcludedFromBackupKey ensuring that your file(s) have an application-controlled lifetime. As you can see from the above, that may not always be the case: there are many, many common scenarios where that resource attribute value can disappear, and with it your files.
It is also worth noting that NSFileProtection attributes work the same way, and can disappear in the same scenarios (and a few more).
TL;DR; What should I do?
Based on your question, code, and the description of the behavior you are seeing:
Setting the NSURLIsExcludedFromBackupKey value on the directory containing the file(s) you are interested in preserving may not be enough to prevent them from being purged. It would be wise to set NSURLIsExcludedFromBackupKey on every access to the actual files, rather than just a parent directory. Also attempt to ensure this resource value is set after any write to the file, especially through a high level API that may be doing atomic writes, etc.
All NSFileManager and file reading/writing operations should use file coordination. Even in an application that is single threaded there will be other processes interacting with "your" files. Processes like the daemons that run backups or purge files during low space conditions. Between your -fileExistsAtPath: and the -setResourceValue:forKey:error: another process could alter, delete, or move your file and its attributes. -setResourceValue:forKey:error: will actually return YES and no error in many cases where it did nothing, like the file not existing.
Files and directories marked with NSURLIsExcludedFromBackupKey are the responsibility of the application to manage. The application should still purge those files or their contents at some appropriate time, or set limits on their growth. If you look at the per-application disk usage information on a device, you can probably guess the names of some applications that do not do this correctly.
Test update scenarios as described in TechNote 2285: Testing iOS App Updates. often. Ideally the iOS Simulator would have a "Simulate Low Disk Space" capability similar to simulating memory warnings, but at this time it does not.
If at all possible, alter application logic to recreate these files if they go missing.
In the documentation you linked it is stated that
Critical data should be stored in the /Documents directory. Critical data is any data that cannot be recreated by your app, such as user documents and other user-generated content.
It is also mentioned that
Cached data should be stored in the /Library/Caches directory. Examples of files you should put in the Caches directory include (but are not limited to) database cache files and downloadable content, such as that used by magazine, newspaper, and map apps. Your app should be able to gracefully handle situations where cached data is deleted by the system to free up disk space.
The directory you are using is not explicitly mentioned for storing user data, it is used by the system and is not save for your data. It's guarantied to be untouched by an update of your app, but that's it
To find the documents folder you could do something like
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsFolderPath = [paths firstObject];
I'm looking for a better way to download a bunch of files from my iCloud container to my sandbox. This is what I currently use:
for (FileAttachments *fileLink in items) {
NSURL *cloudFileURL = [fileLink urlForCloudAttachmentWithName];
NSURL *fileURL = [backupCloudLocalDirectoryURL URLByAppendingPathComponent: fileLink.fileName];
NSFileCoordinator *fileCoordinator = [[NSFileCoordinator alloc] initWithFilePresenter:nil];
[fileCoordinator coordinateReadingItemAtURL:fileURL options:NSFileCoordinatorReadingWithoutChanges error:&error
byAccessor:^(NSURL *newURL) {
NSError *blockError = nil;
[fileManager copyItemAtURL:cloudFileURL toURL:fileURL error:&blockError];
}];
}
}
Is there any problem with making a copy of the iCloud item this way? In production, I have some users complaining that all their files weren't downloaded. Is it better to use NSFileManager's startDownloadingUbiquitousItemAtURL:error instead? If so, why?
Thanks.
It's still not completely clear to me from comments how you're discovering the URLs, however some important details that affect your situation are:
Using coordinateReadingItemAtURL:block: has nothing to do with downloading files from iCloud. The purpose of NSFileCoordinator is to coordinate among readers and writers of a file so that, for example, you don't get two threads trying to write to the same file at the same time. You use file coordinators with iCloud because the iCloud system needs to read and write files and so does your app. Using a coordinator avoids corrupting the file, but again, has nothing to do with downloading the file.
To download a file from iCloud you need to use startDownloadingUbiquitousItemAtURL:error: and then wait until the file downloads. The normal flow here is:
a. Use NSMetadataQuery to find files that exist in the iCloud account
b. Use startDownloadingUbiquitousItemAtURL:error: to make sure they're available on the local device.
The reason you need to use this call is simply because that's how iCloud works on iOS. Files don't download until you ask them to download, and this is how you ask. [On OS X it's different, everything automatically downloads.]
You cannot simply copy from an iCloud URL to another location unless you already know that the file has been downloaded. The file-copy operation does not download the file, so if the file isn't available locally, the copy will fail.
You must use a metadata query to identify the files and their download status then, if they have not been downloaded initiate the download, and using the metadata query determine when the download is complete and then copy the file from the ubiquity container to the apps sandbox directory using a file coordinator. If you try copying the file while it's partially downloaded you may get some strange results.
I had the same problem with you.
My case is that: When network disconnects, iCloud service copy file from iCloud container to sandbox. When this line executes, it can not go into the block to copy file. This is the reason why this file can not be copied.
fileCoordinator coordinateReadingItemAtURL:fileURL options:NSFileCoordinatorReadingWithoutChanges error:&error byAccessor:^(NSURL *newURL)
My solution is: Before copying file from iCLoud container to sandbox, you must check network. If it is not available, don't execute this code (return method). If network connects, execute fileCoordinator and copy.
More info: When copy file from ICloud container to sandbox, method fileManager copyItemAtURL:toURL:error: is OK because I implement this method and it's good.
Hope this works.
I am trying to code a hard disk based restore function into an existing demo Photo Application for iOS devices. The idea is to allow users to restore the last applied effects and all, even after restarting the application/unexpected crash etc.
I am saving one image file (the latest effects applied to it) into the NSCachesDirectory:
NSData* data = UIImagePNGRepresentation(image);
[data writeToFile:[self getFileAtIndex:getPath] atomically:YES];
On going to the recover option, the file saved at this path is recovered using:
image = [[UIImage imageWithContentsOfFile:[self getFileAtIndex:getPath]]retain];
The code is working fine in my test device (iPhone 4s). But somehow I doubt if this is the best practice for coding a restore function.
Moving into a real world app, if things were stored in NSCachesDirectory, do you think it will be backed up into iCloud? Is there any possibility of a conflict with other apps?
How exactly is NSCachesDirectory directory managed for multiple apps accessing it simultaneously?
I would like to know the best practice for doing this.
Please help/comment.
As Mr. Bonjour mentioned in the comment, on iOS, every app has its separate file system, so there can never be any conflict with other apps.
Caches directory is not backed up on iCloud and you shouldn't rely on files in caches directory to persist across launches. You should use caches directory for temporary files only. If you allow restore only during one session, then using caches directory is fine, but if you want to allow restore across multiple launches, you should store them in library/documents directory. But be aware that Apple has a strict review policy for storing files in Documents directory since it takes up space on user's cloud storage.