I'm looking into using UIDocument en NSFileWrapper to store 'projects' that contain quite a few large video files and some small text files. There are a few problems that I run into, and I'm starting to wonder if UIDocument is still the right strategy.
Performance
As far as I can tell, NSFileWrapper loads everything in memory. This can be a problem when working with large video files. I think it's possible to work around this by using custom save and load methods that forego the standard NSFileWrapper.
Metadata
I want to display a list of all the documents along with some metadata. This can for example include a preview image, number of recorded scenes, length of videos etc. The only way to fetch this data now is to open each document and retrieve it. Probably quite slow, especially with large documents.
Solutions?
I see two solutions now: ditch UIDocument altogether and go for a custom architecture, or use some kind of centralized metadata file. Drawbacks of the latter is that I have to manage metadata in two separate places and that I need to keep them in sync manually.
Is UIDocument still the way to go here, and if so: What could be a way to solve these problems?
Based on the comments the asker found a way to move forward as such:
drop UIDocument in favor of a Core Data solution. I now save all my
data using Core Data and manually manage the large files on the
filesystem. It works pretty well for me, and I'm glad I made the
switch. The only thing I had to give up was easy iCloud syncing. But
with files as large as these, that wasn't really feasible anyway. It
seems you can create your own file wrapper class to work around some
performance problems with large files.
Related
I'm working on an iOS app that will need to save data onto files. I chose to go for a Document Based app, precisely an app based on a UIDocumentBrowserViewController so that I can easily save and load files from the system's Files app.
Since the data I need to save/load on a file is quite complex: big hierarchy of different objects, with meta-data, image files, etc. I'm wondering what is the best technology to use going forward.
I came across NSFileWrapperand its ability to save different files as one. And I could definitely use that. But I also saw UIManagedDocument and the ability to use Core Data in my project while maybe saving the content of the Core Data database (I know it's not quite a database, but you know what I mean) into a file that I could write somewhere in the File App.
Is this a behavior I can expect?
To reformulate: I'm wondering if I can read/write files through a UIDocumentBrowserViewController, with data described by a UIManagedDocument that works with Core Data.
Thank you in advance. 🙂
As you have discovered, UIManagedDocument is there for your kind of application. And it does feature methods to write and read additional content like the metadata or image files you have, within the document package.
That being said, I have never used UIManagedDocument, and have never seen it used by others. A quick search of GitHub finds only this one project with two contributors who wrote a wrapper around it in 2013. Also, there does not seem to be any sample code from Apple, and the remark in the the writeAdditionalContent(_:to:originalContentsURL:) documentation that Additional content is not supported on iCloud leaves me a little concerned, but maybe it's a good sign that the Core Data team knows where to draw the line.
I have used the macOS counterpart of UIManagedDocument, NSPersistentDocument. It is in a similar situation of not being used very much, but with many more known technical issues. So a few years ago I switched to BSManagedDocument, which purportedly mimics UIManagedDocument to support Core Data in all its modern glory. I have been happy with BSManagedDocument.
In summary, if I was in your situation, yes I would give UIManagedDocument a try. But don't be surprised if you need to use a DTS support incident or two during your development.
Im using a strategy where I'm saving images and pdfs as NSData in the respective managed objects where they belong. I'm having a problem syncing with Ensembles that the pdf doesn't always carry over from one device to another. Now I'm not sure if this is due to some flaws in my code or if it's not a good way of syncing chunks of data like this. Does anyone have experience of this?
I'm using Ensembles 2.2 syncing through CloudKit.
Ensembles should handle this fine. I use it for exactly this purpose, syncing image data including PDF.
I would look closer at the handling of the data. Is the value transformer working (if you are using one)? Is the device capable of unpacking and displaying the PDF data?
An alternative to syncing the PDF directly is transforming to a format like PNG before putting it in your store.
Transformable data type is really just binary under the covers with some additional metadata. Have you tested a simple lightweight migration on an existing store? I suspect the migration would work and would leave the existing data in the store.
If you are looking to get the existing binary data actually moved out of the SQLite file then you are looking at something a bit more involved.
A heavy migration will accomplish what you are looking for but if the stores are large it may take took long and potentially not provide enough feedback for a good user experience. I personally do not use heavy migrations, ever, on IOS but it will accomplish your goal.
An export/import will also work. I generally recommend export/import when a lightweight migration won't work. It involves a medium amount of code but in the end you own the code, understand the entire process and can tweak it to your exact needs.
We've recently converted our code to use UIDocument instead of manipulating files on the file system directly, and we've encountered some performance issues as a result. We are wondering whether we are using this class incorrectly, if anyone else had these issues, and what are the common ways to address them.
Our app
We have a "shoebox app" that manages a bunch of documents, each consisting of multiple image files that can be quite heavy, a small metadata file and a small preview image. The user may have many documents on her device (1000+ documents). Each document's files are grouped in an directory and we use NSFileWrapper to read and write them.
When our app starts, it needs the metadata of all the documents in order to show a document index, and a subset of the preview images. More preview images are loaded as the user scrolls.
In order to get that information, we open all the documents, read their metadata and preview image if we need to, close them, and then open again on demand.
Problem #1: Slow loading time
It takes a lot of time to open all the documents and read their metadata. I think there are several factors contributing to this problem:
- Each document open action is relatively slow
- The open document blocks and the completion blocks are executed on the same queue, which makes the operation's latency very bad (my document is open, but the completion block has to wait for X open document blocks before it can run)
We thought about solving this problem using a separate index file, but this approach has the drawback that we will need to manage the metadata in two places and that we will need to keep it synched with the file system in case iCloud changes the files.
Problem #2: Threading
Each open document creates its own "File Access Thread". When we open many documents concurrently, the overhead crushes the app.
We solved this issue by synching the open operations using a semaphore. This approach has the drawback that it slows down the loading even more.
Questions
Is there some fundamental problem with the way we are using UIDocument? If not:
Has anyone encountered a similar loading time problem? What is the common way to solve it? Do other apps keep an index file?
Is there a way to make UI document use a thread pool? If not, how do you control resource usage?
Thanks!
Ok, no good news here.
We tried consulting with friends in the industry, profiling UIDocument and using modified implementations of it that alter various aspects of its operation in order to see if we can improve its performance but to no avail.
My conclusion is that UIDocument is not suitable for this kind of usage - it is just not designed to support the latency and throughput requirements we have for open operations. UIDocument should only be used when you want to open a small number of files at any given moment (much like word processors and drawing apps).
Admittedly, this is exactly what Apple's documentation says, but I guess we had to learn just how serious they were the hard way :)
We ended up using some "tricks" to improve the user experience, and will move away from UIDocument as soon as we can.
So my recommendation is that only if:
Your app resembles a document based app in nature, meaning you will not have more than a few documents open at any given moment
You do not need the information inside the documents in order to "discover" them and show them to the user, or can afford the overhead of managing a separate index file
You really need the auto saving/undo/synchronization/iCloud abilities of this class
then use it. Otherwise consider other solutions.
A a side note which is not directly related to the question but I will add here as a public service: UIDocument's async model required some major changes in the app architecture when we moved from direct file access. If you plan on making this move, evaluate the work you will need to do carefully.
Good luck future programmers.
The document classes have methods to read asynchronously. Have your tried that?
This really sounds like something more suited to Core Data or SQLite, for the metadata. At the very least, cache the metadata in Core Data (a single store for the entire app, not one per document).
I am making a simple iPhone app that will basically be an editor.
As such, I need some way to store the documents the user creates.
Since on iPhone, the concept of the filesystem is not present for the user, I searched around to see what I should use.
I found this question & answer that basically says to use Core Data, but I recently found out about NSFileManager.
My question simply is, for user-created documents, what is the best storage system to use? Traditional files by using NSFileManager? Core Data? Something else?
Personally, I would use CoreData because it will abstract away all of the file-management code for you. If you are making simple text documents then this isn't such a big deal, but if you are working with a complex document architecture (i.e., a collection a numerous objects) then it can save you a lot of effort.
If the user wants to export their document it is very easy to write a function to do so with your CoreData objects.
The only downside to CoreData is that if you are using non-standard attributes it can get a little bit tricky, but it is certainly not a deal breaker in most cases.
People create document formats without CoreData all of the time, so there are plenty of examples out there, and it will just come down to personal preference. There really isn't any generalized right answer to this - it a design decision that should be evaluated on a per-app basis.
If all of your data for displaying the file is contained in one long string (like HTML) then I would recommend that you use the file manager, since it will be easy to get a list of files in a certain directory to display to the user for opening. However, if they are not self contained (like NSAttributedString, which has many stored formatting regions along with the actual content) then you should use CoreData, as it will be easier to keep all the pieces together.
I'm developing an app which needs to show some logos. These logos are just 8kb PNG files, and I'm just going to handle a little amount of them (10-20 at most). However, these are downloaded from the Internet because they might change. So, what I'm trying to achieve is, making the app to download them (done), storing them into the file system, and only downloading again whenever they change (might be months).
Everyone seems to use Core Data, which in my opinion is something designed for bigger and more complex things, because my files will always have the same name plus don't have relations between them.
Is the file system the way to go? Any good tutorial?
Yes, the file system is probably your best option for this. You say that you've already implemented the downloading. How have you done so? With NSURLConnection? If so, then at some point, you have an NSData object. This has a couple of write... methods you can use to save the data to a file on the filesystem. Be sure to save the files in the right place, as your app is sandboxed and you can't write anywhere you like.
The advantage Core Data brings is efficiency. Using NSFetchedResultsController to display your logos in a tableview gets you optimized object loading and memory management. It will automatically load only the items which can be displayed on one screen, and as the user flicks through the table it will handle releasing items which move offscreen. Implementing that on your own is not a simple task.
If you want to build and display your data without Core Data, you'll probably want to use NSKeyValueCoder, which will allow you to easily write an array or dictionary of objects (including nested arrays, dictionaries, and images).