We have a mobile application that currently uses a Realm database in conjunction with photos that are stored in the device’s document folder.
We are looking for a way to allow our users to use the app and share data between different devices are were very interested in the new Realm Mobile Platform. One thing we can’t quite figure out though, is how it would work with the photos. Would we need to manually synch those between devices through another service? Or is there a way to make it work with Realm Mobile Platform? Any ideas of how we could make this work would be greatly appreciated.
Storing the photos in the database would work, but might increase the size by a lot. I imagine you also want to preserve offline capabilities - e.g. if a user adds an image while offline, they'll expect that image to eventually sync.
One solution would be to have a per-device realm (i.e. give it a unique name to avoid syncing with anything but the server) that could be used as an image upload queue with a single class (pseudocode):
class Image : RealmObject {
byte[] Data
// Other properties to map this image to whatever
// object it belongs to.
}
Then, your objects with images will look like
class ObjectWithImage : RealmObject {
string ImageUrl
// Other properties
}
So now, when you add a new ObjectWithImage, you'll add a corresponding Image object to the image Realm. Then, on the server, you can listen for changes on the image Realm (requires at least Professional edition) and when a new Image gets synchronized, you can upload the data to AWS S3/Azure blob/other hosting, update the corresponding ObjectWithImage's ImageUrl, and delete the processed Image object. Then on the device you can use an image caching library, e.g. SDWebImage to download the image from the url and store it locally for faster retrieval next time.
The benefits of this solution are:
The shared realm will only contain url's, reducing both storage and bandwidth requirements thus making synchronization faster.
The app will work while offline, eventually syncing both the shared realm and the image realm when connection is restored.
The image Realm will not be synced to other devices, so there will not be any unnecessary traffic.
And, as any solution goes, there are some drawbacks:
It will not work with the Developer edition as it requires modifying the objects on the server.
It may require some more advanced transaction error handling, as you want to commit two transactions in two different realms in a (preferably) atomic fashion.
Related
I searched across many threads and issues but I doesn't found the answer.
When I use the Realm Objects Server and the clients connects to the server, the whole DB is synced across all the clients?
In other words, if I have a public DB with millions of objects, relations, etc all the clients have a copy of the whole DB on their devices?
I'll need Realm sync feature but I don't know how to sync occurs. Is the sync incremental? Every user has the objects that device needs to make queries?
My app will increment the size every hour, and the sync feature of the Realm is perfect for me, but I have doubts about the size of DB over the time and how clients sync a lot of data.
Thanks in advance!
At present, yes. Public Realms are synchronized in their entirety to every client's device. The entire list will initially be downloaded when the client's device connects for the first time, and from that point on, any additional changes will be synchronized at the time they are made on the server.
That being said, all Realm files are compacted (ie, all empty allocated space is removed and all strings are condensed), and then compressed with gzip before being downloaded to the client, so as long as the public Realm doesn't contain large binary blobs, even very large files should come down quite quickly.
It's on the roadmap to add partial replication to the Realm Mobile Platform. This will allow only portions of a single Realm file to be synchronized to specific clients. However there are no concrete plans for when this will be released.
For the time being, the easiest thing to do would be to maintain the master list of data on the server, and copy only the desired data to each user's private Realm. However since this would require custom logic to be executed on the server, it would require the Professional or Enterprise editions of the Realm Mobile Platform.
I couldn't find resources discussing the difference between the three download methods in the firebase storage documentation and pros/cons of each. I would like some clarification about the firebase storage documentation.
My App
Displays 100 images ranging from 10 KB-500 KB in size on a table view
Will be used in a location where internet connection and/or phone service could be very weak
Could be used by many users
3 methods for downloading from Firebase storage
Download to NSData in memory
This is the easiest way to quickly download a file, but it must load entire contents of your file into memory. If you request a file larger than your app's available memory, your app will crash. To protect against memory issues, make sure to set the max size to something you know your app can handle, or use another download method.
Question: I tried this method to display 100 images that were 10KB-500KB in size on my table view cells. Although my app didn't crash, as I scrolled through my table, my memory usage increased to 268 mb. Would this method not be recommended for displaying a lot of images?
Download to an NSURL representing a file on device
The writeToFile:completion: method downloads a file directly to a local device. Use this if your users want to have access to the file while offline or to share in a different app.
Question: Does that mean all images from firebase storage will be downloaded on user's phone? Does that mean that the app will be taking up a large percentage of the available storage on the phone?
Generate an NSURL representing the file online
If you already have download infrastructure based around URLs, or just want a URL to share, you can get the download URL for a file by calling the downloadURLWithCompletion: method on a storage reference.
Question: Does this method require a strong internet connection and/or phone service connection to work?
Generally, your memory usage should not be affected by the method of retrieval. As long as you're displaying the 100 images, their data will be stored in the memory and should have the same size if they're identically formatted/compressed.
Either way you go with, I suggest you implement pagination (for your convenience, this question's answer might serve as a good implementation reference/guide) to possibly decrease the memory and network usage.
Now, down to comparing the methods:
Method 1
...but it must load entire contents of your file into memory.
This line might throw some people off thinking it's a
memory-inefficient solution, when all it really means is that you
cannot retrieve parts of the data, you can only download the entire
file. In the case of storing images, you probably would want that for
the data to make sense.
If your application needs to download the images every time the users
access it (i.e if your images are regularly updated), then this
method will probably suit you best. The images will be downloaded
every time the application starts, then they'll get discarded when
you kill it.
You stated that a part of your user base might have a weak internet
connection and so the next method might be more efficient and
user-friendly
Method 2
First off, the answers to your questions:
Yes. The images downloaded using this method will be stored on the users' devices.
The images should take up about the same size they're taking on Firebase storage.
Secondly, if you plan to use this method, then I suggest you store a
timestamp (or any sort of marker) in your database for when the last
change to the images occurred. Then, every time the app opens up, do
the following flow:
If no images are downloaded -> download images and store the database timestamp locally
If the local timestamp does not equal the timestamp on the database -> download images and store the new timestamp locally
Else -> use the images you already have, they should be identical to the ones in Firebase storage
That would be the best way to go if your network usage priority is
higher than that of the local storage.
And finally...
Method 3 (not really)
This is not a data download method, this simply generates a
download URL given a reference to the child. You can then use that
URL to download the data in your app or elsewhere as long as the used
app or API is authorized to access your Firebase storage.
Update:
The URL is generated from a Firebase reference (FIRDatabase.database().reference().child("exampleReference")) and would look like this: (Note: this is a fake link that will not actually work, just used for illustration purpose)
https://firebasestorage.googleapis.com/v0/b/projectName.appspot.com/o/somePathHere%2FchildName%2FsomeOtherChildName%2FimageName.jpg?alt=media&token=1a8f83a7-95xf-4d3s-nf9b-99a274927bcb
If you simply try to access that link you generate through any regular web-browser (assuming you don't have any Firebase rule that conflicts with that in your project), you can directly download that image from anywhere, not just through your app.
So in conclusion, this "Method" does not download data from Firebase storage, it just returns a download URL for your data in case you want a direct link.
My app is a mood diary and to save data I've chosen to use Core Data (for strings, images, etc.); to allow the user to restore his diary I've implemented iCloud, that works well with Core Data.
Everything works well if I have not much entries, but when the images saved are too much the app is slow to load data and encounters memory warnings.
To save my images I've chosen Transformable data type and not Binary Data; I know that is not the best way to save images (saving url is surely better), but I need to sync my data on iCloud, and saving images as Transformable allows me to sync data in a simple way (thanks to the possibility offered by Apple Api to link Core Data and iCloud).
What can I do to avoid this memory warnings and sync my app pics on iCloud?
I've considered the possibility to save my pics on a custom photo album (if iCloud is activated for Photo app my app pics would be synchronized), but I need to save them with custom name to retrieve them from camera roll to my app, and for the moment I don't find any solution to save pic with custom name in a custom photo album.
Saving photos in document directory (and saving urls in my core data entity) would be the right choice for local database, but my app pics would not be synchronized.
There are a few things you can try.
First, add an image thumbnail property which stores a smaller version of the image. Use that whenever possible. Loading a bunch of full-size photos needs a lot of memory, so change to loading smaller images whenever your UI allows smaller sizes.
Beyond that you can change how you handle images using one of the following strategies. In ascending order of complexity (and effectiveness):
Make sure "Allows external storage" is enabled for the image property in your data model. This will let Core Data put the images outside the persistent store without requiring you to manage those files. This will save on memory if, for example, you sometimes fetch data but aren't using the image property.
Change the data model so that the image is saved in a different entity, with a relationship linking it to your current entity. This should make it easier to avoid "accidentally" loading images when you're not using them.
Put the images in separate files and keep only the file names in Core Data. You can still sync the images via iCloud, because you can sync files directly via iCloud outside of Core Data. But you'll need extra code to manage uploading/downloading the images. You'll also need to make sure you can handle the case where Core Data has finished syncing but the image is not available yet.
On this list, #1 is easiest but will probably have the least effect. Using #3 should be very effective but will require the most work.
So my iOS app has just been rejected because it's storing too much non-user-created data in the SQLite database, which lives in the Documents folder.
The app basically involves a relatively large library of images -- around 60-or-so megabytes of them to start, and there are also in-app purchases which each add an additional 60-or-so mb of images. Furthermore, the user can add their own images to the library.
Right now everything (images and all) is stored in an SQLite database, which is generated when the app is first launched. As the user adds more images, or purchases image packs, those images are added to the database. To the user, all the images (user-generated or not) behave the same in the app.
But Apple won't allow this: I can't have all that data stored in the SQLite database in the Documents folder unless I set it specifically NOT to back up to iCloud, as it's all recreatable data.
But if I set it to not back up, then the user-generated data won't back up either, which I definitely don't want.
Any suggestions how I might "split up" the database, such that all the user-generated stuff can be backed up, but the included-or-purchased stuff isn't?
The reason you are being rejected is not following the Data Storage Guidelines. Data created or edited by the user belongs within NSDocumentsDirectory, while application data should be stored elsewhere (i.e., the Application Support directory). These requirements are a result of how iCloud backup and disk space purging work on iOS.
For a Core Data application, this means your persistent stores must be split into two different sets of files, in two different locations. This, in turn, ends up driving much of the application architecture and data model. To have relationships between the user data and application data, for instance, you must use two different managed object model configurations and the relationship must be a fetched property.
There is more detail on how to implement this in this answer.
An alternative for your specific case would be to save the images on the file system, in the caches directory or elsewhere. User images could exist in NSDocumentsDirectory while application images could exist in NSCachesDirectory. This would remove the images from Core Data and instead your model objects would have the path to the image on the file system. This would be a short term fix to get you through submission, and would probably work.
One option will be, storing your data to a server and calling is using web service. During first launch. Or as per requirement.
Newbie question.
I will need to have a data base from about 200 UIImages (single of them less than 500kb size) for iPad app. Customer want to have possibility to change set of this images from time to time without releasing new version of app in appstore and app must work without connection to the web (local data base on a device). I don't see how this can be done simultaneously, I see only one common option here:
Image data base would be stored on a server, what app customer will be able to change anytime. User will need to have web connection and every time he will start the application - existing data base will load into the app.
Main questions here:
is it possible to update data base on user's device without releasing new version of app and what data base managing system is more proper to this situation(SQLite, MySQL etc...)?
Q : is it possible to update data base on user's device without releasing new version of app?
A : Yes. It is possible.
SQLite will be perfect for you.
The photographs reside on the web server.
A number of start-off photographs may reside within the boundle so that the app is not really empty at start.
However, when downloading the app, the user must be online. In most cases he would still be online directly afterwards when he launches the app for the first time.
The server provides two services:
A quick one that just provides a version number of the
photo-database content and/or the date of the last change to the
photographs on the server.
The app frequently (not more than daily I would say) checks wether there are new images on the server or not.
If they are then the user is asked, whether he wants to download them.
If the user says YES then the app sends the version number and/or last date and/or IDs of all local photographs to the server and the
server provides the information about which photographs have been
added and where to download that very photograph and which have to
be deleted.
Then you add or delete or update the photographs from the download source given by the server. (That may well be an URL to the
very same server of course.)
For 200 data sets I would strongly suggest core data with SQLite - the standard stuff.
You may then think of holding the image data in the file system or in NSData properties within the database.