Our main app allows our users to post media (videos and images) as well as documents on the timeline with a file size limit of 500 megabytes on a timeline.
We're currently working on a Share Extension to allow users to share files throughout the OS to that timeline. However we're running into the issue that the Share Extension has a hard memory limit of 120 megabytes.
The current implementation in our main app requires that the files selected by the user get converted to a Data object before getting compressed and then uploaded to the API via multipart form data. However to achieve this we must load the files into memory where we run into the hard memory limit.
Apple documentation is very brief and there's not a lot to be found on SO or elsewhere on how to achieve this. There are some workarounds found by storing these files (or references) in UserDefaults and then opening the main app to handle them but that kind of defeats the purpose of sharing something quickly via the extension.
What would be a way around this limit to allow us to upload these large files?
Late to this question I ran into the same issue now. The issue is the memory limit imposed by Apple on File extensions, so you can not use a data object in memory if that object is more than around 50 - 100 MB. I changed the upload to an NSURLSessionUploadTask uploadTaskWithRequest:fromFile:completionHandler:, where I upload the file from the file system directly. If you can not change that you need to provide the multipart form data and boundaries in the delegate of uploadTaskWithStreamedRequest: but consider that also here you can not read the whole file at once but have to stream it as well.
Related
I couldn't find resources discussing the difference between the three download methods in the firebase storage documentation and pros/cons of each. I would like some clarification about the firebase storage documentation.
My App
Displays 100 images ranging from 10 KB-500 KB in size on a table view
Will be used in a location where internet connection and/or phone service could be very weak
Could be used by many users
3 methods for downloading from Firebase storage
Download to NSData in memory
This is the easiest way to quickly download a file, but it must load entire contents of your file into memory. If you request a file larger than your app's available memory, your app will crash. To protect against memory issues, make sure to set the max size to something you know your app can handle, or use another download method.
Question: I tried this method to display 100 images that were 10KB-500KB in size on my table view cells. Although my app didn't crash, as I scrolled through my table, my memory usage increased to 268 mb. Would this method not be recommended for displaying a lot of images?
Download to an NSURL representing a file on device
The writeToFile:completion: method downloads a file directly to a local device. Use this if your users want to have access to the file while offline or to share in a different app.
Question: Does that mean all images from firebase storage will be downloaded on user's phone? Does that mean that the app will be taking up a large percentage of the available storage on the phone?
Generate an NSURL representing the file online
If you already have download infrastructure based around URLs, or just want a URL to share, you can get the download URL for a file by calling the downloadURLWithCompletion: method on a storage reference.
Question: Does this method require a strong internet connection and/or phone service connection to work?
Generally, your memory usage should not be affected by the method of retrieval. As long as you're displaying the 100 images, their data will be stored in the memory and should have the same size if they're identically formatted/compressed.
Either way you go with, I suggest you implement pagination (for your convenience, this question's answer might serve as a good implementation reference/guide) to possibly decrease the memory and network usage.
Now, down to comparing the methods:
Method 1
...but it must load entire contents of your file into memory.
This line might throw some people off thinking it's a
memory-inefficient solution, when all it really means is that you
cannot retrieve parts of the data, you can only download the entire
file. In the case of storing images, you probably would want that for
the data to make sense.
If your application needs to download the images every time the users
access it (i.e if your images are regularly updated), then this
method will probably suit you best. The images will be downloaded
every time the application starts, then they'll get discarded when
you kill it.
You stated that a part of your user base might have a weak internet
connection and so the next method might be more efficient and
user-friendly
Method 2
First off, the answers to your questions:
Yes. The images downloaded using this method will be stored on the users' devices.
The images should take up about the same size they're taking on Firebase storage.
Secondly, if you plan to use this method, then I suggest you store a
timestamp (or any sort of marker) in your database for when the last
change to the images occurred. Then, every time the app opens up, do
the following flow:
If no images are downloaded -> download images and store the database timestamp locally
If the local timestamp does not equal the timestamp on the database -> download images and store the new timestamp locally
Else -> use the images you already have, they should be identical to the ones in Firebase storage
That would be the best way to go if your network usage priority is
higher than that of the local storage.
And finally...
Method 3 (not really)
This is not a data download method, this simply generates a
download URL given a reference to the child. You can then use that
URL to download the data in your app or elsewhere as long as the used
app or API is authorized to access your Firebase storage.
Update:
The URL is generated from a Firebase reference (FIRDatabase.database().reference().child("exampleReference")) and would look like this: (Note: this is a fake link that will not actually work, just used for illustration purpose)
https://firebasestorage.googleapis.com/v0/b/projectName.appspot.com/o/somePathHere%2FchildName%2FsomeOtherChildName%2FimageName.jpg?alt=media&token=1a8f83a7-95xf-4d3s-nf9b-99a274927bcb
If you simply try to access that link you generate through any regular web-browser (assuming you don't have any Firebase rule that conflicts with that in your project), you can directly download that image from anywhere, not just through your app.
So in conclusion, this "Method" does not download data from Firebase storage, it just returns a download URL for your data in case you want a direct link.
I'm building a mobile app using Meteor. To allow for offline usage of the app, I want the app to be able to download a large-ish json file while online, then access the data in the json file, written to MongoDB, while offline.
This works fine. However, in the downloaded json file, there are plenty of references to online images that won't display in the app once the app is offline.
So, I want to be able to download (a selection of) the images referenced in the json file to the app, so that the app can access them even when offline.
(Downloading images could happen in the background for as long as a connection is available.)
There's an implementation of imgCache.js available on Atmosphere, which fails to initialize for me.
I suppose it's theoretically possible to individually load each image to a canvas, save the canvas content to MongoDB, then load the content when needed. Info on some of this is here. But, this feels rather convoluted and, if really feasible, I would expect someone to have done this before with success.
How can I do achieve caching of images for offline use in Meteor?
So, you've probably already read this article about application cache.
If the images are static, you can just include them in the manifest. Be sure you understand the manifest and cache expirations (see the article).
If the images are dynamic, you'll find some techniques to store images in local storage
If that's the case, this may be what you want.
We're using Parse in our iOS app and we've discovered that our app is using an enormous amount of space -around 2.3GB in some devices- in storage. After downloading app data to my Mac, I've realized almost all of that data is the cached images in a folder called PFFileStaging, it contains highest resolution PNGs of all the PFFiles that the user has viewed in our app, ever. How can we disable this behavior, at least limit it? Is that the intended behavior? I heavily doubt using GBs of space is the intended behavior. Is this a bug?
Unfortunately this is not cache related (as per Parse's engineers: "this is used to ensure that no concurrent modification happens to the file after you request uploading").
They're planning to implement automatic trimming of PFFileStaging folder on every app start (as per road plan this should appear in next version 1.8.2).
See the the whole thread on GitHub.
Most apps I see have loading screens, presumably to prevent hiccups in the game and to pre-load all of the data needed later in the game. I am currently working on a game that skips every few frames, presumably because the image data for the objects being rendered to the screen are not being held in memory. What do programmers actually do to hold this data in memory throughout the time the app is running? I can think of using arrays, such as NSArray or NSMutableArray to hold the images, however I still struggle to see how that would improve the performance of the app. Perhaps I am totally going off on a tangent here. A word of advice from someone with experience in the area would be greatly appreciated.
EDIT:
For the person who voted to close the question, my question, more specifically, is this: What is a typical method in which an ios game loads its data into active memory to enable easy and low-overhead access during run time?
I have seen several application that downloads configuration and metadata on application load, some also download images and audio files.
Usually they store the data in the documents or cache folder, so the they will not have to wait for download during the game.
Another technique I encountered is downloading a zip file and expanding it in the cache directory. Most of these apps do not load the files the next time unless something new is needed.
Do not load the images to the memory, not inside an nsarray nor nsmutabearray. You do not have enough memory to cache images like this. You should just predownload them
Edit:
You can use the application loading stage to download more resources from your servers (images, audio, plist files) but after they are download and stored in the file system they should be treated like resources that exists in your IPA. They should be loaded into memory only when they are needed
Problem statement:
I want to upload a large binary (such as an audio clip) from an iOS app to S3, and I'd like to make the app's handling of disconnects (or low connectivity) as robust as possible, preferably by uploading the binary as a series of chunks.
Unfortunately, neither the AWSiOS SDK, nor ASI's S3 framework seem to support to multi-part uploads, or indicate that they plan to add support. I realize that I can initiate a 'longish' upload using beginBackgroundTaskWithExpirationHandler: and that'll give me a window of time to complete the upload (currently 600 seconds, I believe), but what's to be done if I'm not in a situation to complete said upload within that timeframe?
Aside from worrying about completing tasks within that time frame, is their a 'best practice' for how an app should resume uploads, or even just break a larger upload into smaller chunks?
I've thought about writing a library to talk to S3's REST API specifically for multi-part uploads, but this seems like a problem other have either been solved, or realized needn't be solved (perhaps for being completely in appropriate for the platform).
Another (overly complicated) solution would be chunking the file on the device, uploading those to S3 (or elsewhere) and have them re-assembled on S3 via a server process. This seems even more unpalatable than rolling my own library for multi-part upload.
How are others handling this problem?
Apparently I was looking at some badly out of date documentation.
in AmazonS3Client see:
- (S3MultipartUpload * AmazonS3Client)initiateMultipartUploadWithKey:(NSString *)theKey withBucket:(NSString *)theBucket
Which will give you a S3MultipartUpload which will contain an uploadId.
You can then put together an S3UploadPartRequest using initWithMultipartUpload: (S3MultipartUpload *) multipartUpload and send that as you usually would.
S3UploadPartRequest contains an int property partNumber where you can specify the part # you're uploading.
you can write some code to do so, you can refer code from http://dextercoder.blogspot.in/2012/02/multipart-upload-to-amazon-s3-in-three.html. Core java code, steps can be used for iOS.