I have been searching these threads and other sites but have not come across a way to do this both efficiently and memory friendly. And so, here is my story:
My IOS (iPad) app uses sprite sheets (a large image, such as 2k x 2k at 16 bpp, which is composed of many smaller sprites). I have created a sprite atlas class which manages these sheets, handle sprite animations, and other features.
The idea (in the load method) is to load in the sheet from the file system, split it apart into UIImages (one per sprite) using CGImageCreateWithImageInRect, and then dispose of the loaded sheet. Seems simple enough.
Note that the sheet is loaded into a UIImage by initWithContentsOfFile or imageNamed (more on that below).
The desire is to save file memory by using sprite sheets, and then save runtime memory by only retaining the actual sprites themselves as UIImages. In my experiments thus far this is what I find happening:
If I use initWithContentsOfFile I see (from Instruments) that it appears to do a file open, fstat64, and close for the file for EACH sprite in it. This takes an horrendous amount of time to load all the sprites. It actually seems to load the entire sheet, grab one sprite, close the sheet, then load the entire sheet again for the next sprite, until they are all created. Also it appears to consume lots of memory (proven by the "received memory warning" after just the second sheet loaded, as well as by Instruments allocations).
Next I tried imageNamed (which caches the sheet). The file loading occurs once and so it is MUCH faster. All seems good and in fact and I can go until many sheets are loaded. But eventually the dreaded "received memory warning" appears... and a few seconds later the app crashes. It appears to be the case that the cached image (even though the pointer is set to null after pulling out the sprites) never goes away. I have read several posts that also state this seems to be its behavior (although other posts say other things, so that is not conclusive - does anyone know definitively?).
And so it appears that neither method is what is needed. What I want is to load the file, pull out each sprite into its own UIImage, then have the UIImage for the big sheet file released completely.
I have read one site that talked about using the initWithContentsOfFile approach to set up their own caching system (rather than trust the IOS plan with imageNamed) so they can release the image when desired. However, I don't think they had sprite sheets in mind.
And so, I turn this over to the experts out there to see if there are some ideas on how to get both fast load times AND use minimal memory.
[and yes, I know that IOS 7 has SpriteKit. But this needs to also work on IOS 6.x.]
One interesting data point is that on an original iPad the imageNamed version actually works fine with no "received memory warning". It might have been IOS 5.x. But the app will crash on the iPad 2 device.
I am not including code here because what I am after is an understanding of the mechanics involved with how memory is used with these functions related to image handling.
And while I am at it, can someone please clarify this point:
True or False: When using CGImageCreateWithImageInRect, what it does is actually creates from new memory a bitmap the size of the rectangle specified and then COPIES from the original UIImage the pixels into this new memory (as opposed to setting up the bitmap format and having a pointer point into the original UIImage's pixel data). I think this is True, but want verification.
Thanks!
Related
My app is saving and retrieving data from Parse.com. And showing images, buttons, scrollviews, etc.. (the normal stuff). Then when I got near finishing my app, it started to receive memory warnings and the app started crashing often. I checked it in the Instruments and noticed the live bytes was extremely high at some points, and I can't figure out why.
Is the app crashing because of the high live bytes? What should value of the live bytes be?
Obiously something is going on in the VM. But I have no idea what this is. What is the VM: CG raster data? And this: VM: CG Image? I am not using CGImages only UIImages
Is the app crashing because of the high live bytes?
Yes.
What should value of the live bytes be?
There's not fixed number. The limits change from OS version to OS version, and sometimes depend on the device and what else is going on at the moment. The right thing to do is (a) try not to use so much, and (b) heed the warnings and dispose of stuff you don't need.
Obiously something is going on in the VM. But I have no idea what this is. What is the VM: CG raster data? And this: VM: CG Image? I am not using CGImages only UIImages
A UIImage is just a wrapper around a CGImage.
You have too many images alive at the same time. That's the problem you have to fix.
So, how many is too many? It depends on how big they are.
Also, note that the "raster data" is the decompressed size. A 5Mpix RGBA 8bpp image takes 20MB of RAM for its raster data, whether the file is 8MB or 8KB.
I still feel the number is too high though, or is 30-40 MB an okey number handling 3-6 full-screen sized images at a time? This is when tested on a 4 year old iPhone4, iOS 7. If that matters.
On an iPhone 4, "full-screen" means 640x960 pixels. 8bpp RGBA means 4 bytes per pixel. So, with 6 such images, that's 640*960*4*6 = 14MB. So, that's the absolute minimum storage you should expect if you've loaded and drawn 6 full-screen images.
So, why do you actually see more than twice that?
Well, as Images and Memory Management in the class reference says:
In low-memory situations, image data may be purged from a UIImage object to free up memory on the system. This purging behavior affects only the image data stored internally by the UIImage object and not the object itself. When you attempt to draw an image whose data has been purged, the image object automatically reloads the data from its original file. This extra load step, however, may incur a small performance penalty.
So think of that 14MB as basically a cache that iOS uses to speed things up, in case you want to draw the images again. If you run a little low on memory, it'll purge the cache automatically, so you don't have to worry about it.
So, that leaves you with 16-24MB, which is presumably used by the buffers of your UI widgets and layers and by the compositor behind the scenes. That's a bit more than the theoretical minimum of 14MB, but not horribly so.
If you want to reduce memory usage further, what you probably need to do is not draw all 6 images. If they're full-screen, there's no way the user can see more than 1 or 2 at a time. So, you could load and render them on demand instead of preloading them (or, if you can predict which one will usually be needed next, preload 1 of them instead of all of them), and destroy them when they're no longer visible. Since you'd then only have 2 images instead of 6, that should drop your memory usage from 16-24MB + a 14MB cache to 5-9MB + a 5MB cache. This obviously means a bit more CPU—it probably won't noticeably affect responsiveness or battery drain, but you'd want to test that. And, more importantly, it will definitely make your code more complicated.
Obviously, if it's appropriate for your images, you could also do things like using non-Retina images (which will cut memory by 75%) or dropping color depth from RGBA-8 to ARGB-1555 (50%), but most images don't look as good that way (which is why we have high-color Retina displays).
I'm trying to optimize an iOS application which contains a lot of images and code. I have reduced the size of images with some programs but using instruments reveals that the application is still taking among 70-90mb of cache memory.
I have read that loading the resources(images) by demand and discard them when are not longer needed would be a good solution. How can i do it?
I have also a question:
When we use:
UIImage *aux = [UIImage imagenamed:#"image.png"];
and after we write aux=nil;
the image is discarded from cache?
Are only some of those images visible at a time? Write a system that loads only those images currently visible (and perhaps some that you application thinks might become visible soon). When you get a memory warning from the system, look for some images you've loaded in the past that haven't been visible for a while and release them.
To answer your second question, yes, setting a reference to nil will release it IF you are using ARC (Automatic Reference Counting), and if the reference you set to nil is the only reference to that object. All references to an object must go away before it will be released.
I would look at some of the solutions available, such as Path's FastImageCache, and see if they meet your needs. FastImageCache stores uncompressed images on disk in a format similar to Sprite Sheets (used by 2D games) so they can be loaded quickly when needed. The emphasis here is on improving scrolling performance, so if that's not an issue for you, this might not be the right tool for the job.
You might also look at this thread, although this is aimed at caching web images.
You might also take a look at The Tumblr Image Cache
I'm looking for solution of animation about 50 images on retina iPad each has 2048*1536 size. I want to animate them on finger move(change images on uiimageview sync with touches moved event). Images loads slowly and animation freezes. I want to find any solution to solve that problem. Thanks.
There are a couple of issues that make this situation very hard to deal with. First, the memory usage of 50 full screen images is very large. For some background on how much memory that actually requires, see this blog post Video and Memory usage on iOS devices. The second issue you have run into is CPU usage. A retina iPad has multiple CPUs, but decoding huge PNG images still takes a lot of CPU cycles and that will prevent the animations from running smoothly. So, the only way you will get this to work well is to avoid decoding the image data at runtime and also avoid holding all the decoded data in memory because that would crash the device. The best solution is to simply mmap() all the decoded data and decode it ahead of time, that makes it possible to blit image data into CoreGraphics without actually having to copy the data. If you would like to use my library that does all that, it is linked at the bottom of the blog post.
I'm working on an iPad-only iOS app that essentially downloads large, high quality images (JPEG) from Dropbox and shows the selected image in a UIScrollView and UIImageView, allowing the user to zoom and pan the image.
The app is mainly used for showing the images to potential clients who are interested in buying them as framed prints. The way it works is that the image is first shown, zoomed and panned to show the potential client if they like the image. If they do like it, they can decide if they want to crop a specific area (while keeping to specific aspect ratios/sizes) and the final image (cropped or not) is then sent as an email attachment to production.
The problem I've been facing for a while now, is that even though the app will only be running on new iPads (ie. more memory etc.), I'm unable to find a method of handling the images so that the app doesn't get a memory warning and then crash.
Most of the images are sized 4256x2832, which brings the memory usage to at least 40MB per image. While I'm only displaying one image at a time, image cropping (which is the main memory/crash problem at the moment) is creating a new cropped image, which in turn momentarily bumps the apps total RAM usage to about 120MB, causing a crash.
So in short: I'm looking for a way to manage very large images, have the ability to crop them and after cropping still have enough memory to send them as email attachments.
I've been thinking about implementing a singleton image manager, which all the views would use and it would only contain one big image at a time, but I'm not sure if that's the right way to go, or even if it'd help in any way.
One way to deal with this is to tile the image. You can save the large decompressed image to "disk" as a series of tiles, and as the user pans around pull out only the tiles you need to actually display. You only ever need 1 tile in memory at a time because you draw it to the screen, then throw it out and load the next tile. (You'll probably want to cache the visible tiles in memory, but that's an implementation detail. Even having the whole image as tiles may relieve memory pressure as you don't need one large contiguous block.) This is how applications like Photoshop deal with this situation.
I ended up sort of solving the problem. Since I couldn't resize the original files in Dropbox (the client has their reasons), I went ahead and used BOSImageResizeOperation, which is essentially just a fast, thread-safe library for quickly resizing images.
Using this library, I noticed that images that previously took 40-60MB of memory per image, now only seemed to take roughly half that. Additionally, the resizing is so quick that the original image gets released from memory so fast, that iOS doesn't execute a memory warning.
With this, I've gotten further with the app and I appreciate all the idea, suggestions and comments. I'm hoping this will get the app done and I can get as far away from large image handling as possible, heh.
I have an iOS opengl es 2.0 app that needs to use a TON of large textures. Ideally 4096x4096. I have a struct array that contains all the info about the texture, and as I need to use each one I glGenTextures a new texture id and load the image there, free up the uiimage, etc. That all works great.
My app uses a bunch of textures for UI, image processing, etc. About 4-5 of the 15 I'm using for all of that are 4k x 4k. Rest are smaller. And then these load-as-needed textures are also 4k.
On loading about the 4th-5th of those the app crashes HARD. No console or debug. Just quits to the springboard in the middle of trying to load the next texture.
I don't have a memory leak - I ran instruments. I'm using ARC. I can post the crash report from the Organizer but it doesn't have much info. Just that my app's rpages was 170504.
I could post the image load code but its the same code I've used on all my apps for years. The new thing is pushing the system that hard and trying to load that many large textures.
Q1: Anyone have experience with using a ton of large textures?
So I resolved to the fact that I'll have to do preview res stuff at 1024x1024 and then final res stuff at 4096. The 1k images are now loading as needed and staying loaded. The 4k images will all be loaded one at a time into the same texture to be used and then move on to the next.
I wrote into my image loader a preview parameter and when set it shrinks the image to fit in 1024 during the load. Now Instead of crashing on the 4th or 5th I can add textures 'all day'. My GUESS is that I could do 16x as many as before. But I only need like 20-30 at a time. (only!) So far I've tried 20 with no memory warnings or crashes.
However.. if the app keeps running, because my textures are loaded at unique texture ids, at some point I would hit that spot where I need to unload one that's no longer needed to load the next one. This is probably very simple, but....
Q2: How do I free up a texture that's at an texture id when I no longer need it?
Q3: Will a memory warning tell me that I need to free up an open gl texture?
Q4: Aren't textures loaded on the PVR chip? Are they or how are they even taking up the phone's memory?
Thanks!
Removing Texture:
You have to use this GL call from the main thread.
glDeleteTextures(1, &_texture);
Memory warning is a general call to the application. It will not give you specific information. It is always better to remove unwanted textures from the memory if they are not needed anymore. Eg: We usually remove textures used in menu when the user moves to the In-Game screens, they are reloaded again when the user navigates back. This is much easier to manage memory than waiting for the system to call memory warning.
When you load PNG image, the data is decompressed and stored raw as array of colors per pixel. A 1K texture will use 4 mb despite of content/colors in the image. PVR is a hardware decompression chip which will decompress realtime when the image is used by the GPU, and the image file size you see is what memory it uses.