My app is saving and retrieving data from Parse.com. And showing images, buttons, scrollviews, etc.. (the normal stuff). Then when I got near finishing my app, it started to receive memory warnings and the app started crashing often. I checked it in the Instruments and noticed the live bytes was extremely high at some points, and I can't figure out why.
Is the app crashing because of the high live bytes? What should value of the live bytes be?
Obiously something is going on in the VM. But I have no idea what this is. What is the VM: CG raster data? And this: VM: CG Image? I am not using CGImages only UIImages
Is the app crashing because of the high live bytes?
Yes.
What should value of the live bytes be?
There's not fixed number. The limits change from OS version to OS version, and sometimes depend on the device and what else is going on at the moment. The right thing to do is (a) try not to use so much, and (b) heed the warnings and dispose of stuff you don't need.
Obiously something is going on in the VM. But I have no idea what this is. What is the VM: CG raster data? And this: VM: CG Image? I am not using CGImages only UIImages
A UIImage is just a wrapper around a CGImage.
You have too many images alive at the same time. That's the problem you have to fix.
So, how many is too many? It depends on how big they are.
Also, note that the "raster data" is the decompressed size. A 5Mpix RGBA 8bpp image takes 20MB of RAM for its raster data, whether the file is 8MB or 8KB.
I still feel the number is too high though, or is 30-40 MB an okey number handling 3-6 full-screen sized images at a time? This is when tested on a 4 year old iPhone4, iOS 7. If that matters.
On an iPhone 4, "full-screen" means 640x960 pixels. 8bpp RGBA means 4 bytes per pixel. So, with 6 such images, that's 640*960*4*6 = 14MB. So, that's the absolute minimum storage you should expect if you've loaded and drawn 6 full-screen images.
So, why do you actually see more than twice that?
Well, as Images and Memory Management in the class reference says:
In low-memory situations, image data may be purged from a UIImage object to free up memory on the system. This purging behavior affects only the image data stored internally by the UIImage object and not the object itself. When you attempt to draw an image whose data has been purged, the image object automatically reloads the data from its original file. This extra load step, however, may incur a small performance penalty.
So think of that 14MB as basically a cache that iOS uses to speed things up, in case you want to draw the images again. If you run a little low on memory, it'll purge the cache automatically, so you don't have to worry about it.
So, that leaves you with 16-24MB, which is presumably used by the buffers of your UI widgets and layers and by the compositor behind the scenes. That's a bit more than the theoretical minimum of 14MB, but not horribly so.
If you want to reduce memory usage further, what you probably need to do is not draw all 6 images. If they're full-screen, there's no way the user can see more than 1 or 2 at a time. So, you could load and render them on demand instead of preloading them (or, if you can predict which one will usually be needed next, preload 1 of them instead of all of them), and destroy them when they're no longer visible. Since you'd then only have 2 images instead of 6, that should drop your memory usage from 16-24MB + a 14MB cache to 5-9MB + a 5MB cache. This obviously means a bit more CPU—it probably won't noticeably affect responsiveness or battery drain, but you'd want to test that. And, more importantly, it will definitely make your code more complicated.
Obviously, if it's appropriate for your images, you could also do things like using non-Retina images (which will cut memory by 75%) or dropping color depth from RGBA-8 to ARGB-1555 (50%), but most images don't look as good that way (which is why we have high-color Retina displays).
Related
I am planning to ship an app with at least 20 pictures that can be at big as 10mb each. They are pictures that the user is likely to zoom in quite a bit therefore it is a requirement to keep the resolution quite high. We are still trying to make them smaller, but even so, its unlikely that they are going to be less than 7mb each.
The images can be shipped with the app as well as additional pictures can also be downloaded. The requirement is for the pictures to be available offline once the user downloads them as the app is to be used in remote areas by researchers.
What is the best mechanism to store them and how should I store them in iOS Swift 3?
Thanks for your help in advance.
You can store your pictures in document directory of the app and store the path in SQLite DB.
There are not rules but a set of best practices.
To store them I suggest you to save them directly into your resources, not Xcode image asset, this is because "image asset" can only be called with the imageNamed: method of UIImage, that has the side effect of cache images.
Then you can create a plist file with an array of image names, and fetch your info from here. If you need something more complicated there is Core Data, but I can't see an application of it with your spec.
What is the size of an uncompressed image in memory? An approx
equation n_pixel_height * n_pixel_width * n_channels in bytes (supposing 8 bit for channel)
If your images are about 10Mb in jpg they are compressed, thus means that they will take a huge amount of memory. memory on this kind of devices is a precious and short resource.
If your app exceeds the memory limit, after a set of callback such as didReceiveMemoryWarning, if you don't free enough memory, you app will be killed.
Alway try on device in this case and not on simulator because the simulator use your mac resources.
Now how to handle big images?
You can use CATiledLayer, you can find a lot of tutorials online. CATiledLayer as the name suggest creates a tiles of layer, each tile should correspond at a piece of your image, and it draws them only when they are visible.
Unfortunately it draws asynchronously this means that your tiles can be shown not exactly at the same time, there some strategy to avoid that issue, one is explained in an apple sample code.
Of course there is a problem that needs to be solved, how can I cut my large images into tiles?
You can do programmatically or provide them already cut as resources of your application
I'm looking for solution of animation about 50 images on retina iPad each has 2048*1536 size. I want to animate them on finger move(change images on uiimageview sync with touches moved event). Images loads slowly and animation freezes. I want to find any solution to solve that problem. Thanks.
There are a couple of issues that make this situation very hard to deal with. First, the memory usage of 50 full screen images is very large. For some background on how much memory that actually requires, see this blog post Video and Memory usage on iOS devices. The second issue you have run into is CPU usage. A retina iPad has multiple CPUs, but decoding huge PNG images still takes a lot of CPU cycles and that will prevent the animations from running smoothly. So, the only way you will get this to work well is to avoid decoding the image data at runtime and also avoid holding all the decoded data in memory because that would crash the device. The best solution is to simply mmap() all the decoded data and decode it ahead of time, that makes it possible to blit image data into CoreGraphics without actually having to copy the data. If you would like to use my library that does all that, it is linked at the bottom of the blog post.
I'm working on an iPad-only iOS app that essentially downloads large, high quality images (JPEG) from Dropbox and shows the selected image in a UIScrollView and UIImageView, allowing the user to zoom and pan the image.
The app is mainly used for showing the images to potential clients who are interested in buying them as framed prints. The way it works is that the image is first shown, zoomed and panned to show the potential client if they like the image. If they do like it, they can decide if they want to crop a specific area (while keeping to specific aspect ratios/sizes) and the final image (cropped or not) is then sent as an email attachment to production.
The problem I've been facing for a while now, is that even though the app will only be running on new iPads (ie. more memory etc.), I'm unable to find a method of handling the images so that the app doesn't get a memory warning and then crash.
Most of the images are sized 4256x2832, which brings the memory usage to at least 40MB per image. While I'm only displaying one image at a time, image cropping (which is the main memory/crash problem at the moment) is creating a new cropped image, which in turn momentarily bumps the apps total RAM usage to about 120MB, causing a crash.
So in short: I'm looking for a way to manage very large images, have the ability to crop them and after cropping still have enough memory to send them as email attachments.
I've been thinking about implementing a singleton image manager, which all the views would use and it would only contain one big image at a time, but I'm not sure if that's the right way to go, or even if it'd help in any way.
One way to deal with this is to tile the image. You can save the large decompressed image to "disk" as a series of tiles, and as the user pans around pull out only the tiles you need to actually display. You only ever need 1 tile in memory at a time because you draw it to the screen, then throw it out and load the next tile. (You'll probably want to cache the visible tiles in memory, but that's an implementation detail. Even having the whole image as tiles may relieve memory pressure as you don't need one large contiguous block.) This is how applications like Photoshop deal with this situation.
I ended up sort of solving the problem. Since I couldn't resize the original files in Dropbox (the client has their reasons), I went ahead and used BOSImageResizeOperation, which is essentially just a fast, thread-safe library for quickly resizing images.
Using this library, I noticed that images that previously took 40-60MB of memory per image, now only seemed to take roughly half that. Additionally, the resizing is so quick that the original image gets released from memory so fast, that iOS doesn't execute a memory warning.
With this, I've gotten further with the app and I appreciate all the idea, suggestions and comments. I'm hoping this will get the app done and I can get as far away from large image handling as possible, heh.
I have an iOS opengl es 2.0 app that needs to use a TON of large textures. Ideally 4096x4096. I have a struct array that contains all the info about the texture, and as I need to use each one I glGenTextures a new texture id and load the image there, free up the uiimage, etc. That all works great.
My app uses a bunch of textures for UI, image processing, etc. About 4-5 of the 15 I'm using for all of that are 4k x 4k. Rest are smaller. And then these load-as-needed textures are also 4k.
On loading about the 4th-5th of those the app crashes HARD. No console or debug. Just quits to the springboard in the middle of trying to load the next texture.
I don't have a memory leak - I ran instruments. I'm using ARC. I can post the crash report from the Organizer but it doesn't have much info. Just that my app's rpages was 170504.
I could post the image load code but its the same code I've used on all my apps for years. The new thing is pushing the system that hard and trying to load that many large textures.
Q1: Anyone have experience with using a ton of large textures?
So I resolved to the fact that I'll have to do preview res stuff at 1024x1024 and then final res stuff at 4096. The 1k images are now loading as needed and staying loaded. The 4k images will all be loaded one at a time into the same texture to be used and then move on to the next.
I wrote into my image loader a preview parameter and when set it shrinks the image to fit in 1024 during the load. Now Instead of crashing on the 4th or 5th I can add textures 'all day'. My GUESS is that I could do 16x as many as before. But I only need like 20-30 at a time. (only!) So far I've tried 20 with no memory warnings or crashes.
However.. if the app keeps running, because my textures are loaded at unique texture ids, at some point I would hit that spot where I need to unload one that's no longer needed to load the next one. This is probably very simple, but....
Q2: How do I free up a texture that's at an texture id when I no longer need it?
Q3: Will a memory warning tell me that I need to free up an open gl texture?
Q4: Aren't textures loaded on the PVR chip? Are they or how are they even taking up the phone's memory?
Thanks!
Removing Texture:
You have to use this GL call from the main thread.
glDeleteTextures(1, &_texture);
Memory warning is a general call to the application. It will not give you specific information. It is always better to remove unwanted textures from the memory if they are not needed anymore. Eg: We usually remove textures used in menu when the user moves to the In-Game screens, they are reloaded again when the user navigates back. This is much easier to manage memory than waiting for the system to call memory warning.
When you load PNG image, the data is decompressed and stored raw as array of colors per pixel. A 1K texture will use 4 mb despite of content/colors in the image. PVR is a hardware decompression chip which will decompress realtime when the image is used by the GPU, and the image file size you see is what memory it uses.
I apologize upfront for the long question, with multiple subquestions, but the question is really as stated in the title. All that follows is a detailed breakup of different aspects of the question.
In my universal ios game built using cocos2d, I have four categories of images - I want to determine which of them should go into spritesheets and which are better off loaded as individual images. My current guess is that only character animations that run throughout the game provide value being loaded in memory as spritesheets:
Character animations that run throughout the game play (except
when user is in menus): for these, I assume that having the images
in a spritesheet will reduce runtime memory usage (due to the
padding of individual files to power of two byte boundaries), hence
they are candidates for a spritesheet. Is that correct?
Small images (about 200 of them) of which 0 to 4 are displayed at any
time, picked at random, during game play. For these, I am not sure
if it is worth having all 200 images loaded in memory when only at
most any 4 are used at a time. So it may be better to directly
access them as images. Is that correct?
A few (about 20) small menu elements like buttons that are used only in static menus: since the menu items are used only during menu display, I assume they are not of much value in improving memory access via spritesheets. Is that correct?
A few large images that are used as backgrounds for the menus, the game play scene, etc. Most of these images are as large as the screen resolution. Since the screen resolution is roughly equal to the maximum size of a single image (for example, for ipad retina, 4096 x 4096 image size versus screen size of 2048 x 1536), there isn't much gain in using spritesheets as at most 1 or 2 large images would fit in one spritesheet. Also, since each of these large files is used only in one scene, loading all these large images as spritesheets in memory upfront seems like an unnecessary overhead. Hence, directly access them as spritesheets. Is that correct?
A couple of common related questions:
a) Packing images used across different scenes into the same spritesheet makes us load them into memory even when only a subset of the images is used in that scene. I assume that is a bad idea. Correct?
b) There is a stutter in the game play only on older devices (iPad 1 and iPhone 3gs). Will spritesheets help reduce such stutter?
c) I assume that spritesheets are only going to benefit runtime memory footprint, and not so much the on disk size of the app archive. I noticed, for instance, that a set of files which are 11.8 MB in size, when put in a spritesheet are 11 Mb - not much of a compression benefit. Is that a valid assumption?
Please let me know your thoughts on my rationale above.
Thanks
Anand
Rule of thumb: everything goes in a spritesheet (texture atlas), unless there's a good reason not to.
Definitely texture atlas.
Cocos2d's caching mechanism will cause all individual images to be cached and remain in memory, so eventually they'll use more than if they were in a texture atlas. I would only use single images if they're rarely needed (ie menu icons) and I would also remove them from CCTextureCache just after they've been added to the scene, to make sure they will be removed from memory when leaving that menu screen.
Assumption may be wrong. A 320x480 image uses 512x512 as a texture in memory. If you have multiple of them, you may be better off having them all in a single texture atlas. Unless you enabled NPOT support in cocos2d. Again, don't forget that CCTextureCache is caching the textures.
Keep in mind that large textures benefit a lot from spritebatching. So even if there may only be 2-3 images in the texture atlas, it can make a difference in performance.
a) It really depends. If your app is running low on memory, you might want to have separate texture atlases. Otherwise it would be wasteful not to pack the images into a single texture atlas (if possible).
b) It depends on what's causing the stutter. Most common issue: loading resources (ie images) during gameplay, or creating and removing many nodes every frame instead of re-using existing ones.
c) Probably. It may depend on the texture atlas tool (I recommend TexturePacker). It definitely depends on the file format. If you can pack multiple PNG into a single .pvr.ccz texture atlas, you'll save a lot of memory (and gain loading speed).
For more info refer to my cocos2d memory optimization blog post.