How to find UIImage Bottleneck - ios

I have an app that uses UIImage objects. Up to this point, I've been using image objects initialized using something like this:
UIImage *image = [UIImage imageNamed:imageName];
using an image in my app bundle. I've been adding functionality to allow users to use imagery from the camera or their library using UIImagePickerController. These images, obviously, can't be in my app bundle, so I initialize the UIImage object a different way:
UIImage *image = [UIImage imageWithContentsOfFile:pathToFile];
This is done after first resizing the image to a size similar to the other files in my app bundle, in both pixel dimensions and total bytes, both using Jpeg format (interestingly, PNG was much slower, even for the same file size). In other words, the file pointed to by pathToFile is a file of similar size as an image in the bundle (pixel dimensions match, and compression was chosen so byte count was similar).
The app goes through a loop making small pieces from the original image, among other things that are not relevant to this post. My issue is that going through the loop using an image created the second way takes much longer than using an image created the first way.
I realize the first method caches the image, but I don't think that's relevant, unless I'm not understanding how the caching works. If it is the relevant factor, how can I add caching to the second method?
The relevant portion of code that is causing the bottleneck is this:
[image drawInRect:self.imageSquare];
Here, self is a subclass of UIImageView. Its property imageSquare is simply a CGRect defining what gets drawn. This portion is the same for both methods. So why is the second method so much slower with similar sized UIImage object?
Is there something I could be doing differently to optimize this process?
EDIT: I change access to the image in the bundle to imageWithContentsOfFile and the time to perform the loop changed from about 4 seconds to just over a minute. So it's looking like I need to find some way to do caching like imageNamed does, but with non-bundled files.

UIImage imageNamed doesn't simply cache the image. It caches an uncompressed image. The extra time spent was not caused by reading from local storage to RAM but by decompressing the image.
The solution was to create a new uncompressed UIImage object and use it for the time sensitive portion of the code. The uncompressed object is discarded when that section of code is complete. For completeness, here is a copy of the class method to return an uncompressed UIImage object from a compressed one, thanks to another thread. Note that this assumes data is in CGImage. That is not always true for UIImage objects.
+(UIImage *)decompressedImage:(UIImage *)compressedImage
{
CGImageRef originalImage = compressedImage.CGImage;
CFDataRef imageData = CGDataProviderCopyData(
CGImageGetDataProvider(originalImage));
CGDataProviderRef imageDataProvider = CGDataProviderCreateWithCFData(imageData);
CFRelease(imageData);
CGImageRef image = CGImageCreate(
CGImageGetWidth(originalImage),
CGImageGetHeight(originalImage),
CGImageGetBitsPerComponent(originalImage),
CGImageGetBitsPerPixel(originalImage),
CGImageGetBytesPerRow(originalImage),
CGImageGetColorSpace(originalImage),
CGImageGetBitmapInfo(originalImage),
imageDataProvider,
CGImageGetDecode(originalImage),
CGImageGetShouldInterpolate(originalImage),
CGImageGetRenderingIntent(originalImage));
CGDataProviderRelease(imageDataProvider);
UIImage *decompressedImage = [UIImage imageWithCGImage:image];
CGImageRelease(image);
return decompressedImage;
}

Related

Should I use UIImage or CGImage in a Swift iOS App?

All the code I can find revolves around loading images directly into visual controls.
However, I have my own cache system (converting a project from another language) and so I as efficient as possible want the following:
Load jpg/png images - probably into a bitmap / cgimage. (his can either be from the file system or from images downloaded online)
Possibly save image back as a compressed/resized png/jpg file
Supply an image reference for a visual control
I am new to swift and ios platform, but as far as I can tell, cgimage is as close as it gets? However, there does not appear to be a way to load an image from he file system when using cgimage... But i have found people discussing ways for e.g. UIImage, so I am now doubting my initial impression ha cgimage was the best match for my needs.
It is easy to get confused between UIImage, CGImage and CIImage. The difference is following:
UIImage: UIImage object is a high-level way to display image data. You can create images from files, from Quartz image objects, or from raw image data you receive. They are immutable and must specify an image’s properties at initialization time. This also means that these image objects are safe to use from any thread.
Typically you can take NSData object containing a PNG or JPEG representation image and convert it to a UIImage.
CGImage: A CGImage can only represent bitmaps. Operations in CoreGraphics, such as blend modes and masking require CGImageRefs. If you need to access and change the actual bitmap data, you can use CGImage. It can also be converted to NSBitmapImageReps.
CIImage: A CIImage is an immutable object that represents an image. It is not an image. It only has the image data associated with it. It has all the information necessary to produce an image.
You typically use CIImage objects in conjunction with other Core Image classes such as CIFilter, CIContext, CIColor, and CIVector. You can create CIImage objects with data supplied from variety of sources such as Quartz 2D images, Core Videos image, etc.
It is required to use the various GPU optimized Core Image filters. They can also be converted to NSBitmapImageReps. It can be based on the CPU or the GPU.
In conclusion, UIImage is what you are looking for. Reasons are:
You can get image from device memory and assign it to UIImage
You can get image from URL and assign it to UIImage
You can write UIImage in your desired format to device memory
You can resize image assigned to UIImage
Once you have assigned an image to UIImage, you can use that instance in controls directly. e.g. setting background of a button, setting as image for UIImageView
Would have added code samples but all these are basic questions which have been already answered on Stackoverflow, so there is no point. Not to mention adding code will make this unnecessarily large.
Credit for summarizing differences: Randall Leung
You can load your image easily into an UIImage object...
NSData *data = [NSData dataWith...];
UIImage *image = [UIImage imageWithData:data];
If you then want to show it in a view, you can use an UIImageView:
UIImageView *imageView = [[UIImageView alloc] init]; // or whatever
...
imageView.image = image;
See more in UIImage documentation.
Per the documentation at: https://developer.apple.com/documentation/uikit/uiimage
let uiImage = uiImageView.image
let cgImage = uiImage.cgImage

iOS: Instruments shows imageio_png_data is 300x larger in size than its actual image size

I have an image that is only 28KB in size:
I'm adding it to my view using this code:
UIImageView *background = [UIImageView new];
background.frame = CGRectMake(0, 0, 1080, 1920);
background.image = [UIImage imageNamed:#"Submit.png"];
[self.view addSubview:background];
Now I'm profiling with Instruments Allocation and "Marking Generation" right before and right after the image is allocated:
Instruments indicates that it took 7.92MB to load the image into memory.
I'm seeing the same issue with other images as well.
Why is ImageIO_PNG_Data at 7.92MB when the image is only 28KB in size?
#matt and #dan really did a good job of explaining why an uncompressed image should take up literally 300X memory of the actual PNG image size to display on the screen. What makes this issue worse is that the iOS caches these images and does NOT release them from cache EVER, even on memory warnings.
So here's a way to prevent image caching on iOS to save up a ton of memory, just use imageWithContentsOfFile instead of imageNamed:
Replace:
background.image = [UIImage imageNamed:#"Submit.png"];
With:
background.image = [UIImage imageWithContentsOfFile:[[[NSBundle mainBundle] bundlePath] stringByAppendingString:#"/Submit.png"]];
and now the ImageIO_PNG_Data's will be released when the view controller is dismissed.
It's all right here:
https://developer.apple.com/library/ios/documentation/UIKit/Reference/UIImage_Class/#//apple_ref/occ/clm/UIImage/imageNamed:
If you have an image file that will only be displayed once and wish to
ensure that it does not get added to the system’s cache, you should
instead create your image using imageWithContentsOfFile:. This will
keep your single-use image out of the system image cache, potentially
improving the memory use characteristics of your app.
It's because a PNG is compressed data describing what the image looks like, so a PNG that is nothing but a solid color is tiny because it is easy to describe. But the bitmap is the bitmap - just a grid of pixels - and depends purely on the dimensions of the image (which, in your case, is immense).

imageWithContentsOfFile memory issue

My project is having image sharing functionality. In this functionality my app asks the user from which library you want to share.
I wrote below code to assign image, retrieved from Default library/Custom library.
Device default library
imgVwMediaFile.image = [UIImage imageWithCGImage:[asset thumbnail]; //(ALAsset *)asset
Custom library (images are at Documents/image/user)
imgVwMediaFile.image = [UIImage imageWithContentsOfFile:path];
//(NSString*)path
App displyas this images in collection view having custom cell.
When i select app custom library then images are retrieves from Documents directory ,but imagewithcontentofFile cosuming around 90 to 100 Mb memory.
In other case when i select app default libray, it is not cosuming more then 8 or 10 Mb memory.
I tried diffrent code from stack Q/A for custom library, but still memory issue is there.
-1-
CGImageRef iref = [[UIImage imageNamed:asset] CGImage] ;
imgVwMediaFile.image=[UIImage imageWithCGImage:iref];
iref=nil
-2-
NSData *imageData = [[NSData alloc] initWithContentsOfFile:path];
imgVwMediaFile.image=[UIImage imageWithData:imageData];
imageData=nil
-3-
imgVwMediaFile.image=[UIImage imageNamed:path];
So please guide me that, how can i load images from document dir ?
What are the best way to load images from document dir without increasing memory load ?
The problem is that yhe images you have saved to disk are large, so loading lots of the, to display in a collection takes a lot of memory. You use thumbnails from the built in library so you see a different result.
Ideally you would save a folder of thumbnails in your custom library and display those, then, when an image is selected, get the corresponding full size image to use (this is the approach basically all photo libraries use). Alternatively you can resize the images on-the-fly, but your scrolling is basically guaranteed to suck.
UIImage *img = [UIImage imageWithContentsOfFile:path];

Load large UIImages asynchronously

I have an NSManagedObject which has pictures that are stored somewhere like /var/mobile/Applications/.../.../uniqueIDforNSMO/Pictures/
I have no problem getting these pictures off the disk by finding them based on the NSMO's uniqueID, I have a DataController that will pull them for me, and that is very performant. But I run into issues when I try to add these pictures to a view, the UI becomes blocked. I suspect the problem is that these images are very high resolution (they are images from the iPad camera roll).
Displaying 5 images takes about 3 seconds, leaving the UI blocked. Displaying even just 2 images blocks the damn UI. Here's how I add them, on a background thread:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
UIImage *image = [self.photos objectAtIndex:index];
dispatch_async(dispatch_get_main_queue(), ^{
cell.imageView.image = image;
});
});
If I set the imageView.image without dispatching it takes even longer.
Could the problem be that the images are so large, while the cell's imageViews are only 150x150? Or is it because I only have a reference to the image at first, so the realization of the image is the slow part?
Can anyone suggest anything here? I have tried resizing the images before they are returned in the array and that did not help.
The problem is that the images are being lazily loaded - UIImage only loads the image into memory when it needs to be drawn. What you can do is eagerly load the image on a background thread:
Create a CGBitmapContextRef using UIGraphicsBeginImageContext
Draw the image into the context
Get a new image from the context using UIGraphicsGetImageFromCurrentImageContext
Pass the image back to the main thread
As they are high-resolution images, I would suggest that you should make persistent thumbnails cache for that. Generating thumbnail real-time every time would waste too much CPU time unnecessarily.

Resize UIImage in Place

I have a category (very popular code found on web) to UIImage to do various image manipulation.
- (UIImage *)imageScaledToSize:(CGSize)newSize {
UIGraphicsBeginImageContext(newSize);
[self drawInRect:CGRectMake(0, 0, newSize.width, newSize.height)];
UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
return newImage;
}
One aspect that I am making heavy use of is scaling an image down. My app can take quite large images and scales them down to a "working" size. However, there are still times when the app crashes due to memory. This is because the category creates a new scaled image from the original. Therefore, the original HUGE image is still resident while the new smaller (but still big) image is created.
So, my question is, is there a way to load this large original image and rescale it in place? That is, rescale the original without creating a new image, and not allocing more memory?
Yes, and there is even a complete working Apple sample project that does this for you.
As far as I know there is no limitation on what size image it can scale down. Of course though, the larger the image the more time consuming the process is.

Resources