Translating CIDetector (face detection) results into UIImageView coordinates - ios

I've been struggling to translate the CIDetector (face detection) results into coordinates relative to the UIImageView displaying the image so I can draw the coordinates using CGPaths.
I've looked at all questions here and all the tutorials I could find and most of them use small images that are not scaled when displayed in a UIImageView (example). The problem I am having is with using large images which are scaled using aspectFit when displayed in a UIImageView and determining the correct scale + translation values.
I am getting inconsistent results when testing with images of different sizes/aspect ratios, so I think my routine is flawed. I'd been struggling with this for a while so if anyone has some tips or can x-ray what I am doing wrong, that would be a great help.
What I am doing:
get the face coordinates
use the frameForImage routine below (found here on SO) to get the scale and bounds of the UIImageView image
create transform for scale + translation
apply transform to the CIDetector result
// my routine for determining transform values
NSDictionary* data = [self frameForImage:self.imageView.image inImageViewAspectFit:self.imageView];
CGRect scaledImageBounds = CGRectFromString([data objectForKey:#"bounds"]);
float scale = [[data objectForKey:#"scale"] floatValue];
CGAffineTransform transform = CGAffineTransformMakeScale(scale, -scale);
transform = CGAffineTransformTranslate(transform,
scaledImageBounds.origin.x / scale,
-(scaledImageBounds.origin.y / scale + scaledImageBounds.size.height / scale));
CIDetector results transformed using:
mouthPosition = CGPointApplyAffineTransform(mouthPosition, transform);
// example of bad result: scale seems incorrect
// routine below found here on SO for determining bound for image scaled in UIImageView using 'aspectFit`
-(NSDictionary*)frameForImage:(UIImage*)image inImageViewAspectFit:(UIImageView*)myImageView
{
float imageRatio = image.size.width / image.size.height;
float viewRatio = myImageView.frame.size.width / myImageView.frame.size.height;
float scale;
CGRect boundingRect;
if(imageRatio < viewRatio)
{
scale = myImageView.frame.size.height / image.size.height;
float width = scale * image.size.width;
float topLeftX = (myImageView.frame.size.width - width) * 0.5;
boundingRect = CGRectMake(topLeftX, 0, width, myImageView.frame.size.height);
}
else
{
scale = myImageView.frame.size.width / image.size.width;
float height = scale * image.size.height;
float topLeftY = (myImageView.frame.size.height - height) * 0.5;
boundingRect = CGRectMake(0, topLeftY, myImageView.frame.size.width, height);
}
NSDictionary * data = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithFloat:scale], #"scale",
NSStringFromCGRect(boundingRect), #"bounds",
nil];
return data;
}

I completely understand what you are trying to do, but let me offer you a different way to achieve what you want.
you have an over sized image
you know the size of the imageView
ask the image for its CGImage
determine a 'scale' factor, which is the imageView width divided by the image width
multiple this value and your image height, then subtract the result from the imageViewHeight, to get the "empty" height in the imageView, lets call this 'fillHeight'
divide 'fillHeight' by 2 and round to get the 'offset' value used below
using context provided by UIGraphicsBeginImageContextWithOptions(imageView.size, NO, 0), paint the background whatever color you want, then draw your CGImage
CGContextDrawImage (context, CGRectMake(0, offset, imageView.size.width, rintf( image.size.height*scale )), [image CGImage]);
get this new image using:
UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
return image;
set the image: imageView.image = image;
Now you can exactly map back to your image as you know the EXACT scaling ratio and offsets.

This might be the simple answer you are looking for. If you're x and y coordinates are inverted, you can mirror them yourself. In the below snippet im looping through my returned features and needing to invert the y coordinates, and x coordinate if it's front-facing camera:
for (CIFaceFeature *f in features) {
float newy = -f.bounds.origin.y + self.frame.size.height - f.bounds.size.height;
float newx = f.bounds.origin.x;
if( isMirrored ) {
newx = -f.bounds.origin.x + self.frame.size.width - f.bounds.size.width;
}
[[soups objectAtIndex:rnd] drawInRect:CGRectMake(newx, newy, f.bounds.size.width, f.bounds.size.height)];
}

Related

iOS, Thumbnail from image, Big Nerd Ranch, chapter 19

I am going through 19th chapter of Big Nerd Ranch, iOS textbook and can not understand several parts of the function that takes in a big image and creates a thumbnail out of it. Have a look:
- (void)setThumbnailFromImage:(UIImage *)image
{
CGSize origImageSize = image.size;
// The rectangle of the thumbnail
CGRect newRect = CGRectMake(0, 0, 40, 40);
// Figure out a scaling ratio to make sure we maintain the same aspect ratio
float ratio = MAX(newRect.size.width / origImageSize.width,
newRect.size.height / origImageSize.height);
// Create a transparent bitmap context with a scaling factor
// equal to that of the screen
UIGraphicsBeginImageContextWithOptions(newRect.size, NO, 0.0);
// Create a path that is a rounded rectangle
UIBezierPath *path = [UIBezierPath bezierPathWithRoundedRect:newRect
cornerRadius:5.0];
// Make all subsequent drawing clip to this rounded rectangle
[path addClip];
// Center the image in the thumbnail rectangle
CGRect projectRect;
projectRect.size.width = ratio * origImageSize.width;
projectRect.size.height = ratio * origImageSize.height;
projectRect.origin.x = (newRect.size.width - projectRect.size.width) / 2.0;
projectRect.origin.y = (newRect.size.height - projectRect.size.height) / 2.0;
[image drawInRect:projectRect];
// Get the image from the image context; keep it as our thumbnail
UIImage *smallImage = UIGraphicsGetImageFromCurrentImageContext();
self.thumbnail = smallImage;
// Cleanup image context resources; we're done
UIGraphicsEndImageContext();
}
From my understanding we are getting the MAX of the two ratios and then we put a smaller edge of the original image equal to newRect's edge (which is 40 in our case), the other edge seemingly should stick out of the newRect since the edge would be larger than the edge of newRect, when we UIGraphicsGetImageFromCurrentImageContext(). That's my vague 'understanding'.
Could anyone please explain what is this whole code doing in a detailed way, especially, the centering part? If you know some tutorials that might be relevant, it would also be great.
I just took or added to the previous comments and tried to explain each part more clearly. You seemed to get the basic idea, so I hope this helps solidify everything.
- (void)setThumbnailFromImage:(UIImage *)image
{
CGSize origImageSize = image.size;
//Create new rectangle of your desired size
CGRect newRect = CGRectMake(0, 0, 40, 40);
//Divide both the width and the height by the width and height of the original image to get the proper ratio.
//Take whichever one is greater so that the converted image isn't distorted through incorrect scaling.
float ratio = MAX(newRect.size.width / origImageSize.width,
newRect.size.height / origImageSize.height);
// Create a transparent bitmap context with a scaling factor
// equal to that of the screen
// Basically everything within this builds the image
UIGraphicsBeginImageContextWithOptions(newRect.size, NO, 0.0);
// Create a path that is a rounded rectangle -- essentially a frame for the new image
UIBezierPath *path = [UIBezierPath bezierPathWithRoundedRect:newRect
cornerRadius:5.0];
// Applying path
[path addClip];
// Center the image in the thumbnail rectangle
CGRect projectRect;
// Scale the image with previously determined ratio
projectRect.size.width = ratio * origImageSize.width;
projectRect.size.height = ratio * origImageSize.height;
// I believe the anchor point of the new image is (0.5, 0.5), so here he is setting the position to be in the middle
// Half of the width and height added to whatever origin you have (in this case 0) will give the proper coordinates
projectRect.origin.x = (newRect.size.width - projectRect.size.width) / 2.0;
projectRect.origin.y = (newRect.size.height - projectRect.size.height) / 2.0;
// Add the scaled image
[image drawInRect:projectRect];
// Retrieving the image that has been created and saving it in memory
UIImage *smallImage = UIGraphicsGetImageFromCurrentImageContext();
self.thumbnail = smallImage;
// Cleanup image context resources; we're done
UIGraphicsEndImageContext();
}

Cropping Images faster

I am trying to crop my images in iOS for awhile now. The code I have works well but isn't fast enough. When I supply it with around 20-25 images it takes 7-10 seconds to process it. I have tried every possible way to fix this, but haven't been successful. I am not sure what am I missing.
- (UIImage *)squareImageWithImage:(UIImage *)image scaledToSize:(CGSize)targetSize {
UIImage *sourceImage = image;
UIImage *newImage = nil;
CGSize imageSize = sourceImage.size;
CGFloat width = imageSize.width;
CGFloat height = imageSize.height;
CGFloat targetWidth = targetSize.width;
CGFloat targetHeight = targetSize.height;
CGFloat scaleFactor = 0.0;
CGFloat scaledWidth = targetWidth;
CGFloat scaledHeight = targetHeight;
CGPoint thumbnailPoint = CGPointMake(0.0,0.0);
if (CGSizeEqualToSize(imageSize, targetSize) == NO)
{
CGFloat widthFactor = targetWidth / width;
CGFloat heightFactor = targetHeight / height;
if (widthFactor > heightFactor)
{
scaleFactor = widthFactor; // scale to fit height
}
else
{
scaleFactor = heightFactor; // scale to fit width
}
scaledWidth = width * scaleFactor;
scaledHeight = height * scaleFactor;
// center the image
if (widthFactor > heightFactor)
{
thumbnailPoint.y = (targetHeight - scaledHeight) * 0.5;
}
else
{
if (widthFactor < heightFactor)
{
thumbnailPoint.x = (targetWidth - scaledWidth) * 0.5;
}
}
}
UIGraphicsBeginImageContext(targetSize); // this will crop
CGRect thumbnailRect = CGRectZero;
thumbnailRect.origin = thumbnailPoint;
thumbnailRect.size.width = scaledWidth;
thumbnailRect.size.height = scaledHeight;
[sourceImage drawInRect:thumbnailRect];
newImage = UIGraphicsGetImageFromCurrentImageContext();
if(newImage == nil)
{
NSLog(#"could not scale image");
}
//pop the context to get back to the default
UIGraphicsEndImageContext();
return newImage;
}
Your original question was not correctly stating the problem and so it now does: the reason that these operations take so long is the number of CPU cycles needed to scale an image (not crop it, which is much simpler and faster). When scaling, the system needs to use blending of some number of pixels that surround an area, which consume lots of cpu cycles. You can make this go faster by using a combination of techniques, but there is no single answer.
1) Use blocks and dispatch these image operations on a concurrent dispatch queue, to get parallelism. I believe the latest iPad has 4 cores that you can put to use that way. [UIGraphicsBeginImageContext is thread safe].
2) Get the ContextRef pointer, and set the interpolation setting to the lowest setting:
UIGraphicsBeginImageContext(targetSize);
CGContextRef context = UIGraphicsGetCurrentContext();
CGContextSetInterpolationQuality(context, kCGInterpolationLow);
...
3) Cheat - don't scale except by powers of two. In this technique, you would determine the "best" power of two to shrink the image by, the expand the width and height to fit your target size. If you can use a power of two, you can use the CGImageRef from the UIImage, get the pixel pointer, and copy every other pixel / every other row, and create a smaller image really quickly (using CGImageCreate). It may not be as high a quality that you would get with letting the system scale your image, but it will be faster. This is obviously a fair amount of code, but you can make the operations really fast this way.
4) Redefine your task. Instead of trying to resize a group of images, change your app so that you only show one or two resized images at a time, and while the user is looking at them, do the other image operations on a background queue. This is for completeness, I assume you already thought of this.
PS: if this works for you, no need for the bounty, help someone else instead.
Use drawInRect is slowly. You can try CGImageCreateWithImageInRect which will be faster(at least 10x faster than drawInRect).
CGImageRef imageRef = CGImageCreateWithImageInRect([self CGImage], theRect); // e.g. theRect={{100,100, {200, 200}}
UIImage *finalImage = [UIImage imageWithCGImage:imageRef];

Capture a Screen Shot of UIView - Slow Performance

I have a Drawing App of sorts, I would like to create a Snapshot of the Canvas UIView (both on and off screen) and then scale it down. The code I have for doing that take bloody for ever on an iPad 3. Simulator there is no delay. The Canvas is 2048x2048.
Is there another way I should be doing this? Or something I have a miss in the code?
Thank you!
-(UIImage *) createScreenShotThumbnailWithWidth:(CGFloat)width{
// Size of our View
CGSize size = editorContentView.bounds.size;
//First Grab our Screen Shot at Full Resolution
UIGraphicsBeginImageContext(size);
[editorContentView.layer renderInContext:UIGraphicsGetCurrentContext()];
UIImage *screenShot = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
//Calculate the scal ratio of the image with the width supplied.
CGFloat ratio = 0;
if (size.width > size.height) {
ratio = width / size.width;
} else {
ratio = width / size.height;
}
//Setup our rect to draw the Screen shot into
CGSize newSize = CGSizeMake(ratio * size.width, ratio * size.height);
//Send back our screen shot
return [self imageWithImage:screenShot scaledToSize:newSize];
}
Did you use the "Time Profiler" Instrument ("Product" Menu -> "Profile") to check where in your code you spend the most of your time? (use it with your Device of course, not the Simulator, to have realistic profiling). I'd guess it is not in the image capture portion you quoted in your question, but in your rescaling method imageWithImage:scaledToSize: method.
Instead of rendering the image at its whole size in a context, then rescaling the image to the final size, you should render the layer in the context directly at the expected size by applying some affine transform to the context.
So simply use CGContextConcatCTM(someScalingAffineTransform); on UIGraphicsGetCurrentContext() right after your line UIGraphicsBeginImageContext(size);, to apply an scaling affine transform that will make the layer be rendered at a different scale/size.
This way it will be directly rendered as the expected size which will be much faster, instead of being rendered at 100% and then having you to rescale it afterwards in a time-consuming way
Thank you AliSoftware, Here is the Code I ended up using:
-(UIImage *) createScreenShotThumbnailWithWidth:(CGFloat)width{
if (IoUIDebug & IoUIDebugSelectorNames) {
NSLog(#"%# - %#", INTERFACENAME, NSStringFromSelector(_cmd) );
}
// Size of our View
CGSize size = editorContentView.bounds.size;
//Calculate the scal ratio of the image with the width supplied.
CGFloat ratio = 0;
if (size.width > size.height) {
ratio = width / size.width;
} else {
ratio = width / size.height;
}
CGSize newSize = CGSizeMake(ratio * size.width, ratio * size.height);
//Create GraphicsContext with our new size
UIGraphicsBeginImageContext(newSize);
//Create Transform to scale down the Context
CGAffineTransform transform = CGAffineTransformIdentity;
transform = CGAffineTransformScale(transform, ratio, ratio);
//Apply the Transform to the Context
CGContextConcatCTM(UIGraphicsGetCurrentContext(),transform);
//Render our Image into the the Scaled Graphic Context
[editorContentView.layer renderInContext:UIGraphicsGetCurrentContext()];
//Save a copy of the Image of the Graphic Context
UIImage* screenShot = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
return screenShot;
}

Is it possible to display an image in Core Graphics with an Aspect Fit resize?

A CALayer can do it, and a UIImageView can do it. Can I directly display an image with aspect-fit with Core Graphics? The UIImage drawInRect does not allow me to set the resize mechanism.
If you're already linking AVFoundation, an aspect-fit function is provided in that framework:
CGRect AVMakeRectWithAspectRatioInsideRect(CGSize aspectRatio, CGRect boundingRect);
For instance, to scale an image to fit:
UIImage *image = …;
CRect targetBounds = self.layer.bounds;
// fit the image, preserving its aspect ratio, into our target bounds
CGRect imageRect = AVMakeRectWithAspectRatioInsideRect(image.size,
targetBounds);
// draw the image
CGContextDrawImage(context, imageRect, image.CGImage);
You need to do the math yourself. For example:
// desired maximum width/height of your image
UIImage *image = self.imageToDraw;
CGRect imageRect = CGRectMake(10, 10, 42, 42); // desired x/y coords, with maximum width/height
// calculate resize ratio, and apply to rect
CGFloat ratio = MIN(imageRect.size.width / image.size.width, imageRect.size.height / image.size.height);
imageRect.size.width = imageRect.size.width * ratio;
imageRect.size.height = imageRect.size.height * ratio;
// draw the image
CGContextDrawImage(context, imageRect, image.CGImage);
Alternatively, you can embed a UIImageView as a subview of your view, which gives you easy to use options for this. For similar ease of use but better performance, you can embed a layer containing the image in your view's layer. Either of these approaches would be worthy of a separate question, if you choose to go down that route.
Of course you can. It'll draw the image in whatever rect you pass. So just pass an aspect-fitted rect. Sure, you have to do a little bit of math yourself, but that's pretty easy.
here's the solution
CGSize imageSize = yourImage.size;
CGSize viewSize = CGSizeMake(450, 340); // size in which you want to draw
float hfactor = imageSize.width / viewSize.width;
float vfactor = imageSize.height / viewSize.height;
float factor = fmax(hfactor, vfactor);
// Divide the size by the greater of the vertical or horizontal shrinkage factor
float newWidth = imageSize.width / factor;
float newHeight = imageSize.height / factor;
CGRect newRect = CGRectMake(xOffset,yOffset, newWidth, newHeight);
[image drawInRect:newRect];
-- courtesy https://stackoverflow.com/a/1703210

Where on the screen is my UIImage?

I have a UIImage contained in a UIImageView. It's set to use the UIViewContentModeScaleAspectFit contentMode. The UIImageView is the size of the screen. The image is not, hence the scaleAspectFit mode. What I can't figure out is, where on the screen is the UIImage? What's it's frame? Where does the top left of the image appear on the screen? I can see it on the screen, but not in code. This should be simple, but I can't figure it out.
Try this in your UIImageView:
It will will compute the frame of the image, assuming you are using UIViewContentModeScaleAspectFit.
- (CGRect) imageFrame
{
float horizontalScale = self.frame.size.width / self.image.size.width;
float verticalScale = self.frame.size.height / self.image.size.height;
float scale = MIN(horizontalScale, verticalScale);
float xOffset = (self.frame.size.width - self.image.size.width * scale) / 2;
float yOffset = (self.frame.size.height - self.image.size.height * scale) / 2;
return CGRectMake(xOffset,
yOffset,
self.image.size.width * scale,
self.image.size.height * scale);
}
What it does is works out how much you need to shrink/enlarge the UIImage to fit it in the UIImageView in each dimension, and then picks the smallest scaling factor, to ensure that the image fits in the allotted space.
With that you know the size of the UI Image, and it's easy to calculate the X, Y offset w.r.t. the containing frame.

Resources