CGImageCreateWithImageInRect() returning nil - ios

I'm trying to crop an image into a square, but once I actually try to do the crop by using CGImageCreateWithImageInRect(), this line crashes. I set breakpoints and made sure that the arguments passed into this function are not nil.
I'm fairly new to programming and Swift, but have searched around and haven't found any solution to my problem.
The failure reason:
fatal error: unexpectedly found nil while unwrapping an Optional value
func cropImageToSquare(imageData: NSData) -> NSData {
let image = UIImage(data: imageData)
let contextImage : UIImage = UIImage(CGImage: image!.CGImage!)
let contextSize: CGSize = contextImage.size
let imageDimension: CGFloat = contextSize.height
let posY : CGFloat = (contextSize.height + (contextSize.width - contextSize.height)/2)
let rect: CGRect = CGRectMake(0, posY, imageDimension, imageDimension)
// error on line below: fatal error: unexpectedly found nil while unwrapping an Optional value
let imageRef: CGImageRef = CGImageCreateWithImageInRect(contextImage.CGImage, rect)!
let croppedImage : UIImage = UIImage(CGImage: imageRef, scale: 1.0, orientation: image!.imageOrientation)
let croppedImageData = UIImageJPEGRepresentation(croppedImage, 1.0)
return croppedImageData!
}

Your code uses a lot of force-unwrapping with !s. I would recommend avoiding this — the compiler is trying to help you write code that won't crash. Use optional chaining with ?, and if let / guard let, instead.
The ! on that particular line is hiding an issue where CGImageCreateWithImageInRect might return nil. The documentation explains that this happens when the rect is not correctly inside the image bounds. Your code works for images in portrait orientation, but not landscape.
Furthermore, there's a convenient function provided by AVFoundation which can automatically find the right rectangle for you to use, called AVMakeRectWithAspectRatioInsideRect. No need to do the calculations manually :-)
Here's what I would recommend:
import AVFoundation
extension UIImage
{
func croppedToSquare() -> UIImage
{
guard let cgImage = self.CGImage else { return self }
// Note: self.size depends on self.imageOrientation, so we use CGImageGetWidth/Height here.
let boundingRect = CGRect(
x: 0, y: 0,
width: CGImageGetWidth(cgImage),
height: CGImageGetHeight(cgImage))
// Crop to square (1:1 aspect ratio) and round the resulting rectangle to integer coordinates.
var cropRect = AVMakeRectWithAspectRatioInsideRect(CGSize(width: 1, height: 1), boundingRect)
cropRect.origin.x = ceil(cropRect.origin.x)
cropRect.origin.y = ceil(cropRect.origin.y)
cropRect.size.width = floor(cropRect.size.width)
cropRect.size.height = floor(cropRect.size.height)
guard let croppedImage = CGImageCreateWithImageInRect(cgImage, cropRect) else {
assertionFailure("cropRect \(cropRect) was not inside \(boundingRect)")
return self
}
return UIImage(CGImage: croppedImage, scale: self.scale, orientation: self.imageOrientation)
}
}
// then:
let croppedImage = myUIImage.croppedToSquare()

Related

Pixellating a UIImage returns UIImage with a different size

I'm using an extension to pixellate my images like the following:
func pixellated(scale: Int = 8) -> UIImage? {
guard let ciImage = CIImage(image: self), let filter = CIFilter(name: "CIPixellate") else { return nil }
filter.setValue(ciImage, forKey: kCIInputImageKey)
filter.setValue(scale, forKey: kCIInputScaleKey)
guard let output = filter.outputImage else { return nil }
return UIImage(ciImage: output)
}
The problem is the image represented by self here has not the same size than the one I create using UIImage(ciImage: output).
For example, using that code:
print("image.size BEFORE : \(image.size)")
if let imagePixellated = image.pixellated(scale: 48) {
image = imagePixellated
print("image.size AFTER : \(image.size)")
}
will print:
image.size BEFORE : (400.0, 298.0)
image.size AFTER : (848.0, 644.0)
Not the same size and not the same ratio.
Any idea why?
EDIT:
I added some prints in the extension as following:
func pixellated(scale: Int = 8) -> UIImage? {
guard let ciImage = CIImage(image: self), let filter = CIFilter(name: "CIPixellate") else { return nil }
print("UIIMAGE : \(self.size)")
print("ciImage.extent.size : \(ciImage.extent.size)")
filter.setValue(ciImage, forKey: kCIInputImageKey)
filter.setValue(scale, forKey: kCIInputScaleKey)
guard let output = filter.outputImage else { return nil }
print("output : \(output.extent.size)")
return UIImage(ciImage: output)
}
And here are the outputs:
UIIMAGE : (250.0, 166.5)
ciImage.extent.size : (500.0, 333.0)
output : (548.0, 381.0)
You have two problems:
self.size is measured in points. self's size in pixels is actually self.size multiplied by self.scale.
The CIPixellate filter changes the bounds of its image.
To fix problem one, you can simply set the scale property of the returned UIImage to be the same as self.scale:
return UIImage(ciImage: output, scale: self.scale, orientation: imageOrientation)
But you'll find this still isn't quite right. That's because of problem two. For problem two, the simplest solution is to crop the output CIImage:
// Must use self.scale, to disambiguate from the scale parameter
let floatScale = CGFloat(self.scale)
let pixelSize = CGSize(width: size.width * floatScale, height: size.height * floatScale)
let cropRect = CGRect(origin: CGPoint.zero, size: pixelSize)
guard let output = filter.outputImage?.cropping(to: cropRect) else { return nil }
This will give you an image of the size you want.
Now, your next question may be, "why is there a thin, dark border around my pixellated images?" Good question! But ask a new question for that.

Swift 3 - How do I improve image quality for Tesseract?

I am using Swift 3 to build a mobile app that allows the user to take a picture and run Tesseract OCR over the resulting image.
However, I've been trying to increase the quality of scan and it doesn't seem to be working much. I've segmented the photo into a more "zoomed in" region that I want to recognize and even tried making it black and white. Are there any strategies for "enhancing" or optimizing the picture quality/size so that Tesseract can recognize it better? Thanks!
tesseract.image = // the camera photo here
tesseract.recognize()
print(tesseract.recognizedText)
I got these errors and have no idea what to do:
Error in pixCreateHeader: depth must be {1, 2, 4, 8, 16, 24, 32}
Error in pixCreateNoInit: pixd not made
Error in pixCreate: pixd not made
Error in pixGetData: pix not defined
Error in pixGetWpl: pix not defined
2017-03-11 22:22:30.019717 ProjectName[34247:8754102] Cannot convert image to Pix with bpp = 64
Error in pixSetYRes: pix not defined
Error in pixGetDimensions: pix not defined
Error in pixGetColormap: pix not defined
Error in pixClone: pixs not defined
Error in pixGetDepth: pix not defined
Error in pixGetWpl: pix not defined
Error in pixGetYRes: pix not defined
Please call SetImage before attempting recognition.Please call SetImage before attempting recognition.2017-03-11 22:22:30.026605 EOB-Reader[34247:8754102] No recognized text. Check that -[Tesseract setImage:] is passed an image bigger than 0x0.
ive been using tesseract fairly successfully in swift 3 using the following:
func performImageRecognition(_ image: UIImage) {
let tesseract = G8Tesseract(language: "eng")
var textFromImage: String?
tesseract?.engineMode = .tesseractCubeCombined
tesseract?.pageSegmentationMode = .singleBlock
tesseract?.image = imageView.image
tesseract?.recognize()
textFromImage = tesseract?.recognizedText
print(textFromImage!)
}
I also found pre-processing the image helped too. I added the following extension to UIImage
import UIKit
import CoreImage
extension UIImage {
func toGrayScale() -> UIImage {
let greyImage = UIImageView()
greyImage.image = self
let context = CIContext(options: nil)
let currentFilter = CIFilter(name: "CIPhotoEffectNoir")
currentFilter!.setValue(CIImage(image: greyImage.image!), forKey: kCIInputImageKey)
let output = currentFilter!.outputImage
let cgimg = context.createCGImage(output!,from: output!.extent)
let processedImage = UIImage(cgImage: cgimg!)
greyImage.image = processedImage
return greyImage.image!
}
func binarise() -> UIImage {
let glContext = EAGLContext(api: .openGLES2)!
let ciContext = CIContext(eaglContext: glContext, options: [kCIContextOutputColorSpace : NSNull()])
let filter = CIFilter(name: "CIPhotoEffectMono")
filter!.setValue(CIImage(image: self), forKey: "inputImage")
let outputImage = filter!.outputImage
let cgimg = ciContext.createCGImage(outputImage!, from: (outputImage?.extent)!)
return UIImage(cgImage: cgimg!)
}
func scaleImage() -> UIImage {
let maxDimension: CGFloat = 640
var scaledSize = CGSize(width: maxDimension, height: maxDimension)
var scaleFactor: CGFloat
if self.size.width > self.size.height {
scaleFactor = self.size.height / self.size.width
scaledSize.width = maxDimension
scaledSize.height = scaledSize.width * scaleFactor
} else {
scaleFactor = self.size.width / self.size.height
scaledSize.height = maxDimension
scaledSize.width = scaledSize.height * scaleFactor
}
UIGraphicsBeginImageContext(scaledSize)
self.draw(in: CGRect(x: 0, y: 0, width: scaledSize.width, height: scaledSize.height))
let scaledImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return scaledImage!
}
func orientate(img: UIImage) -> UIImage {
if (img.imageOrientation == UIImageOrientation.up) {
return img;
}
UIGraphicsBeginImageContextWithOptions(img.size, false, img.scale)
let rect = CGRect(x: 0, y: 0, width: img.size.width, height: img.size.height)
img.draw(in: rect)
let normalizedImage : UIImage = UIGraphicsGetImageFromCurrentImageContext()!
UIGraphicsEndImageContext()
return normalizedImage
}
}
And then called this before passing the image to performImageRecognition
func processImage() {
self.imageView.image! = self.imageView.image!.toGrayScale()
self.imageView.image! = self.imageView.image!.binarise()
self.imageView.image! = self.imageView.image!.scaleImage()
}
Hope this helps

Tesseract OCR w/ iOS & Swift returns error or gibberish

I used this tutorial to get Tesseract OCR working with Swift: http://www.piterwilson.com/blog/2014/10/18/minimal-tesseact-ocr-setup-in-swift/
It works fine if I upload the demo image and call
tesseract.image = UIImage(named: "image_sample.jpg");
But if I use my camera code and take a picture of that same image and call
tesseract.image = self.image.blackAndWhite();
the result is either gibberish like
s I 5E251 :Ec
‘-. —7.//:E*髧
a g :_{:7 IC‘
J 7 iii—1553‘
: fizzle —‘;-—:
; ~:~./: -:-‘-
‘- :~£:': _-'~‘:
: 37%; §:‘—_
: ::::E 7,;.
1f:,:~ ——,
Or it returns a BAD_EXC_ACCESS error. I haven't been able to reproduce the reasoning behind why it gives the error or the gibberish. This is the code of my camera capture (photo taken()) and the processing step (nextStepTapped()):
#IBAction func photoTaken(sender: UIButton) {
var videoConnection = stillImageOutput.connectionWithMediaType(AVMediaTypeVideo)
if videoConnection != nil {
// Show next step button
self.view.bringSubviewToFront(self.nextStep)
self.nextStep.hidden = false
// Secure image
stillImageOutput.captureStillImageAsynchronouslyFromConnection(videoConnection) {
(imageDataSampleBuffer, error) -> Void in
var imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(imageDataSampleBuffer)
self.image = UIImage(data: imageData)
//var dataProvider = CGDataProviderCreateWithCFData(imageData)
//var cgImageRef = CGImageCreateWithJPEGDataProvider(dataProvider, nil, true, kCGRenderingIntentDefault)
//self.image = UIImage(CGImage: cgImageRef, scale: 1.0, orientation: UIImageOrientation.Right)
}
// Freeze camera preview
captureSession.stopRunning()
}
}
#IBAction func nextStepTapped(sender: UIButton) {
// Save to camera roll & proceeed
//UIImageWriteToSavedPhotosAlbum(self.image.blackAndWhite(), nil, nil, nil)
//UIImageWriteToSavedPhotosAlbum(self.image, nil, nil, nil)
// OCR
var tesseract:Tesseract = Tesseract();
tesseract.language = "eng";
tesseract.delegate = self;
tesseract.image = self.image.blackAndWhite();
tesseract.recognize();
NSLog("%#", tesseract.recognizedText);
}
The image saves to the Camera Roll and is completely legible if I uncomment the commented lines. Not sure why it won't work. It has no problem reading the text on the image if it's uploaded directly into Xcode as a supporting file, but if I take a picture of the exact same image on my screen then it can't read it.
Stumbled upon this tutorial: http://www.raywenderlich.com/93276/implementing-tesseract-ocr-ios
It happened to mention scaling the image. They chose the max dimension as 640. I was taking my pictures as 640x480, so I figured I didn't need to scale them, but I think this code essentially redraws the image. For some reason now my photos OCR fairly well. I still need to work on image processing for smaller text, but it works perfectly for large text. Run my image through this scaling function and I'm good to go.
func scaleImage(image: UIImage, maxDimension: CGFloat) -> UIImage {
var scaledSize = CGSize(width: maxDimension, height: maxDimension)
var scaleFactor: CGFloat
if image.size.width > image.size.height {
scaleFactor = image.size.height / image.size.width
scaledSize.width = maxDimension
scaledSize.height = scaledSize.width * scaleFactor
} else {
scaleFactor = image.size.width / image.size.height
scaledSize.height = maxDimension
scaledSize.width = scaledSize.height * scaleFactor
}
UIGraphicsBeginImageContext(scaledSize)
image.drawInRect(CGRectMake(0, 0, scaledSize.width, scaledSize.height))
let scaledImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return scaledImage
}

UIImage Orientation Swift

I have written this code to capture an image using the AVFoundation library in Swift:
#IBAction func cameraButtonWasPressed(sender: AnyObject) {
if let videoConnection = stillImageOutput.connectionWithMediaType(AVMediaTypeVideo){
stillImageOutput.captureStillImageAsynchronouslyFromConnection(videoConnection){
(imageSampleBuffer : CMSampleBuffer!, _) in
let imageDataJpeg = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(imageSampleBuffer)
var pickedImage: UIImage = UIImage(data: imageDataJpeg)!
let library = ALAssetsLibrary()
library.writeImageToSavedPhotosAlbum(pickedImage.CGImage,
metadata:nil,
completionBlock:nil)
}
}
}
It works fine, but when I go to the photo library the image shows rotated 90 degrees counter clockwise.
Can someone give me an hint on where to dig to fix this?
Maybe this Swift code can help you.
//correctlyOrientedImage.swift
import UIKit
extension UIImage {
public func correctlyOrientedImage() -> UIImage {
if self.imageOrientation == UIImageOrientation.Up {
return self
}
UIGraphicsBeginImageContextWithOptions(self.size, false, self.scale)
self.drawInRect(CGRectMake(0, 0, self.size.width, self.size.height))
var normalizedImage:UIImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
return normalizedImage;
}
}
I saw it somewhere in stack overflow and incorporated in my project as well.
You should be using a slightly different writeImage method:
(1) get the orientation from the UIImage imageOrientation property (an enum), and cast it to ALAssetOrientation (an enum with the same Int values as UIImageOrientation)
var orientation : ALAssetOrientation = ALAssetOrientation(rawValue:
pickedImage.imageOrientation.rawValue)!
(2) use a similar-but-different method on ALAssetLibrary
library.writeImageToSavedPhotosAlbum(
pickedImage.CGImage,
orientation: orientation,
completionBlock:nil)
This works for me in Objective-C ... I have had a quick go at translating to Swift (as above) but I am getting compiler warnings.
Cannot invoke 'writeImageToSavedPhotosAlbum' with an argument list of type '(CGImage!, orientation: ALAssetOrientation, completionBlock: NilLiteralConvertible)'
Perhaps you could try (I don't have the time to construct a full AVFoundation pipeline in Swift to test this definitively)
If you can't get that to work, the other solution is to extract the exif metadata from the sampleBuffer and pass it through to the method you are already using
library.writeImageToSavedPhotosAlbum(pickedImage.CGImage,
metadata:nil,
completionBlock:nil
Swift 5
extension UIImage {
public func correctlyOrientedImage() -> UIImage {
if self.imageOrientation == UIImage.Orientation.up {
return self
}
UIGraphicsBeginImageContextWithOptions(self.size, false, self.scale)
self.draw(in: CGRect(x: 0, y: 0, width: self.size.width, height: self.size.height))
let normalizedImage = UIGraphicsGetImageFromCurrentImageContext()!;
UIGraphicsEndImageContext();
return normalizedImage;
}
}

Custom CIFilter subclass returns image with scale out of whack

I'm new to writing CIFilters, and I'm stuck on this problem. Here is my source image being displayed in a UIImageView with contentMode set to Aspect Fit:
Here is the image returned from my CIFilter object being displayed in the same UIImageView:
I've tried copying over the original scale and orientation from my source image to the UIImage being constructed from the CIImage returned from the filter with no luck.
What might be causing this?
I'm thinking I'm doing something wrong in my CIFilter class. I am starting to suspect something in my outputImage method:?
func outputImage() -> CIImage? {
if let inputImage = inputImage {
let dod = inputImage.extent()
if let kernel = kernel {
let args = [inputImage as AnyObject]
let dod = inputImage.extent().rectByInsetting(dx: -1, dy: -1)
return kernel.applyWithExtent(dod, roiCallback: {
(index, rect) in
return rect.rectByInsetting(dx: -1, dy: -1)
}, arguments: args)
}
}
return nil
}

Resources