Issues with cropping an image - ios

Question

I am having difficulties with cropping an image; two things are happening currently that I wish to improve:
1) The quality of the photo degrades once it is cropped.
2) The view of the orientation is not correct after the photo is taken.
In Summary:
What is occurring, is the photo quality after cropping is not of correct standard and when the image appears in the ImageView it is rotated 90 degrees; why are these occurring? I am trying to crop an image based on the view of the captured stream.
Here is the cropping of the image:
func crop(capture : UIImage) -> UIImage {
let crop = cameraView.bounds
//CGRect(x:0,y:0,width:50,height:50)
let cgImage = capture.cgImage!.cropping(to: crop)
let image : UIImage = UIImage(cgImage: cgImage!)
return image
}
Here is where I am calling the crop
func capture(_ captureOutput: AVCapturePhotoOutput, didFinishProcessingPhotoSampleBuffer photoSampleBuffer: CMSampleBuffer?, previewPhotoSampleBuffer: CMSampleBuffer?, resolvedSettings: AVCaptureResolvedPhotoSettings, bracketSettings: AVCaptureBracketedStillImageSettings?, error: Error?) {
if let photoSampleBuffer = photoSampleBuffer {
let photoData = AVCapturePhotoOutput.jpegPhotoDataRepresentation(forJPEGSampleBuffer: photoSampleBuffer, previewPhotoSampleBuffer: previewPhotoSampleBuffer)
var imageTaken = UIImage(data: photoData!)
//post photo
let croppedImage = self.crop(capture: imageTaken!)
imageTaken = croppedImage
self.imageView.image = imageTaken
// UIImageWriteToSavedPhotosAlbum(imageTaken!, nil, nil, nil)
}
}
& Here is the whole class
import UIKit
import AVFoundation
class CameraVC: UIViewController, UIImagePickerControllerDelegate, UINavigationControllerDelegate, AVCapturePhotoCaptureDelegate {
var captureSession : AVCaptureSession?
var stillImageOutput : AVCapturePhotoOutput?
var previewLayer : AVCaptureVideoPreviewLayer?
#IBOutlet weak var imageView: UIImageView!
#IBOutlet var cameraView: UIView!
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view.
}
override func viewDidLayoutSubviews() {
super.viewDidLayoutSubviews()
previewLayer?.frame = cameraView.bounds
}
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
captureSession = AVCaptureSession()
captureSession?.sessionPreset = AVCaptureSessionPreset1920x1080
stillImageOutput = AVCapturePhotoOutput()
let backCamera = AVCaptureDevice.defaultDevice(withMediaType: AVMediaTypeVideo)
do {
let input = try AVCaptureDeviceInput(device: backCamera)
if (captureSession?.canAddInput(input))!{
captureSession?.addInput(input)
if (captureSession?.canAddOutput(stillImageOutput) != nil){
captureSession?.addOutput(stillImageOutput)
previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
previewLayer?.videoGravity = AVLayerVideoGravityResizeAspect
previewLayer?.connection.videoOrientation = AVCaptureVideoOrientation.portrait
cameraView.layer.addSublayer(previewLayer!)
captureSession?.startRunning()
let captureVideoLayer: AVCaptureVideoPreviewLayer = AVCaptureVideoPreviewLayer.init(session: captureSession!)
captureVideoLayer.frame = self.cameraView.bounds
captureVideoLayer.videoGravity = AVLayerVideoGravityResizeAspectFill
self.cameraView.layer.addSublayer(captureVideoLayer)
}
}
} catch {
print("An error has occured")
}
}
#IBAction func takePhoto(_ sender: UIButton) {
didPressTakePhoto()
}
func didPressTakePhoto(){
if let videoConnection = stillImageOutput?.connection(withMediaType: AVMediaTypeVideo) {
videoConnection.videoOrientation = AVCaptureVideoOrientation.portrait
let settingsForCapture = AVCapturePhotoSettings()
settingsForCapture.flashMode = .auto
settingsForCapture.isAutoStillImageStabilizationEnabled = true
settingsForCapture.isHighResolutionPhotoEnabled = false
stillImageOutput?.capturePhoto(with: settingsForCapture, delegate: self)
}
}
func capture(_ captureOutput: AVCapturePhotoOutput, didFinishProcessingPhotoSampleBuffer photoSampleBuffer: CMSampleBuffer?, previewPhotoSampleBuffer: CMSampleBuffer?, resolvedSettings: AVCaptureResolvedPhotoSettings, bracketSettings: AVCaptureBracketedStillImageSettings?, error: Error?) {
if let photoSampleBuffer = photoSampleBuffer {
let photoData = AVCapturePhotoOutput.jpegPhotoDataRepresentation(forJPEGSampleBuffer: photoSampleBuffer, previewPhotoSampleBuffer: previewPhotoSampleBuffer)
var imageTaken = UIImage(data: photoData!)
//post photo
let croppedImage = self.crop(capture: imageTaken!)
imageTaken = croppedImage
self.imageView.image = imageTaken
// UIImageWriteToSavedPhotosAlbum(imageTaken!, nil, nil, nil)
}
}
func crop(capture : UIImage) -> UIImage {
let crop = cameraView.bounds
//CGRect(x:0,y:0,width:50,height:50)
let cgImage = capture.cgImage!.cropping(to: crop)
let image : UIImage = UIImage(cgImage: cgImage!)
return image
}
}

Answer 1

An alternative is to use a Core Image filter called CIPerspectiveCorrect. Since it uses a CIImage - which isn't a true image but a "recipe" for an image - it doesn't suffer from degradation.
Basically, turn your UIImage/CGImage into a CIImage, pick any 4 points in it, and crop. It needn't be a parallelogram (or CGRect), just 4 points. There are two differences of note when using CI filters:
Instead of using a CGRect, you use a CIVector. A vector can have 2, 3, 4, even more parameters depending on the filter. In this case you want 4 CIVectors with 2 parameters each, corresponding to top left (TL), top right (TR), bottom left (BL), and bottom right (BR).
CI images have their point of origin (X/Y == 0/0) at their bottom left, not top left. This basically means your Y coordinate is upside down from CG or UI images.
Here's some sample code. First, some sample declarations, including a CI context:
let uiTL = CGPoint(x: 50, y: 50)
let uiTR = CGPoint(x: 75, y: 75)
let uiBL = CGPoint(x: 100, y: 300)
let uiBR = CGPoint(x: 25, y: 200)
var ciImage:CIImage!
var ctx:CIContext!
#IBOutlet weak var imageView: UIImageView!
In viewDidLoad we set the context and get our CIImage from the UIImageView:
override func viewDidLoad() {
super.viewDidLoad()
ctx = CIContext(options: nil)
ciImage = CIImage(image: imageView.image!)
}
UIImageViews have a frame or CGRect, and UIImages have a size or CGSize. CIImages have an extent, which is basically your CGSize. But remember, the Y axis is flipped, and it is possible for an infinite extent! (This isn't thecae for a UIImage source though.) Here's some helper functions to convert things:
func createScaledPoint(_ pt:CGPoint) -> CGPoint {
let x = (pt.x / imageView.frame.width) * ciImage.extent.width
let y = (pt.y / imageView.frame.height) * ciImage.extent.height
return CGPoint(x: x, y: y)
}
func createVector(_ point:CGPoint) -> CIVector {
return CIVector(x: point.x, y: ciImage.extent.height - point.y)
}
func createPoint(_ vector:CGPoint) -> CGPoint {
return CGPoint(x: vector.x, y: ciImage.extent.height - vector.y)
}
Here's the actual call to CIPerspectiveCorrection. If I remember correctly, a change in Swift 3 is to use AnyObject. While more strongly-typed variables worked in previous versions of Swift, they cause dumps now:
func doPerspectiveCorrection(
_ image:CIImage,
context:CIContext,
topLeft:AnyObject,
topRight:AnyObject,
bottomRight:AnyObject,
bottomLeft:AnyObject)
-> UIImage {
let filter = CIFilter(name: "CIPerspectiveCorrection")
filter?.setValue(topLeft, forKey: "inputTopLeft")
filter?.setValue(topRight, forKey: "inputTopRight")
filter?.setValue(bottomRight, forKey: "inputBottomRight")
filter?.setValue(bottomLeft, forKey: "inputBottomLeft")
filter!.setValue(image, forKey: kCIInputImageKey)
let cgImage = context.createCGImage((filter?.outputImage)!, from: (filter?.outputImage!.extent)!)
return UIImage(cgImage: cgImage!)
}
Now that we have our CIImage, we create the four CIVectors. In this sample project I hard-coded the 4 CGPoints and chose to create the CIVector in viewWillLayoutSubviews, the earliest I have the UI frames:
override func viewWillLayoutSubviews() {
let ciTL = createVector(createScaledPoint(uiTL))
let ciTR = createVector(createScaledPoint(uiTR))
let ciBR = createVector(createScaledPoint(uiBR))
let ciBL = createVector(createScaledPoint(uiBL))
imageView.image = doPerspectiveCorrection(CIImage(image: imageView.image!)!,
context: ctx,
topLeft: ciTL,
topRight: ciTR,
bottomRight: ciBR,
bottomLeft: ciBL)
}
If you put this code into a project, load in your image into a UIImageView, figure out what 4 CGPoints you want and run this, you should see your capped image. Good luck!

Answer 2

Adding to #dfd answer. If you just don't want to use fancy core image filter follow the work around here.
the quality of the photo after I crop it is poor?
Though you use highest possible session preset AVCaptureSessionPreset1920x1080 you are converting the sample image buffer to JPEG format and then cropping it. So obviously there will be some loss on quality.
To get nice quality cropped image try session preset AVCaptureSessionPresetHigh to let AVFoundation decide the high quality video and dngPhotoDataRepresentation(forRawSampleBuffer:previewPhotoSampleBuffer:) to get the image in DNG format.
The view and the orientation is not correct after I take the photo?
didFinishProcessingPhotoSampleBuffer delegate gives your sample buffer which is always 90 degree rotated to left. So you might want to rotate 90 degree right by yourself.
init your UIImage with cgImage and orientation as right and scale as 1.0.
init(cgImage: CGImage, scale: CGFloat, orientation: UIImageOrientation)