I was able to identify squares from a images using VNDetectRectanglesRequest. Now I want those rectangles to store as separate images (UIImage or cgImage). Below is what I tried.
let rectanglesDetection = VNDetectRectanglesRequest { request, error in
rectangles = request.results as! [VNRectangleObservation]
rectangles.sort{$0.boundingBox.origin.y > $1.boundingBox.origin.y}
for rectangle in rectangles {
let rect = rectangle.boundingBox
let imageRef = cgImage.cropping(to: rect)
let image = UIImage(cgImage: imageRef!, scale: image!.scale, orientation: image!.imageOrientation)
checkBoxImages.append(image)
}
Can anybody point out what's wrong or what should be the best approach?
Update 1
At this stage, I'm testing with an image that I added to the assets.
With this image I get 7 rectangles as observations as each for each cell and one for the table margin.
My task is to identify the text inside in each rectangle and my approach is to send VNRecognizeTextRequest for each rectangle that has been identified. My real scenario is little complicated than this but I want to at least achieve this before going forward.
Update 2
for rectangle in rectangles {
let trueX = rectangle.boundingBox.minX * image!.size.width
let trueY = rectangle.boundingBox.minY * image!.size.height
let width = rectangle.boundingBox.width * image!.size.width
let height = rectangle.boundingBox.height * image!.size.height
print("x = " , trueX , " y = " , trueY , " width = " , width , " height = " , height)
let cropZone = CGRect(x: trueX, y: trueY, width: width, height: height)
guard let cutImageRef: CGImage = image?.cgImage?.cropping(to:cropZone)
else {
return
}
let croppedImage: UIImage = UIImage(cgImage: cutImageRef)
croppedImages.append(croppedImage)
}
My image width and height is
width = 406.0 height = 368.0
I've taken my debug interface for you to get a proper understand.
As #Lasse mentioned, this is my actual issue with screenshots.
This is just a guess since you didn't state what the actual problem is, but probably you're getting a zero-sized image for each VNRectangleObservation.
The reason is: Vision uses a normalized coordinate space from 0.0 to 1.0 with lower left origin.
So in order to get the correct rectangle of your original image, you need to convert the rect from Normalized Space to Image Space. Luckily there is VNImageRectForNormalizedRect(::_:) to do just that.
Related
I have a UIScrollView which contains a UIImage. On top of that is a box that the user can move the image, so that that portion is cropped.
This screenshot explains it better:
So they can scroll the image around until the portion they want is inside that box.
I then want to be able to crop the scrollView/UIImage to exactly that size and store the cropped image.
It shouldn't be very hard but I've spent ages trying screenshots, UIGraphicsContext, etc. and cant seem to get anything to work.
Thanks for the help.
I finally figured out how to get it to work. Here is the code:
func croppedImage() -> UIImage {
let cropSize = CGSize(width: 280, height: 280)
let scale = (imageView.image?.size.height)! / imageView.frame.height
let cropSizeScaled = CGSize(width: cropSize.width * scale, height: cropSize.height * scale)
if #available(iOS 10.0, *) {
let r = UIGraphicsImageRenderer(size: cropSizeScaled)
let x = -scrollView.contentOffset.x * scale
let y = -scrollView.contentOffset.y * scale
return r.image { _ in
imageView.image!.draw(at: CGPoint(x: x, y: y))
}
} else {
return UIImage()
}
}
So it first calculates the scale of the imageView and the actual image.
Then it creates a CGSize of that crop box as shown in the photo. However, the width and height must be scaled by the scale factor. (e.g. 280 * 6.5)
You must check if the phone is running iOS 10.0 for UIGraphicsImageRender - if not, it won't work.
Initialise this with the crop box size.
The image must then be offset, and this is calculated by getting the scrollView's content offset, negating it, and multiplying by the scale factor.
Then return the image drawn at that point!
Given a CGRect, I want to use GPUImage to crop a video. For example, if the rect is (0, 0, 50, 50), the video would be cropped at (0,0) with a length of 50 on each side.
What's throwing me is that GPUImageCropFilter doesn't take a rectangle, rather a normalized crop region with values ranging from 0 to 1. My intuition was to to this:
let assetSize = CGSizeApplyAffineTransform(videoTrack.naturalSize, videoTrack.preferredTransform)
let cropRect = CGRect(x: frame.minX/assetSize.width,
y: frame.minY/assetSize.height,
width: frame.width/assetSize.width,
height: frame.height/assetSize.height)
to calculate the crop region based on the size of the incoming asset. Then:
// Filter
let cropFilter = GPUImageCropFilter(cropRegion: cropRect)
let url = NSURL(fileURLWithPath: "\(NSTemporaryDirectory())\(String.random()).mp4")
let movieWriter = GPUImageMovieWriter(movieURL: url, size: assetSize)
movieWriter.encodingLiveVideo = false
movieWriter.shouldPassthroughAudio = false
// add targets
movieFile.addTarget(cropFilter)
cropFilter.addTarget(movieWriter)
cropFilter.forceProcessingAtSize(frame.size)
cropFilter.setInputRotation(kGPUImageRotateRight, atIndex: 0)
What should the movie writer size be? Shouldn't it be the size of the frame I want to crop with? And should I be using forceProcessingAtSize with the size value of my crop frame?
A complete code example would be great; I've been trying for hours and I can't seem to get the section of the video that I want.
FINAL:
if let videoTrack = self.asset.tracks.first {
let movieFile = GPUImageMovie(asset: self.asset)
let transformedRegion = CGRectApplyAffineTransform(region, videoTrack.preferredTransform)
// Filters
let cropFilter = GPUImageCropFilter(cropRegion: transformedRegion)
let url = NSURL(fileURLWithPath: "\(NSTemporaryDirectory())\(String.random()).mp4")
let renderSize = CGSizeApplyAffineTransform(videoTrack.naturalSize, CGAffineTransformMakeScale(transformedRegion.width, transformedRegion.height))
let movieWriter = GPUImageMovieWriter(movieURL: url, size: renderSize)
movieWriter.transform = videoTrack.preferredTransform
movieWriter.encodingLiveVideo = false
movieWriter.shouldPassthroughAudio = false
// add targets
// http://stackoverflow.com/questions/37041231/gpuimage-crop-to-cgrect-and-rotate
movieFile.addTarget(cropFilter)
cropFilter.addTarget(movieWriter)
movieWriter.completionBlock = {
observer.sendNext(url)
observer.sendCompleted()
}
movieWriter.failureBlock = { _ in
observer.sendFailed(.VideoCropFailed)
}
disposable.addDisposable {
cropFilter.removeTarget(movieWriter)
movieWriter.finishRecording()
}
movieWriter.startRecording()
movieFile.startProcessing()
}
As you note, the GPUImageCropFilter takes in a rectangle in normalized coordinates. You're on the right track, in that you just need to convert your CGRect in pixels to normalized coordinates by dividing the X components (origin.x and size.width) by the width of the image and the Y components by the height.
You don't need to use forceProcessingAtSize(), because the crop will automatically output an image of the appropriate cropped size. The movie writer's size should be matched to this cropped size, which you should know from your original CGRect.
The one complication you introduce is the rotation. If you need to apply a rotation in addition to your crop, you might want to check and make sure that you don't need to swap your X and Y for your crop region. This should be apparent in the output if the two need to be swapped.
There were some bugs with applying rotation at the same time as a crop a while ago, and I can't remember if I fixed all those. If I didn't, you could insert a dummy filter (gamma or brightness set to default values) before or after the crop and apply the rotation at that stage.
I am trying to make a simple Crop functionality with Swift. I am trying with CGImageCreateWithImageInRect function - which works perfectly but produce inferior quality. Am I missing something ?
func retriveCroppedImage(){
let yratio: CGFloat = imgviewrect.size.height / chosenImage.size.height
let xratio: CGFloat = imgviewrect.size.width / chosenImage.size.width
var cliprect = CGRectMake(centerpoint.x - vWidth/2, centerpoint.y - vHeight/2, vWidth, vHeight)
print("cliprect top \(cliprect.size)")
cliprect.size.height = cliprect.size.height / xratio;
cliprect.size.width = cliprect.size.width / xratio;
cliprect.origin.x = cliprect.origin.x / xratio + imgviewrect.origin.x / xratio
cliprect.origin.y = cliprect.origin.y / yratio - imgviewrect.origin.y / xratio
print("cliprect On Image \(cliprect)")
let imageRef = CGImageCreateWithImageInRect(chosenImage.CGImage, cliprect )
croppedImg = UIImage(CGImage: imageRef!, scale: UIScreen.mainScreen().scale, orientation: chosenImage.imageOrientation)
print("Operation complete");
}
Screen shots : Main VC
after cropping I get Cropped Image
After trying all the options - I found accidentally I set Alpha in Image View on the story board. There was nothing wrong with the CGImageCreateWithImageInRect function. Now my cropping app is working as desired. But thank you all for the suggestions.
I am trying to create a simple crop feature that takes into effect device screen-density and zoom-scale
I basically modeled it after the code in this tutorial:
https://www.youtube.com/watch?v=hz9pMw4Y2Lk
func cropImage(sender:AnyObject!) { //triggered by a button
let myScale = UIScreen.mainScreen().scale
var height = self.scrollView.bounds.height
var width = self.scrollView.bounds.width
UIGraphicsBeginImageContextWithOptions(CGSizeMake(width, height), true, myScale)
let offset = scrollView.contentOffset
CGContextTranslateCTM(UIGraphicsGetCurrentContext(), -offset.x, -offset.y)
scrollView.layer.renderInContext(UIGraphicsGetCurrentContext())
let image = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
//i would like to check here if target image is >300x300px
if image.size.width > 300 && image.size.height > 300{
println("image correct")
println(image.size)
} else {
println("nope")
println(image.size)
}
}
So far I will always end up having an image that is bounds.height/width -which means that on a 320 device incl. a 8xp leading/trailing gap, the user might never be able to create a "correct image".
I understand why it happens, but I do not understand where I should be multiplying with device-scale factor and/or zoom-factor of the UIScrollView.
For example having a camera picture imported at ScrollView zoom-scale 0.0 - i want to keep it at ~8MP'ish.
I'm using the CIPerspectiveCorrection Filter and my problem is that my returned image results are mirrored, upside down, and the points used for the perspective correction seem to be referencing the wrong axis, or axis direction.
In order to isolate the issue I have been working with a test image that is 1024 x 1024 and I am passing in a perfectly rectangular area. I'm still ending up with images flipped vertically and horizontally.
Here is my function that returns a cropped CIImage instance given an image and set of points:
private func _getCroppedImageWithImage(image:CIImage, topLeft:CGPoint, topRight:CGPoint, botLeft:CGPoint, botRight:CGPoint) -> CIImage {
var rectCoords = NSMutableDictionary(capacity: 4)
rectCoords["inputTopLeft"] = CIVector(CGPoint:topLeft)
rectCoords["inputTopRight"] = CIVector(CGPoint:topRight)
rectCoords["inputBottomLeft"] = CIVector(CGPoint:botLeft)
rectCoords["inputBottomRight"] = CIVector(CGPoint:botRight)
return image.imageByApplyingFilter("CIPerspectiveCorrection", withInputParameters: rectCoords)
}
And here is where I am calling this function:
func testCrop() {
let ciInputImage = CIImage(image:UIImage(named:"test-pattern.jpg")!)
println("source image is \(ciInputImage)") //<CIImage: 0x170212290 extent [0 0 1024 1024]>
let ptBotLeft = CGPointMake(32.0,992.0)
let ptBotRight = CGPointMake(992.0,992.0)
let ptTopRight = CGPointMake(992.0,32.0)
let ptTopLeft = CGPointMake(32.0,32.0)
let croppedImage = _getCroppedImageWithImage(ciInputImage, topLeft: ptTopLeft, topRight: ptTopRight, botLeft: ptBotLeft, botRight: ptBotRight)
println("cropped image \(croppedImage)") //<CIImage: 0x174204a60 extent [0 0 960 960]>
let croppedImageCG = CIContext(options: nil).createCGImage(croppedImage, fromRect: croppedImage.extent())
let imageVC = ImageViewController(image: UIImage(CGImage: croppedImageCG))
presentViewController(imageVC, animated: true, completion: nil)
}
Has anyone encountered problems like this before?
Here is the source image
And here is the final image displayed in a UIImageView with contentMode set to scaleAspectFit
OK, my issue, I am pretty sure, is that CoreImage uses the Cartesian coordinate system. Y is up. (zero, zero) is at the bottom left.