I have a PKCanvas, what i am trying to do is to take the pkcanvas drawing that the user has made ad compare to another image.
However i keep getting nil returned section.
Why is this? my drawing and picture elements are returning nil. but my image2 element is returning the users drawing as expected.
in the code below i am just comparing the drawing the user made against itself (just for testing purposes)
Also "nothing here" does get printed.
//function type for ciimage
func featureprintObservationForImage(_ image: CIImage?) -> VNFeaturePrintObservation? {
guard let ciImage = image else {
print("nothing here")
return nil
}
let requestHandler = VNImageRequestHandler(ciImage: ciImage, options: [:])
let request = VNGenerateImageFeaturePrintRequest()
do {
try requestHandler.perform([request])
return request.results?.first as? VNFeaturePrintObservation
} catch {
print("Vision error: \(error)")
return nil
}
}
#IBAction func finishCalculation(_ sender: Any) {
//geting drawing
UIGraphicsBeginImageContextWithOptions(theCanvasView.bounds.size, false, UIScreen.main.scale)
theCanvasView.drawHierarchy(in: theCanvasView.bounds, afterScreenUpdates: true)
let image2 = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
let picture = featureprintObservationForImage(image2?.ciImage)
let drawing = featureprintObservationForImage(image2?.ciImage)
var distance = Float(0)
do {
try picture!.computeDistance(&distance, to: drawing!)
} catch {
print(error)
}
print("picture to drawing")
print(distance)
}
Related
How to make the animation consistent?
func addBlurTo(_ image: UIImage) -> UIImage? {
guard let ciImg = CIImage(image: image) else { return nil }
let blur = CIFilter(name: "CIGaussianBlur")
blur?.setValue(ciImg, forKey: kCIInputImageKey)
blur?.setValue(1 , forKey: kCIInputRadiusKey)
if let outputImg = blur?.outputImage {
return UIImage(ciImage: outputImg)
}
return nil
}
And for Button:
#IBAction func Button7(_ sender: UIButton) {
UIView.transition(with: imageView, duration: 1, options: .transitionCrossDissolve, animations: {
guard let testImage = self.imageView.image else {return}
self.imageView.image = self.addBlurTo(testImage)
}, completion: nil)
}
You need to change your function a bit to the following and it should work. The problem is the conversion from a ciImage to a UIImage without using context. See here:
If the UIImage object was initialized using a CIImage object, the value of the property is NULL.
and here:
Generating qr code with swiftui shows empty picture
func addBlurTo(_ image: UIImage) -> UIImage? {
guard let ciImg = CIImage(image: image) else { return nil }
let blur = CIFilter(name: "CIGaussianBlur")
blur?.setValue(ciImg, forKey: kCIInputImageKey)
blur?.setValue(1 , forKey: kCIInputRadiusKey)
if let outputImg = blur?.outputImage {
let context = CIContext()
guard let cgOutput = context.createCGImage(outputImg, from: outputImg.extent) else { return nil }
return UIImage(cgImage: cgOutput)
}
return nil
}
If I use the image before it is saved it is normal. But if I save it and use it later is is 90 degrees turned. How can I make sure it doesn't save sideways?
func saveEvent(_ center1: CLLocation, title2: String, imagePicked1: UIImage)
{
let data = UIImagePNGRepresentation(imagePicked1);///
let url = NSURL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent(NSUUID().uuidString+".dat")
do {
try data!.write(to: url!, options: [])
} catch let e as NSError {
print("Error! \(e)");
return
}
let image11 = CKAsset(fileURL: url!)
self.eventRecord.setObject(image11 as CKAsset, forKey: "Picture")
let publicData = CKContainer.default().publicCloudDatabase
publicData.save(self.eventRecord, completionHandler: { record, error in
if error == nil
{
print("Image saved")
}else{
print(error!)
}
})
}
If you need to save your PNG with correct rotation you will need to redraw your image if its orientation it is not .up. You can redraw it as follow:
extension UIImage {
func png(isOpaque: Bool = true) -> Data? { flattened(isOpaque: isOpaque)?.pngData() }
func flattened(isOpaque: Bool = true) -> UIImage? {
if imageOrientation == .up { return self }
UIGraphicsBeginImageContextWithOptions(size, isOpaque, scale)
defer { UIGraphicsEndImageContext() }
draw(in: CGRect(origin: .zero, size: size))
return UIGraphicsGetImageFromCurrentImageContext()
}
}
edit/update:
For iOS10+ tvOS10+ you can use UIGraphicsImageRenderer:
extension UIImage {
func png(isOpaque: Bool = true) -> Data? { flattened(isOpaque: isOpaque).pngData() }
func flattened(isOpaque: Bool = true) -> UIImage {
if imageOrientation == .up { return self }
let format = imageRendererFormat
format.opaque = isOpaque
return UIGraphicsImageRenderer(size: size, format: format).image { _ in draw(at: .zero) }
}
}
Playground testing:
Usage for images without transparency:
let image = UIImage(data: try! Data(contentsOf: URL(string: "https://i.stack.imgur.com/varL9.jpg")!))!
if let data = image.png() {
let imageFromPNGData = UIImage(data: data)
}
With transparency :
if let data = image.png(isOpaque: false) {
let imageFromPNGData = UIImage(data: data)
}
Just convert the image to JPEG data instead. No need to redraw your image:
let imageData = image.jpegData(compressionQuality: 1.0)
You can use this as well to prevent it from changing of orientation.
func rotateImage(image: UIImage) -> UIImage? {
if (image.imageOrientation == UIImage.Orientation.up ) {
return image
}
UIGraphicsBeginImageContext(image.size)
image.draw(in: CGRect(origin: CGPoint.zero, size: image.size))
let copy = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return copy
}
The following code does not seem be able to extract depth data from an image (which contains depth information). The code returns nil after the "guard { auxdepthinfo -...)" line. In the following code, the image is a reference to a portrait image stored in the photoAlbum.
struct DepthReader {
var image: UIImage
func depthDataMap() -> CVPixelBuffer? {
//create data from UIImage
guard let imageDat: Data = UIImageJPEGRepresentation(image, 1.0) else {
return nil
}
//create source
guard let source = CGImageSourceCreateWithData(imageDat as CFData, nil) else {
return nil
}
//extract auxData disparity data
guard let auxDataInfo = CGImageSourceCopyAuxiliaryDataInfoAtIndex(source, 0, kCGImageAuxiliaryDataTypeDepth) as? [AnyHashable : Any] else {
return nil
}
// This is the star of the show!
var depthData: AVDepthData
do {
// Get the depth data from the auxiliary data info
depthData = try AVDepthData(fromDictionaryRepresentation: auxDataInfo)
} catch {
return nil
}
// Make sure the depth data is the type we want
if depthData.depthDataType != kCVPixelFormatType_DisparityFloat32 {
depthData = depthData.converting(toDepthDataType: kCVPixelFormatType_DisparityFloat32)
}
return depthData.depthDataMap
}
}
If I use the image before it is saved it is normal. But if I save it and use it later is is 90 degrees turned. How can I make sure it doesn't save sideways?
func saveEvent(_ center1: CLLocation, title2: String, imagePicked1: UIImage)
{
let data = UIImagePNGRepresentation(imagePicked1);///
let url = NSURL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent(NSUUID().uuidString+".dat")
do {
try data!.write(to: url!, options: [])
} catch let e as NSError {
print("Error! \(e)");
return
}
let image11 = CKAsset(fileURL: url!)
self.eventRecord.setObject(image11 as CKAsset, forKey: "Picture")
let publicData = CKContainer.default().publicCloudDatabase
publicData.save(self.eventRecord, completionHandler: { record, error in
if error == nil
{
print("Image saved")
}else{
print(error!)
}
})
}
If you need to save your PNG with correct rotation you will need to redraw your image if its orientation it is not .up. You can redraw it as follow:
extension UIImage {
func png(isOpaque: Bool = true) -> Data? { flattened(isOpaque: isOpaque)?.pngData() }
func flattened(isOpaque: Bool = true) -> UIImage? {
if imageOrientation == .up { return self }
UIGraphicsBeginImageContextWithOptions(size, isOpaque, scale)
defer { UIGraphicsEndImageContext() }
draw(in: CGRect(origin: .zero, size: size))
return UIGraphicsGetImageFromCurrentImageContext()
}
}
edit/update:
For iOS10+ tvOS10+ you can use UIGraphicsImageRenderer:
extension UIImage {
func png(isOpaque: Bool = true) -> Data? { flattened(isOpaque: isOpaque).pngData() }
func flattened(isOpaque: Bool = true) -> UIImage {
if imageOrientation == .up { return self }
let format = imageRendererFormat
format.opaque = isOpaque
return UIGraphicsImageRenderer(size: size, format: format).image { _ in draw(at: .zero) }
}
}
Playground testing:
Usage for images without transparency:
let image = UIImage(data: try! Data(contentsOf: URL(string: "https://i.stack.imgur.com/varL9.jpg")!))!
if let data = image.png() {
let imageFromPNGData = UIImage(data: data)
}
With transparency :
if let data = image.png(isOpaque: false) {
let imageFromPNGData = UIImage(data: data)
}
Just convert the image to JPEG data instead. No need to redraw your image:
let imageData = image.jpegData(compressionQuality: 1.0)
You can use this as well to prevent it from changing of orientation.
func rotateImage(image: UIImage) -> UIImage? {
if (image.imageOrientation == UIImage.Orientation.up ) {
return image
}
UIGraphicsBeginImageContext(image.size)
image.draw(in: CGRect(origin: CGPoint.zero, size: image.size))
let copy = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return copy
}
I'm using CoreML and Vision to analyze a photo taken with the camera or imported from the library. Once the photo is obtained I run some code to make sure the photo is valid and if it is it returns true otherwise it returns false. I use it like so:
func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
if let error = error {
// display alert there is a problem
return
}
guard let imageData = photo.fileDataRepresentation(), let previewImage = UIImage(data: imageData) else {
// display alert there is a problem
return
}
if useVisionAndCoreMLToCheckIfIsImageValid(image: previewImage) {
tableData.append(previewImage)
} else {
// display alert image was not valid
}
}
The problem is there are 4 points inside the useVisionAndCoreMLToCheckIfIsImageValid function that can go wrong and I need to return false so I can jump out of the function and if it is valid there is 1 point where it can go right and I need to return true. But since the function returns a Bool I keep getting errors when trying to return true or false at those points:
How can I get rid of the above errors?
func useVisionAndCoreMLToCheckIfIsImageValid(image: UIImage) -> Bool {
if let cgImage = image.cgImage {
let foodModel = CustomFoodModel()
guard let model = try? VNCoreMLModel(for: foodModel.model) else {
return false
}
let request = VNCoreMLRequest(model: model) { [weak self](request, error) in
if let error = error {
// 1st point - if there is an error return false
return false
}
guard let results = request.results as? [VNClassificationObservation], let topResult = results.first else {
// 2nd point - if there is a nil value here return false
return false
}
if topResult.confidence > 0.8 {
// 3rd point - if confidence is greater then 80% return true
return true
} else {
// 4th point - if confidence is less then 80% return false
return false
}
}
let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])
DispatchQueue.global(qos: .userInitiated).async { [weak self] in
do {
try handler.perform([request])
} catch let err as NSError {
// 5th point - if there is a try error return false
return false
}
}
}
// if the cgImage is nil return false
return false
}
The return statements you get errors for are indeed return statements for the closures e.g. - VNCoreMLRequest(model: model) { - return for this block - },
VNCoreMLRequest(model: model) { -return for this block- } and not for the useVisionAndCoreMLToCheckIfIsImageValid function itself.
You need to rework your strategy and understand the async picture in your case.
I strongly recommend to first get some knowledge on the topic rather than copy-paste a solution you don't truly understand.
Some blog post on the async programming:
https://ashfurrow.com/blog/comparative-asynchronous-programming/
I was initially going to use completionHandlers instead of a bool but I thought there was an easier way which there isn't. #vadian sent me a link in the comments to a similar SO question that basically said what I'm trying to do cannot be done because as DenislavaShentova's answer said the return statements with error are for the blocks and not the function itself.
Here's the useVisionAndCoreMLToCheckIfIsImageValid code signature using 2 completionHandlers instead of returning a Bool
func useVisionAndCoreMLToCheckIfIsImageValid(image: UIImage, falseCompletion: #escaping ()->(), trueCompletion: #escaping ()->()) {
if let cgImage = image.cgImage {
let foodModel = CustomFoodModel()
guard let model = try? VNCoreMLModel(for: foodModel.model) else {
falseCompletion()
return
}
let request = VNCoreMLRequest(model: model) { [weak self](request, error) in
if let error = error {
// 1st point - run code for false
falseCompletion()
return
}
guard let results = request.results as? [VNClassificationObservation], let topResult = results.first else {
// 2nd point - run code for false
falseCompletion()
return
}
if topResult.confidence > 0.8 {
// 3rd point - run code for false
trueCompletion()
} else {
// 4th point - run code for false
falseCompletion()
}
}
let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])
DispatchQueue.global(qos: .userInitiated).async { [weak self] in
do {
try handler.perform([request])
} catch let err as NSError {
// 5th point - run code for false
falseCompletion()
}
}
}
// if the cgImage is nil run code for false
falseCompletion()
}
and here is the function is use:
func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
if let error = error {
// display alert there is a problem
return
}
guard let imageData = photo.fileDataRepresentation(), let previewImage = UIImage(data: imageData) else {
// display alert there is a problem
return
}
useVisionAndCoreMLToCheckIfIsImageValid(image: previewImage, falseCompletion: thereIsAProblem, trueCompletion: imageIsValid)
}
func imageIsValid() {
tableData.append(previewImage)
}
func thereIsAProblem() {
// do something
}