How can I convert from UIImage to HEIF / HEIC Data in Swift? - ios

Is NSKeyedArchiver appropriate to convert UIImage to Data?
do {
let data = try NSKeyedArchiver.archivedData(withRootObject: UIImage(named: somePath), requiringSecureCoding: true)
...
} catch {
print(error)
}
Or is it overkill and using pngData() is more appropriate?
let image = UIImage(named: somePath)
let data = image?.pngData()
and how can I convert from UIImage to HEIF / HEIC Data ?
The goal is to save the image to the device's file system.

No. Never use NSKeyedArchiver to convert your image to Data. Choose an image format (HEIC, PNG, JPEG, etc) and get its data representation. You should only use PNG when saving images to use in your UI. Most of the time jpeg is the preferred choice. If the device supports HEIC it is an option considering the image quality and reduced data size.
If you need to check if the user device supports HEIC type you can do it as follow:
var isHeicSupported: Bool {
(CGImageDestinationCopyTypeIdentifiers() as! [String]).contains("public.heic")
}
If you need to convert your image to HEIC you need to get a CGImage from your UIImage and convert your UIImage's imageOrientation to CGImagePropertyOrientation to preserve the orientation when creating its data representation:
extension UIImage {
var heic: Data? { heic() }
func heic(compressionQuality: CGFloat = 1) -> Data? {
guard
let mutableData = CFDataCreateMutable(nil, 0),
let destination = CGImageDestinationCreateWithData(mutableData, "public.heic" as CFString, 1, nil),
let cgImage = cgImage
else { return nil }
CGImageDestinationAddImage(destination, cgImage, [kCGImageDestinationLossyCompressionQuality: compressionQuality, kCGImagePropertyOrientation: cgImageOrientation.rawValue] as CFDictionary)
guard CGImageDestinationFinalize(destination) else { return nil }
return mutableData as Data
}
}
extension CGImagePropertyOrientation {
init(_ uiOrientation: UIImage.Orientation) {
switch uiOrientation {
case .up: self = .up
case .upMirrored: self = .upMirrored
case .down: self = .down
case .downMirrored: self = .downMirrored
case .left: self = .left
case .leftMirrored: self = .leftMirrored
case .right: self = .right
case .rightMirrored: self = .rightMirrored
#unknown default:
fatalError()
}
}
}
extension UIImage {
var cgImageOrientation: CGImagePropertyOrientation { .init(imageOrientation) }
}
Usage for lossless compression:
if isHeicSupported, let heicData = image.heic {
// write your heic image data to disk
}
or adding compression to your image:
if isHeicSupported, let heicData = image.heic(compressionQuality: 0.75) {
// write your compressed heic image data to disk
}

Related

UIImage is rotated 90 degrees when creating from url and set to the pasteboard

What do I simply do?
let pasteboard = UIPasteboard.general
let base64EncodedImageString = "here_base_64_string_image"
let data = Data(base64Encoded: base64EncodedImageString)
let url = data?.write(withName: "image.jpeg")
pasteboard.image = UIImage(url: url) //and now when I try to paste somewhere that image for example in imessage, it is rotated... why?
What may be important:
It happens only for images created by camera.
However, if use exactly the same process (!) to create activityItems for UIActivityViewController and try to use iMessage app, then it works... why? What makes the difference?
I use above two simple extensions for UIImage and Data:
extension Data {
func write(withName name: String) -> URL {
let url = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent(name)
do {
try write(to: url, options: NSData.WritingOptions.atomic)
return url
} catch {
return url
}
}
}
extension UIImage {
convenience init?(url: URL?) {
guard let url = url else {
return nil
}
do {
self.init(data: try Data(contentsOf: url))
} catch {
return nil
}
}
}
Before server returns base64EncodedString I upload an image from camera like this:
func imagePickerController(
_ picker: UIImagePickerController,
didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey: Any]
) {
let image = info[.originalImage] as? UIImage
let encodedBase64 = image?.jpegData(compressionQuality: 0.9)?.base64EncodedString() ?? ""
//upload encodedBase64 to the server... that is all
}
I am not sure but I think UIPasteBoard converts your image to PNG and discards its orientation. You can explicitly tell the kind of data you are adding to the pasteboard but I am not sure if this would work for your scenery.
extension Data {
var image: UIImage? { UIImage(data: self) }
}
setting your pasteboard data
UIPasteboard.general.setData(jpegData, forPasteboardType: "public.jpeg")
loading the data from pasteboard
if let pbImage = UIPasteboard.general.data(forPasteboardType: "public.jpeg")?.image {
}
Or Redrawing your image before setting your pasteboard image property
extension UIImage {
func flattened(isOpaque: Bool = true) -> UIImage? {
if imageOrientation == .up { return self }
UIGraphicsBeginImageContextWithOptions(size, isOpaque, scale)
defer { UIGraphicsEndImageContext() }
draw(in: CGRect(origin: .zero, size: size))
return UIGraphicsGetImageFromCurrentImageContext()
}
}
UIPasteboard.general.image = image.flattened()

Rotate image in share extension

I have this extension and it works perfect in app target but crash in share extension when trying to rotate image captured on camera. How to rotate image in share extension? Or maybe it possible to load image from Photo Library already in right orientation.
extension UIImage {
func fixOrientation() -> UIImage {
switch imageOrientation {
case .up:
return self
default:
UIGraphicsBeginImageContextWithOptions(size, false, scale)
draw(in: CGRect(origin: .zero, size: size)) //Thread 1: EXC_RESOURCE RESOURCE_TYPE_MEMORY (limit=120 MB, unused=0x0)
let result = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return result!
}
}
}
Crash screenshots:
First of all it is clear that you have a memory crash. According to App Extension Programming Guide:
Memory limits for running app extensions are significantly lower than the memory limits imposed on a foreground app. On both platforms, the system may aggressively terminate extensions because users want to return to their main goal in the host app.
And from error it is clear that you exceed 120 mb. But you might wonder what is took so much memory.
According to Optimizing Images
Written by Jordan Morgan:
iOS essentially derives its memory hit from an image’s dimensions - whereas the actual file size has much less to do with it.
So if we calculate size or 4032 x 3024 photo it will be... 46mb for 4 bit color and 79mb for 8 bit color. Pretty big, but still less that a limit...
Thing is - you have two copies of your image. One is original and second one - rotated.
To solve this issue you need load only rotated image into memory, without original. This can be done with Image I/O Framework:
extension UIImage {
static func imageWithFixedOrientation(at url: URL) -> UIImage? {
guard let imageSource = CGImageSourceCreateWithURL(url as CFURL, nil) else { return nil }
guard let imageProperties = CGImageSourceCopyPropertiesAtIndex(imageSource, 0, nil) as? Dictionary<CFString, Any> else { return nil }
guard
let width = imageProperties[kCGImagePropertyPixelWidth] as? CGFloat,
let height = imageProperties[kCGImagePropertyPixelHeight] as? CGFloat
else { return nil }
let options: [NSString: Any] = [
kCGImageSourceThumbnailMaxPixelSize: max(width, height),
kCGImageSourceCreateThumbnailFromImageAlways: true,
kCGImageSourceCreateThumbnailWithTransform: true
]
guard let cgImage = CGImageSourceCreateThumbnailAtIndex(imageSource, 0, options as CFDictionary) else { return nil }
return UIImage(cgImage: cgImage)
}
}
In sample app:
extension ViewController: UIImagePickerControllerDelegate & UINavigationControllerDelegate {
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey : Any]) {
picker.dismiss(animated: true)
guard let url = info[.imageURL] as? URL else { return }
let image = UIImage.imageWithFixedOrientation(at: url)
}
}
it reduced memory peaks from 180+mb to just 80mb.
You may try to use more optimized UIGraphicsImageRendererFormat instead of using old UIGraphicsBeginImageContextWithOptions, you can find more information in this post. So your code will look something like below:
func fixOrientation() -> UIImage {
let format = UIGraphicsImageRendererFormat()
format.scale = scale
format.prefersExtendedRange = true
let renderer = UIGraphicsImageRenderer(size: size, format: format)
let image = renderer.image(actions: { context in
var workSize = size;
workSize.width = floor(workSize.width / scale)
workSize.height = floor(workSize.height / scale)
// if the orientation is already correct
// if image.imageOrientation == .up { draw image }
var transform = CGAffineTransform.identity
//TO-DO - set transform depends on current image orientation
//transform =
let ctx = context.cgContext
ctx.concatenate(transform)
guard let cgImageCopy = cgImage else {
return
}
switch imageOrientation {
case .left, .leftMirrored, .right, .rightMirrored:
ctx.draw(cgImageCopy, in: CGRect(x: 0.0, y:0.0, width: workSize.height, height: workSize.width))
break;
default:
ctx.draw(cgImageCopy, in: CGRect(origin: .zero, size: workSize))
break;
}
})
return image
}
Did you try using the function you provided within an didFinishPicking delegate method?
func photoLibrary() {
if UIImagePickerController.isSourceTypeAvailable(.photoLibrary) {
let myPickerController = UIImagePickerController()
myPickerController.delegate = self
myPickerController.sourceType = .photoLibrary
self.present(myPickerController, animated: true, completion: nil)
}
}
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey : Any]) {
picker.dismiss(animated: true)
guard let image = info[.originalImage] as? UIImage else {
print("No image found")
return
}
// Flip the image here
self.mainImageView.image = image.fixOrientation()
}

Firebase MLKit text recognition fail with the front camera iOS

I have integrated the Firebase MLKit SDK as per the document, texts are detected correctly with the back camera photo. When I am using a captured photo from the front camera, texts are detecting wrongly and bad results are coming out.
#IBAction func findTextDidTouch(_ sender: UIButton) {
runTextRecognition(with:imageView.image!)
}
func runCloudTextRecognition(with image: UIImage) {
let visionImage = VisionImage(image: image)
cloudTextDetector.detect(in: visionImage) { features, error in
if let error = error {
print("Received error: \(error)")
return
}
self.processCloudResult(from: features, error: error)
}
}
for text in features {
if let block = text as? VisionTextBlock {
print(block.text)
}
}
I am not able to figure out. Do I need change with the camera or firebase ml kit?
You need to provide image orientation as well. Adding code snippet may help you.
let visionImage = VisionImage(image: image)
let metadata = VisionImageMetadata()
metadata.orientation = self.detectorOrientation(in: image)
visionImage.metadata = metadata
textDetector.process(visionImage) { (features, error) in
if features != nil
{
self.processResult(from: [features!], error: error)
}
}
//Detects orientation of the selected or captured image
func detectorOrientation(in image: UIImage) -> VisionDetectorImageOrientation {
switch image.imageOrientation {
case .up:
return .topLeft
case .down:
return .bottomRight
case .left:
return .leftBottom
case .right:
return .rightTop
case .upMirrored:
return .topRight
case .downMirrored:
return .bottomLeft
case .leftMirrored:
return .leftTop
case .rightMirrored:
return .rightBottom
}
}

Not detecting QR Code from a static image

I want to detect qrcode from still image
Here is the code
let ciimage = <Load image from asset and covert to CIImage>
let detector = CIDetector(ofType: CIDetectorTypeQRCode, context:nil, options:nil)
let features = detector.featuresInImage(ciimage)
print(features.count)
The image is correct CIImage instance which has QRCode but I always get no features, Have any one had the same issue?
Thank you very much.
I think I found answer
That code only works in Playground or real iPhone, it never works on simulator
I generated QRCode image using CIQRCodeGenerator CoreImage filter.
I hope someone can check my answer here and give me advice.
Below is source code what I used
enum QRCodeUtilError:ErrorType{
case GenerationFailed
}
extension UIImage {
/**
This function trys to return CIImage even the image is CGImageRef based.
- Returns: CIImage equal to current image
*/
func ciImage() -> CoreImage.CIImage?{
// if CIImage is not nil then return it
if self.CIImage != nil {
return self.CIImage
}
guard let cgImage = self.CGImage else {
return nil
}
return CoreImage.CIImage(CGImage: cgImage)
}
}
/**
QRCode utility class
*/
class QRCodeUtil{
/**
Generates 'H' mode qrcode image
- Parameter qrCode: QRCode String
- Returns: Optional UIImage instance of QRCode
*/
class func imageForQrCode(qrCode:String) throws -> UIImage{
guard let filter = CIFilter(name: "CIQRCodeGenerator") else {
throw QRCodeUtilError.GenerationFailed
}
guard let data = qrCode.dataUsingEncoding(NSISOLatin1StringEncoding) else {
throw QRCodeUtilError.GenerationFailed
}
filter.setValue(data, forKey: "inputMessage")
filter.setValue("H", forKey: "inputCorrectionLevel")
guard let outputImage = filter.outputImage else {
throw QRCodeUtilError.GenerationFailed
}
/*
Convert the image to CGImage based because
it crashes when saving image using
UIImagePNGRepresentation or UIImageJPEGRepresentation
if image is CIImage based.
*/
let context = CIContext(options: nil)
let cgImage = context.createCGImage(outputImage, fromRect: outputImage.extent)
return UIImage(CGImage: cgImage)
}
/**
Detects QRCode from string
- Parameter qrImage: UIImage which has QRCode
- Returns: QRCode detected
*/
class func qrCodeFromImage(qrImage:UIImage) -> String {
let detector = CIDetector(ofType: CIDetectorTypeQRCode, context: nil, options: [CIDetectorAccuracy:CIDetectorAccuracyHigh])
guard let ciImage = qrImage.ciImage() else {
return ""
}
guard let feature = detector.featuresInImage(ciImage).last as? CIQRCodeFeature else {
return ""
}
return feature.messageString
}
}

Convert UIImage to NSData and convert back to UIImage in Swift?

I'm trying to save a UIImage to NSData and then read the NSData back to a new UIImage in Swift. To convert the UIImage to NSData I'm using the following code:
let imageData: NSData = UIImagePNGRepresentation(myImage)
How do I convert imageData (i.e., NSData) back to a new UIImage?
UIImage(data:imageData,scale:1.0) presuming the image's scale is 1.
In swift 4.2, use below code for get Data().
image.pngData()
Thanks. Helped me a lot. Converted to Swift 3 and worked
To save: let data = UIImagePNGRepresentation(image)
To load: let image = UIImage(data: data)
Use imageWithData: method, which gets translated to Swift as UIImage(data:)
let image : UIImage = UIImage(data: imageData)
Now in Swift 4.2 you can use pngData() new instance method of UIImage to get the data from the image
let profileImage = UIImage(named:"profile")!
let imageData = profileImage.pngData()
Details
Xcode 10.2.1 (10E1001), Swift 5
Solution 1
guard let image = UIImage(named: "img") else { return }
let jpegData = image.jpegData(compressionQuality: 1.0)
let pngData = image.pngData()
Solution 2.1
extension UIImage {
func toData (options: NSDictionary, type: CFString) -> Data? {
guard let cgImage = cgImage else { return nil }
return autoreleasepool { () -> Data? in
let data = NSMutableData()
guard let imageDestination = CGImageDestinationCreateWithData(data as CFMutableData, type, 1, nil) else { return nil }
CGImageDestinationAddImage(imageDestination, cgImage, options)
CGImageDestinationFinalize(imageDestination)
return data as Data
}
}
}
Usage of solution 2.1
// about properties: https://developer.apple.com/documentation/imageio/1464962-cgimagedestinationaddimage
let options: NSDictionary = [
kCGImagePropertyOrientation: 6,
kCGImagePropertyHasAlpha: true,
kCGImageDestinationLossyCompressionQuality: 0.5
]
// https://developer.apple.com/documentation/mobilecoreservices/uttype/uti_image_content_types
guard let data = image.toData(options: options, type: kUTTypeJPEG) else { return }
let size = CGFloat(data.count)/1000.0/1024.0
print("\(size) mb")
Solution 2.2
extension UIImage {
func toJpegData (compressionQuality: CGFloat, hasAlpha: Bool = true, orientation: Int = 6) -> Data? {
guard cgImage != nil else { return nil }
let options: NSDictionary = [
kCGImagePropertyOrientation: orientation,
kCGImagePropertyHasAlpha: hasAlpha,
kCGImageDestinationLossyCompressionQuality: compressionQuality
]
return toData(options: options, type: .jpeg)
}
func toData (options: NSDictionary, type: ImageType) -> Data? {
guard cgImage != nil else { return nil }
return toData(options: options, type: type.value)
}
// about properties: https://developer.apple.com/documentation/imageio/1464962-cgimagedestinationaddimage
func toData (options: NSDictionary, type: CFString) -> Data? {
guard let cgImage = cgImage else { return nil }
return autoreleasepool { () -> Data? in
let data = NSMutableData()
guard let imageDestination = CGImageDestinationCreateWithData(data as CFMutableData, type, 1, nil) else { return nil }
CGImageDestinationAddImage(imageDestination, cgImage, options)
CGImageDestinationFinalize(imageDestination)
return data as Data
}
}
// https://developer.apple.com/documentation/mobilecoreservices/uttype/uti_image_content_types
enum ImageType {
case image // abstract image data
case jpeg // JPEG image
case jpeg2000 // JPEG-2000 image
case tiff // TIFF image
case pict // Quickdraw PICT format
case gif // GIF image
case png // PNG image
case quickTimeImage // QuickTime image format (OSType 'qtif')
case appleICNS // Apple icon data
case bmp // Windows bitmap
case ico // Windows icon data
case rawImage // base type for raw image data (.raw)
case scalableVectorGraphics // SVG image
case livePhoto // Live Photo
var value: CFString {
switch self {
case .image: return kUTTypeImage
case .jpeg: return kUTTypeJPEG
case .jpeg2000: return kUTTypeJPEG2000
case .tiff: return kUTTypeTIFF
case .pict: return kUTTypePICT
case .gif: return kUTTypeGIF
case .png: return kUTTypePNG
case .quickTimeImage: return kUTTypeQuickTimeImage
case .appleICNS: return kUTTypeAppleICNS
case .bmp: return kUTTypeBMP
case .ico: return kUTTypeICO
case .rawImage: return kUTTypeRawImage
case .scalableVectorGraphics: return kUTTypeScalableVectorGraphics
case .livePhoto: return kUTTypeLivePhoto
}
}
}
}
Usage of solution 2.2
let compressionQuality: CGFloat = 0.4
guard let data = image.toJpegData(compressionQuality: compressionQuality) else { return }
printSize(of: data)
let options: NSDictionary = [
kCGImagePropertyHasAlpha: true,
kCGImageDestinationLossyCompressionQuality: compressionQuality
]
guard let data2 = image.toData(options: options, type: .png) else { return }
printSize(of: data2)
Problems
Image representing will take a lot of cpu and memory resources. So, in this case it is better to follow several rules:
- do not run jpegData(compressionQuality:) on main queue
- run only one jpegData(compressionQuality:) simultaneously
Wrong:
for i in 0...50 {
DispatchQueue.global(qos: .utility).async {
let quality = 0.02 * CGFloat(i)
//let data = image.toJpegData(compressionQuality: quality)
let data = image.jpegData(compressionQuality: quality)
let size = CGFloat(data!.count)/1000.0/1024.0
print("\(i), quality: \(quality), \(size.rounded()) mb")
}
}
Right:
let serialQueue = DispatchQueue(label: "queue", qos: .utility, attributes: [], autoreleaseFrequency: .workItem, target: nil)
for i in 0...50 {
serialQueue.async {
let quality = 0.02 * CGFloat(i)
//let data = image.toJpegData(compressionQuality: quality)
let data = image.jpegData(compressionQuality: quality)
let size = CGFloat(data!.count)/1000.0/1024.0
print("\(i), quality: \(quality), \(size.rounded()) mb")
}
}
Links
UTI Image Content Types
CGImageDestinationAddImage(::_:)
Thinking about Memory: Converting UIImage to Data in Swift
Different resize technics
To save as data:
From StoryBoard, if you want to save "image" data on the imageView of MainStoryBoard, following codes will work.
let image = UIImagePNGRepresentation(imageView.image!) as NSData?
To load "image" to imageView:
Look at exclamation point "!", "?" closely whether that is quite same as this one.
imageView.image = UIImage(data: image as! Data)
"NSData" type is converted into "Data" type automatically during this process.
Image to Data:-
if let img = UIImage(named: "xxx.png") {
let pngdata = img.pngData()
}
if let img = UIImage(named: "xxx.jpeg") {
let jpegdata = img.jpegData(compressionQuality: 1)
}
Data to Image:-
guard let image = UIImage(data: pngData) else { return }
For safe execution of code, use if-let block with Data to prevent app crash & , as function UIImagePNGRepresentation returns an optional value.
if let img = UIImage(named: "TestImage.png") {
if let data:Data = UIImagePNGRepresentation(img) {
// Handle operations with data here...
}
}
Note: Data is Swift 3+ class. Use Data instead of NSData with
Swift 3+
Generic image operations (like png & jpg both):
if let img = UIImage(named: "TestImage.png") { //UIImage(named: "TestImage.jpg")
if let data:Data = UIImagePNGRepresentation(img) {
handleOperationWithData(data: data)
} else if let data:Data = UIImageJPEGRepresentation(img, 1.0) {
handleOperationWithData(data: data)
}
}
*******
func handleOperationWithData(data: Data) {
// Handle operations with data here...
if let image = UIImage(data: data) {
// Use image...
}
}
By using extension:
extension UIImage {
var pngRepresentationData: Data? {
return UIImagePNGRepresentation(self)
}
var jpegRepresentationData: Data? {
return UIImageJPEGRepresentation(self, 1.0)
}
}
*******
if let img = UIImage(named: "TestImage.png") { //UIImage(named: "TestImage.jpg")
if let data = img.pngRepresentationData {
handleOperationWithData(data: data)
} else if let data = img.jpegRepresentationData {
handleOperationWithData(data: data)
}
}
*******
func handleOperationWithData(data: Data) {
// Handle operations with data here...
if let image = UIImage(data: data) {
// Use image...
}
}
Swift 5
let the image you create as UIImage be image
image.pngData() as NSData?
Use this for a simple solution
static var UserProfilePhoto = UIImage()
guard let image = UIImage(named: "Photo") else { return }
guard let pngdata = image.pngData() else { return }
UserProfilePhoto = UIImage(data: pngdata)!

Resources