When fetching images and video with Photo Kit my images and video thumbnails are displaying in low res within my LazyVGrid.
They also display in lo-res when I display the image inside a larger Image view.
All of my images and video are stored on iCloud so I set isNetworkAccessAllowed = true AND deliveryMode = .highQualityFormat in PHImageRequestOptions().
I have tried changing the targetSize in the requestImage method to different values but that didn't change the quality either.
What am I doing wrong?
#Published var fetchedMediaArray : [MediaAsset] = []
// model for Assets
struct MediaAsset: Identifiable, Hashable {
var id = UUID()
var image : UIImage
var selected: Bool
var asset: PHAsset
}
func requestAuth() {/*auth code*/ }
func fetchPhotosAndVideos() {
let opt = PHFetchOptions()
opt.includeHiddenAssets = false
opt.sortDescriptors = [NSSortDescriptor(key: "creationDate", ascending: false)]
opt.predicate = NSPredicate(format: "mediaType == %d || mediaType == %d",
PHAssetMediaType.image.rawValue,
PHAssetMediaType.video.rawValue)
let req = PHAsset.fetchAssets(with: opt)
DispatchQueue.main.async {
let options = PHImageRequestOptions()
options.isSynchronous = true
options.deliveryMode = .highQualityFormat
options.isNetworkAccessAllowed = true
for j in 0..<req.count {
PHCachingImageManager.default().requestImage(for: req[j], targetSize: CGSize(width: 100, height: 100), contentMode: .default, options: options) { (image, _) in
let data1 = MediaAsset(image: image!, selected: false, asset: req[j])
self.fetchedMediaArray.append(data1)
}
}
}
}
When Photos are being downloaded from cloud, it may call your completion handler multiple times.
From the PHImageManager.requestImage(for:asset) docs -
Discussion
For an asynchronous request, Photos may call your result handler block more than once. Photos first calls the block to provide a low-quality image suitable for displaying temporarily while it prepares a high-quality image. (If low-quality image data is immediately available, the first call may occur before the method returns.) When the high-quality image is ready, Photos calls your result handler again to provide it. If the image manager has already cached the requested image at full quality, Photos calls your result handler only once. The PHImageResultIsDegradedKey key in the result handler’s info parameter indicates when Photos is providing a temporary low-quality image.
What you can do in your callback handler -
If your asset is being returned in target/requested size(or bigger than requested), you can consider it done.
If your asset is being returned in smaller than target/requested size, you should check for PHImageResultIsDegradedKey in the info parameter and wait for Photos to call your completion next time.
Related
I am trying to section user's Photo Library pictures in a Collection View based on their creation date in ascending order. I am using this method which obviously is painfully slow especially when the number of pictures is high.
First I fetch the PHAssets in sorted order:
let allPhotosOptions = PHFetchOptions()
allPhotosOptions.sortDescriptors = [NSSortDescriptor(key: "creationDate", ascending: true)]
inFetchResult = PHAsset.fetchAssets(with: allPhotosOptions)
and then I copy the assets inside an array:
inFetchResult.enumerateObjects { (asset, index, stop) in
let yearComponent = Calendar.current.component(.year, from: asset.creationDate!)
let monthComponent = Calendar.current.component(.month, from: asset.creationDate!)
let monthName = DateFormatter().monthSymbols[monthComponent - 1]
var itemFound = false
for (index, _) in self.dateArray.enumerated() {
if self.dateArray[index].date == "\(monthName) \(yearComponent)" {
self.dateArray[index].assets.append(asset)
itemFound = true
break
} else {
continue
}
}
if !itemFound {
self.dateArray.append((date: "\(monthName) \(yearComponent)", assets: [asset]))
}
}
I then use this array as my data source.
Is there a better way to do this? I have tried dictionaries but they change the order of the objects. I also have considered finding a way to only add the assets to my dateArray when they are going to be displayed on view, however, collection view needs to know the total number of sections up front so I must go through all of the pictures and check their dates before loading the view.
You can fetch photo assets and do all the sorting logic in background
thread like in the below code, once you will done with the heavy
processing then you can access main Thread to update the UI.
let allPhotosOptions = PHFetchOptions()
allPhotosOptions.sortDescriptors = [NSSortDescriptor(key: "creationDate", ascending: true)]
DispatchQueue.global(qos: .userInitiated).async {
let photos = PHAsset.fetchAssets(with: .image, options: nil)
var result = [Any]()
photos.enumerateObjects({ asset, _, _ in
// do the fetched photos logic in background thread
})
DispatchQueue.main.async {
// once you will get all the data, you can do UI related stuff here like
// reloading data, assigning data to UI elements etc.
}
}
After reading a number of answers on SO and other articles (see below) what is the best method to manage memory when you are loading multiple images into an animation array?
http://www.alexcurylo.com/2009/01/13/imagenamed-is-evil/
Here are my goals:
Create Animation Array (see code below)
After the Animation plays flush the cache and load another animation (rinse, wash, repeat, you get it :)
As Apple states
https://forums.developer.apple.com/thread/17888
Using the UIImage(named: imageName) caches the images, but in my case after playing 2-3 animations in a row iOS terminates the App (the OS does not respond to a low memory warning and instead terminates the App - see end of code below)
I don't want to cache the images and rather we could either:
Flush the memory each time an animation completes and then load a new animation; or
Flush the memory when the user moves to a new Scene
Here is my code:
// create the Animation Array for each animation
func createImageArray(total: Int, imagePrefix: String) -> [UIImage]{
var imageArray: [UIImage] = []
for imageCount in 0..<total {
let imageName = "\(imagePrefix)-\(imageCount).png"
let image = UIImage(named: imageName)! // here is where we need to address memory and not cache the images
//let image = UIImage(contentsOfFile: imageName)! // maybe this?
imageArray.append(image)
}
return imageArray
}
// here we set the animate function values
func animate(imageView: UIImageView, images: [UIImage]){
imageView.animationImages = images
imageView.animationDuration = 1.5
imageView.animationRepeatCount = 1
imageView.startAnimating()
}
// Here we call the animate function
animation1 = createImageArray(total: 28, imagePrefix: "ImageSet1")
animation2 = createImageArray(total: 53, imagePrefix: "ImageSet2")
animation3 = createImageArray(total: 25, imagePrefix: "ImageSet3")
func showAnimation() {
UIView.animate(withDuration: 1, animations: {
animate(imageView: self.animationView, images: self.animation1)
}, completion: { (true) in
//self.animationView.image = nil // Maybe we try the following?
//self.animationView.removeFromSuperview()
//self.animationView = nil
})
}
Based on SO responses, it looks like this may be the best method to prevent the images from being cached, but it doesn't seem to work in my code:
let image = UIImage(contentsOfFile: imageName)!
I have also tried this but it doesn't seem to work either:
func applicationDidReceiveMemoryWarning(application: UIApplication) {
NSURLCache.sharedURLCache().removeAllCachedResponses()
}
I also tried the following article (removeFromSuperview) in the completion block but I couldn't get this to work either (see my code above):
https://www.hackingwithswift.com/example-code/uikit/how-to-animate-views-using-animatewithduration
New Code:
// create the Animation Array for each animation
func createImageArray(total: Int, imagePrefix: String) -> [UIImage]{
var imageArray: [UIImage] = []
for imageCount in 0..<total {
let imageName = "\(imagePrefix)-\(imageCount).png"
//let image = UIImage(named: imageName)! // replaced with your code below
if let imagePath = Bundle.mainBundle.path(forResource: "ImageSet1" ofType: "png"),
let image = UIImage(contentsOfFile: imagePath) {
//Your image has been loaded }
imageArray.append(image)
}
return imageArray }
It's pretty simple. UIImage(named:) caches images, and UIImage(contentsOfFile:) does not.
If you don't want your images to be cached, use UIImage(contentsOfFile:) instead. If you can't get that to work then post your code and we'll help you debug it.
Be aware that UIImage(contentsOfFile:) does not look for files in your app bundle. It expects a full path to the image file. You will need to use Bundle methods to find the path to the file and then pass that path to UIImage(contentsOfFile:):
if let imagePath = Bundle.mainBundle.path(forResource: "ImageSet1" ofType: "png"),
let image = UIImage(contentsOfFile: imagePath) {
//Your image has been loaded
}
Your code is loading all the images for all 3 animations into an array of images and never releasing them, so the fact that the system caches those images seems pretty irrelevant. In a low memory condition the system should flush the cached copies of the images, but your code will still hold all those images in your arrays so the memory won't get freed. It looks to me like it's your code that's causing the memory problem, not the system's images caching.
Your code might look like this:
func createImageArray(total: Int, imagePrefix: String) -> [UIImage]{
var imageArray: [UIImage] = []
for imageCount in 0..<total {
let imageName = "\(imagePrefix)-\(imageCount)"
if let imagePath = Bundle.main.path(forResource: imageName,
ofType: "png"),
let image = UIImage(contentsOfFile: imagePath) {
imageArray.append(image)
}
}
return imageArray
}
I am attempting to build a video merging app that allows users to select several short clips from a collection view and then generates a preview of the videos all merged into one. I am using the Photos framework (PHCachingImageManager) to populate the collection view and am passing an array of the selected PHAssets to the function below in order to request low quality AVAssets (for merging & generating the preview).
The problem is, I need to keep the AVAssets in the order in which the user selected them, but the "requestAVAsset" function is asynchronous and the completion handler is often called multiple times. I've never used Dispatch Groups before, but attempted to use them below...and the AVAssets are still out of order sometimes.
func requestAVAssets(assets: [PHAsset]) -> [AVAsset] {
var videoArray: [AVAsset] = []
let dispatchGroup = DispatchGroup()
let videoOptions = PHVideoRequestOptions()
videoOptions.isNetworkAccessAllowed = true
videoOptions.deliveryMode = .fastFormat
for asset in assets {
dispatchGroup.enter()
self.imageManager.requestAVAsset(forVideo: asset, options: videoOptions, resultHandler: { (video, audioMix, info) in
guard video != nil else { return }
videoArray.append(video!)
dispatchGroup.leave()
})
}
dispatchGroup.wait()
return videoArray
}
I'm guessing I've either misplaced some code or am approaching this in entirely the wrong way! Any suggestions are appreciated.
If you capture the current index while you're iterating the AVAssets, you can insert rather than append. That's how I do it, at least.
func requestAVAssets(assets: [PHAsset]) -> [AVAsset] {
var videoArray = [AVAsset?](repeating: nil, count: assets.count)
let videoOptions = PHVideoRequestOptions()
videoOptions.isNetworkAccessAllowed = true
videoOptions.deliveryMode = .fastFormat
for (i, asset) in assets.enumerated() {
self.imageManager.requestAVAsset(forVideo: asset, options: videoOptions, resultHandler: { (video, audioMix, info) in
guard let video = video else { return }
videoArray.remove(at: i)
videoArray.insert(video, at: i)
})
}
return videoArray.flatMap { $0 }
}
Giving the array the desired number of items as nil will stop it from erroring when inserting items, and then when the download is complete, remove the existing nil value and replace it with the actual AVAsset.
Finally, flatMap the resulting array to unpack the optionals (and optionally check that you have the desired number of items by comparing it with the incoming assets array).
Dodging the dispatch question entirely because it's late and I've had a bad day, but what if you kept the "correct" index associated with the video, and then sorted on that? I think something like this would work.
struct SelectedVideo {
let index: Int
let asset: AVAsset
}
func requestAVAssets(assets: [PHAsset]) -> [AVAsset] {
var videoArray: [SelectedVideo] = []
let dispatchGroup = DispatchGroup()
let videoOptions = PHVideoRequestOptions()
videoOptions.isNetworkAccessAllowed = true
videoOptions.deliveryMode = .fastFormat
for (index, asset) in assets.enumerated() {
dispatchGroup.enter()
self.imageManager.requestAVAsset(forVideo: asset, options: videoOptions, resultHandler: { (videoMb, audioMixMb, infoMb) in
guard let video = videoMb else return
videoArray.append(SelectedVideo(index, video))
dispatchGroup.leave()
})
}
dispatchGroup.wait()
return videoArray.sort { $0.index < $1.index}.map({$0.video})
}
This is kind of a hack (haven't even tried compiling it), but like I said, it's been a bad day.
A couple minor changes to note: I changed the params to the closure to say "Mb" which means "maybe" and is a nice convention I've seen for naming optionals passed to closures. Also, instead of "guard video != nil" followed by force-unwrapping, it's much preferred to do a "guard let video = videoMb", and then video is non-optional.
For a small project, I'm making an iOS app which should do two things:
take a picture
take a hash from the picture data (and print it to the xcode console)
Then, I want to export the picture to my laptop and confirm the hash. I tried exporting via AirDrop, Photos.app, email and iCloud (Photos.app compresses the photo and iCloud transforms it into an .png).
Problem is, I can't repodruce the hash. This means that the exported picture differs from the picture in the app. There are some variables I tried to rule out one by one. To get NSData from a picture, one can use the UIImagePNGRepresentation and UIImageJPEGRepresentation functions, forcing the image in a format representation before extracting the data. To be honest, I'm not completely sure what these functions do (other than transforming to NSData), but they do something different from the other because they give a different result compared to each other and compared to the exported data (which is .jpg).
There are some things unclear to me what Swift/Apple is doing to my (picture)data upon exporting. I read in several places that Apple transforms (or deletes) the EXIF but to me it is unclear what part. I tried to anticipate this by explicitly removing the EXIF data myself before hashing in both the app (via function ImageHelper.removeExifData (found here) and via exiftools on the command line), but to no avail.
I tried hashing an existing photo on my phone. I had a photo send to me by mail but hashing this in my app and on the command line gave different results. A string gave similar results in the app and on command line so the hash function(s) are not the problem.
So my questions are:
Is there a way to prevent transformation when exporting a photo
Are there alternatives to UIImagePNGRepresentation / UIImageJPEGRepresentation functions
(3. Is this at all possible or is iOS/Apple too much of a black box?)
Any help or pointers to more documentation is greatly appreciated!
Here is my code
//
// ViewController.swift
// camera test
import UIKit
import ImageIO
// extension on NSData format, to enable conversion to String type
extension NSData {
func toHexString() -> String {
var hexString: String = ""
let dataBytes = UnsafePointer<CUnsignedChar>(self.bytes)
for (var i: Int=0; i<self.length; ++i) {
hexString += String(format: "%02X", dataBytes[i])
}
return hexString
}
}
// function to remove EXIF data from image
class ImageHelper {
static func removeExifData(data: NSData) -> NSData? {
guard let source = CGImageSourceCreateWithData(data, nil) else {
return nil
}
guard let type = CGImageSourceGetType(source) else {
return nil
}
let count = CGImageSourceGetCount(source)
let mutableData = NSMutableData(data: data)
guard let destination = CGImageDestinationCreateWithData(mutableData, type, count, nil) else {
return nil
}
// Check the keys for what you need to remove
// As per documentation, if you need a key removed, assign it kCFNull
let removeExifProperties: CFDictionary = [String(kCGImagePropertyExifDictionary) : kCFNull, String(CGImagePropertyOrientation): kCFNull]
for i in 0..<count {
CGImageDestinationAddImageFromSource(destination, source, i, removeExifProperties)
}
guard CGImageDestinationFinalize(destination) else {
return nil
}
return mutableData;
}
}
class ViewController: UIViewController, UINavigationControllerDelegate, UIImagePickerControllerDelegate, MFMailComposeViewControllerDelegate {
#IBOutlet weak var imageView: UIImageView!
// creats var for picture
var imagePicker: UIImagePickerController!
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view, typically from a nib.
}
override func didReceiveMemoryWarning() {
super.didReceiveMemoryWarning()
// Dispose of any resources that can be recreated.
}
// calls Camera function and outputs picture to imagePicker
#IBAction func cameraAction(sender: UIButton) {
imagePicker = UIImagePickerController()
imagePicker.delegate = self
imagePicker.sourceType = .Camera
presentViewController(imagePicker, animated: true, completion: nil)
}
// calls camera app, based on cameraAction
func imagePickerController(picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : AnyObject]) {
imagePicker.dismissViewControllerAnimated(true, completion: nil)
imageView.image = info[UIImagePickerControllerOriginalImage] as? UIImage
}
// calls photoHash function based on button hashAction
#IBAction func hashAction(sender: AnyObject) {
photoHash()
}
// converts latest picture to binary to sha256 hash and outputs to console
func photoHash(){
let img = ImageHelper.removeExifData(UIImagePNGRepresentation(imageView.image!)!)
let img2 = ImageHelper.removeExifData(UIImageJPEGRepresentation(imageView.image!, 1.0)!)
let imgHash = sha256_bin(img!)
let imgHash2 = sha256_bin(img2!)
print(imgHash)
print(imgHash2)
// write image to photo library
UIImageWriteToSavedPhotosAlbum(imageView.image!, nil, nil, nil)
}
// Digests binary data from picture into sha256 hash, output: hex string
func sha256_bin(data : NSData) -> String {
var hash = [UInt8](count: Int(CC_SHA256_DIGEST_LENGTH), repeatedValue: 0)
CC_SHA256(data.bytes, CC_LONG(data.length), &hash)
let res = NSData(bytes: hash, length: Int(CC_SHA256_DIGEST_LENGTH))
let resString = res.toHexString()
return resString
}
}
Specifications:
MacBook Pro retina 2013, OS X 10.11.5
xcode version 7.3.1
swift 2
iphone 5S
hash on command line via shasum -a 256 filename.jpg
Since posting my question last week I learned that Apple seperates the image data from the meta data (image data is stored in UIIMage object), so hashing the UIImage object will never result in a hash that is the same as a hash digested on the command line (or in python or where ever). This is because for python/perl/etc, the meta data is present (even with a tool as Exiftool, the exif data is standardized but still there, whereas in the app environment, the exif data is simply not there, I guess this has something to do with low level vs high level languages but not sure).
Although there are some ways to access the EXIF data (or meta data in general) of a UIImage, it is not easy. This is a feature to protect the privacy (among other things) of the user.
Solution
I have found a solution to our specific problem via a different route: turns out that iOS does save all the image data and meta data in one place on disk for a photo. By using the Photos API, I can get access to these with this call (I found this in an answer on SO, but I just don't remember how I ended up there. If you recognise this snippet, please let me know):
func getLastPhoto() {
let fetchOptions = PHFetchOptions()
fetchOptions.sortDescriptors = [NSSortDescriptor(key: "creationDate", ascending: true)]
let fetchResult = PHAsset.fetchAssetsWithMediaType(PHAssetMediaType.Image, options: fetchOptions)
if let lastAsset: PHAsset = fetchResult.lastObject as? PHAsset {
let manager = PHImageManager.defaultManager()
let imageRequestOptions = PHImageRequestOptions()
manager.requestImageDataForAsset(lastAsset, options: imageRequestOptions) {
(let imageData: NSData?, let dataUTI: String?,
let orientation: UIImageOrientation,
let info: [NSObject : AnyObject]?) -> Void in
// Doing stuff to the NSDAta in imageData
}
}
By sorting on date in reverse order the first entry is (obviously) the most recent photo. And as long as I don't load it into an imageView, I can do with the data what I want (sending it to a hash function in this case).
So the flow is as follows: user takes photo, photo is saved to the library and imported to the imageView. The user then presses the hash button upon which the most recently added photo (the one in the imageView) is fetched from disk with meta data and all. I can then export the photo from the library by airdrop (for now, https request in later stadium) and reproduce the hash on my laptop.
I'm writing an iOS app share extension, and I wanted to obtain a large preview image. After some effort, I was able to make this code work:
class ShareViewController: SLComposeServiceViewController {
// This is the result handler for the call to loadPreviewImageWithOptions
let imageHandler: NSItemProviderCompletionHandler = { [unowned self]
(result: NSSecureCoding?, error: NSError!) in
if result is UIImage
{
let image = result as! UIImage
dispatch_async(dispatch_get_main_queue(), { () -> Void in
self.imageView.image = image
self.imageView.contentMode = UIViewContentMode.ScaleAspectFill
self.imageView.clipsToBounds = true
})
}
}
// Find the shared item and preview it.
for item: AnyObject in self.extensionContext!.inputItems
{
let inputItem = item as! NSExtensionItem
for provider: AnyObject in inputItem.attachments!
{
let provider = provider as! NSItemProvider
// I want a preview image as large as the device.
var options_dict = [NSObject:AnyObject]()
options_dict[NSItemProviderPreferredImageSizeKey] = NSValue(CGSize: CGSize(width: 960, height:540))
provider.loadPreviewImageWithOptions(options_dict, completionHandler: imageHandler)
}
}
...
}
I obtain an image, but its size is always 84 x 79 pixels. From the NSItemProvider documentation the options dictionary should support preview image size:
options - A dictionary of keys and values that provide information about the item, such as the size of an image. For a list of possible keys, see Options Dictionary Key.
And under Options Dictonary Key on the same page:
NSItemProviderPreferredImageSizeKey -
A key specifying the dimensions of an image in pixels. The value of this key is an NSValue object containing a CGSize or NSSize data type.
There is one clue:
Keys are used in the dictionary passed to the options parameter of a NSItemProviderLoadHandler block.
So maybe I have to call or override loadItemForTypeIdentifier with the size option, and then call loadPreviewImageWithOptions? I'm trying this now.