I am attempting to build a video merging app that allows users to select several short clips from a collection view and then generates a preview of the videos all merged into one. I am using the Photos framework (PHCachingImageManager) to populate the collection view and am passing an array of the selected PHAssets to the function below in order to request low quality AVAssets (for merging & generating the preview).
The problem is, I need to keep the AVAssets in the order in which the user selected them, but the "requestAVAsset" function is asynchronous and the completion handler is often called multiple times. I've never used Dispatch Groups before, but attempted to use them below...and the AVAssets are still out of order sometimes.
func requestAVAssets(assets: [PHAsset]) -> [AVAsset] {
var videoArray: [AVAsset] = []
let dispatchGroup = DispatchGroup()
let videoOptions = PHVideoRequestOptions()
videoOptions.isNetworkAccessAllowed = true
videoOptions.deliveryMode = .fastFormat
for asset in assets {
dispatchGroup.enter()
self.imageManager.requestAVAsset(forVideo: asset, options: videoOptions, resultHandler: { (video, audioMix, info) in
guard video != nil else { return }
videoArray.append(video!)
dispatchGroup.leave()
})
}
dispatchGroup.wait()
return videoArray
}
I'm guessing I've either misplaced some code or am approaching this in entirely the wrong way! Any suggestions are appreciated.
If you capture the current index while you're iterating the AVAssets, you can insert rather than append. That's how I do it, at least.
func requestAVAssets(assets: [PHAsset]) -> [AVAsset] {
var videoArray = [AVAsset?](repeating: nil, count: assets.count)
let videoOptions = PHVideoRequestOptions()
videoOptions.isNetworkAccessAllowed = true
videoOptions.deliveryMode = .fastFormat
for (i, asset) in assets.enumerated() {
self.imageManager.requestAVAsset(forVideo: asset, options: videoOptions, resultHandler: { (video, audioMix, info) in
guard let video = video else { return }
videoArray.remove(at: i)
videoArray.insert(video, at: i)
})
}
return videoArray.flatMap { $0 }
}
Giving the array the desired number of items as nil will stop it from erroring when inserting items, and then when the download is complete, remove the existing nil value and replace it with the actual AVAsset.
Finally, flatMap the resulting array to unpack the optionals (and optionally check that you have the desired number of items by comparing it with the incoming assets array).
Dodging the dispatch question entirely because it's late and I've had a bad day, but what if you kept the "correct" index associated with the video, and then sorted on that? I think something like this would work.
struct SelectedVideo {
let index: Int
let asset: AVAsset
}
func requestAVAssets(assets: [PHAsset]) -> [AVAsset] {
var videoArray: [SelectedVideo] = []
let dispatchGroup = DispatchGroup()
let videoOptions = PHVideoRequestOptions()
videoOptions.isNetworkAccessAllowed = true
videoOptions.deliveryMode = .fastFormat
for (index, asset) in assets.enumerated() {
dispatchGroup.enter()
self.imageManager.requestAVAsset(forVideo: asset, options: videoOptions, resultHandler: { (videoMb, audioMixMb, infoMb) in
guard let video = videoMb else return
videoArray.append(SelectedVideo(index, video))
dispatchGroup.leave()
})
}
dispatchGroup.wait()
return videoArray.sort { $0.index < $1.index}.map({$0.video})
}
This is kind of a hack (haven't even tried compiling it), but like I said, it's been a bad day.
A couple minor changes to note: I changed the params to the closure to say "Mb" which means "maybe" and is a nice convention I've seen for naming optionals passed to closures. Also, instead of "guard video != nil" followed by force-unwrapping, it's much preferred to do a "guard let video = videoMb", and then video is non-optional.
Related
I have about 200 audio assets about 5 seconds each that are loaded into an AVQueuePlayer. I do this so it plays all the assets as one full song, but allowing the user to view or jump to a specific verse.
The issue is the main thread gets locked when I load them in like this:
let items = urls
.map(AVAsset.init)
.map(AVPlayerItem.init)
let player = AVQueuePlayer(items: items)
Then I discovered I must asynchronously load them into the player so tried something like this:
let items = urls
.map(AVAsset.init)
.map(AVPlayerItem.init)
let player = AVQueuePlayer()
// Below does NOT compile!!
await withThrowingTaskGroup(of: Void.self) { group in
for var item in items {
group.addTask { _ = try await item.asset.load(.duration) }
for try await result in group {
player?.insert(item, after: nil)
}
}
}
I'm only supporting iOS 16+ so I'm trying to make use of the new Swift Concurrency APIs available in AVFoundation. I tried following this document but there's a lot of gaps I can't quite get. What is the proper way of loading up that many assets into the queue player without locking up the main thread?
Since it seems that you are trying to directly affect items and player I would use actor. To keep everything together and synchronized.
To bypass captured var 'item' in concurrently-executing code you can use .enumerated()
actor PlayerLoader{
let items = [URL(string: "https://v.ftcdn.net/01/53/82/41/700_F_153824165_dN7n9QftImnClb7z1IJIjbLgLlkHyYDS_ST.mp4")!]
.map(AVAsset.init)
.map(AVPlayerItem.init)
let player = AVQueuePlayer()
func loadItems() async throws{
try await withThrowingTaskGroup(of: Void.self) { group in
for (idx, _) in items.enumerated() {
group.addTask { [weak self] in
_ = try await self?.items[idx].asset.load(.duration)
}
for try await _ in group {
player.insert(items[idx], after: nil)
}
}
}
}
}
Then just call loadItems with Task or Task.detached
let loader = PlayerLoader()
let task = Task{ //or Task.detached depending on where you will be calling this line of code
do{
try await loader.loadItems()
print("total number of items loaded \(loader.player.items().count)")
}catch{
print(error)
throw error
}
}
How about something like this?
import AVFoundation
let items = [URL(string: "https://google.com")!]
let player = AVQueuePlayer()
try await withThrowingTaskGroup(of: AVPlayerItem.self) { group in
for item in items {
group.addTask {
AVPlayerItem(asset: AVAsset(url: item))
}
}
for try await item in group {
player.insert(item, after: nil)
print(item)
}
}
I apologize if this question is simple or the problem is obvious as I am still a beginner in programming.
I am looping over an array and trying to make an async Firestore call. I am using a DispatchGroup in order to wait for all iterations to complete before calling the completion.
However, the Firestore function is not even getting called. I tested with print statements and the result is the loop iterations over the array have gone through with an enter into the DispatchGroup each time and the wait is stuck.
func getUserGlobalPlays(username: String, fixtureIDs: [Int], completion: #escaping (Result<[UserPlays]?, Error>) -> Void) {
let chunkedArray = fixtureIDs.chunked(into: 10)
var plays: [UserPlays] = []
let group = DispatchGroup()
chunkedArray.forEach { ids in
group.enter()
print("entered")
DispatchQueue.global().async { [weak self] in
self?.db.collection("Users").document("\(username)").collection("userPlays").whereField("fixtureID", in: ids).getDocuments { snapshot, error in
guard let snapshot = snapshot, error == nil else {
completion(.failure(error!))
return
}
for document in snapshot.documents {
let fixtureDoc = document.data()
let fixtureIDx = fixtureDoc["fixtureID"] as! Int
let choice = fixtureDoc["userChoice"] as! Int
plays.append(UserPlays(fixtureID: fixtureIDx, userChoice: choice))
}
group.leave()
print("leaving")
}
}
}
group.wait()
print(plays.count)
completion(.success(plays))
}
There are a few things going on with your code I think you should fix. You were dangerously force-unwrapping document data which you should never do. You were spinning up a bunch of Dispatch queues to make the database calls in the background, which is unnecessary and potentially problematic. The database call itself is insignificant and doesn't need to be done in the background. The snapshot return, however, can be done in the background (which this code doesn't do, so you can add that if you wish). And I don't know how you want to handle errors here. If one document gets back an error, your code sends back an error. Is that how you want to handle it?
func getUserGlobalPlays(username: String,
fixtureIDs: [Int],
completion: #escaping (_result: Result<[UserPlays]?, Error>) -> Void) {
let chunkedArray = fixtureIDs.chunked(into: 10)
var plays: [UserPlays] = []
let group = DispatchGroup()
chunkedArray.forEach { id in
group.enter()
db.collection("Users").document("\(username)").collection("userPlays").whereField("fixtureID", in: id).getDocuments { snapshot, error in
if let snapshot = snapshot {
for doc in snapshot.documents {
if let fixtureIDx = doc.get("fixtureIDx") as? Int,
let choice = doc.get("choice") as? Int {
plays.append(UserPlays(fixtureID: fixtureIDx, userChoice: choice))
}
}
} else if let error = error {
print(error)
// There was an error getting this one document. Do you want to terminate
// the entire function and pass back an error (through the completion
// handler)? Or do you want to keep going and parse whatever data you can
// parse?
}
group.leave()
}
}
// This is the completion handler of the Dispatch Group.
group.notify(queue: .main) {
completion(.success(plays))
}
}
When fetching images and video with Photo Kit my images and video thumbnails are displaying in low res within my LazyVGrid.
They also display in lo-res when I display the image inside a larger Image view.
All of my images and video are stored on iCloud so I set isNetworkAccessAllowed = true AND deliveryMode = .highQualityFormat in PHImageRequestOptions().
I have tried changing the targetSize in the requestImage method to different values but that didn't change the quality either.
What am I doing wrong?
#Published var fetchedMediaArray : [MediaAsset] = []
// model for Assets
struct MediaAsset: Identifiable, Hashable {
var id = UUID()
var image : UIImage
var selected: Bool
var asset: PHAsset
}
func requestAuth() {/*auth code*/ }
func fetchPhotosAndVideos() {
let opt = PHFetchOptions()
opt.includeHiddenAssets = false
opt.sortDescriptors = [NSSortDescriptor(key: "creationDate", ascending: false)]
opt.predicate = NSPredicate(format: "mediaType == %d || mediaType == %d",
PHAssetMediaType.image.rawValue,
PHAssetMediaType.video.rawValue)
let req = PHAsset.fetchAssets(with: opt)
DispatchQueue.main.async {
let options = PHImageRequestOptions()
options.isSynchronous = true
options.deliveryMode = .highQualityFormat
options.isNetworkAccessAllowed = true
for j in 0..<req.count {
PHCachingImageManager.default().requestImage(for: req[j], targetSize: CGSize(width: 100, height: 100), contentMode: .default, options: options) { (image, _) in
let data1 = MediaAsset(image: image!, selected: false, asset: req[j])
self.fetchedMediaArray.append(data1)
}
}
}
}
When Photos are being downloaded from cloud, it may call your completion handler multiple times.
From the PHImageManager.requestImage(for:asset) docs -
Discussion
For an asynchronous request, Photos may call your result handler block more than once. Photos first calls the block to provide a low-quality image suitable for displaying temporarily while it prepares a high-quality image. (If low-quality image data is immediately available, the first call may occur before the method returns.) When the high-quality image is ready, Photos calls your result handler again to provide it. If the image manager has already cached the requested image at full quality, Photos calls your result handler only once. The PHImageResultIsDegradedKey key in the result handler’s info parameter indicates when Photos is providing a temporary low-quality image.
What you can do in your callback handler -
If your asset is being returned in target/requested size(or bigger than requested), you can consider it done.
If your asset is being returned in smaller than target/requested size, you should check for PHImageResultIsDegradedKey in the info parameter and wait for Photos to call your completion next time.
I'm working my way through the Stanford CS193P Course on iTunesU. Stuck on this one part - There's one section where the instructor has the following comment:
When a drop happens, you'll have to collect both the aspect ratio (from
the UIImage) and the URL before you can add an item. You could do this
straightforwardly with a couple of local variables that are captured by
the closures used to load up the drag and drop data
I assume this to mean that I have to update my model item in the performDrop method of the UICollectionViewDropDelegate protocol. Therefore I will have to call loadObject for both the NSURL item and the UIImage item. However, since the completion handlers for loadObject are off the main thread, I don't have access to the same local variable for both. Therefore if I first load the UIImage, get the aspect ratio, and save it to a local variable in the performDrop method, that variable is still empty when I call loadObject on the URL, which is when I create the model based on the URL and the aspect ratio. I tried nesting the loadObject calls, but the 2nd call was not firing. My code is below:
let placeholderContext = coordinator.drop(item.dragItem, to:
UICollectionViewDropPlaceholder(insertionIndexPath: destinationIndexPath, reuseIdentifier: placeholderCellReuseIdentifier))
var aspectRatio: [CGFloat] = []
item.dragItem.itemProvider.loadObject(ofClass: UIImage.self) { (provider, error) in
if var image = provider as? UIImage {
aspectRatio.append(image.size.width / image.size.height)
}
DispatchQueue.main.async {
item.dragItem.itemProvider.loadObject(ofClass: NSURL.self, completionHandler: { (provider, error) in
print(provider)
if let error = error {
print(error.localizedDescription)
}
print("load urls in completion")
})
}
}
print("main thread: \(aspectRatio)")
item.dragItem.itemProvider.loadObject(ofClass: NSURL.self) { (provider, error) in
print("loaded urls")
DispatchQueue.main.async {
if let url = provider as? URL {
placeholderContext.commitInsertion(dataSourceUpdates: { (insertionIndexPath) in
self.galleryItems.items.insert(GalleryItem(imageURL: url.imageURL, aspectRatio: 1.0), at: insertionIndexPath.item)
})
} else {
placeholderContext.deletePlaceholder()
}
}
}
Am I misguided here or is there something simple I am missing? How can I get the aspect ratio from loading the UIImage first, and then use that in the URL load object completion handler, in the way as is described in the homework?
This is a question about the objc.io video about networking.
Note: I have seen similar questions, but none of the suggestions worked
for me.
In the sample code, the main call is:
Webservice().load(resource: Episode.all) { result in
print(result!)
}
and the load function looks like this:
func load<A>(resource: Resource<A>, completion: #escaping (A?) -> ()) {
URLSession.shared.dataTask(with: resource.url as URL) { data, _, _ in
guard let data = data else {
completion(nil)
return
}
completion(resource.parse(data as NSData))
}.resume()
}
For simplicity they just print the result (an array of dictionaries), but I don't understand how I can further use it, because it is inside the closure. This way I can eg show the array in a UITableView or something like that.
So what I would like to do is something like:
var episodes = [Episode]()
Webservice().load(resource: Episode) { result in
episodes = result!
}
print(episodes)
But this results in an empty array being printed.
I also tried using
DispatchQueue.main.async {
episodes = result!
}
But again, nothing is printed.
How can I accomplish that?