how to quickly play numbers of videos on collectionView like Tiktok? [closed] - ios

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I'm building a video list view(using collectionView) like the Tiktok app. I'm adding AVPlayerLayer on imageView(for each cellItem) and playing AVPlayer on that, taking a few time to load the video layer. Can anyone suggest how we can fetch video data for the player before going on the video page to make the video page more smooth???
Please check the below code what I'm doing wrong in that??
func setupVideoFor(url: String, completion: #escaping COMPLETION_HANDLER = {_ in}) {
if self.videoCache.object(forKey: url as NSString) != nil {
return
}
guard let URL = URL(string: url) else {
return
}
didVideoStartPlay = completion
let asset = AVURLAsset(url: URL)
let requestedKeys = ["playable"]
asset.loadValuesAsynchronously(forKeys: requestedKeys) { [weak self] in
guard let strongSelf = self else {
return
}
/**
Need to check whether asset loaded successfully, if not successful then don't create
AVPlayer and AVPlayerItem and return without caching the videocontainer,
so that, the assets can be tried to be downloaded again when need be.
*/
var error: NSError? = nil
let status = asset.statusOfValue(forKey: "playable", error: &error)
switch status {
case .loaded:
break
case .failed, .cancelled:
print("Failed to load asset successfully")
return
default:
print("Unkown state of asset")
return
}
let player = AVPlayer()
let item = AVPlayerItem(asset: asset)
DispatchQueue.main.async {
let videoContainer = VideoContainer(player: player, item: item, url: url)
strongSelf.videoCache.setObject(videoContainer, forKey: url as NSString)
videoContainer.player.replaceCurrentItem(with: videoContainer.playerItem)
/**
Try to play video again in case when playvideo method was called and
asset was not obtained, so, earlier video must have not run
*/
if strongSelf.videoURL == url, let layer = strongSelf.currentLayer {
strongSelf.duration = asset.duration
strongSelf.playVideo(withLayer: layer, url: url)
}
}
}
}

It depends on a few factors...
AVPlayer is a way to control what happens to an AVPlayerItem, and AVPlayerLayer is just the display layer for that.
You want to look into AVPlayerItem. You can initialize a number of AVPlayerItem objects without passing them to the AVPlayer. You can observe each of their status properties (with KVO) to know when they are ready to play. You could do this before showing any video layer at all, then pass the ready AVPlayerItem objects to the AVPlayer, and that could give the perception of speeded up video.
Also, you might consider looking at your video's HLS manifest. You can check errors of the manifest itself with mediastreamvalidator which can be found (along with other tools) over here. https://developer.apple.com/documentation/http_live_streaming/about_apple_s_http_live_streaming_tools
This tool will inspect how the playlist is set up, and report any number of errors, including ones that would affect performance. For example, if the initial bitrate (what the player will try to play before it figures out data about network conditions, etc) is set too high, this could lead to long loading times.

Related

Playing and caching a remote asset with AVPlayer

I am trying to use AVPlayer to play/cache a remote asset using two tools on Github: CachingPlayerItem with Cache. I found the solution elsewhere(scroll down), which nearly gets me there, My issue now is that I have to tap twice on the remote audio asset (a hyperlink in Firebase) to get it to stream. For some mysterious reason, AVPlayer will not play the remote asset unless it is cached in my case. I am aware that I can directly stream the url using AVPlayerItem(url:) but that is not the solution I am seeking; the sample code for CachingPlayerItem say that should not be necessary.
In my tinkering, I think something is happening with the async operations that are performed when I call playerItem.delegate = self. Maybe I am misunderstanding how this asynchronous delegate operation is working... Any clarity and pointers would be appreciated.
import AVKit
import Cache
class AudioPlayer: AVPlayer, ObservableObject, AVAudioPlayerDelegate {
let diskConfig = DiskConfig(name: "DiskCache")
let memoryConfig = MemoryConfig(expiry: .never, countLimit: 10, totalCostLimit: 10)
lazy var storage: Cache.Storage<String, Data>? = {
return try? Cache.Storage(diskConfig: diskConfig, memoryConfig: memoryConfig, transformer: TransformerFactory.forData())
}()
/// Plays audio either from the network if it's not cached or from the cache.
func startPlayback(with url: URL) {
let playerItem: CachingPlayerItem
do {
let result = try storage!.entry(forKey: url.absoluteString)
// The video is cached.
playerItem = CachingPlayerItem(data: result.object, mimeType: "audio/mp4", fileExtension: "m4a")
} catch {
// The video is not cached.
playerItem = CachingPlayerItem(url: url)
}
playerItem.delegate = self // Seems to be the problematic line if the result is not cached.
self.replaceCurrentItem(with: playerItem) // This line is different from what you do. The behaviour doesnt change whether I have AVPlayer as private var.
self.automaticallyWaitsToMinimizeStalling = false
self.play()
}
}
extension AudioPlayer: CachingPlayerItemDelegate {
func playerItem(_ playerItem: CachingPlayerItem, didFinishDownloadingData data: Data) {
// Video is downloaded. Saving it to the cache asynchronously.
storage?.async.setObject(data, forKey: playerItem.url.absoluteString, completion: { _ in })
print("Caching done!")
}
}

How to control AVPlayer buffering

I'm using an AVPlayer to play a remote progressive download (i.e. non-HLS) video. But, I can't figure out how to control its buffering behavior.
I would like to pre-fetch 2 seconds of the video before it's ready to play, and also to stop buffering when the video is paused.
Here's my setup:
let asset = AVURLAsset(url: url)
let playerItem = AVPlayerItem(asset: asset)
let player = AVPlayer()
I tried the following, without success:
// doesn't start buffering
playerItem.preferredForwardBufferDuration = 2.0
// doesn't stop buffering
playerItem.preferredForwardBufferDuration = 2.0
player.replaceCurrentItem(with: playerItem)
I tried player.automaticallyWaitsToMinimizeStalling = true in both cases, and in combination with various player.pause() or player.rate = 0 - doesn't work.
A potential approach that comes to mind is to observe for loadedTimeRanges until the first 2 seconds loaded and set current item of the player to nil.
let c = playerItem.publisher(for: \.loadedTimeRanges, options: .new)
.compactMap { $0.first as? CMTimeRange }
.sink {
if $0.duration.seconds - $0.start.seconds > 2 {
player.replaceCurrentItem(with: nil)
}
}
This would work for pre-buffer, but it doesn't work for pausing, because it makes the video blank instead of paused. (And at this point, I feel I'm attempting to reimplement/interfere with some core buffering functionality)

HLS caching using AVAssetDownloadTask

I am following Apple's documentation on caching HLS (.m3u8) video.
https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/MediaPlaybackGuide/Contents/Resources/en.lproj/HTTPLiveStreaming/HTTPLiveStreaming.html
Under Playing Offline Assets in the documentation, it is instructed to use AVAssetDownloadTask's asset to simultaneously start playing.
func downloadAndPlayAsset(_ asset: AVURLAsset) {
// Create new AVAssetDownloadTask for the desired asset
// Passing a nil options value indicates the highest available bitrate should be downloaded
let downloadTask = downloadSession.makeAssetDownloadTask(asset: asset,
assetTitle: assetTitle,
assetArtworkData: nil,
options: nil)!
// Start task
downloadTask.resume()
// Create standard playback items and begin playback
let playerItem = AVPlayerItem(asset: downloadTask.urlAsset)
player = AVPlayer(playerItem: playerItem)
player.play()
}
The issue is that the same asset is downloaded twice.
Right after AVPlayer is initialized it starts to buffer the asset. Initially, I assumed that the data from the buffer must be used to create cache but AVAssetDownloadTask doesn't start to download the data for caching until AVPlayer finishes playing the asset. The buffered data is basically discarded.
I used KVO on currentItem.loadedTimeRanges to check state of buffer.
playerTimeRangesObserver = currentPlayer.observe(\.currentItem?.loadedTimeRanges, options: [.new, .old]) { (player, change) in
let time = self.currentPlayer.currentItem?.loadedTimeRanges.firs.
if let t = time {
print(t.timeRangeValue.duration.seconds)
}
}
Below method to check the downloading status of AVAssetDownloadTask.
/// Method to adopt to subscribe to progress updates of an AVAssetDownloadTask.
func urlSession(_ session: URLSession, assetDownloadTask: AVAssetDownloadTask, didLoad timeRange: CMTimeRange, totalTimeRangesLoaded loadedTimeRanges: [NSValue], timeRangeExpectedToLoad: CMTimeRange) {
// This delegate callback should be used to provide download progress for your AVAssetDownloadTask.
guard let asset = activeDownloadsMap[assetDownloadTask] else { return }
var percentComplete = 0.0
for value in loadedTimeRanges {
let loadedTimeRange: CMTimeRange = value.timeRangeValue
percentComplete +=
loadedTimeRange.duration.seconds / timeRangeExpectedToLoad.duration.seconds
}
print("PercentComplete for \(asset.stream.name) = \(percentComplete)")
}
Is this the right behaviour or am I doing something wrong?
I want to be able to use the video data that is being cached (AVAssetDownloadTask downloading is in progress) to play in AVPlayer.
Your AVAssetDownloadTask must be configured to download differing HLS variants than your AVPlayerItem is requesting.
If you already have some data downloaded by AVAssetDownloadTask, your AVPlayerItem will subsequently use it.
But if you already have some data downloaded by AVPlayerItem, your AVAssetDownloadTask may ignore it, as it needs to satisfy the requirements of your download configuration.

Swift 4 - Trying to display AWS video url from backend in AVPlayerViewController not working, get weird error

I have a tableview which sets a UIImage to hold either an image url from AWS or a thumbnail generated from a video URL also from AWS. The video url refuses to display in my tableview and it throws this error in the debugger.
2017-12-29 12:20:37.053337-0800 VideoFit[3541:1366125] CredStore - performQuery - Error copying matching creds. Error=-25300, query={
class = inet;
"m_Limit" = "m_LimitAll";
"r_Attributes" = 1;
sync = syna;
}
When I click the cell to display either the image or video url, it segues to the video player correctly and then I get the error again and the triangular start button has a line through it signifying that there is no video to be played.
But when I print the url it has successfully passed so that is not the issue, the issue is AVPlayer can't handle my AWS video url for some reason. It is an https link so that means it must be secure but I already set my arbitrary loads to true
<key>NSAppTransportSecurity</key>
<dict>
<key>NSAllowsArbitraryLoads</key>
<true/>
</dict>
Here is some code for displaying my videos in the videoPlayer VC and also the thumbnail generator function, perhaps there is some issue lying with these?
import UIKit
import AVKit
import MediaPlayer
class VideoPlayerVC: AVPlayerViewController {
var urlToPlay: String?
override func viewDidLoad() {
super.viewDidLoad()
print("Here is the url ---> \(String(describing: urlToPlay))")
playVideo()
}
private func playVideo() {
guard let urlFromString = urlToPlay else { print("No url to play") ;return }
let url = URL(string: urlFromString)
print("Here is the url to play ---> \(String(describing: url))")
let asset: AVURLAsset = AVURLAsset(url: url!)
let item: AVPlayerItem = AVPlayerItem(asset: asset)
let player: AVPlayer = AVPlayer(playerItem: item)
self.player = player
self.showsPlaybackControls = true
self.player?.play()
}
}
This is how I make the thumbnail, when my cellforRow atIndexPath method runs it throws this error for every video object in the tableview.
override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
if let cell = tableView.dequeueReusableCell(withIdentifier: "sortedExerciseCell") as? SortedExerciseCell! {
// Do if check for videoURI and imageURI
if selectedExerciseArray[indexPath.row].imageURI != nil {
if let imageURI = URL(string: selectedExerciseArray[indexPath.row].imageURI!) {
print("It's a photo!")
// Using DispatchQueue.global(qos: .background).async loads cells in background
DispatchQueue.global(qos: .background).async {
let data = try? Data(contentsOf: imageURI)
DispatchQueue.main.async {
cell.exerciseImg.image = UIImage(data: data!)
}
}
}
} else {
if let videoURI = URL(string: selectedExerciseArray[indexPath.row].videoURI!) {
print("It's a video!")
print(videoURI)
DispatchQueue.global(qos: .background).async {
DispatchQueue.main.async {
cell.exerciseImg.image = self.thumbnailForVideoAtURL(url: videoURI)
// for every video object the above error is thrown!!!
}
}
}
}
cell.exerciseName.text = selectedExerciseArray[indexPath.row].name
return cell
} else {
return UITableViewCell()
}
}
// Used to just display the video thumbnail in cell, when clicked on will display video as needed
private func thumbnailForVideoAtURL(url: URL) -> UIImage? {
let asset = AVAsset(url: url)
let assetImageGenerator = AVAssetImageGenerator(asset: asset)
do {
print("Doing the video thumbnail from backend")
let imageRef = try assetImageGenerator.copyCGImage(at: CMTimeMake(1, 60) , actualTime: nil)
return UIImage(cgImage: imageRef)
} catch {
print("This is failing for some reason")
print("error")
return nil
}
}
I've looked at similar questions on stack overflow about this but none seem to give a complete answer on how to solve this problem, most chalking it up to an iOS 11 bug that can't be fixed or transport security (Already have arbitrary loads on so this can't be the issue..) anyone else have any approaches that might work?
Important note - My backend developer can only view the video url's from a webpage, in other words he must make a basic website to display the video after downloading it from the web. I'm not sure if this is standard procedure for handling video url's from AWS but I decided to try loading the url with "loadHTMLString" in a custom UIViewController conforming to WKUIDelegate, I see the video but the same situation happens where the triangular start button is crossed out signifying no video can be played. I'm not really sure what else I can try at this moment, any help is appreciated.
Here is one of the links I've pulled from my backend.
https://videofitapptestbucket.s3.us-west-2.amazonaws.com/100001427750
It seems that your problem is in file extension. DMS is not recognized by OS, it throws when creating image with assetgenerator.
Error Domain=AVFoundationErrorDomain Code=-11828 "Cannot Open" UserInfo={NSLocalizedFailureReason=This media format is not supported., NSLocalizedDescription=Cannot Open, NSUnderlyingError=0x6040000534d0 {Error Domain=NSOSStatusErrorDomain Code=-12847 "(null)"}}
Rename your files on server. The mp4 extension seems to work just fine with your file.
Also, subclassing AVPlayerViewController is generally not that great idea as Apple says that it will result in undefined behavior. (read: random mess). I would suggest to use class that have AVPlayer inside.
Try:
if let path = urlToPlay {
let url = URL(string: path)!
let videoPlayer = AVPlayer(url: url)
self.player = videoPlayer
DispatchQueue.main.async {
self.player?.play()
}
}
Call it in viewDidAppear() method

How to make AVPlayer appear instantly instead of after video ends?

Our app lets users record a video, after which the app adds subtitles and exports the edited video.
The goal is to replay the video immediately, but the AVPlayer only appears after the video finishes (and only plays audio, which is a separate issue).
Here's what happens now: we show a preview so the user can see what he is recording in real-time. After the user is done recording, we want to play back the video for review. Unfortunately, no video appears, and only audio plays back. An image representing some frame of the video appears when the audio is done playing back.
Why is this happening?
func exportDidFinish(exporter: AVAssetExportSession) {
println("Finished exporting video")
// Save video to photo album
let assetLibrary = ALAssetsLibrary()
assetLibrary.writeVideoAtPathToSavedPhotosAlbum(exporter.outputURL, completionBlock: {(url: NSURL!, error: NSError!) in
println("Saved video to album \(exporter.outputURL)")
self.playPreview(exporter.outputURL)
if (error != nil) {
println("Error saving video")
}
})
}
func playPreview(videoUrl: NSURL) {
let asset = AVAsset.assetWithURL(videoUrl) as? AVAsset
let playerItem = AVPlayerItem(asset: asset)
player = AVPlayer(playerItem: playerItem)
let playerLayer = AVPlayerLayer(player: player)
playerLayer.frame = view.frame
view.layer.addSublayer(playerLayer)
player.play()
}
Perhaps this can help:
let assetLibrary = ALAssetsLibrary()
assetLibrary.writeVideoAtPathToSavedPhotosAlbum(exporter.outputURL, completionBlock: {(url: NSURL!, error: NSError!) in
if (error != nil) {
println("Error saving video")
}else{
println("Saved video to album \(url)")
self.playPreview(url)
}
})
Send "url" to "playPreview" leaving "completionBlock" and not that which comes from "AVAssetExportSession"
Perhaps...!
The answer was we had an incorrectly composed video in the first place, as described here: AVAssetExportSession export fails non-deterministically with error: "Operation Stopped, NSLocalizedFailureReason=The video could not be composed.".
The other part of the question (audio playing long before images/video appears) was answered here: Long delay before seeing video when AVPlayer created in exportAsynchronouslyWithCompletionHandler
Hope these help someone avoid the suffering we endured! :)

Resources