Video Streaming Fails when using AVAssetResourceLoader - ios

We are using video streaming in our Swift IOS application and it works very well. The problem is that we would like to use AVAssetResourceLoader so we can make the requests to the streaming server using our own URLSession rather than whatever AVPlayer uses (I have been completely unable to find out what session AVPlayer uses or how to influence what session it uses.)
The exact behavior is that shouldWaitForLoadingOfRequestedResource is called once for two bytes on on the .m3u8 file then it is called again asking for the whole file, and then (based on the start time) the correct .ts file is requested. After we fetch the .ts file the video player simply doesn't do anything further.
This happens whether or not we use a sample streaming video file "https://tungsten.aaplimg.com/VOD/bipbop_adv_example_v2/master.m3u8" or our own server. The sequence is identical if we don't use AVAssetResourceLoader (we can tell from our server.) right up until the .ts file is requested. At that point when we don't use the custom loader the AvPlayer brings up the video and keeps requesting .ts files. If we comment out all other interactions with the AVPlayer including setting the initial time the behavior is identical so I am only going to include the code from viewDidLoad and shouldWaitForLoadingOfRequestedResource.
Again if we simply remove the "xyzzy" prefix so that AVAssetResourceLoader isn't used, everything works. Also, I guess importantly, if we target a video file that is not a streaming file everything works either way.
One more thing our transformation of the mime type for the .ts file produced some kind of weird dynamic uti, but this doesn't seem to have anything to do with the problem because even if we hardcode the uti the same thing happens.
override func viewDidLoad() {
super.viewDidLoad()
avPlayer = AVPlayer()
avPlayerLayer = AVPlayerLayer(player: avPlayer)
videoView.layer.insertSublayer(avPlayerLayer, at: 0)
videoView.backgroundColor = UIColor.black
url = URL(string: "xyzzy" + currentPatient.videoURL())!
let asset = AVURLAsset(url: url)
asset.resourceLoader.setDelegate(self, queue: DispatchQueue.main)
let item = AVPlayerItem(asset: asset)
let avPlayerItem = item
avPlayer.replaceCurrentItem(with: avPlayerItem)
videoScrollView.delegate = self
videoScrollView.minimumZoomScale = 1.0
videoScrollView.maximumZoomScale = 6.0
}
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
let urlString = loadingRequest.request.url?.absoluteString
let urlComponents = urlString?.components(separatedBy: "xyzzy")
let url = URL(string: urlComponents![1])
let request = loadingRequest.dataRequest!
let infoRequest = loadingRequest.contentInformationRequest
let task = globalSession.dataTask(with: url!) { (data, response, error) in
self.avPlayerThread.async {
if error == nil && data != nil {
let uti = UTTypeCreatePreferredIdentifierForTag(kUTTagClassMIMEType, response?.mimeType as! CFString, nil)
if let infoRequest = infoRequest {
infoRequest.contentType = uti?.takeRetainedValue() as? String
if request.requestsAllDataToEndOfResource == false {
infoRequest.contentLength = Int64(request.requestedLength)
} else {
infoRequest.contentLength = Int64((data?.count)!)
}
infoRequest.isByteRangeAccessSupported = true
}
if infoRequest == nil || request.requestsAllDataToEndOfResource == true {
loadingRequest.dataRequest?.respond(with: data!)
}
loadingRequest.finishLoading()
} else {
print ("error \(error)")
loadingRequest.finishLoading(with: error)
}
}
}
task.resume()
return true
}

Related

AVPlayer playing wrong video file

I am having a weird situation and have no clue how to handle this , I am downloading the videos from firestorage and caching into device for future use , meanwhile the background thread is already doing its job , I am passing a video url to the function to play the video. The issue is that sometimes avplayer is playing the right video and sometimes taking some other video url from the cache.
you can find the code in below :
func cacheVideo(for exercise: Exercise) {
print(exercise.imageFileName)
guard let filePath = filePathURL(for: exercise.imageFileName) else { return }
if fileManager.fileExists(atPath: filePath.path) {
// print("already exists")
} else {
exercise.loadRealURL { (url) in
print(url)
self.getFileWith(with: url, saveTo: filePath)
}
}
}
writing file here
func getFileWith(with url: URL, saveTo saveFilePathURL: URL) {
DispatchQueue.global(qos: .background).async {
print(saveFilePathURL.path)
if let videoData = NSData(contentsOf: url) {
videoData.write(to: saveFilePathURL, atomically: true)
DispatchQueue.main.async {
// print("downloaded")
}
} else {
DispatchQueue.main.async {
let error = NSError(domain: "SomeErrorDomain", code: -2001 /* some error code */, userInfo: ["description": "Can't download video"])
print(error.debugDescription)
}
}
}
}
now playing the video using this
func startPlayingVideoOnDemand(url : URL) {
activityIndicatorView.startAnimating()
activityIndicatorView.isHidden = false
print(url)
let cachingPlayerItem = CachingPlayerItem(url: url)
cachingPlayerItem.delegate = self
cachingPlayerItem.download()
// cachingPlayerItem.preferredPeakBitRate = 0
let avasset = AVAsset(url: url)
let playerItem = AVPlayerItem(asset: avasset)
let player = AVPlayer(playerItem: playerItem)
player.automaticallyWaitsToMinimizeStalling = false
initializeVideoLayer(for: player)
}
any suggestions would be highly appreciated.
this was solved because the data model which i was using to download bunch of videos files was accessed in background thread and meanwhile i was trying to assign the url to the same data model class in order to fetch the video and play in avplayer. Hence this was the issue and resolved by simply adding a new attribute into data model for assigning the url to play right away.

Azure Media Service Fairplay DRM AVPlayer swift implementation

I am trying to play a Fairplay DRM protected (encrypted through Azure Media Services) HLS video stream on iOS Device.
I have used the code and process described in the following links:
https://icapps.com/blog/how-integrate-basic-hls-stream-fairplay
https://gist.github.com/fousa/5709fb7c84e5b53dbdae508c9cb4fadc
Following is the code I have written for this.
import UIKit
import AVFoundation
class ViewController: UIViewController, AVAssetResourceLoaderDelegate {
#IBOutlet weak var videoView: UIView!
var player: AVPlayer!
override func viewDidLoad() {
super.viewDidLoad()
let streamURL = "someexampleurl.com/stream.m3u8"
if let url = URL(string: streamURL) {
//2. Create AVPlayer object
let asset = AVURLAsset(url: url)
let queue = DispatchQueue(label: "Some queue")
asset.resourceLoader.setDelegate(self, queue: queue)
let playerItem = AVPlayerItem(asset: asset)
player = AVPlayer(playerItem: playerItem)
//3. Create AVPlayerLayer object
let playerLayer = AVPlayerLayer(player: player)
playerLayer.frame = self.videoView.bounds //bounds of the view in which AVPlayer should be displayed
playerLayer.videoGravity = .resizeAspect
//4. Add playerLayer to view's layer
self.videoView.layer.addSublayer(playerLayer)
//5. Play Video
player.play()
}
// Do any additional setup after loading the view.
}
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
// We first check if a url is set in the manifest.
guard let url = loadingRequest.request.url else {
print("🔑", #function, "Unable to read the url/host data.")
loadingRequest.finishLoading(with: NSError(domain: "com.error", code: -1, userInfo:
nil))
return false
}
print("🔑", #function, url)
// When the url is correctly found we try to load the certificate date. Watch out! For this
// example the certificate resides inside the bundle. But it should be preferably fetched from
// the server.
guard
let certificateURL = Bundle.main.url(forResource: "certfps", withExtension: "cer"),
let certificateData = try? Data(contentsOf: certificateURL) else {
print("🔑", #function, "Unable to read the certificate data.")
loadingRequest.finishLoading(with: NSError(domain: "com.error", code: -2, userInfo: nil))
return false
}
// Request the Server Playback Context.
let contentId = "xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
guard
let contentIdData = contentId.data(using: String.Encoding.utf8),
let spcData = try? loadingRequest.streamingContentKeyRequestData(forApp: certificateData, contentIdentifier: contentIdData, options: nil),
let dataRequest = loadingRequest.dataRequest else {
loadingRequest.finishLoading(with: NSError(domain: "com.error", code: -3, userInfo: nil))
print("🔑", #function, "Unable to read the SPC data.")
return false
}
// Request the Content Key Context from the Key Server Module.
let ckcURL = URL(string: "https://xxxxx.keydelivery.northeurope.media.azure.net/FairPlay/?kid=xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx")!
var request = URLRequest(url: ckcURL)
request.httpMethod = "POST"
let assetIDString = "xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
let postString = "spc=\(spcData.base64EncodedString())&assetId=\(assetIDString)"
request.setValue(String(postString.count), forHTTPHeaderField: "Content-Length")
request.setValue("application/x-www-form-urlencoded", forHTTPHeaderField: "Content-Type")
request.httpBody = postString.data(using: .ascii, allowLossyConversion: true)
let session = URLSession(configuration: URLSessionConfiguration.default)
let task = session.dataTask(with: request) { data, response, error in
if let data = data {
// The CKC is correctly returned and is now send to the `AVPlayer` instance so we
// can continue to play the stream.
if var responseString = String(data: data, encoding: .utf8) {
responseString = responseString.replacingOccurrences(of: "<ckc>", with: "").replacingOccurrences(of: "</ckc>", with: "")
var ckcData = Data(base64Encoded: responseString)!
dataRequest.respond(with: ckcData)
loadingRequest.finishLoading()
} else {
// print("Error encountered while fetching FairPlay license for URL: \(self.drmUrl), \(error?.localizedDescription ?? "Unknown error")")
}
task.resume()
return true
}
}
Everything above works but in the CKC response I get
{
"Error": {
"Message": "Failed content key policy evaluation.",
"Code": "AuthorizationPolicyEvaluationFailure"
}
}
Can anyone please here let me know what I am missing here, this is my first time trying this out
so I could be making a very obvious mistake so please bear with that.
Any help regarding this would be really great (I have been hitting my head on this for multiple days now.)
Thanks.
One thing that will probably help with troubleshooting is to enable the license delivery logging. You can do this in the Azure portal by going to your Media Services account, in the Monitoring section go to Diagnostic settings. Click 'Add diagnostic setting'. Give the setting a name and then, at least initially, tell it to archive to a storage account. Log the 'KeyDeliveryRequests'. Once you save this reproduce the issue. Then go to your Storage account and look for the log result. The Storage container ‘insights-logs-keydeliveryrequests’ will contain the logs.
you can add request header parameter like "authorization" (probably a base 64 token called JWT), "mimetype" in making CKC request, it would work.
Finally, I figured the thing I was missing was not passing the JWT in the "Authorization" header for the CKC request.
Passing the JWT did the trick. :)
Note: JWT stands for the JSON web token generated during the media encryption in azure media services.

Can we manually download/cache the transport stream while streaming m3u8 file

I am trying to play transport stream from m3u8 file. My requirement is to process the downloaded data before giving it to AVPlayer. For that I am using a proxy server[GCDWebserver] to intercept all the request. In proxy server I will download the data process it and feed it back.
I was able to download the media file and also have tried returning data using GCDWebServerDataResponse(data: apiData, contentType: apiResponse.mimeType ?? ""). But player is not playing the media content.
I am using GCDWebserver as my proxy server.
I have created an instance of AVPlayerItem with the following url
http://34.55.7.151:8080/
which is actually my local servers ip & port number.
/// Initialise AVPlayer Item
let url = URL(string: "http://10.155.177.151:8080/")!
let playerItem = AVPlayerItem(url: url)
player = AVPlayer(playerItem: playerItem)
player.automaticallyWaitsToMinimizeStalling = false
let playerViewController = AVPlayerViewController()
playerViewController.player = player
DispatchQueue.main.async {
self.present(playerViewController, animated: true) {
playerViewController.player!.play()
}
}
/// Http proxy handler
webServer.addDefaultHandler(forMethod: "GET", request: GCDWebServerRequest.self) { (request, completion) in
let mediaUrl = URL(string: "http://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4")!
if self.session == nil {
self.session = URLSession(configuration: URLSessionConfiguration.default)
}
let dataTask = self.session?.dataTask(with: mediaUrl, completionHandler: { (data, response, error) in
if let apiData = data, let apiResponse = response {
completion(GCDWebServerDataResponse(data: apiData, contentType: apiResponse.mimeType ?? ""))
} else {
completion(GCDWebServerDataResponse(text: "Error"))
}
})
dataTask!.resume()
}
Something similar has been implemented here: https://github.com/StyleShare/HLSCachingReverseProxyServer
It downloads the segments while streaming, if the segment is available locally, it will use that instead.

Load local encrypted video file Asset into AVPlayer

I want to try read the Data from a encrypted "video" file, decrypt the data and make it run on AVPlayer.
My app can download a video, the video comes encrypted from my API. All I have to do is: decrypt the file with a cypher and then it becomes a valid mp4 file.
Theses steps are covered. I did the process and did write the decrypted data in a file, played it on my machine and passed it with file URL to AVPlayer and it played as well.
However, I do not want have to save the decrypted file. I did a bit of research and came up with AVAssetResourceLoaderDelegate. Tried to implement it but without success. The examples I saw was hitting a online URL so I'm not entirely sure if its possible to do it local as I tried.
Any one can help me?
guard var components = URLComponents.init(url: video.url, resolvingAgainstBaseURL: false) else { return }
components.scheme = "encryptedVideo"
guard let url = components.url else { return }
let asset = AVURLAsset(url: fileDecryptedURL)
asset.resourceLoader.setDelegate(self, queue: DispatchQueue.global(qos: .background))
let playerItem = AVPlayerItem(asset: asset)
let player = AVPlayer(playerItem: playerItem)
player.actionAtItemEnd = .none
self.avPlayerViewController?.player = player
self.avPlayerViewController?.player?.play()
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
if let dataRequest = loadingRequest.dataRequest,
let url = loadingRequest.request.url,
let contentRequest = loadingRequest.contentInformationRequest {
guard var components = URLComponents.init(url: url, resolvingAgainstBaseURL: false) else { return false }
components.scheme = "file"
guard let localUrl = components.url else { return false }
let storageProvider = StorageVideo()
let dataMaybe = storageProvider.videoData(url: localUrl)
guard let encryptedData = dataMaybe,
let decryptedData = try? RNCryptor.decrypt(data: encryptedData,
withPassword: "omitted") else {
return false
}
contentRequest.contentType = AVFileType.mp4.rawValue
contentRequest.contentLength = Int64(decryptedData.count)
// contentRequest.isByteRangeAccessSupported = true
dataRequest.respond(with: decryptedData)
loadingRequest.finishLoading()
// Did this to save the file to see if it was ok. I could play the file in my machine
// If I pass this file url to a asset (above step) it load as well
// if let documentDirectory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first {
// let fileDecryptedURL = documentDirectory.appendingPathComponent("test").appendingPathExtension("mp4")
// try? decryptedData.write(to: fileDecryptedURL)
// }
}
return true
}

iOS download and play mp3 file

I am looking for the way how can I download and play mp3 file simultaneously.
I can download it, save to local storage and play after.
But, how can I start downloading and playing it simultaneously, and after it will completely download - save it to local storage. Which tools should I use for it?
Currently I use TCBlobDownload to download, after it I save it and AVAudioPlay to play.
You can use AVAssetResourceLoader to plays an audio file as soon as there is enough data while continuing to download.
Configure the delegate of resourceloader
var playerAsset: AVAsset!
if fileURL.pathExtension.count == 0 {
var components = URLComponents(url: fileURL, resolvingAgainstBaseURL: false)!
components.scheme = "fake" // make custom URL scheme
components.path += ".mp3"
playerAsset = AVURLAsset(url: components.url!)
(playerAsset as! AVURLAsset).resourceLoader.setDelegate(self, queue: DispatchQueue.global())
} else {
playerAsset = AVAsset(url: fileURL)
}
let playerItem = AVPlayerItem(asset: playerAsset)
then, read audio's data and responds to the resource loader
// MARK: - AVAssetResourceLoaderDelegate methods
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
if let url = loadingRequest.request.url {
var components = URLComponents(url: url, resolvingAgainstBaseURL: false)!
components.scheme = NSURLFileScheme // replace with the real URL scheme
components.path = String(components.path.dropLast(4))
if let attributes = try? FileManager.default.attributesOfItem(atPath: components.url!.path),
let fileSize = attributes[FileAttributeKey.size] as? Int64 {
loadingRequest.contentInformationRequest?.isByteRangeAccessSupported = true
loadingRequest.contentInformationRequest?.contentType = "audio/mpeg3"
loadingRequest.contentInformationRequest?.contentLength = fileSize
let requestedOffset = loadingRequest.dataRequest!.requestedOffset
let requestedLength = loadingRequest.dataRequest!.requestedLength
if let handle = try? FileHandle(forReadingFrom: components.url!) {
handle.seek(toFileOffset: UInt64(requestedOffset))
let data = handle.readData(ofLength: requestedLength)
loadingRequest.dataRequest?.respond(with: data)
loadingRequest.finishLoading()
return true
} else {
return false
}
} else {
return false
}
} else {
return false
}
}
And if you want to do this with objective c, refer this

Resources