AVAssetResourceLoaderDelegate implementation for an asset of unknown length - ios

Question

My iOS app uses AVPlayer to play streaming audio from my server and storing it on a device.
I implemented AVAssetResourceLoaderDelegate, so I could intercept the stream. I change my scheme (from http to a fake scheme, so that AVAssetResourceLoaderDelegate method gets called:
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool
I followed this tutorial:
http://blog.jaredsinclair.com/post/149892449150/implementing-avassetresourceloaderdelegate-a
Over there, I put the original scheme back, and create a session for pulling audio from the server. Everything works perfectly when my server provides the Content-Length (size of the audio file in bytes) header for the streamed audio file.
But sometimes I stream audio files where I cannot provide their length ahead of time (let's say a live podcast stream). In this case, AVURLAsset sets length to -1 and fails with:
"Error Domain=AVFoundationErrorDomain Code=-11849 \"Operation Stopped\" UserInfo={NSUnderlyingError=0x61800004abc0 {Error Domain=NSOSStatusErrorDomain Code=-12873 \"(null)\"}, NSLocalizedFailureReason=This media may be damaged., NSLocalizedDescription=Operation Stopped}"
And I cannot bypass this error. I tried to go a hacky way, provide fake
Content-Length: 999999999, but in this case, once the entire audio stream is downloaded, my session fails with:
Loaded so far: 10349852 out of 99999999
The request timed out.
//Audio file got downloaded, its size is 10349852
//AVPlayer tries to get the next chunk and then fails with request times out
Have anyone ever faced this problem before?
P.S. If I keep original http scheme in AVURLAsset, AVPlayer knows how to handle this scheme, so it plays audio file just fine (even w/o Content-Length), I do not know how it does that w/o failing. Also, in this case, my AVAssetResourceLoaderDelegate is never used, so I cannot intercept and copy the content of the audio file to a local storage.
Here is the implementation:
import AVFoundation
#objc protocol CachingPlayerItemDelegate {
// called when file is fully downloaded
#objc optional func playerItem(playerItem: CachingPlayerItem, didFinishDownloadingData data: NSData)
// called every time new portion of data is received
#objc optional func playerItemDownloaded(playerItem: CachingPlayerItem, didDownloadBytesSoFar bytesDownloaded: Int, outOf bytesExpected: Int)
// called after prebuffering is finished, so the player item is ready to play. Called only once, after initial pre-buffering
#objc optional func playerItemReadyToPlay(playerItem: CachingPlayerItem)
// called when some media did not arrive in time to continue playback
#objc optional func playerItemDidStopPlayback(playerItem: CachingPlayerItem)
// called when deinit
#objc optional func playerItemWillDeinit(playerItem: CachingPlayerItem)
}
extension URL {
func urlWithCustomScheme(scheme: String) -> URL {
var components = URLComponents(url: self, resolvingAgainstBaseURL: false)
components?.scheme = scheme
return components!.url!
}
}
class CachingPlayerItem: AVPlayerItem {
class ResourceLoaderDelegate: NSObject, AVAssetResourceLoaderDelegate, URLSessionDelegate, URLSessionDataDelegate, URLSessionTaskDelegate {
var playingFromCache = false
var mimeType: String? // is used if we play from cache (with NSData)
var session: URLSession?
var songData: NSData?
var response: URLResponse?
var pendingRequests = Set<AVAssetResourceLoadingRequest>()
weak var owner: CachingPlayerItem?
//MARK: AVAssetResourceLoader delegate
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest) -> Bool {
if playingFromCache { // if we're playing from cache
// nothing to do here
} else if session == nil { // if we're playing from url, we need to download the file
let interceptedURL = loadingRequest.request.url!.urlWithCustomScheme(scheme: owner!.scheme!).deletingLastPathComponent()
startDataRequest(withURL: interceptedURL)
}
pendingRequests.insert(loadingRequest)
processPendingRequests()
return true
}
func startDataRequest(withURL url: URL) {
let request = URLRequest(url: url)
let configuration = URLSessionConfiguration.default
configuration.requestCachePolicy = .reloadIgnoringLocalAndRemoteCacheData
configuration.timeoutIntervalForRequest = 60.0
configuration.timeoutIntervalForResource = 120.0
session = URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
let task = session?.dataTask(with: request)
task?.resume()
}
func resourceLoader(_ resourceLoader: AVAssetResourceLoader, didCancel loadingRequest: AVAssetResourceLoadingRequest) {
pendingRequests.remove(loadingRequest)
}
//MARK: URLSession delegate
func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) {
(songData as! NSMutableData).append(data)
processPendingRequests()
owner?.delegate?.playerItemDownloaded?(playerItem: owner!, didDownloadBytesSoFar: songData!.length, outOf: Int(dataTask.countOfBytesExpectedToReceive))
}
func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive response: URLResponse, completionHandler: #escaping (URLSession.ResponseDisposition) -> Void) {
completionHandler(URLSession.ResponseDisposition.allow)
songData = NSMutableData()
self.response = response
processPendingRequests()
}
func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError err: Error?) {
if let error = err {
print(error.localizedDescription)
return
}
processPendingRequests()
owner?.delegate?.playerItem?(playerItem: owner!, didFinishDownloadingData: songData!)
}
//MARK:
func processPendingRequests() {
var requestsCompleted = Set<AVAssetResourceLoadingRequest>()
for loadingRequest in pendingRequests {
fillInContentInforation(contentInformationRequest: loadingRequest.contentInformationRequest)
let didRespondCompletely = respondWithDataForRequest(dataRequest: loadingRequest.dataRequest!)
if didRespondCompletely {
requestsCompleted.insert(loadingRequest)
loadingRequest.finishLoading()
}
}
for i in requestsCompleted {
pendingRequests.remove(i)
}
}
func fillInContentInforation(contentInformationRequest: AVAssetResourceLoadingContentInformationRequest?) {
// if we play from cache we make no URL requests, therefore we have no responses, so we need to fill in contentInformationRequest manually
if playingFromCache {
contentInformationRequest?.contentType = self.mimeType
contentInformationRequest?.contentLength = Int64(songData!.length)
contentInformationRequest?.isByteRangeAccessSupported = true
return
}
// have no response from the server yet
if response == nil {
return
}
let mimeType = response?.mimeType
contentInformationRequest?.contentType = mimeType
if response?.expectedContentLength != -1 {
contentInformationRequest?.contentLength = response!.expectedContentLength
contentInformationRequest?.isByteRangeAccessSupported = true
} else {
contentInformationRequest?.isByteRangeAccessSupported = false
}
}
func respondWithDataForRequest(dataRequest: AVAssetResourceLoadingDataRequest) -> Bool {
let requestedOffset = Int(dataRequest.requestedOffset)
let requestedLength = dataRequest.requestedLength
let startOffset = Int(dataRequest.currentOffset)
// Don't have any data at all for this request
if songData == nil || songData!.length < startOffset {
return false
}
// This is the total data we have from startOffset to whatever has been downloaded so far
let bytesUnread = songData!.length - Int(startOffset)
// Respond fully or whaterver is available if we can't satisfy the request fully yet
let bytesToRespond = min(bytesUnread, requestedLength + Int(requestedOffset))
dataRequest.respond(with: songData!.subdata(with: NSMakeRange(startOffset, bytesToRespond)))
let didRespondFully = songData!.length >= requestedLength + Int(requestedOffset)
return didRespondFully
}
deinit {
session?.invalidateAndCancel()
}
}
private var resourceLoaderDelegate = ResourceLoaderDelegate()
private var scheme: String?
private var url: URL!
weak var delegate: CachingPlayerItemDelegate?
// use this initializer to play remote files
init(url: URL) {
self.url = url
let components = URLComponents(url: url, resolvingAgainstBaseURL: false)!
scheme = components.scheme
let asset = AVURLAsset(url: url.urlWithCustomScheme(scheme: "fakeScheme").appendingPathComponent("/test.mp3"))
asset.resourceLoader.setDelegate(resourceLoaderDelegate, queue: DispatchQueue.main)
super.init(asset: asset, automaticallyLoadedAssetKeys: nil)
resourceLoaderDelegate.owner = self
self.addObserver(self, forKeyPath: "status", options: NSKeyValueObservingOptions.new, context: nil)
NotificationCenter.default.addObserver(self, selector: #selector(didStopHandler), name:NSNotification.Name.AVPlayerItemPlaybackStalled, object: self)
}
// use this initializer to play local files
init(data: NSData, mimeType: String, fileExtension: String) {
self.url = URL(string: "whatever://whatever/file.\(fileExtension)")
resourceLoaderDelegate.songData = data
resourceLoaderDelegate.playingFromCache = true
resourceLoaderDelegate.mimeType = mimeType
let asset = AVURLAsset(url: url)
asset.resourceLoader.setDelegate(resourceLoaderDelegate, queue: DispatchQueue.main)
super.init(asset: asset, automaticallyLoadedAssetKeys: nil)
resourceLoaderDelegate.owner = self
self.addObserver(self, forKeyPath: "status", options: NSKeyValueObservingOptions.new, context: nil)
NotificationCenter.default.addObserver(self, selector: #selector(didStopHandler), name:NSNotification.Name.AVPlayerItemPlaybackStalled, object: self)
}
func download() {
if resourceLoaderDelegate.session == nil {
resourceLoaderDelegate.startDataRequest(withURL: url)
}
}
override init(asset: AVAsset, automaticallyLoadedAssetKeys: [String]?) {
fatalError("not implemented")
}
// MARK: KVO
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
delegate?.playerItemReadyToPlay?(playerItem: self)
}
// MARK: Notification handlers
func didStopHandler() {
delegate?.playerItemDidStopPlayback?(playerItem: self)
}
// MARK:
deinit {
NotificationCenter.default.removeObserver(self)
removeObserver(self, forKeyPath: "status")
resourceLoaderDelegate.session?.invalidateAndCancel()
delegate?.playerItemWillDeinit?(playerItem: self)
}
}

Answer 1

You can not handle this situation as for iOS this file is damaged because header is incorrect. System think that you are going to play regular audio file but it doesn't have all info about it. You don't know what audio duration will be, only if you have a live streaming. Live streaming on iOS is done using HTTP live streaming protocol.
Your iOS code is correct. You have to modify your backend and provide m3u8 playlist for live streaming audios, then iOS will accept it as a live stream and audio player will start tracks.
Some related info can be found here. As an iOS developer with good experience in streaming audio / video I can tell you that code to play live / VOD is the same.

Answer 2

But sometimes I stream audio files where I cannot provide their length ahead of time (let's say a live podcast stream). In this case, AVURLAsset sets length to -1 and fails with
In this scenario you should let the player re-request this data later on and set renewalDate property of contentInformationRequest for the given part to some point in future when this data will be available.
If it's just an inifinte live stream, you always provide the length of aquired portion, and set new renewDate for the next renewal cycle (according to my observation natively AVPlayer just updates this data with fixed period of time, say, every 4-6 seconds). The server usually provides such information with "Expires" http header. You can rely on this information yourself and implement something like this (borrowed from my own question on apple developers forum):
if let httpResonse = response as? HTTPURLResponse, let expirationValue = httpResonse.value(forHTTPHeaderField: "Expires") {
let dateFormatter = DateFormatter()
dateFormatter.locale = Locale(identifier: "en_US_POSIX")
dateFormatter.dateFormat = "EEE, dd MMM yyyy HH:mm:ss zzz"
if let expirationDate = dateFormatter.date(from: expirationValue) {
let renewDate = max(expirationDate, Date(timeIntervalSinceNow: 8))
contentInformationRequest.renewalDate = renewDate
}
}
This line let renewDate = max(expirationDate, Date(timeIntervalSinceNow: 8)) adds 8 seconds grace period for the
player to load videos. Otherwise it does not keep up with the pace of
renewals, and video loads in poor quality.
Or just update it periodically if you know in advance it's a live asset without fixed length and your server doesn't privde the required information:
contentInformationRequest.renewalDate = Date(timeIntervalSinceNow: 8)