What is the recommended way for handling error in AVAssetResourceLoaderDelegate? specifically network requests in shouldWaitForLoadingOfRequestedResource / shouldWaitForRenewalOfRequestedResource?
However, there is no way for my app to know something went wrong until I start playing the video. I tried loadingRequest.finishLoading(with: err) and return false in the shouldWaitForLoadingOfRequestedResource delegate call, but nothing was reported until the download finishes and the user starts playing the video.
the code goes like this..
let urlAsset = AVURLAsset(url: url)
asset?.resourceLoader.preloadsEligibleContentKeys = true
asset?.resourceLoader.setDelegate(self.fairPlayAssetLoaderDelegate, queue: DispatchQueue.global(qos: .default)) // fairPlayAssetLoaderDelegate is an object that implements AVAssetResourceLoaderDelegate
// send urlAsset to AVAssetDownloadURLSession.makeAssetDownloadTask
after setting preloadsEligibleContentKeys = true, it seems like shouldWaitForLoadingOfRequestedResource is called right away after setting the resource loader delegate in AVURLAsset. but there is no way to tell the resource loader has successfully got all the information it wants.
I tried using loadValuesAsynchronously on AVURLAsset, but that's a no go either.
AVAssetDownloadURLSession doesn't always report the error either..
Thanks in advance.
Related
I am trying to find a way to pause any playing media on the device, so I was thinking of triggering the same logic that is fired when a user press the headphone "middle button"
I managed to prevent music from resuming (after I pause it within my app, which basically start an AVAudioSession for recording) by NOT setting the AVAudioSession active property to false and leave it hanging, but I am pretty sure thats a bad way to do it. If I deactivate it the music resumes. The other option I am thinking of is playing some kind of silent loop that would "imitate" the silence I need to do. But I think if what I am seeking is doable, it would be the best approach as I understood from this question it cannot be done using the normal means
func stopAudioSession() {
let audioSession = AVAudioSession.sharedInstance(
do {
if audioSession.secondaryAudioShouldBeSilencedHint{
print("someone is playing....")
}
try audioSession.setActive(false, options: .notifyOthersOnDeactivation)
isSessionActive = false
} catch let error as NSError {
print("Unable to deactivate audio session: \(error.localizedDescription)")
print("retying.......")
}
}
In this code snippet as the function name implies I set active to false, tried to find other options but I could not find another way of stopping my recording session and prevent resume of the other app that was already playing
If someone can guide me to which library I should look into, if for example I can tap into the H/W part and trigger it OR if I can find out which library is listening to this button press event and handling the pause/play functionality
A friend of mine who is more experienced in IOS development suggested the following workaround and it worked - I am posting it here as it might help someone trying to achieve a similar behaviour.
In order to stop/pause what is currently being played on a user device, you will need to add a music player into your app. then at the point where you need to pause/stop the current media, you just initiate the player, play and then pause/stop it - simple :)
like so:
let musicPlayer = MPMusicPlayerApplicationController.applicationQueuePlayer
func stopMedia(){
MPMediaLibrary.requestAuthorization({(newPermissionStatus: MPMediaLibraryAuthorizationStatus) in
self.musicPlayer.setQueue(with: .songs())
self.musicPlayer.play()
print("Stopping music player")
self.musicPlayer.pause()
print("Stopped music player")
})
}
the part with MPMediaLibrary.requestAuthorization is needed to avoid an authorisation error when accessing user's media library.
and of course you will need to add the Privacy - Media Library Usage Description
key into your Info.plist file
I am developing a Voice to Text application using iOS SFSpeechRecognizer API.
Found a great tutorial here: and it worked fine.
I wanted to process the text and perform some action as soon as the voice input is stopped. So, was curious whether there is a delegate method available for SFSpeechRecognizer which can recognise when the voice input is stopped so that I can capture the input and process further?
So, was curious whether there is a delegate method available for SFSpeechRecognizer which can recognise when the voice input is stopped so that I can capture the input and process further?
Not built into the SFSpeechRecognizer API, no. On the contrary, that is exactly why you must provide interface that allows the user to tell the recognizer that the input is finished (e.g. a Done button of some sort). Your app will be rejected if you omit that interface.
A possible solution maybe to use a third party library like FDSoundActivatedRecorder which start recording when sound is detected and
stops recording when the user is done talking.
Then you can use the recorded audio as in this link to convert it to text in a go.
func transcribeAudio(url: URL) {
// create a new recognizer and point it at our audio
let recognizer = SFSpeechRecognizer()
let request = SFSpeechURLRecognitionRequest(url: url)
// start recognition!
recognizer?.recognitionTask(with: request) { [unowned self] (result, error) in
// abort if we didn't get any transcription back
guard let result = result else {
print("There was an error: \(error!)")
return
}
// if we got the final transcription back, print it
if result.isFinal {
// pull out the best transcription...
print(result.bestTranscription.formattedString)
}
}
}
I am writing an iOS app in Swift for HLS live streaming. I want my app logic to be notified when each HLS segment request is initiated during playback (and what the respective URL is). I have tried to observe changes to various properties of AVPlayer and AVPlayerItem using KVO. Though, it only informs me of when the playback is initiated. For example, adding the following observer triggers a invocation of observeValue method when playback starts, but I have not found a way to be continuously notified of each segment request.
playerItem.addObserver(self, forKeyPath: "status", options:NSKeyValueObservingOptions(), context: nil)
Is there a method with KVO that should allow me to be notified of each segment request? Are there other objects/API:s not related to AVFoundation that I should consider?
/George
I follow Fabian's approach when I need to debug HLS streams. It shows useful info every time that there is an update related to the current stream being played. Here's the code I use, hope it helps anyone facing a similar issue!
func trackPlayerLogs() {
NotificationCenter.default.addObserver(self,
selector: #selector(handleAVPlayerAccess),
name: NSNotification.Name.AVPlayerItemNewAccessLogEntry,
object: nil)
}
func handleAVPlayerAccess(notification: Notification) {
guard let playerItem = notification.object as? AVPlayerItem,
let lastEvent = playerItem.accessLog()?.events.last else {
return
}
let indicatedBitrate = lastEvent.indicatedBitrate
print("--------------PLAYER LOG--------------")
print("EVENT: \(lastEvent)")
print("INDICATED BITRATE: \(indicatedBitrate)")
print("PLAYBACK RELATED LOG EVENTS")
print("PLAYBACK START DATE: \(lastEvent.playbackStartDate)")
print("PLAYBACK SESSION ID: \(lastEvent.playbackSessionID)")
print("PLAYBACK START OFFSET: \(lastEvent.playbackStartOffset)")
print("PLAYBACK TYPE: \(lastEvent.playbackType)")
print("STARTUP TIME: \(lastEvent.startupTime)")
print("DURATION WATCHED: \(lastEvent.durationWatched)")
print("NUMBER OF DROPPED VIDEO FRAMES: \(lastEvent.numberOfDroppedVideoFrames)")
print("NUMBER OF STALLS: \(lastEvent.numberOfStalls)")
print("SEGMENTS DOWNLOADED DURATION: \(lastEvent.segmentsDownloadedDuration)")
print("DOWNLOAD OVERDUE: \(lastEvent.downloadOverdue)")
print("--------------------------------------")
}
I don't know of an easy way to be notified of each segment request while it's happening. You should look at AVPlayerItem's accessLog property and look at the AVPlayerItemAccessLogEvents in the log. These will describe both network and playback events. I highly recommend this approach if it fits your needs.
Another way is to set up your app as an HTTP server and point an AVURLAsset/AVPLayerItem at the local server which will then have to translate those requests to an external server. This is much more complex, difficult, error-prone, and is nearly guaranteed to have bad performance. Please do not do this.
addendum:
You may be tempted to look at AVAssetResourceLoaderDelegate as it says you can handle resource requests on behalf of an AVURLAsset. Unfortunately it does not go through the loader for segments. It seems to be for playlists, decryption keys and other such assets.
I'm having trouble with an app I'm building, the app objective is to play audio files, it works by requesting an audio file from a public API, playing it and wait until it ends, after it ends it requests another audio and starts over.
Here's a shortened version of the code that does this, I omitted the error checking part for simplicity
func requestEnded(audioSource: String) {
let url = URL(string: "https://example.com/audios/" + audioSource)
audio = AVPlayer(url: url!)
NotificationCenter.default.addObserver(self,selector: #selector(MediaItem.finishedPlaying(_:)), name: NSNotification.Name.AVPlayerItemDidPlayToEndTime, object: audio?.currentItem)
audio?.play()
}
#objc func finishedPlaying(_ notification: NSNotification) {
print("Audio ended")
callAPI()
}
func callAPI() {
// do all the http request stuff
requestEnded(audioSource: "x.m4a")
}
// callAPI() is called when the app is initialized
It works well when the screen is unlocked. When I lock the phone the current audio keeps playing but when it ends finishedPlaying() never gets called (the print statement is not shown on the console).
How can I make it so the app would know the audio ended and trigger another one all while locked?, In the android version I got around the screenlock problem by using a partial wakelock which made it run normally even with the screen off.
It has to be done this way because the API decides the audio on realtime and it's all done on the backend so no way to buffer more than one audio without breaking the requirements of the app
What are my options here?, any help is appreciated.
My end goal is to make http requests in the background while handling the response in memory. From my understanding, background requests have to be made with a custom delegate (which means I can't use dataTaskWithRequest(request, completionHandler)), and to deal with the response in memory I have to use a data task (which means I can't use a download task along with URLSession(session, downloadTask, didFinishDownloadingToURL)).
According to this: https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/URLLoadingSystem/NSURLSessionConcepts/NSURLSessionConcepts.html#//apple_ref/doc/uid/10000165i-CH2-SW1 it doesn't look like there are any delegate methods that get called for data tasks upon completion. Is the only way to work with the response through a delegate to work with individual NSData fragments through URLSession(session, dataTask, data)? https://developer.apple.com/library/ios/documentation/Foundation/Reference/NSURLSessionDataDelegate_protocol/index.html#//apple_ref/occ/intfm/NSURLSessionDataDelegate/URLSession:dataTask:didReceiveData: There's no delegate method for handling the entire final response as a single NSData instance?
For small quantities of data like API calls or small images like avatars meant to be displayed immediately, dataTaskWithRequest(request, completionHandler) is the method you want. It sets up an asynchronous task, meaning that when you start the task, execution will return to your code immediately and the task will take care of downloading the data and buffering it in memory "in the background" while your app is running. Once all the download task is complete, it will call your completionHandler to let it know that it's done and that the data is ready. It will pass the data to your handler as an argument.
For larger files like podcasts, videos and large images, you'll want iOS to download the file for you even when the user starts looking at another app and your app is suspended. You will then want to use a NSURLSessionDownloadTask with a background session configuration backgroundSessionConfigurationWithIdentifier: and a custom delegate. Your custom delegate will need to implement the method URLSession:downloadTask:didFinishDownloadingToURL:. When this method is called, you can read the contents of the file at the url that will be passed to you using code like this:
let data = NSData(contentsOfURL: url)
The reason background downloads that persist after an iOS app is quit are handled like this is that iOS wants to be able to continue downloading multiple files on behalf of different apps like podcasts, videos, etc. If the user is on a high speed network, downloading multiple large files in memory can quickly consume all of the devices memory, so they get stored as they are downloaded instead. In the same vein, you should keep your file sizes in mind before reading the entire file into memory with NSData(contentsOfURL:).
Here's a working example of how everything fits together. Paste this in an iOS playground and look at the image you'll get:
import UIKit
class MyDelegate: NSObject, NSURLSessionDelegate, NSURLSessionDownloadDelegate {
func URLSession(session: NSURLSession, downloadTask: NSURLSessionDownloadTask, didFinishDownloadingToURL location: NSURL) {
if let data = NSData(contentsOfURL: location) {
// work with data ...
UIImage(data: data)
}
}
}
let configuration = NSURLSessionConfiguration.backgroundSessionConfigurationWithIdentifier("my-session-identifier")
let delegate = MyDelegate()
let session = NSURLSession(configuration: configuration, delegate: delegate, delegateQueue: nil)
let url = NSURL(string: "https://pbs.twimg.com/profile_images/473550724433858560/tuHsaI2U.png")!
let task = session.downloadTaskWithURL(url)
task.resume()
// this allows the code to run in a playground
import XCPlayground
XCPlaygroundPage.currentPage.needsIndefiniteExecution = true
If I correctly understand the docs you're supposed to implement the
func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?)
method from URLSessionTaskDelegate. It "tells the delegate that the task finished transferring data" (per docs).
It will be called with nil value for error, but you also have to check task.response for potential problems.
Before that you should manually collect pieces of your data coming via the URLSession:dataTask:didReceiveData: calls:
This delegate method may be called more than once, and each call provides only data received since the previous call. The app is responsible for accumulating this data if needed.
There's no delegate methods that would combine all the data into a single object for you, you have to do it manually.