Why is AVPlayer Boundary Time Observer not working? - ios

I am trying to observe a time in the timeline of my AVPlayer.
I tried this on the main queue; which did not work. I then switched to a background queue, as advised from this stack overflow post; which did not with either. Looking for a working solution or an explanation as to why this isn't working.
//add boundary time notification to global queue
avLayer.player!.addBoundaryTimeObserver(forTimes: [NSValue(time: avLayer.player!.currentItem!.duration)], queue: DispatchQueue.main){
self.avLayer.player!.pause()
}
//add boundary time notification to background queue
avLayer.player!.addBoundaryTimeObserver(forTimes: [NSValue(time: avLayer.player!.currentItem!.duration)], queue: DispatchQueue.global(qos: .userInteractive)){
self.avLayer.player!.pause()
}
Update: After retaining a strong reference to the return value of the observer, I set a breakpoint in the callback. It is still not working.
//add boundary time notification
boundaryTimeObserver = avLayer.player!.addBoundaryTimeObserver(forTimes: [NSValue(time: avLayer.player!.currentItem!.duration)], queue: DispatchQueue.main){
self.avLayer.player!.pause()
}

2019 simple example ..
var player = AVPlayer()
var token: Any?
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
let u = "https... "
let playerItem = AVPlayerItem(url: URL(string: u)!)
player = AVPlayer(playerItem: playerItem)
player.play()
token = player.addBoundaryTimeObserver(
forTimes: [0.5 as NSValue],
queue: DispatchQueue.main) { [weak self] in
self?.spinner.stopAnimating()
print("The audio is in fact beginning about now...")
}
}
Works perfectly.
Important .. it won't find "0"
Use a small value to find the "beginning" as a quick solution.

There may be two problems:
As the documentation for addBoundaryTimeObserver states:
You must maintain a strong reference to the returned value as long as you want the time observer to be invoked by the player
As your initial code does not keep a reference to the returned internal opaque time observer, the observer probably is released immediately and thus is never called.
Make sure the time you register for observing actually has the correct value:
playerItem.duration may be indefinite (see documentation of this property)
even the duration of the playerItem's asset may be unknown, or an in-precise estimation, depending on the type of the asset and loading state (again, see documentation of AVAsset.duration on this).
As a consequence, the time you register for observing may never be reached (note that the time can easily be checked by inserting a CMTimeShow(duration))
Approaches to resolve this:
if you just want to stop the player when the playerItem's end is reached, setting player.actionAtItemEnd to pause may be sufficient
if you need to execute some custom logic when the item's end is reached, register an observer for AVPlayerItemDidPlayToEndTime notifications with the playerItem as object. This mechanism is independent from possibly in-precise durations and so hopefully more reliable

Related

AVPlayerItem isPlaybackBufferFull always false

Trying to learn AVFoundation by debugging and understanding all properties of AVPlayerItem.
I've noticed something when I run a video playing on simulator (or device), that AVPlayerItem's property isPlaybackBufferFull is always false.
On the player view, I am printing all the properties and updating them thanks to KVO.
On the debug view, the loaded time ranges gets updated correctly. When it's fully loaded, the property isPlaybackBufferFull doesn't get set to true.
let newItem: AVPlayerItem = .init(url: url)
player.replaceCurrentItem(with: newItem)
self.isPlaybackBufferFullObserver = item.observe(\.isPlaybackBufferFull, changeHandler: self.onIsPlaybackBufferFullObserverChanged)
private func onIsPlaybackBufferFullObserverChanged(playerItem: AVPlayerItem, change: NSKeyValueObservedChange<Bool>) {
if playerItem.isPlaybackBufferFull {
// Breakpoint should normally stops here
}
}
Am I missing anything? Shouldn't isPlaybackBufferFull be true at some point?

Alamofire request blocking during application lifecycle

I'm having troubles with Alamofire using Operation and OperationQueue.
I have an OperationQueue named NetworkingQueue and I push some operation (wrapping AlamofireRequest) into it, everything works fine, but during application living, at one moment all Alamofire request are not sent. My queue is getting bigger and bigger and no request go to the end.
I do not have a scheme to reproduce it anytime.
Does anybody have a clue for helping me?
Here is a sample of code
The BackgroundAlamoSession
let configuration = URLSessionConfiguration.background(withIdentifier: "[...].background")
self.networkingSessionManager = Alamofire.SessionManager(configuration: configuration)
AbstractOperation.swift
import UIKit
import XCGLogger
class AbstractOperation:Operation {
private let _LOGGER:XCGLogger = XCGLogger.default
enum State:String {
case Ready = "ready"
case Executing = "executing"
case Finished = "finished"
var keyPath: String {
get{
return "is" + self.rawValue.capitalized
}
}
}
override var isAsynchronous:Bool {
get{
return true
}
}
var state = State.Ready {
willSet {
willChangeValue(forKey: self.state.rawValue)
willChangeValue(forKey: self.state.keyPath)
willChangeValue(forKey: newValue.rawValue)
willChangeValue(forKey: newValue.keyPath)
}
didSet {
didChangeValue(forKey: oldValue.rawValue)
didChangeValue(forKey: oldValue.keyPath)
didChangeValue(forKey: self.state.rawValue)
didChangeValue(forKey: self.state.keyPath)
}
}
override var isExecuting: Bool {
return state == .Executing
}
override var isFinished:Bool {
return state == .Finished
}
}
A concrete Operation implementation
import UIKit
import XCGLogger
import SwiftyJSON
class FetchObject: AbstractOperation {
public let _LOGGER:XCGLogger = XCGLogger.default
private let _objectId:Int
private let _force:Bool
public var object:ObjectModel?
init(_ objectId:Int, force:Bool) {
self._objectId = objectId
self._force = force
}
convenience init(_ objectId:Int) {
self.init(objectId, force:false)
}
override var desc:String {
get{
return "FetchObject(\(self._objectId))"
}
}
public override func start(){
self.state = .Executing
_LOGGER.verbose("Fetch object operation start")
if !self._force {
let objectInCache:objectModel? = Application.main.collections.availableObjectModels[self._objectId]
if let objectInCache = objectInCache {
_LOGGER.verbose("object with id \(self._objectId) founded on cache")
self.object = objectInCache
self._LOGGER.verbose("Fetch object operation end : success")
self.state = .Finished
return
}
}
if !self.isCancelled {
let url = "[...]\(self._objectId)"
_LOGGER.verbose("Requesting object with id \(self._objectId) on server")
Application.main.networkingSessionManager.request(url, method : .get)
.validate()
.responseJSON(
completionHandler: { response in
switch response.result {
case .success:
guard let raw:Any = response.result.value else {
self._LOGGER.error("Error while fetching json programm : Empty response")
self._LOGGER.verbose("Fetch object operation end : error")
self.state = .Finished
return
}
let data:JSON = JSON(raw)
self._LOGGER.verbose("Received object from server \(data["bId"])")
self.object = ObjectModel(objectId:data["oId"].intValue,data:data)
Application.main.collections.availableobjectModels[self.object!.objectId] = self.object
self._LOGGER.verbose("Fetch object operation end : success")
self.state = .Finished
break
case .failure(let error):
self._LOGGER.error("Error while fetching json program \(error)")
self._LOGGER.verbose("Fetch object operation end : error")
self.state = .Finished
break
}
})
} else {
self._LOGGER.verbose("Fetch object operation end : cancel")
self.state = .Finished
}
}
}
The NetworkQueue
class MyQueue {
public static let networkQueue:SaootiQueue = SaootiQueue(name:"NetworkQueue", concurent:true)
}
How I use it in another operation and wait for for result
let getObjectOperation:FetchObject = FetchObject(30)
SaootiQueue.networkQueue.addOperations([getObjectOperation], waitUntilFinished: true)
How I use it the main operation using KVO
let getObjectOperation:FetchObject = FetchObject(30)
operation.addObserver(self, forKeyPath: #keyPath(Operation.isFinished), options: [.new], context: nil)
operation.addObserver(self, forKeyPath: #keyPath(Operation.isCancelled), options: [.new], context: nil)
queue.addOperation(operation)
//[...]
override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
if let operation = object as? FetchObject {
operation.removeObserver(self, forKeyPath: #keyPath(Operation.isFinished))
operation.removeObserver(self, forKeyPath: #keyPath(Operation.isCancelled))
if keyPath == #keyPath(Operation.isFinished) {
//Do something
}
}
A few clarifications:
My application is a radio player and I need, while playing music and the background, to fetch the currently playing program. This is why I need background Session.
In fact I also use the background session for all the networking I do when the app is foreground. Should I avoid that ?
The wait I'm using is from another queue and is never used in the main queue (I know it is a threading antipattern and I take care of it).
In fact it is used when I do two networking operation and the second one depends of the result of the second. I put a wait after the first operation to avoid KVO observing. Should I avoid that ?
Additional edit:
When I say "My queue is getting bigger and bigger and no request go to the end", it means that at one moment during application livecycle, random for the moment (I can not find a way to reproduce it at every time), Alamofire request don't reach the response method.
Because of that the Operation wrapper don't end and the queue is growing.
By the way I'm working on converting Alamofire request into URLRequest for having clues and I founded some problem on using the main queue. I have to sort what is due to the fact that Alamofire use the main queue for reponse method and I'll see if I find a potential deadlock
I'll keep you informed. Thanks
There are minor issues, but this operation implementation looks largely correct. Sure, you should make your state management thread-safe, and there are other stylistic improvements you could make, but I don't think this is critical to your question.
What looks worrisome is addOperations(_:waitUntilFinished:). From which queue are you waiting? If you do that from the main queue, you will deadlock (i.e. it will look like the Alamofire requests never finish). Alamofire uses the main queue for its completion handlers (unless you override the queue parameter of responseJSON), but if you're waiting on the main thread, this can never take place. (As an aside, if you can refactor so you never explicitly "wait" for operations, that not only avoids the deadlock risk, but is a better pattern in general.)
I also notice that you're using Alamofire requests wrapped in operations in conjunction with a background session. Background sessions are antithetical to operations and completion handler closure patterns. Background sessions continue after your app has been jettisoned and you have to rely solely upon the SessionDelegate closures that you set when you first configure your SessionManager when the app starts. When the app restarts, your operations and completion handler closures are long gone.
Bottom line, do you really need background session (i.e. uploads and downloads that continue after your app terminates)? If so, you may want to lose this completion handler and operation based approach. If you don't need this to continue after the app terminates, don't use background sessions. Configuring Alamofire to properly handle background sessions is a non-trivial exercise, so only do so if you absolutely need to. Remember to not conflate background sessions and the simple asynchronous processing that Alamofire (and URLSession) do automatically for you.
You asked:
My application is a radio player and I need, while playing music and the background, to fetch the currently playing program. This is why I need background Session.
You need background sessions if you want downloads to proceed while the app is not running. If your app is running in the background, though, playing music, you probably don't need background sessions. But, if the user chooses to download a particular media asset, you may well want background session so that the download proceeds when the user leaves the app, whether the app is playing music or not.
In fact I also use the background session for all the networking I do when the app is foreground. Should I avoid that ?
It's fine. It's a little slower, IIRC, but it's fine.
The problem isn't that you're using background session, but that you're doing it wrong. The operation-based wrapping of Alamofire doesn't make sense with a background session. For sessions to proceed in the background, you are constrained as to how you use URLSession, namely:
You cannot use data tasks while the app is not running; only upload and download tasks.
You cannot rely upon completion handler closures (because the entire purpose of background sessions is to keep them running when your app terminates and then fire up your app again when they're done; but if the app was terminated, your closures are all gone).
You have to use delegate based API only for background sessions, not completion handlers.
You have to implement the app delegate method to capture the system provided completion handler that you call when you're done processing background session delegate calls. You have to call that when your URLSession tells you that it's done processing all the background delegate methods.
All of this is a significant burden, IMHO. Given that the system is keeping you app alive for background music, you might contemplate using a standard URLSessionConfiguration. If you're going to use background session, you might need to refactor all of this completion handler-based code.
The wait I'm using is from another queue and is never used in the main queue (I know it is a threading antipattern and I take care of it).
Good. There's still serious code smell from ever using "wait", but if you are 100% confident that it's not deadlocking here, you can get away with it. But it's something you really should check (e.g. put some logging statement after the "wait" and make sure you're getting past that line, if you haven't already confirmed this).
In fact it is used when I do two networking operation and the second one depends of the result of the second. I put a wait after the first operation to avoid KVO observing. Should I avoid that ?
Personally, I'd lose that KVO observing and just establish addDependency between the operations. Also, if you get rid of that KVO observing, you can get rid of your double KVO notification process. But I don't think this KVO stuff is the root of the problem, so maybe you defer that.

Swift: How to stop scheduleBuffer completion handler being called when interrupted?

I have a AVAudioPlayerNode that I'm scheduling a lot of buffers on using the following:
node.scheduleBuffer(buffer, at: nil, options: .interrupts, completionHandler: completeFunc)
But I'm having a bit of a problem. If I play another buffer interrupting the currently playing buffer, the completion handler for the buffer getting interrupted is still being called. I guess this makes sense, but I can't seem to find a way to check if the file actually COMPLETED playing or if it was interrupted. Is there a way I can do this?
Any answers will help!
You will need an interruption state variable that is referenced in the completion handler and toggled by the interrupting code. Here's some shorthand code that makes some assumptions on how you're looping through buffers:
var node: AVAudioPlayerNode!
var loopingBuffers: [AVAudioPCMBuffer] = [buffer1, buffer2, buffer3]
var interruptBuffers = false // state variable to handle the interruption
func scheduleNextBuffer() {
/* code to find the next buffer in the loopingBuffers queue */
// assuming you have instantiated an AVAudioPlayerNode elsewhere
node.scheduleBuffer(nextBuffer, completionHandler: bufferCompletion)
}
func bufferCompletion() {
// check the state variable to make certain the completionHandler isn't being triggered by a scheduling interruption
guard !interruptBuffers else { return }
scheduleNextBuffer()
}
func interruptBuffers() {
let newBuffers: [AVAudioPCMBuffer] = [buffer4, buffer5, buffer6]
interruptBuffers = true
node.scheduleBuffer(buffer4, at: nil, options: .interrupts, completionHandler: bufferCompletion)
// cleanup your state and queue up your new buffers
interruptBuffers = false
loopingBuffers = newBuffers
}

How to get animation to work at exact points during playback of a music file?

Question:
In Swift code, apart from using an NSTimer, how can I get animations
to start at exact points during playback of a music file played using AVFoundation?
Background
I have a method that plays a music file using AVFoundation (below). I also have UIView animations that I want to start at exact points during the music file being played.
One way I could achieve this is using an NSTimer, but that has the potential to get out of sync or not be exact enough.
Is there a method that I can tap into AVFoundation accessing the music file's time elapsed (time counter), so when certain points during the music playback arrive, animations start?
Is there an event / notification that AVFoundation triggers that gives a constant stream of time elapsed since the music file has started playing?
For example
At 0:52.50 (52 seconds and 1/2), call startAnimation1(), at 1:20.75 (1 minute, 20 seconds and 3/4), call startAnimation2(), and so on?
switch musicPlayingTimeElapsed {
case 0:52.50:
startAnimation1()
case 1:20.75:
startAnimation2()
default:
()
}
Playing music using AVFoundation
import AVFoundation
var myMusic : AVAudioPlayer?
func playMusic() {
if let musicFile = self.setupAudioPlayerWithFile("fileName", type:"mp3") {
self.myMusic = musicFile
}
myMusic?.play()
}
func setupAudioPlayerWithFile(file:NSString, type:NSString) -> AVAudioPlayer? {
let path = NSBundle.mainBundle().pathForResource(file as String, ofType: type as String)
let url = NSURL.fileURLWithPath(path!)
var audioPlayer:AVAudioPlayer?
do {
try audioPlayer = AVAudioPlayer(contentsOfURL: url)
} catch {
print("AVAudioPlayer not available")
}
return audioPlayer
}
If you use AVPlayer instead of AVAudioPlayer, you can use the (TBH slightly awkward) addBoundaryTimeObserverForTimes method:
let times = [
NSValue(CMTime:CMTimeMake(...)),
NSValue(CMTime:CMTimeMake(...)),
NSValue(CMTime:CMTimeMake(...)),
// etc
];
var observer: AnyObject? = nil // instance variable
self.observer = self.player.addBoundaryTimeObserverForTimes(times, queue: nil) {
switch self.player.currentTime() {
case 0:52.50:
startAnimation1()
case 1:20.75:
startAnimation2()
default:
break
}
}
// call this to stop observer
self.player.removeTimeObserver(self.observer)
The way I solve this is to divide the music up into separate segments beforehand. I then use one of two approaches:
I play the segments one at a time, each in its own audio player. The audio player's delegate is notified when a segment finishes, and so starting the next segment — along with accompanying action — is up to me.
Alternatively, I queue up all the segments onto an AVQueuePlayer. I then use KVO on the queue player's currentItem. Thus, I am notified exactly when we move to a new segment.
You might try using Key Value Observing to observe the duration property of your sound as it plays. When the duration reaches your time thresholds you'd trigger each animation. You'd need to make the time thresholds match times >= the trigger time, since you will likely not get a perfect match with your desired time.
I don't know how well that would work however. First, I'm not sure if the sound player's duration is KVO-compliant.
Next, KVO is somewhat resource-intensive, and if your KVO listener gets called thousands of times a second it might bog things down. It would at least be worth a try.

Why is AVPlayer not continuing to next song while in background?

I have been streaming music from remote source using AVPlayer. I get URLs, use one to create an AVPlayerItem, which i then associate with my instance of AVPlayer. I add an observer to the item that I associate with the player to observe when the item finishes playing ( AVPlayerItemDidPlayToEndTimeNotification ). When the observer notifies me at the item end, I then create a new AVPlayerItem and do it all over again. This works well in the foreground AND in the background on iOS 9.2.
Problem: Since I have updated to iOS 9.3 this does not work in the background. Here is the relevant code:
var portionToBurffer = Double()
var player = AVPlayer()
func prepareAudioPlayer(songNSURL: NSURL, portionOfSongToBuffer: Double){
self.portionToBuffer = portionOfSongToBuffer
//create AVPlayerItem
let createdItem = AVPlayerItem(URL: songNSURL)
//Associate createdItem with AVPlayer
player = AVPlayer(playerItem: createdItem)
//Add item end observer
NSNotificationCenter.defaultCenter().addObserver(self, selector: "playerItemDidReachEnd:", name: AVPlayerItemDidPlayToEndTimeNotification, object: player.currentItem)
//Use KVO to see how much is loaded
player.currentItem?.addObserver(self, forKeyPath: "loadedTimeRanges", options: .New, context: nil)
}
override func observeValueForKeyPath(keyPath: String?, ofObject object: AnyObject?, change: [String : AnyObject]?, context: UnsafeMutablePointer<Void>) {
if keyPath == "loadedTimeRanges" {
if let loadedRangeAsNSValueArray = player.currentItem?.loadedTimeRanges {
let loadedRangeAsCMTimeRange = loadedRangeAsNSValueArray[0].CMTimeRangeValue
let endPointLoaded = CMTimeRangeGetEnd(loadedRangeAsCMTimeRange)
let secondsLoaded = CMTimeGetSeconds(endPointLoaded)
print("the endPointLoaded is \(secondsLoaded) and the duration is \(CMTimeGetSeconds((player.currentItem?.duration)!))")
if secondsLoaded >= portionToBuffer {
player.currentItem?.removeObserver(self, forKeyPath: "loadedTimeRanges")
player.play()
}
}
}
}
func playerItemDidReachEnd(notification: NSNotification){
recievedItemEndNotification()
}
func recievedItemEndNotification() {
//register background task
bgTasker.registerBackgroundTask()
if session.playlistSongIndex == session.playlistSongTitles.count-1 {
session.playlistSongIndex = 0
} else {
session.playlistSongIndex += 1
}
prepareAudioPlayer(songURL: session.songURLs[session.playlistSongIndex], portionOfSongToBuffer: 30.00)
}
I have set breakpoints to see that player.play() IS being called when in the background. When i print player.rate it reads 0.0. I have checked the property playbackLikelyToKeepUp of the AVPlayerItem and it is true. I have confirmed also that the new URL is successfully used to create the new AVPlayerItem and associated with the AVPlayer when the app is in the background. I have turned audio and airplay background capabilities on and I have even opened up a finite length background task (in code above as bgTasker.registerBackgroundTask). No idea what is going on.
I found THIS but i'm not sure it helps. Any advice would be great, thanks
When the observer notifies me at the item end, I then create a new AVPlayerItem and do it all over again
But the problem is that meanwhile play stops, and the rule is that background playing is permitted only so long as you were playing in the foreground and continue to play in the background.
I would suggest using AVQueuePlayer instead of AVPlayer. This will allow you to enqueue the next item while the current item is still playing — and thus, we may hope, this will count as continuing to play.
I encountered the similar problem, and I searched lots of websites on google, but didn't find the answer.
The Phenomenon
The problem of my app is that when I start play an audio, and turn the app to background, it will finish the playing of the current audio, and when playing the second audio, it will load some data, but then it stopped, if I turn the app to foreground, it will play the audio.
Solution
My solution is to add the following call.
UIApplication.shared.beginReceivingRemoteControlEvents()
So enable background audio capabilities is not enough, we need to begin receiving remote controller events.

Resources