Handling AVPlayer stalls - ios

I'm trying to catch a moment when AVPlayer is unable to continue playback in case no more media available (too slow network, signal loss, etc). As described in documentation and different examples I'm using KVO to detect this:
item = [[AVPlayerItem alloc] initWithURL:audioURL];
player = [AVPlayer playerWithPlayerItem:item];
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(onItemNotification:) name:AVPlayerItemPlaybackStalledNotification object:item];
[item addObserver:self forKeyPath:#"playbackBufferEmpty" options:NSKeyValueObservingOptionNew context:nil];
[item addObserver:self forKeyPath:#"playbackLikelyToKeepUp" options:NSKeyValueObservingOptionNew context:nil];
...
- (void) onItemNotification:(NSNotification*)not
{
NSLog(#"Item notification: %#", not.name);
}
...
- (void) observeValueForKeyPath:(NSString*)keyPath ofObject:(id)object change:(NSDictionary*)change context:(void*)context
{
NSLog(#"Observe keyPath %#", keyPath);
}
I'm starting a playback and turn WiFi off after that. Unfortunately neither 'playbackBufferEmpty' nor 'AVPlayerItemPlaybackStalledNotification' comes. At the moment when playback stops I receive only one AVPlayerItemTimeJumpedNotification and that's all.
However there were at least 2 times when I got these notifications. But I can't figure out how to get them every time when playback is stalled.
Am I doing something wrong?

First try to disconnect internet from your router and you will get playbackBufferEmpty notification.
To handle network switching you will need to implemented Reachability

There are 2 cases where player can get stuck: it never starts or it runs out of buffered data. I use the following to handle both cases:
When you create a AVPlayer instance:
[_player addObserver:self forKeyPath:#"rate" options:0 context:nil];
[_player.currentItem addObserver:self forKeyPath:#"status" options:0 context:nil];
This is the handler:
-(void)observeValueForKeyPath:(NSString*)keyPath
ofObject:(id)object
change:(NSDictionary*)change
context:(void*)context {
if ([keyPath isEqualToString:#"status"]) {
if (_player.status == AVPlayerStatusFailed) {
//Failed to start.
//Description from the docs:
// Indicates that the player can no longer play AVPlayerItem instances because of an error. The error is described by
// the value of the player's error property.
}
} else if ([keyPath isEqualToString:#"rate"]) {
if (_player.rate == 0 && //playback rate is 0
CMTIME_COMPARE_INLINE(_player.currentItem.currentTime, >, kCMTimeZero) && //video has started
CMTIME_COMPARE_INLINE(_player.currentItem.currentTime, <, _player.currentItem.duration) && //video hasn't reached the end
_isPlaying) { //instance variable to track playback state
//Video stalled. Possible connection loss or connection is too slow.
}
}
}
Don't forget to remove observers when you are done:
[_player.currentItem removeObserver:self forKeyPath:#"status"];
[_player removeObserver:self forKeyPath:#"rate"];
See my answer here to see how I handle stalled video: AVPlayer stops playing and doesn't resume again

Related

IOS Detect call interruption while streaming video

I'm streaming video using AVPlayer and i want to pause the video when GSM call comes and resume when the call will end and control is back to my app. How can i achieve this?
I figured it out, i did it by using AVAudioSessionInterruptionNotification. Below is code snippet
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(routeInterrypt:) name:AVAudioSessionInterruptionNotification object:nil];
-(void)routeInterrypt:(NSNotification *)notification {
NSDictionary *dic = notification.userInfo;
int changeReason= [dic[AVAudioSessionRouteChangeReasonKey] intValue];
if (changeReason == AVAudioSessionRouteChangeReasonUnknown) {
if (_state == TYVideoPlayerStateContentPlaying || _state == TYVideoPlayerStateBuffering) {
[self pauseContent];
return;
}
} }

Memory Leak: AVplayerViewController

I am facing memory leaks when we play a video and return back to the parent window. See the screenshot below of readings from allocations tool. Every time, when I pop the view controller (showing video) there are some objects related to AVFoundation holding the memory.
Interestingly the responsible library of all these objects is AVFoundation. None of the increase in memory is due to objects created in the APP. It is very highly unlikely that there is some problem with such a popular framework. I saw a few examples of AVPlayerViewController over the web but they seem to have the same problem.
Does anyone have any idea what/where is the problem? If anyone wants to replicate this then he can download any of the 2 projects given above. You have to make minor changes in the storyboard for creating root view controller with the navigation controller.
http://www.modejong.com/blog/post13_iOS8_SunSpot/index.html
https://github.com/coolioxlr/PageView-AVPlayer
This is how I am clearing the memory:
-(void) dealloc{
[self clearCurrentVideo];
}
-(void)clearCurrentVideo {
[_playerItem removeObserver:self forKeyPath:#"status"];
[_currentVideoPlayerViewController.player removeObserver:self forKeyPath:#"rate"];
[_currentVideoPlayerViewController.player pause];
_currentVideoPlayerViewController.delegate=nil;
_currentVideoPlayerViewController.player=nil;
_playerItem=nil;
_currentVideoPlayerViewController = nil;
}
This is how I load the asset for videos:
-(void)playtheAsset:(AVAsset *)asset{
[asset loadValuesAsynchronouslyForKeys:#[#"playable"] completionHandler:
^{
dispatch_async( dispatch_get_main_queue(),
^{
[self loadTheAsset:asset withKeys:#[#"playable"]];
});
}];
}
- (void)loadTheAsset:(AVAsset *)asset withKeys:(NSArray *)requestedKeys{
/* Make sure that the value of each key has loaded successfully. */
for (NSString *thisKey in requestedKeys)
{
NSError *error = nil;
AVKeyValueStatus keyStatus = [asset statusOfValueForKey:thisKey error:&error];
if (keyStatus == AVKeyValueStatusFailed)
{
//[self assetFailedToPrepareForPlayback:error];
if([thisKey isEqualToString:#"playable"]){
[self showNetworkErrorLabel];
}
return;
} else if ((keyStatus == AVKeyValueStatusLoaded) || ( keyStatus == AVKeyValueStatusLoading )){
[self removeNetworkLabel ];
}
}
/* Use the AVAsset playable property to detect whether the asset can be played. */
if (!asset.playable)
{
/* Generate an error describing the failure. */
NSString *localizedDescription = NSLocalizedString(#"Item cannot be played", #"Item cannot be played description");
NSString *localizedFailureReason = NSLocalizedString(#"The assets tracks were loaded, but could not be made playable.", #"Item cannot be played failure reason");
NSDictionary *errorDict = [NSDictionary dictionaryWithObjectsAndKeys:
localizedDescription, NSLocalizedDescriptionKey,
localizedFailureReason, NSLocalizedFailureReasonErrorKey,
nil];
NSError *assetCannotBePlayedError = [NSError errorWithDomain:#"StitchedStreamPlayer" code:0 userInfo:errorDict];
NSLog(#"%#",assetCannotBePlayedError);
[self showNetworkErrorLabel];
/* Display the error to the user. */
[self assetFailedToPrepareForPlayback:assetCannotBePlayedError];
return;
}
/* At this point we're ready to set up for playback of the asset. */
/* Stop observing our prior AVPlayerItem, if we have one. */
if (_playerItem)
{
/* Remove existing player item key value observers and notifications. */
[_playerItem removeObserver:self forKeyPath:#"status"];
[[NSNotificationCenter defaultCenter] removeObserver:self
name:AVPlayerItemDidPlayToEndTimeNotification
object:_playerItem];
[[NSNotificationCenter defaultCenter] removeObserver:self
name:AVPlayerItemPlaybackStalledNotification
object:_playerItem];
}
/* Create a new instance of AVPlayerItem from the now successfully loaded AVAsset. */
_playerItem = [AVPlayerItem playerItemWithAsset:asset];
/* Observe the player item "status" key to determine when it is ready to play. */
[_playerItem addObserver:self
forKeyPath:#"status"
options:NSKeyValueObservingOptionInitial | NSKeyValueObservingOptionNew
context:AVPlayerDemoPlaybackViewControllerStatusObservationContext];
/* When the player item has played to its end time we'll toggle
the movie controller Pause button to be the Play button */
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(playerItemDidReachEnd:)
name:AVPlayerItemDidPlayToEndTimeNotification
object:_playerItem];
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(playerItemFailedToPlayToEndTime:) name:AVPlayerItemPlaybackStalledNotification object:_playerItem];
// Remove the movie player view controller from the "playback did finish" notification observers
// Observe ourselves so we can get it to use the crossfade transition
[[NSNotificationCenter defaultCenter] removeObserver:_currentVideoPlayerViewController
name:kPlayerViewDismissedNotification
object:_currentVideoPlayerViewController.player];
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(videoFinishedCallback:)
name:kPlayerViewDismissedNotification
object:_currentVideoPlayerViewController.player];
/* Create new player, if we don't already have one. */
if (!_currentVideoPlayerViewController.player)
{
/* Get a new AVPlayer initialized to play the specified player item. */
_currentVideoPlayerViewController.player=[AVPlayer playerWithPlayerItem:self->_playerItem];
[_currentVideoPlayerViewController.player addObserver:self
forKeyPath:#"rate"
options:NSKeyValueObservingOptionNew
context:AVPlayerDemoPlaybackViewControllerRateObservationContext];
}
}
I couldn't figure out the reason behind this. I tried using AVPlayer instead and created my own UI using (reference Apple AVPlayer Demo app.) and I couldn't find any leak there. It just worked.
If someone gets stuck with similar problem just give a try to reference code from AVPlayer Demo app.
And if someone knows the answer for the issue I on this thread. Please let me know.

How to get a non null userInfo after the first time to my #selector(method)

I have a code like the following:
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(itemPlayEnded:) name:AVPlayerItemDidPlayToEndTimeNotification object:nil];
[[NSNotificationCenter defaultCenter] postNotificationName:AVPlayerItemDidPlayToEndTimeNotification object:nil userInfo:userInfo];
In the selector method:
- (void)itemPlayEnded:(NSNotification *)notification
{
NSLog(#"Entered itemPlayEnded");
AVPlayerItem *p = [notification object];
NSLog(#"userinfo description %#",[[notification userInfo] description]);
}
The first time the itemPlayEnded is accessed the userInfo is non-null. After the first time the value is null.
Can anybody tell me why this is happening? Why null value for userInfo after the first time?
EDIT:
I need to clarify what is happening. I also updated my Notification code to use the lastItem which is a AVPlayerItem.
queuePlayer = [[AVQueuePlayer alloc] init];
NSDictionary *userInfo = #{#"tones": copyoftones};
for (NSString *playThis in listOfTonesToBePlayed) {
NSString *soundPath =[[NSBundle mainBundle] pathForResource:playThis ofType:#"mp3"];
NSURL *soundURL = [NSURL fileURLWithPath:soundPath];
AVPlayerItem *thePlayerItemA = [[AVPlayerItem alloc] initWithURL:soundURL];
lastItem = thePlayerItemA;
[queuePlayer insertItem:thePlayerItemA afterItem:nil];
}
queuePlayer.actionAtItemEnd = AVPlayerActionAtItemEndAdvance;
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(itemPlayEnded:) name:AVPlayerItemDidPlayToEndTimeNotification object:lastItem];
[[NSNotificationCenter defaultCenter] postNotificationName:AVPlayerItemDidPlayToEndTimeNotification object:lastItem userInfo:userInfo];
[queuePlayer play];
What happens.
before the queuePlayer is able to play a tone the itemPlayEnded is entered along with the Dictionary non nil
Next the list of tones are played
The itemPlayEnded is re-entered with the Dictionary nil.
I wanted to use code to reset the Notification inside of the itemPlayEnded method with something like the following (p is a AVPlayerItem which is the lastItem from the code with the Notification):
[[NSNotificationCenter defaultCenter] removeObserver:self name:AVPlayerItemDidPlayToEndTimeNotification object:p];
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(itemPlayEnded:) name:AVPlayerItemDidPlayToEndTimeNotification object:p];
[[NSNotificationCenter defaultCenter] postNotificationName:AVPlayerItemDidPlayToEndTimeNotification object:p userInfo:userInfo];
[queuePlayer play];
But then the itemPlayEnded method is re-entered in a never-ending loop without playing the player. A race condition.
Any suggestions?
EDIT 2:
I have determined that the offending code is:
[[NSNotificationCenter defaultCenter] postNotificationName:AVPlayerItemDidPlayToEndTimeNotification object:lastItem userInfo:userInfo];
some how I need to set the object to use the correct sender. Not sure yet on what to put there.
EDIT 3:
What I really wanted to do was be able to loop thru the complete sequence of sounds or mp3s. I finally figured that what I was trying to do here was not going to work like I wanted it. So, I ended up using the class from https://github.com/dgiovann/AVQueuePlayerPrevious and this worked out great for me!
You are subscribing to the AVPlayerItemDidPlayToEndTimeNotification which is being posted first by yourself (with some user info) and then posted again, this time NOT by you and therefore may or may not contain any user info. Refer to the documentation to see when and why Apple will post this notification and what information you can expect with it.
https://developer.apple.com/library/mac/documentation/AVFoundation/Reference/AVPlayerItem_Class/Reference/Reference.html

How does app like heard works?

I want to make app which continuously records voice in background. App like heard does exactly that.
But I am stuck on interruptions. The app fail to resume recording when it faces interruptions.
I have used this for notification :
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(handleAudioSessionInterruption:)
name:AVAudioSessionInterruptionNotification
object:nil];
and to resume recording I have:
- (void)handleAudioSessionInterruption:(NSNotification*)notification {
NSNumber *interruptionType = [[notification userInfo] objectForKey:AVAudioSessionInterruptionTypeKey];
NSNumber *interruptionOption = [[notification userInfo] objectForKey:AVAudioSessionInterruptionOptionKey];
switch (interruptionType.unsignedIntegerValue) {
case AVAudioSessionInterruptionTypeBegan:{
// • Audio has stopped, already inactive
// • Change state of UI, etc., to reflect non-playing state
NSLog(#"Audio has stopped, already inactive");
NSLog(#"is recording: %#",recorder.isRecording?#"yes":#"no");
[self.recorder pause];
NSLog(#"is recording after pause: %#",recorder.isRecording?#"yes":#"no");
//[recorder pause];
} break;
case AVAudioSessionInterruptionTypeEnded:{
// • Make session active
// • Update user interface
// • AVAudioSessionInterruptionOptionShouldResume option
if (interruptionOption.unsignedIntegerValue == AVAudioSessionInterruptionOptionShouldResume) {
// Here you should continue playback.
//NSLog(#"completed");
NSLog(#"start recording again %d");
//NSLog(#"completed: %d",[recorder record]);
// Set the audio file
[self.recorder record];
NSLog(#"is recording after again : %#",recorder.isRecording?#"yes":#"no");
//[player play];
}
} break;
default:
break;
}
}
Add in the plist the UIBackgroundModes an 'audio' as Item 0. Hope it will help - at least it works for me when playing audio.

Why is AVPlayer failing with 'playbackBufferEmpty' status instantly after 'playbackLikelyToKeepUp'?

I have an audio stream that will play, then KVO observes playbackLikelyToKeepUp twice, then stops playing instantly observing playbackBufferEmpty twice.
Firstly I don't see why observeValueForKeyPath should be called twice for each message. And secondly, the presence of playbackLikelyToKeepUp should indicate playbackBufferEmpty shouldn't happen.
I should note this is happening in the simulator and the device.
- (void)setRetryConnectionTimer:(NSTimer *)newTimer
{
if (_retryConnectionTimer) { // Take care of releasing old Timers here to make things easy.
NSLog(#"[Timer Setter] Timer Exists. Invalidating and releasing.");
[_retryConnectionTimer invalidate];
[_retryConnectionTimer release];
_retryConnectionTimer = nil;
}
_retryConnectionTimer = [newTimer retain];
}
- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object
change:(NSDictionary *)change context:(void *)context {
if (object == self.audioStream && [keyPath isEqualToString:#"status"])
{
if(self.audioStream.status == AVPlayerStatusReadyToPlay)
{
//AVPlayer ready. Kill retry timer.
if(self.retryConnectionTimer){
if([self.retryConnectionTimer isValid]){
NSLog(#"Timer is valid. invalidating and setting to nil.");
[self.retryConnectionTimer invalidate];
self.retryConnectionTimer = nil;
}
}
NSLog(#"Buffering..."); // Allow the buffer to fill some by delaying play message.
[self.audioPlayer performSelector:#selector(play) withObject:nil afterDelay:kBufferSize];
}
}
if ([keyPath isEqualToString:#"playbackBufferEmpty"]){
NSLog(#"BUFFER EMPTY!!!"); // Buffer empty.... why when playbackShouldKeep up was just sent milliseconds ago.
_retries = 0;
[self.audioPlayer pause];
self.retryConnectionTimer = [NSTimer scheduledTimerWithTimeInterval:20.0f
target:self
selector:#selector(tryReconnect:)
userInfo:nil
repeats:YES];
}
if ([keyPath isEqualToString:#"playbackLikelyToKeepUp"]){
NSLog(#"playbackLikelyToKeepUp");
}
}
- (void)tryReconnect:(NSTimer *)sender {
NSLog(#"tryReconnect Called. Retry: %i", _retries);
if (_retries <= kReconnectRetries) {
[self restartStream];
_retries ++;
} else {
NSLog(#"Connection Dropped: invalidating Timer.");
[self.retryConnectionTimer invalidate];
self.retryConnectionTimer = nil;
_retries = 0;
}
}
- (void)restartStream
{
[self.audioStream removeObserver:self forKeyPath:#"status"];
[self.audioStream removeObserver:self forKeyPath:#"playbackBufferEmpty"];
[self.audioStream removeObserver:self forKeyPath:#"playbackLikelyToKeepUp"];
self.audioStream = [AVPlayerItem playerItemWithURL:[NSURL URLWithString:#"http://myaudiostream.com:9094"]];
[self.audioStream addObserver:self forKeyPath:#"status" options:0 context:nil];
[self.audioStream addObserver:self forKeyPath:#"playbackBufferEmpty" options:NSKeyValueObservingOptionNew context:nil];
[self.audioStream addObserver:self forKeyPath:#"playbackLikelyToKeepUp" options:NSKeyValueObservingOptionNew context:nil];
NSLog(#"Playing audioPlayer with new AVPlayerItem");
self.audioPlayer = [AVPlayer playerWithPlayerItem:self.audioStream];
}
From the logs.
2014-01-04 13:32:48.121 Audio Test [833:70b] Playing audioPlayer with new AVPlayerItem
2014-01-04 13:32:55.749 Audio Test [833:70b] Buffering...
2014-01-04 13:32:55.752 Audio Test [833:70b] playbackLikelyToKeepUp
2014-01-04 13:36:00.266 Audio Test [833:70b] playbackLikelyToKeepUp
2014-01-04 13:36:00.267 Audio Test [833:70b] BUFFER EMPTY!!!
2014-01-04 13:36:00.267 Audio Test [833:70b] Creating new timer and assigning to self.retryConnectionTimer
2014-01-04 13:36:03.701 Audio Test [833:70b] BUFFER EMPTY!!!
2014-01-04 13:36:03.702 Audio Test [833:70b] Creating new timer and assigning to self.retryConnectionTimer
2014-01-04 13:36:03.702 Audio Test [833:70b] [Timer Setter] Timer Exists. Invalidating and releasing.
2014-01-04 13:36:07.171 Audio Test [833:70b] playbackLikelyToKeepUp
2014-01-04 13:36:23.697 Audio Test [833:70b] tryReconnect Called. Retry: 0
2014-01-04 13:36:23.698 Audio Test [833:70b] Playing audioPlayer with new AVPlayerItem
2014-01-04 13:36:34.002 Audio Test [833:70b] Buffering...
2014-01-04 13:36:34.002 Audio Test [833:70b] Timer is valid. invalidating and setting to nil.
2014-01-04 13:36:34.002 Audio Test [833:70b] [Timer Setter] Timer Exists. Invalidating and releasing.
Are you actually checking the result? The playbackLikelyToKeepUp is a BOOL so it could very well be NO.

Resources