My iOS app uses an AVPlayer to stream media from the web. I'm using KVO to detect events like buffering (playbackBufferEmpty) and being caught up (playbackLikelyToKeepUp). However, I've noticed an eerily specific bug. Although during most network slowdowns, the playbackBufferEmpty KVO is hit, and when it recovers, the playbackLikelyToKeepUp KVO is hit as expected, I have noticed that (with some frequency) the AVPlayer will buffer for some period of time, recover, the playbackLikelyToKeepUp is hit, the audio plays for maybe 1-3 seconds, and then the audio stops playing again as if it were buffering—without hitting the playbackBufferEmpty KVO again. And it doesn't recover from this.
TL;DR: playbackBufferEmpty (10-20 seconds) -> playbackLikelyToKeepUp -> plays for 1-3s -> stops playing but no KVO is hit.
The worst part is when this happens, the AVPlayer does not auto-start playing again like it does when recovering from buffering, and the app is silent until the AVPlayer is manually stopped/started again. It happens more or less every time I stream, eventually. Is this a known issue with AVPlayer? Is this some third KVO event that I can watch for? Or any tips on how to debug this? Thanks!
EDIT: additional info:
The AVPlayer's error property is nil when this happens, the status property is Ready to Play, and the rate property is 1. AKA Nothing else weird going on, as far as I can tell.
I have also experienced issues with AVPlayer recovering from buffering / network problems, and I too have the exact same scenario where avPlayer.status is .readyToPlay, avPlayer.error is nil, and avPlayer.rate is 1. Talk about unreliable! I've also checked the errorLog() and that doesn't provide anything relevant other than a generic internet connection error.
I try to detect this state by having a timer which, when it times out, I check the current video time and the time when the timer started and if it's the same (video hasn't progressed), we can assume the video really isn't playing when it is claiming that it is.
Then I treat this as an unrecoverable state and reload the entire stream by creating a new AVPlayerItem.
Related
I'm making a metronome app and use AKSamplerMetronome class. I did synchronization between devices with Ableton link so they could play together. I start playing and after a while sound on the device starts to delay and after couple of minutes I can clearly hear it.
The thing is beat count synchronized correctly but the sound that comes from the phone is not.
This bug is not on all devices, on some of them it's working great, no delay.
for synchronization I call setBeatTime once on start
Any suggestions what it might be?
If I create instance of AKSamplerMetronome, wait for couple minutes and then play, sound immediately would be out of sync.
I am trying to build an iOS app which controls a music player which runs on a seperate machine. I would like to use the MPNowPlayingInfoCenter for inspecting and controlling this player. As far as I can tell so far, the app actually has to output audio for this to work (see also this answer).
However, for instance, the Spotify app is actually capable of doing this without playing audio on the iOS device. If you use Spotify Connect to play the audio on a different device, the MPNowPlayingInfoCenter still displays the correct song and the controls are functional.
What's the catch here? What does one (conceptually) have to do to achieve this? I can think of continuously emitting a "silent" audio stream, but that seams a bit brute-force.
Streaming silence will work, but you don't need to stream it all the time. Just long enough to send your Now Playing info. Using AVAudioPlayer, I've found approaches as short as this will send the data (assuming the player is loaded with a 1s silent audio file):
player.play()
let nowPlayingInfoCenter = MPNowPlayingInfoCenter.default()
nowPlayingInfoCenter.nowPlayingInfo = [...]
player.stop()
I was very surprised this worked within a single event loop. Any other approach to playing silence seems to work as well. Again, it can be very brief (in fact the above code in my tests doesn't even get as far as playing the silence).
I'm very interested in whether this works reliably for other people, so please comment if you make discoveries about it.
I've explored the Spotify app a bit. I'm not 100% certain if this is the same technique they're using. They're able to mix with other audio somehow. So you can be playing local audio on the phone and also playing Spotify Connect to some other device, and the "Now Playing" info will kind of stomp on each other. For my purposes, that would actually be better, but I haven't been able to duplicate it. I still have to make the audio session non-mixable for at least a moment (so you get ~ 1s audio drop if you're playing local audio at the same time). I did find that the Spotify app was not highly reliable about playing to my Connect device when I was also playing other audio locally. Sometimes it would get confused and switch around where it wanted to play. (Admittedly this is a bizarre use case; I was intentionally trying to break it to figure out how they're doing it.)
EDIT: I don't believe Spotify is actually doing anything special here. I think they just play silent audio. I think getting two streams playing at the same time has to do with AirPlay rather than Spotify (I can't make it happen with Bluetooth, only AirPlay).
I am using MPMusicPlayerController so my app can play music that the user has bought through iTunes. When I select a song and start to play, there is a lag before the sound starts. I am assuming that the song is being buffered from the cloud.
The problem is that I have not found a way to know when the buffering is complete and the audio actually starts.
To play the song I use:
_mediaController = [MPMusicPlayerController applicationMusicPlayer];
[_mediaController setQueueWithItemCollection:collection];
[_mediaController beginGeneratingPlaybackNotifications];
[_mediaController play];
As soon as I call "play", the playback state change notification is called and the playback state is "MPMusicPlaybackStatePlaying" even though the user can't hear any music. I've noticed that even though the mediaController is in the "playing" playback state, the _mediaController.currentPlaybackTime always equals 0 until the music can be heard at which time the currentPlaybackTime properly tracks with the music.
I thought I could use the [_mediaController prepareToPlay] method to preload the audio file, but when I use that, the MPMediaPlaybackIsPreparedToPlayDidChangeNotification notification is never called. So the mediaController is never marked as "prepared".
With all this being said, I have not found a way to prebuffer songs using the MPMusicPlayerController. I know this issue has been around for a while because there is an old question from a few years ago with essentially the same issue, but no answer. Does anyone know how to make this work?
MPMediaPlaybackIsPreparedToPlayDidChangeNotification looks deprecated.
The MPMediaPlayerController getters and notifications are kind of garbage, and aren't "consistent" at all in the way where you set a value and expect the same value to come back when you grab it again.
I solved this by "buffering" the song first, as my app frequently will start in the middle of a song. So my buffering algo is - I'll play the song, then wait for the playback state changed notification, then pause it again and wait for another notifcation. This process will without a doubt trigger the MPMusicPlayerControllerNowPlayingItemDidChangeNotification, then finally the song is ready to play or be mutated (set currentTime or Rate). This seems to work very well.
The prepareToPlay completion handler is also garbage. It seems to fire before the song is actually ready to play, and the method actually seems to start playback :( , which is more than it leads on. This seems to be commonly reported as a "bug" in the apple dev forums.
In the prepareToPlay callback, setting currentPlaybackTime or Rate won't actually mutate the player - you need to wait for the additional MPMusicPlayerControllerNowPlayingItemDidChangeNotification sometime after playback starts for the first time on the song before you'll have any luck mutating any of the player properties.
currentPlaybackRate and Time also aren't very reliable, although they are more reliable once playback actually starts. I'll cache it to what the user sets it to until playback starts to solve the consistency problem. There is a lot more here to get the currentPlaybackRate or time consistently, let me know if you want a code sample, as grabbing those properties for reading will yield different results depending on the executing thread :(
Checking the playback state isn't reliable either unfortunately. Often I've found that the player reports MPMusicPlaybackStatePlaying when its fully paused and not playing - it stays like this indefinitely in certain situations. I recommend abstracting out any determinations if the MPMediaPlayerController is actually playing based on calls to play or pause and a subsequent follow-up of a MPMusicPlayerControllerPlaybackStateDidChangeNotification confirming the player has started that action.
I should mention I'm on iOS 12.1.3, as Apple seems to intermittently fix a small bug or add more API breakages as time goes on. Since its pretty broken now, any improvements may break any abstraction layer you build to fix it - so I'm testing each iOS release to make sure everything still "works".
Hope this helps a bit, its a real struggle.
I have a 3rd party SDK that handles an audio recording. It has a callback when the recording starts. In the callback I'm trying to play a sound to indicate to the user that the device is now listening (like Siri, or any other speech recognition tends to do), but when I try I get the following error:
AURemoteIO::ChangeHardwareFormats: error -10875
I have tried playing the sound using AudioServicesPlaySystemSound as well as an AVAudioPlayer both with the same result. The sound plays fine at other times, and per the error my assumption is there's an incompatibility between the playback and recording on the hardware level. Can anyone clarify this error, or give me a hint as to a possible workaround?
Make sure that the Audio Session is initialised and configured for Play_and_Record before you start the RemoteIO Audio Unit recording.
You shouldn't and likely can't start playing a sound in a RemoteIO recording callback. Only set a boolean flag in the callback to indicate that a sound should be played. Play your sound from the main UI run loop.
My problem is related specifically to the external SDK and how they handle the audio interface. They override everything when you ask the SDK to start recording, if you take control back you break the recording session. So within the context of that SDK there's no way to work around it unless they fix the SDK.
I am using AVPlayer to play an audio stream, and it's possible to keep it playing in the background. I'm wondering how could I handle a situtation where the user loses internet connectivity, so that I could provide some feedback or maybe try to re-establish the playback after some seconds.
EDIT: I know that the question regards AVPlayer, but if you have an answer with MPMoviePlayerController it might be useful as well. Right now, by using MPMoviePlayerController, I'm trying to get the MPMovieFinishReasonPlaybackError case of the MPMoviePlayerPlaybackDidFinishReasonUserInfoKey, by subscribing to the MPMoviePlayerPlaybackDidFinishNotification but if f.e. my audio is playing in the background and I turn airplane mode on, I never get this notification; I only get MPMovieFinishReasonPlaybackEnded, and I don't know how to separate that from the case that the user stops the audio himself.
I tried looking around for the actual source but I remember reading somewhere that if the audio playback stops (for whatever reason) it kills the background thread. The person writing about the issue talked about possible feeding the stream some empty audio content to keep the thread alive. You might be able to send a local notification from a call back error notifying the user that the audio experienced an error and will have to be manually restarted from within the application. Haven't played around with the API enough to know which callback is the best one to use in this case. If I find the link I'm looking for I'll update.
EDIT: Here's Grant Pannell's take on audio streaming and multitasking.