How to use thread to play next song by MPRemoteCommandCenter? - ios

I want to use MPRemoteCommandCenter to control my music player app. And now it can play and pause music, but can not play next/previous song, only a poor chance can make it.
When user tap next song button in MPRemoteCommandCenter(e.g in the lock screen), it will call startExtendBGJob() function, then I ask for a thread to do the change song job(I think the bug is here, because I'm don't totally understand the background job's anatomy).
func startExtendBGJob(taskBlock: #escaping () -> Void) {
registerBackgroundTask()
DispatchQueue.global(qos: .userInitiated).async {
DLog("APP into BG")
DispatchQueue.global(qos: .default).async {
taskBlock()
}
while self.isPlaying == false || self.tmpPlayer == nil { // waiting for new avplayer been created.
Thread.sleep(forTimeInterval: 1)
}
DispatchQueue.global(qos: .default).asyncAfter(deadline: .now() + 10) {
self.endBackgroundTask()
}
}
}
func registerBackgroundTask() {
bgIdentifier = UIApplication.shared.beginBackgroundTask(expirationHandler: {
[weak self] in
guard let strongSelf = self else { return }
strongSelf.endBackgroundTask()
})
assert(bgIdentifier != UIBackgroundTaskInvalid)
}
func endBackgroundTask() {
UIApplication.shared.endBackgroundTask(bgIdentifier)
bgIdentifier = UIBackgroundTaskInvalid
isExtendingBGJob = false
DLog("App exit BG!")
}
In startNextPlay() function just finding the next song's url, and prepareToPlay() is for creating a new AVPlayer to play next song.
self.tmpPlayer = AVPlayer(url: streamURL)
I'm not english native spearker, thank you so much to read here if you understand what I'm talking about :]. Any help is welcome.

Sorry, It's my fault. It doesn't need any backgournd job.
Just change something like following:
UIApplication.shared.beginReceivingRemoteControlEvents()
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
try AVAudioSession.sharedInstance().setActive(true)
} catch {
DLog("fail to set category: \(error)")
}
Make sure it's AVAudioSessionCategoryPlayback

Related

AVAudioPlayer 'forgets' that its playing when triggered by MPRemoteCommand

I'm trying to play an audiofile and control it's playback via the Remote Command Centre available on the lock screen.
If I do the following:
Begin playback
Pause playback
Lock device
Begin playback from lockscreen (MPRemoteCommandCenter)
It is then impossible to pause playback from lockscreen. The button flickers and nothing happens.
How can I fix this?
Further details below:
It appears that when attempting to pause the audio, the AVAudioPlayer returns 'false' for audioPlayer.isPlaying.
This occurs on iOS13.1 on my iPhoneXR, iPhoneSE and iPhone8. I have no other devices to test against
The logs indicate that AVAudioPlayer.isPlaying initially returns true when playback is started, but subsequently returns false. The player's currentTime also appears stuck at around the time playback was started.
My entire view controller is below (~100 lines). This is the minimum necessary to reproduce the problem.
The this example project demonstrating the error is also available on Github here.
import UIKit
import MediaPlayer
class ViewController: UIViewController {
#IBOutlet weak var playPauseButton: UIButton!
#IBAction func playPauseButtonTap(_ sender: Any) {
if self.audioPlayer.isPlaying {
pause()
} else {
play()
}
}
private var audioPlayer: AVAudioPlayer!
private var hasPlayed = false
override func viewDidLoad() {
super.viewDidLoad()
let fileUrl = Bundle.main.url(forResource: "temp/intro", withExtension: ".mp3")
try! self.audioPlayer = AVAudioPlayer(contentsOf: fileUrl!)
let audioSession = AVAudioSession.sharedInstance()
do { // play on speakers if headphones not plugged in
try audioSession.overrideOutputAudioPort(AVAudioSession.PortOverride.speaker)
} catch let error as NSError {
print("Override headphones failed, probably because none are available: \(error.localizedDescription)")
}
do {
try audioSession.setCategory(.playback, mode: .spokenAudio)
try audioSession.setActive(true)
} catch let error as NSError {
print("Warning: Setting audio category to .playback|.spokenAudio failed: \(error.localizedDescription)")
}
playPauseButton.setTitle("Play", for: .normal)
}
func play() {
playPauseButton.setTitle("Pause", for: .normal)
self.audioPlayer.play()
if(!hasPlayed){
self.setupRemoteTransportControls()
self.hasPlayed = true
}
}
func pause() {
playPauseButton.setTitle("Play", for: .normal)
self.audioPlayer.pause()
}
// MARK: Remote Transport Protocol
#objc private func handlePlay(event: MPRemoteCommandEvent) -> MPRemoteCommandHandlerStatus {
print(".......................")
print(self.audioPlayer.currentTime)
let address = Unmanaged.passUnretained(self.audioPlayer).toOpaque()
print("\(address) not playing: \(!self.audioPlayer.isPlaying)")
guard !self.audioPlayer.isPlaying else { return .commandFailed }
print("attempting to play")
let success = self.audioPlayer.play()
print("play() invoked with success \(success)")
print("now playing \(self.audioPlayer.isPlaying)")
return success ? .success : .commandFailed
}
#objc private func handlePause(event: MPRemoteCommandEvent) -> MPRemoteCommandHandlerStatus {
print(".......................")
print(self.audioPlayer.currentTime)
let address = Unmanaged.passUnretained(self.audioPlayer).toOpaque()
print("\(address) playing: \(self.audioPlayer.isPlaying)")
guard self.audioPlayer.isPlaying else { return .commandFailed }
print("attempting to pause")
self.pause()
print("pause() invoked")
return .success
}
private func setupRemoteTransportControls() {
let commandCenter = MPRemoteCommandCenter.shared()
commandCenter.playCommand.addTarget(self, action: #selector(self.handlePlay))
commandCenter.pauseCommand.addTarget(self, action: #selector(self.handlePause))
var nowPlayingInfo = [String : Any]()
nowPlayingInfo[MPMediaItemPropertyAlbumTitle] = "Major title"
nowPlayingInfo[MPMediaItemPropertyTitle] = "Minor Title"
nowPlayingInfo[MPNowPlayingInfoPropertyElapsedPlaybackTime] = self.audioPlayer.currentTime
nowPlayingInfo[MPMediaItemPropertyPlaybackDuration] = self.audioPlayer.duration
nowPlayingInfo[MPNowPlayingInfoPropertyPlaybackRate] = self.audioPlayer.rate
MPNowPlayingInfoCenter.default().nowPlayingInfo = nowPlayingInfo
}
}
This logs the following (with my // comments added):
.......................
1.438140589569161 // audio was paused here
0x0000000283361cc0 not playing: true // player correctly says its not playing
attempting to play // so it'll start to play
play() invoked with success true // play() successfully invoked
now playing true // and the player correctly reports it's playing
.......................
1.4954875283446711 // The player thinks it's being playing for about half a second
0x0000000283361cc0 playing: false // and has now paused??? WTF?
.......................
1.4954875283446711 // but there's definitely sound coming from the speakers. It has **NOT** paused.
0x0000000283361cc0 playing: false // yet it thinks it's paused?
// note that the memory addresses are the same. This seems to be the same player. ='(
I'm at my wits' end. Help me StackOverflow—You're my only hope.
Edits: I've also tried
Always returning .success
#objc private func handlePlay(event: MPRemoteCommandEvent) -> MPRemoteCommandHandlerStatus {
guard !self.audioPlayer.isPlaying else { return .success }
self.audioPlayer.play()
return .success
}
#objc private func handlePause(event: MPRemoteCommandEvent) -> MPRemoteCommandHandlerStatus {
print(self.audioPlayer.isPlaying)
guard self.audioPlayer.isPlaying else { return .success }
self.pause()
return .success
}
Ignoring the audioPlayer state and just doing as the remote command centre says
#objc private func handlePlay(event: MPRemoteCommandEvent) -> MPRemoteCommandHandlerStatus {
self.audioPlayer.play()
MPNowPlayingInfoCenter.default().nowPlayingInfo?[MPNowPlayingInfoPropertyElapsedPlaybackTime] = self.audioPlayer.currentTime
MPNowPlayingInfoCenter.default().nowPlayingInfo?[MPNowPlayingInfoPropertyPlaybackRate] = self.audioPlayer.rate
return .success
}
#objc private func handlePause(event: MPRemoteCommandEvent) -> MPRemoteCommandHandlerStatus {
self.audioPlayer.pause()
MPNowPlayingInfoCenter.default().nowPlayingInfo?[MPNowPlayingInfoPropertyElapsedPlaybackTime] = self.audioPlayer.currentTime
MPNowPlayingInfoCenter.default().nowPlayingInfo?[MPNowPlayingInfoPropertyPlaybackRate] = self.audioPlayer.rate
return .success
}
Both of these result in the bug persisting the first time pause is tapped on the lock screen. Subsequent taps reset the audio to the original paused position and then work normally.
Update
Replacing AVAudioPlayer with AVPlayer appears to make the problem go away entirely! I'm reasonably sure this is a bug in AVAudioPlayer now.
The necessary steps to switch to AVPlayer are in this public diff
I've used one of my developer-question tickets and submitted this question to Apple. I'll post an answer when I hear back from them.
Update 2
Apple Dev Support confirmed that as of 5 Dec 2019 there's no known workaround for this issue. I've submitted an issue to feedbackassistant.apple.com and will update this answer when something changes.
This is indeed a bug in AVAudioPlayer.
Another workaround if you dont want to switch to AVPlayer is to simply check if playing before pausing and if not, call play just before pause. It's not pretty but it works:
if (!self.player.isPlaying) [self.player play];
[self.player pause];

How to handle network interruption in AVPlayer

I have using reachability to determine network change but once I determine I am trying to reload item but it is not working
// Check if the playback could keep up after a network interruption
private func checkNetworkInterruption() {
guard
let item = playerItem,
!item.isPlaybackLikelyToKeepUp,
reachability?.connection != .unavailable else { return }
self.player?.pause()
// Wait 1 sec to recheck and make sure the reload is needed
DispatchQueue.main.asyncAfter(deadline: DispatchTime.now() + 1) {
if !item.isPlaybackLikelyToKeepUp {
self.reloadItem()
}
self.isPlaying ? self.playSong() : self.player?.pause()
}
}
private func reloadItem() {
player?.replaceCurrentItem(with: nil)
player?.replaceCurrentItem(with: playerItem)
}
AVPlayer can handle the network interrupts by itself, when network fails, the video is paused and keep in buffering status when network is stable, the video is resumed.
but if you need to handle the network interruption, you can try:
private func checkNetworkInterruption() {
guard
let item = playerItem,
!item.isPlaybackLikelyToKeepUp,
reachability?.connection != .unavailable else { return }
self.player?.pause()
// Wait 1 sec to recheck and make sure the reload is needed
DispatchQueue.main.asyncAfter(deadline: DispatchTime.now() + 1) {
if !item.isPlaybackLikelyToKeepUp {
//you can create a new AVItem here
player?.replaceCurrentItem(with: item)
}
self.isPlaying ? self.playSong() : self.player?.pause()
}
}

duckOthers Audio Session Interrupted by Remote Push Notification

I have a workout app that plays short clips of sound every couple of seconds. I have background music enabled so that music from other apps can be played while working out. The problem arises when I get a remote push notification (in my case, Slack) that has a sound, which somehow cancels out my duckingOther audio session and the music from other apps becomes loud again.
Question - How do I reset my duckingOthers audiosession when the user gets this type of interruption?
I set the audio session by calling the below function in didFinishLaunchingWithOptions:
private func setupAudioSession(){
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, with: [.mixWithOthers, .duckOthers, .interruptSpokenAudioAndMixWithOthers])
print("AVAudioSession Category Playback OK")
do {
try AVAudioSession.sharedInstance().setActive(true)
print("AVAudioSession is Active")
} catch let error as NSError {
print(error.localizedDescription)
}
} catch let error as NSError {
print(error.localizedDescription)
}
}
I have tried treating this as a hard interruption (for example a phone call), but when trying to apply the techniques used for this type of interruption, it seems that remote push notifications pass through the cracks. Below is what I used from a different question to try and catch interruptions.
#objc func handleInterruption(notification: Notification) {
guard let userInfo = notification.userInfo,
let typeValue = userInfo[AVAudioSessionInterruptionTypeKey] as? UInt,
let type = AVAudioSessionInterruptionType(rawValue: typeValue) else {
return
}
if type == .began {
// Interruption began, take appropriate actions
print("interruption started")
}
else if type == .ended {
if let optionsValue = userInfo[AVAudioSessionInterruptionOptionKey] as? UInt {
let options = AVAudioSessionInterruptionOptions(rawValue: optionsValue)
if options.contains(.shouldResume) {
// Interruption Ended - playback should resume
print("should resume playback")
setupAudioSession()
} else {
// Interruption Ended - playback should NOT resume
print("should not resume playback")
}
}
}
}
func setupNotifications() {
let notificationCenter = NotificationCenter.default
notificationCenter.addObserver(self,
selector: #selector(handleInterruption),
name: .AVAudioSessionInterruption,
object: nil)
}

Dispatching delegate callback to main queue, from another queue, AVFoundation

I am working on a camera that is wrapped up in a base viewController, with a delegate style interface for callbacks, so all I have to do as a client is subclass the camera view controller, implement the delegate methods, and add the UI buttons.
My question is about recording video. Video recording is started on a unique background task to ensure that the recording can be written to a temporary file. This is done on my
private let sessionQueue = DispatchQueue(label: "com.andrewferrarone.sessionQueue") session queue:
public func startRecording()
{
guard let movieFileOutput = self.movieFileOutput else { return }
//get video preview layer's video orientation on main queue
guard let videoPreviewLayerOrientation = self.previewView.videoPreviewLayer?.connection.videoOrientation else {
print("handleRecord: videoPreviewLayer is nil")
return
}
self.sessionQueue.async {
if !movieFileOutput.isRecording {
if UIDevice.current.isMultitaskingSupported {
self.backgroundRecordingID = UIApplication.shared.beginBackgroundTask(expirationHandler: nil)
}
//update orientation on the movie file output video connection before recording
let movieFileOutputConnection = self.movieFileOutput?.connection(withMediaType: AVMediaTypeVideo)
movieFileOutputConnection?.videoOrientation = videoPreviewLayerOrientation
//start recording to a temporary file:
let outputFileName = UUID().uuidString
let outputFilePath = (NSTemporaryDirectory() as NSString).appendingPathComponent((outputFileName as NSString).appendingPathExtension("mov")!)
movieFileOutput.startRecording(toOutputFileURL: URL(fileURLWithPath: outputFilePath), recordingDelegate: self)
}
}
}
so the recording is setup as a background task dispatched to self.sessionQueue. When I stop recording I receive an AVCaptureFileOutputRecordingDelegate method. In this method, I want to callback my delegate with the filepath, and then cleanup. How do I ensure that the delegate can persist the recording from the temporary file path before cleanup happens and the temporary file is removed?
public func capture(_ captureOutput: AVCaptureFileOutput!, didFinishRecordingToOutputFileAt outputFileURL: URL!, fromConnections connections: [Any]!, error: Error!)
{
//cleanup func for later
func cleanup()
{
let path = outputFileURL.path
if FileManager.default.fileExists(atPath: path) {
do {
try FileManager.default.removeItem(atPath: path)
}
catch let error {
print("Could not remove file at url: \(outputFileURL), error: \(error.localizedDescription)")
}
}
if let currentBackgroundRecordingID = self.backgroundRecordingID {
self.backgroundRecordingID = UIBackgroundTaskInvalid
if currentBackgroundRecordingID != UIBackgroundTaskInvalid {
UIApplication.shared.endBackgroundTask(currentBackgroundRecordingID)
}
}
}
var success = true
if error != nil {
print("Movie file finishing error: \(error)")
success = ((error as NSError).userInfo[AVErrorRecordingSuccessfullyFinishedKey] as AnyObject).boolValue
}
DispatchQueue.main.async {
self.delegate?.camera(self, didFinishRecordingToOutputFileAt: outputFileURL, success: success)
}
cleanup()
}
So I called my delegate back on the main queue with the results, but then I need to call cleanup() should I do this on the main queue right after calling back my delegate? is this safe? or if I leave it the way it is now, then we are on self.sessionQueue, and I am unsure if cleanup() will happen before the delegate method implementation has time to persist the recording. If anyone can give me some insight into what is going on and what would be the safest thing to do here, that would be great. According to apple, the documentation says do not assume that AVCaptureFileOutputRecordingDelegate didFinishRecordingToOutputFileAt method is called on a specific thread. Thanks for your time and help!
How about:
DispatchQueue.main.async {
self.delegate?.camera(self, didFinishRecordingToOutputFileAt: outputFileURL, success: success)
self.sessionQueue.async {
cleanup()
}
}
I think that would be the standard way of handling this situation. When the delegate method finishes, you assume that the delegate is done with the file (or copied it somewhere else).

AVPlayer seek right after it starts to play

I need to jump to particular time in audiofile right after it starts to play. So I use seekToTime method with completion handler
avPlayer.play()
...
avPlayer?.seekToTime(jumpTime, completionHandler: { isComplete in
if isComplete {
MPNowPlayingInfoCenter.defaultCenter().nowPlayingInfo![MPNowPlayingInfoPropertyElapsedPlaybackTime] = CMTimeGetSeconds((self.avPlayer!.currentItem?.currentTime())!)
}
})
The problem is that it needs time to start playing file from the internet. And for some reason the version of seekToTime with completion handler crashes the app, because it's invoked before avPlayer started to play. The version without completion handler works fine.
Error:
Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'AVPlayerItem cannot service a seek request with a completion handler until its status is AVPlayerItemStatusReadyToPlay.'
Is there a way to use a callback with avPlayer.play()?
Yes, there is a way.
You need to observe changes in player:
avPlayer?.addPeriodicTimeObserverForInterval(CMTime(value: 1, timescale: 3), queue: dispatch_get_main_queue()) { [weak self] time in
self?.handleCallback(time)
}
And then you handle the state in the callback:
func handleCallback(time: CMTime) {
if avPlayer?.status == .ReadyToPlay {
// place your seek logic here
}
}
Please look at the Apple's answer for it:
https://developer.apple.com/library/content/qa/qa1820/_index.html
class MyClass {
var isSeekInProgress = false
let player = <#A valid player object #>
var chaseTime = kCMTimeZero
// your player.currentItem.status
var playerCurrentItemStatus:AVPlayerItemStatus = .Unknown
...
func stopPlayingAndSeekSmoothlyToTime(newChaseTime:CMTime)
{
player.pause()
if CMTimeCompare(newChaseTime, chaseTime) != 0
{
chaseTime = newChaseTime;
if !isSeekInProgress
{
trySeekToChaseTime()
}
}
}
func trySeekToChaseTime()
{
if playerCurrentItemStatus == .Unknown
{
// wait until item becomes ready (KVO player.currentItem.status)
}
else if playerCurrentItemStatus == .ReadyToPlay
{
actuallySeekToTime()
}
}
func actuallySeekToTime()
{
isSeekInProgress = true
let seekTimeInProgress = chaseTime
player.seekToTime(seekTimeInProgress, toleranceBefore: kCMTimeZero,
toleranceAfter: kCMTimeZero, completionHandler:
{ (isFinished:Bool) -> Void in
if CMTimeCompare(seekTimeInProgress, chaseTime) == 0
{
isSeekInProgress = false
}
else
{
trySeekToChaseTime()
}
})
}
}

Resources