Background audio doesn't resume after a call - ios

My background audio works fine almost all the time. Screen locked, or mute switch on. But when the user has the application in the background, and it receives a call, even if the user doesn't answer the call, the background audio does not resumes after the interruption ends.
The Music app properly resumes background audio if it was interrupted.
Am I missing some property or do I need to have a callback or set a background task to continue background audio execution after an interruption? Or this is something that we can't do?

We are resuming our audio after an outbound call and inbound call while the app is in the background.
We play audio with AVAudioPlayer and listen to the AVAudioSessionInterruptionNotification. Apple automatically pauses the AVAudioPlayer for you on an interruption and when you tell it to resume after you receive the interruption is over, Apple will set your session active again. See Table 3-2 from Handling Audio Interruptions on recommendations if you are using other types of audio technologies.
Subscribe to the notification:
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(onAudioSessionEvent:) name:AVAudioSessionInterruptionNotification object:nil];
Handle the notification:
- (void) onAudioSessionEvent: (NSNotification *) notification
{
//Check the type of notification, especially if you are sending multiple AVAudioSession events here
if ([notification.name isEqualToString:AVAudioSessionInterruptionNotification]) {
NSLog(#"Interruption notification received!");
//Check to see if it was a Begin interruption
if ([[notification.userInfo valueForKey:AVAudioSessionInterruptionTypeKey] isEqualToNumber:[NSNumber numberWithInt:AVAudioSessionInterruptionTypeBegan]]) {
NSLog(#"Interruption began!");
} else {
NSLog(#"Interruption ended!");
//Resume your audio
}
}
}

In my experience I found I had to "wait a second or two" before attempting to reopen the audio device. I think that after the OS switches back to your app from the call, the phone app is still shutting down.
something like this when you return to foreground after knowing your audio session has been stopped:
dispatch_time_t restartTime = dispatch_time(DISPATCH_TIME_NOW,
1.5LL * NSEC_PER_SEC);
dispatch_after(restartTime, dispatch_get_global_queue(0, 0), ^{
[audioSystem restart];
});

I am using Xamarin so this is C# but the following code is working for me. I first set that my AppDelegate implements the AVAudioSession delegate methods by including IAVAudioSessionDelegate
public partial class AppDelegate: UIApplicationDelegate, IAVAudioSessionDelegate
I added a variable to the AppDelegate class for the audio session
public static AVAudioSession AudioSession;
In the override of the FinishedLaunching method:
AudioSession = AVAudioSession.SharedInstance();
AudioSession.Delegate = this;
error = AudioSession.SetCategory( AVAudioSessionCategory.Playback, AVAudioSessionCategoryOptions.DuckOthers );
AudioSession.SetMode( new NSString( "AVAudioSessionModeMoviePlayback" ), out error );
The two relevant delegate methods in AppDelegate.cs are:
[Export( "beginInterruption" )]
public void BeginInterruption()
{
PlayerViewController.BeginSessionInterruption();
}
[Export( "endInterruptionWithFlags:" )]
public void EndInterruption( AVAudioSessionInterruptionFlags flags )
{ // ignore the flags so we're not dependent on the interrupting event saying that we can resume
PlayerViewController.ResumeAfterSessionInterruption();
}
My AppDelegate also has an override of OnActivated to enable the video tracks if it's a video asset, and an override of DidEnterBackground to disable the media's video tracks but still play the audio.
In the PlayerViewController.BeginSessionInterruption() method, I can't look at Player.Rate to see if the player was running at the time, because the interrupting alarm or phone call has already paused the player. From the "Responding to Interruptions" section of Apple's Audio Session Programming Guide, with my emphasis added:
Your app is active, playing back audio.
A phone call arrives. The system activates the phone app’s audio session.
The system deactivates your audio session. At this point, * playback in your app has stopped *.
The system invokes your interruption listener callback function indicating that your session has been deactivated.
...
My PlayerViewController's Play button has a Paused property to toggle between Play and Pause and draw the appropriate button image. So instead of checking Player.Rate, I look to see if the button's Paused property is false:
public void BeginSessionInterruption()
{
PlayerWasRunningOnInterruption = !btnPlay.Paused;
TogglePlayPause( true ); // put the Play button into the Paused state to agree with the player being stopped
public void ResumeAfterSessionInterruption()
{
NSError error;
AppDelegate.AudioSession.SetActive( true, AVAudioSessionSetActiveOptions.NotifyOthersOnDeactivation, out error ); // always reactivate the audio session
if ( PlayerWasRunningOnInterruption )
{
// rewind a little bit
// call btnPlayClick to resume the playback as if the user pressed the Play button
}
}

Look at:
https://developer.apple.com/documentation/avfoundation/avaudiosession#//apple_ref/doc/uid/TP40008240-CH1-DontLinkElementID_3
https://developer.apple.com/documentation/avfoundation/avaudiosession/responding_to_audio_session_interruptions
Code:
private func setupAudioSession() {
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, with: .mixWithOthers)
try AVAudioSession.sharedInstance().setActive(true)
setupAudioNotifications()
} catch {
print(error)
}
}
private func setupAudioNotifications() {
NotificationCenter.default.addObserver(self, selector: #selector(handleInterruption), name: .AVAudioSessionInterruption, object: nil)
}
#objc func handleInterruption(notification: Notification) {
guard let userInfo = notification.userInfo,
let typeValue = userInfo[AVAudioSessionInterruptionTypeKey] as? UInt,
let type = AVAudioSessionInterruptionType(rawValue: typeValue) else {
return
}
if type == .began {
// Interruption began, take appropriate actions
Player.shared.stop()
} else if type == .ended {
if let optionsValue = userInfo[AVAudioSessionInterruptionOptionKey] as? UInt {
let options = AVAudioSessionInterruptionOptions(rawValue: optionsValue)
if options.contains(.shouldResume) {
// Interruption Ended - playback should resume
Player.shared.start()
} else {
// Interruption Ended - playback should NOT resume
}
}
}
}

if you only detect the call status you can use CTCallCenter, but if you want to detect the interruption (include the call interruption),you can use AVAudioSessionInterruptionNotification, and if you want to support background you should add code like this:
[[AVAudioSession sharedInstance] setActive:YES error:nil];
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:nil];
before you resume play music in background.hope this can help.

Only two options works for me:
reload AVPlayer or AVAudioPlayer after interruption end
func interruptionNotification(_ notification: Notification) {
guard let type = notification.userInfo?[AVAudioSessionInterruptionTypeKey] as? UInt,
let interruption = AVAudioSessionInterruptionType(rawValue: type) else {
return
}
if interruption == .ended && playerWasPlayingBeforeInterruption {
player.replaceCurrentItem(with: AVPlayerItem(url: radioStation.url))
play()
}
}
use
AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, with: .mixWithOthers)

Related

How can i control my headset for my music player?

I am creating an music player iOS app and getting data from firebase. I can able to control play, pause, next and previous in simulator or iPhone. While headset is connect to device play, next and previous functionalities are not working properly.
Here is the code which i've used;
func setupRemoteCommandCenter() {
// Get the shared MPRemoteCommandCenter
let commandCenter = MPRemoteCommandCenter.shared()
// Add handler for Play Command
commandCenter.playCommand.addTarget { event in
player?.play()
print("headset play")
return .success
}
// Add handler for Pause Command
commandCenter.pauseCommand.addTarget { event in
player?.pause()
print("headset pause")
return .success
}
// Add handler for Next Command
commandCenter.nextTrackCommand.addTarget { event in
return .success
}
// Add handler for Previous Command
commandCenter.previousTrackCommand.addTarget { event in
return .success
}
}
And calling setupRemoteCommandCenter function in viewdidload
This document says that to receive player event notifications you need to
begin playing audio
be the "Now playing app"
The definition of "Now playing app" is hard to pin down, but it seems to be any app that has an active, non-mixable audio session and is playing audio (or has very recently played audio, there seems to be a brief grace period here) . One possible non-mixable audio session is:
let session = AVAudioSession.sharedInstance()
do {
try session.setCategory(AVAudioSessionCategoryPlayback)
try session.setActive(true)
} catch let err as NSError {
print("Error setting up non mixable audio session \(err)")
}
p.s. if you want the headset controls to work while the screen is locked (or if you want the lockscreen controls to work for that matter), you will need to add audio to Background Modes, because technically, if the screen is locked then your app has been backgrounded:

iOS Audio not working during call answered when phone is locked. WebRTC used for calling

I am facing a problem with Audio When using Callkit with WebRTC for VOIP call, While answering the call from Lock Screen.
General Functionality :
My app activates the audioSession when it's launched. For an incoming call, SDP Offer & Answer are generated and exchanged. Peer Connection is set up. Both audio and video streams are generated, whether it's audio call or video call. Then Call is reported to callkit by using the following code:
callProvider.reportNewIncomingCall(with: currentCallUUID!, update: update) { error in }
If app is in the foreground, it works fine.
But, when the phone is locked, and user answers the call from lock screen, the Streams are exchanged but no audio comes on either end until user enters into the app himself.
As the user enters into the App, audio becomes active on both the ends.
All the background settings and capabilities are set properly.
I have also referred to the following work around provided by Apple staff. But even it does not work.
https://forums.developer.apple.com/thread/64544
As I mentioned, I am using WebRTC for calling. If I exchange the media streams after the user answers the call( still on Lock Screen) and peer connection is set at that time. It works fine (But it adds the delay in making the call connection).
But if Peer Connection is made before displaying call (say before reporting call to callkit), the audio stops working.
I am able to resolve this issue.
Steps that I followed -
I checked the code related to WebRTC here
I added RTCAudioSession header file which is actually a private class of Webrtc. So every time I receive a call event from signaling, I enable RTCAudiosession and on end of the call, I disable it.
I have to render the incoming streams to a dummy view (Although it is not displayed when the call is going and the app is not yet open, but it is required to make audio working).
I hope this will help if someone is facing the same issue.
#abhimanyu are you still facing the issue or you made it work. I am facing same issue with CallKit.
As per my understanding in WebRTC M60 release they have fixed on issue related to CallKit, which I think created a side effect and caused this issue.
The issue which they have fixed is related to System AudioSession, when ever CallKit presents incoming call UI and play ringer tone CallKit takes control of AudioSession and after user action (accept/ decline) it releases control. In WebRTC M60 release, now they have added observers for this control exchange. That's why it is working if app is in foreground, but if phone is locked and any incoming call is accepted then (I am assuming you are using CallKit UI for call and not redirecting user to App on accept from lock screen) due to Native UI of call it is not possible for WebRTC to activate its own AudioSession instance as call is going through CallKit Screen.
Link for bug which has been fixed on WebRTC M60: https://bugs.chromium.org/p/webrtc/issues/detail?id=7446
If you found any workaround for this issue please let me know.
Please Note that I share my code and its about to my needs and I share for reference. you need to change it according to your need.
when you receive voip notification create new incident of your webrtc handling class, and
add this two lines to code block because enabling audio session from voip notification fails
RTCAudioSession.sharedInstance().useManualAudio = true
RTCAudioSession.sharedInstance().isAudioEnabled = false
didReceive method;
func pushRegistry(_ registry: PKPushRegistry, didReceiveIncomingPushWith payload: PKPushPayload, for type: PKPushType, completion: #escaping () -> Void) {
let state = UIApplication.shared.applicationState
if(payload.dictionaryPayload["hangup"] == nil && state != .active
){
Globals.voipPayload = payload.dictionaryPayload as! [String:Any] // I pass parameters to Webrtc handler via Global singleton to create answer according to sdp sent by payload.
RTCAudioSession.sharedInstance().useManualAudio = true
RTCAudioSession.sharedInstance().isAudioEnabled = false
Globals.sipGateway = SipGateway() // my Webrtc and Janus gateway handler class
Globals.sipGateway?.configureCredentials(true) // I check janus gateway credentials stored in Shared preferences and initiate websocket connection and create peerconnection
to my janus gateway which is signaling server for my environment
initProvider() //Crating callkit provider
self.update.remoteHandle = CXHandle(type: .generic, value:String(describing: payload.dictionaryPayload["caller_id"]!))
Globals.callId = UUID()
let state = UIApplication.shared.applicationState
Globals.provider.reportNewIncomingCall(with:Globals.callId , update: self.update, completion: { error in
})
}
}
func initProvider(){
let config = CXProviderConfiguration(localizedName: "ulakBEL")
config.iconTemplateImageData = UIImage(named: "ulakbel")!.pngData()
config.ringtoneSound = "ringtone.caf"
// config.includesCallsInRecents = false;
config.supportsVideo = false
Globals.provider = CXProvider(configuration:config )
Globals.provider.setDelegate(self, queue: nil)
update = CXCallUpdate()
update.hasVideo = false
update.supportsDTMF = true
}
modify your didActivate and didDeActive delegate functions like below,
func provider(_ provider: CXProvider, didActivate audioSession: AVAudioSession) {
print("CallManager didActivate")
RTCAudioSession.sharedInstance().audioSessionDidActivate(audioSession)
RTCAudioSession.sharedInstance().isAudioEnabled = true
// self.callDelegate?.callIsAnswered()
}
func provider(_ provider: CXProvider, didDeactivate audioSession: AVAudioSession) {
print("CallManager didDeactivate")
RTCAudioSession.sharedInstance().audioSessionDidDeactivate(audioSession)
RTCAudioSession.sharedInstance().isAudioEnabled = false
}
in Webrtc handler class configure media senders and audiosession
private func createPeerConnection(webRTCCallbacks:PluginHandleWebRTCCallbacksDelegate) {
let rtcConfig = RTCConfiguration.init()
rtcConfig.iceServers = server.iceServers
rtcConfig.bundlePolicy = RTCBundlePolicy.maxBundle
rtcConfig.rtcpMuxPolicy = RTCRtcpMuxPolicy.require
rtcConfig.continualGatheringPolicy = .gatherContinually
rtcConfig.sdpSemantics = .planB
let constraints = RTCMediaConstraints(mandatoryConstraints: nil,
optionalConstraints: ["DtlsSrtpKeyAgreement":kRTCMediaConstraintsValueTrue])
pc = sessionFactory.peerConnection(with: rtcConfig, constraints: constraints, delegate: nil)
self.createMediaSenders()
self.configureAudioSession()
if webRTCCallbacks.getJsep() != nil{
handleRemoteJsep(webrtcCallbacks: webRTCCallbacks)
}
}
mediaSenders;
private func createMediaSenders() {
let streamId = "stream"
// Audio
let audioTrack = self.createAudioTrack()
self.pc.add(audioTrack, streamIds: [streamId])
// Video
/* let videoTrack = self.createVideoTrack()
self.localVideoTrack = videoTrack
self.peerConnection.add(videoTrack, streamIds: [streamId])
self.remoteVideoTrack = self.peerConnection.transceivers.first { $0.mediaType == .video }?.receiver.track as? RTCVideoTrack
// Data
if let dataChannel = createDataChannel() {
dataChannel.delegate = self
self.localDataChannel = dataChannel
}*/
}
private func createAudioTrack() -> RTCAudioTrack {
let audioConstrains = RTCMediaConstraints(mandatoryConstraints: nil, optionalConstraints: nil)
let audioSource = sessionFactory.audioSource(with: audioConstrains)
let audioTrack = sessionFactory.audioTrack(with: audioSource, trackId: "audio0")
return audioTrack
}
audioSession ;
private func configureAudioSession() {
self.rtcAudioSession.lockForConfiguration()
do {
try self.rtcAudioSession.setCategory(AVAudioSession.Category.playAndRecord.rawValue)
try self.rtcAudioSession.setMode(AVAudioSession.Mode.voiceChat.rawValue)
} catch let error {
debugPrint("Error changeing AVAudioSession category: \(error)")
}
self.rtcAudioSession.unlockForConfiguration()
}
Please consider that because I worked with callbacks and delegates code includes delegates and callback chunks. you can ignore them accordingly!!
FOR REFERENCE You can also check the example at this link

MPRemoteCommandCenter not generating remote control events MPMusicPlayerController.applicationMusicPlayer()

I'm using MPRemoteCommandCenter and MPMusicPlayerController.applicationMusicPlayer on the iPhone.
I'm trying to receive remote control events when the user is playing music and double taps on the headphone button.
If I use AVAudioPlayer, the remote commands are received perfectly.
However, if I use MPMusicPlayerController with any of its players (systemMusicPlayer, applicationMusicPlayer, or applicationQueuePlayer) the the commands do not get received. They appear to get gobbled up. For example when I double tap the remote, the music will toggle between play and stop. Instead, I need the remote events sent to my app.
Below is a sample app with my code. In the info.plist I've specified the required background mode for an app that plays audio (although its not necessary).
import UIKit
import MediaPlayer
class ViewController: UIViewController {
var mpPlayer:MPMusicPlayerController!
func remoteHandler() {
print("success")
}
override func viewDidLoad() {
super.viewDidLoad()
mpPlayer = MPMusicPlayerController.applicationMusicPlayer()
//mpPlayer = MPMusicPlayerController.systemMusicPlayer()
assert(mpPlayer != nil)
let cc = MPRemoteCommandCenter.shared()
print("cc = \(cc)")
cc.nextTrackCommand.isEnabled = true
cc.nextTrackCommand.addTarget(self, action: #selector(ViewController.remoteHandler))
cc.previousTrackCommand.isEnabled = true
cc.previousTrackCommand.addTarget(self, action: #selector(ViewController.remoteHandler))
cc.playCommand.isEnabled = true
cc.playCommand.addTarget(self, action: #selector(ViewController.remoteHandler))
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
print("AVAudioSession successfully set AVAudioSessionCategoryPlayback")
} catch let error as NSError {
print("AVAudioSession setCategory error: \(error.localizedDescription)")
}
mpPlayer.setQueueWithStoreIDs(["270139033"]) // requires iOS 10.3
mpPlayer.play()
}
}
Output is:
cc = 0x123e086c0
AVAudioSession successfully set AVAudioSessionCategoryPlayback
remoteHandler is never called.
From the Apple Developer web site.
When you use either the system or application player, you do not get
event notifications. Those players automatically handle events.
So there is no way to receive remote control events if you use MPMusicPlayerController. Looking forward to see this feature! Right now MPMusicPlayerController is the only way to play Apple Music songs.

Background Playback Of Audio Stream After Interruption, iOS Swift

I Have an app that uses AVPlayer() to play a music stream from the web. I have everything setup to play in background but when a call comes in (or any other interrupt) the playback will not resume after the interrupt is over. To be specific if the interrupt is dissmised pretty quickly playback will resume but if I have a long phone call for example playback will not resume.
This only happens if my app is in the background as well. If my app is in the foreground when an interrupt comes in everything works.
I have my interrupt notification set up as:
func playInterrupt(notification: NSNotification) {
var info = notification.userInfo!
var intValue: UInt = 0
(info[AVAudioSessionInterruptionTypeKey] as! NSValue).getValue(&intValue)
if let type = AVAudioSessionInterruptionType(rawValue: intValue) {
switch type {
case .began:
print("began")
pause()
case .ended:
print("ended")
play()
}
}
Using
AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, with: AVAudioSessionCategoryOptions.mixWithOthers )
Fixes the problem but then if I start playing from another audio source, they both overlay each other. Ideally I need my app to stop playback in that situation.
Any suggestions on how to achieve this?
Thanks in advance

Using Spotify/background music with camera open

I have an app that needs to have:
Background music playing while using the app (eg. spotify)
Background music playing while watching movie from AVPlayer
Stop the music when recording a video
Like Snapchat, the camera-viewcontroller is part of a "swipeview" and therefore always on.
However, when opening and closing the app, the music makes a short "crack" noise/sound that ruins the music.
I recorded it here:
https://soundcloud.com/morten-stulen/hacky-sound-ios
(3 occurrences)
I use these settings for changing the AVAudiosession in the appdelegate didFinishLaunchingWithOptions:
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord,withOptions:
[AVAudioSessionCategoryOptions.MixWithOthers,
AVAudioSessionCategoryOptions.DefaultToSpeaker])
try AVAudioSession.sharedInstance().setActive(true)
} catch {
print("error")
}
I use the LLSimpleCamera control for video recording and I've set the session there to:
_session.automaticallyConfiguresApplicationAudioSession = NO;
It seems others have the same problem with other camera libraries as well:
https://github.com/rFlex/SCRecorder/issues/127
https://github.com/rFlex/SCRecorder/issues/224
This guy removed the audioDeviceInput, but I kinda need that for recording video.
https://github.com/omergul123/LLSimpleCamera/issues/48
I also tried with Apple's code "AvCam", and I still have the same issue. How does Snapchat do this?!
Any help would be greatly appreciated, and I'll gladly provide more info or code!
I do something similar to what you're wanting, but without the camera aspect, but I think this will do what you want. My app allows background audio that will mix with non-fullscreen video/audio. When the user plays an audio file or a full screen video file, I stop the background audio completely.
The reason I do SoloAmbient then Playback is because I allow my audio to be played in the background when the device is locked. Going SoloAmbient will stop all background music playing and then switching to Playback lets my audio play in the app as well as in the background.
This is why you see a call to a method that sets the lock screen information in the Unload method. In this case, it is nulling it out so that there is no lock screen info.
In AppDelegate.swift
//MARK: Audio Session Mixing
func allowBackgroundAudio()
{
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, withOptions: .MixWithOthers)
} catch {
NSLog("AVAudioSession SetCategory - Playback:MixWithOthers failed")
}
}
func preventBackgroundAudio()
{
do {
//Ask for Solo Ambient to prevent any background audio playing, then change to normal Playback so we can play while locked
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategorySoloAmbient)
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
} catch {
NSLog("AVAudioSession SetCategory - SoloAmbient failed")
}
}
When I want to stop background audio, for example when playing an audio track that should be alone, I do the following:
In MyAudioPlayer.swift
func playUrl(url: NSURL?, backgroundImageUrl: NSURL?, title: String, subtitle: String)
{
ForgeHelper.appDelegate().preventBackgroundAudio()
if _mediaPlayer == nil {
self._mediaPlayer = MediaPlayer()
_mediaPlayer!.delegate = self
}
//... Code removed for brevity
}
And when I'm done with my media playing, I do this:
private func unloadMediaPlayer()
{
if _mediaPlayer != nil {
_mediaPlayer!.unload()
self._mediaPlayer = nil
}
_controlView.updateForProgress(0, duration: 0, animate: false)
ForgeHelper.appDelegate().allowBackgroundAudio()
setLockScreenInfo()
}
Hope this helps you out!

Resources