AudioKit handling of AVAudioSessionInterruption - ios

After receiving a phone call or just having the phone ring our background play enabled AudioKit app goes silent for good and I am not sure how to handle that. The only way to restart sound output is to kill and restart the app. Other interruptions like enabling and using Siri work without a hitch and the app's sound is ducked during the event.
Typically an app can register itself to receive notifications (e.g. NSNotification.Name.AVAudioSessionInterruption) to detect an AVAudioSession interruption, but how does one retrieve the AVSession object that is normally passed into the notification?
NotificationCenter.default.addObserver(self, selector: #selector(AppDelegate.sessionInterrupted(_:)),
name: NSNotification.Name.AVAudioSessionInterruption,
object: MISSING_AK_AVAUDIOSESSION_REF)
Furthermore, if one were able to successfully implement audio interrupt notifications, what happens with AudioKit? It is not designed to be "restarted" or put on hold. Any help would be much appreciated.

This is going to depend on your app how you handle this. At very least, you'll want to do Audiokit.stop() and then Audiokit.start() when the interruption is finished.
You'll want to register for the notification with something like this:
NotificationCenter.default.addObserver(self,
selector: #selector(handleInterruption),
name: .AVAudioSessionInterruption,
object: nil)
Then handle it with something like this:
#objc internal func handleInterruption(_ notification: Notification) {
guard let info = notification.userInfo,
let typeValue = info[AVAudioSessionInterruptionTypeKey] as? UInt,
let type = AVAudioSessionInterruptionType(rawValue: typeValue) else {
return
}
//...handle each type here
}

Related

iOS Audio not working during call answered when phone is locked. WebRTC used for calling

I am facing a problem with Audio When using Callkit with WebRTC for VOIP call, While answering the call from Lock Screen.
General Functionality :
My app activates the audioSession when it's launched. For an incoming call, SDP Offer & Answer are generated and exchanged. Peer Connection is set up. Both audio and video streams are generated, whether it's audio call or video call. Then Call is reported to callkit by using the following code:
callProvider.reportNewIncomingCall(with: currentCallUUID!, update: update) { error in }
If app is in the foreground, it works fine.
But, when the phone is locked, and user answers the call from lock screen, the Streams are exchanged but no audio comes on either end until user enters into the app himself.
As the user enters into the App, audio becomes active on both the ends.
All the background settings and capabilities are set properly.
I have also referred to the following work around provided by Apple staff. But even it does not work.
https://forums.developer.apple.com/thread/64544
As I mentioned, I am using WebRTC for calling. If I exchange the media streams after the user answers the call( still on Lock Screen) and peer connection is set at that time. It works fine (But it adds the delay in making the call connection).
But if Peer Connection is made before displaying call (say before reporting call to callkit), the audio stops working.
I am able to resolve this issue.
Steps that I followed -
I checked the code related to WebRTC here
I added RTCAudioSession header file which is actually a private class of Webrtc. So every time I receive a call event from signaling, I enable RTCAudiosession and on end of the call, I disable it.
I have to render the incoming streams to a dummy view (Although it is not displayed when the call is going and the app is not yet open, but it is required to make audio working).
I hope this will help if someone is facing the same issue.
#abhimanyu are you still facing the issue or you made it work. I am facing same issue with CallKit.
As per my understanding in WebRTC M60 release they have fixed on issue related to CallKit, which I think created a side effect and caused this issue.
The issue which they have fixed is related to System AudioSession, when ever CallKit presents incoming call UI and play ringer tone CallKit takes control of AudioSession and after user action (accept/ decline) it releases control. In WebRTC M60 release, now they have added observers for this control exchange. That's why it is working if app is in foreground, but if phone is locked and any incoming call is accepted then (I am assuming you are using CallKit UI for call and not redirecting user to App on accept from lock screen) due to Native UI of call it is not possible for WebRTC to activate its own AudioSession instance as call is going through CallKit Screen.
Link for bug which has been fixed on WebRTC M60: https://bugs.chromium.org/p/webrtc/issues/detail?id=7446
If you found any workaround for this issue please let me know.
Please Note that I share my code and its about to my needs and I share for reference. you need to change it according to your need.
when you receive voip notification create new incident of your webrtc handling class, and
add this two lines to code block because enabling audio session from voip notification fails
RTCAudioSession.sharedInstance().useManualAudio = true
RTCAudioSession.sharedInstance().isAudioEnabled = false
didReceive method;
func pushRegistry(_ registry: PKPushRegistry, didReceiveIncomingPushWith payload: PKPushPayload, for type: PKPushType, completion: #escaping () -> Void) {
let state = UIApplication.shared.applicationState
if(payload.dictionaryPayload["hangup"] == nil && state != .active
){
Globals.voipPayload = payload.dictionaryPayload as! [String:Any] // I pass parameters to Webrtc handler via Global singleton to create answer according to sdp sent by payload.
RTCAudioSession.sharedInstance().useManualAudio = true
RTCAudioSession.sharedInstance().isAudioEnabled = false
Globals.sipGateway = SipGateway() // my Webrtc and Janus gateway handler class
Globals.sipGateway?.configureCredentials(true) // I check janus gateway credentials stored in Shared preferences and initiate websocket connection and create peerconnection
to my janus gateway which is signaling server for my environment
initProvider() //Crating callkit provider
self.update.remoteHandle = CXHandle(type: .generic, value:String(describing: payload.dictionaryPayload["caller_id"]!))
Globals.callId = UUID()
let state = UIApplication.shared.applicationState
Globals.provider.reportNewIncomingCall(with:Globals.callId , update: self.update, completion: { error in
})
}
}
func initProvider(){
let config = CXProviderConfiguration(localizedName: "ulakBEL")
config.iconTemplateImageData = UIImage(named: "ulakbel")!.pngData()
config.ringtoneSound = "ringtone.caf"
// config.includesCallsInRecents = false;
config.supportsVideo = false
Globals.provider = CXProvider(configuration:config )
Globals.provider.setDelegate(self, queue: nil)
update = CXCallUpdate()
update.hasVideo = false
update.supportsDTMF = true
}
modify your didActivate and didDeActive delegate functions like below,
func provider(_ provider: CXProvider, didActivate audioSession: AVAudioSession) {
print("CallManager didActivate")
RTCAudioSession.sharedInstance().audioSessionDidActivate(audioSession)
RTCAudioSession.sharedInstance().isAudioEnabled = true
// self.callDelegate?.callIsAnswered()
}
func provider(_ provider: CXProvider, didDeactivate audioSession: AVAudioSession) {
print("CallManager didDeactivate")
RTCAudioSession.sharedInstance().audioSessionDidDeactivate(audioSession)
RTCAudioSession.sharedInstance().isAudioEnabled = false
}
in Webrtc handler class configure media senders and audiosession
private func createPeerConnection(webRTCCallbacks:PluginHandleWebRTCCallbacksDelegate) {
let rtcConfig = RTCConfiguration.init()
rtcConfig.iceServers = server.iceServers
rtcConfig.bundlePolicy = RTCBundlePolicy.maxBundle
rtcConfig.rtcpMuxPolicy = RTCRtcpMuxPolicy.require
rtcConfig.continualGatheringPolicy = .gatherContinually
rtcConfig.sdpSemantics = .planB
let constraints = RTCMediaConstraints(mandatoryConstraints: nil,
optionalConstraints: ["DtlsSrtpKeyAgreement":kRTCMediaConstraintsValueTrue])
pc = sessionFactory.peerConnection(with: rtcConfig, constraints: constraints, delegate: nil)
self.createMediaSenders()
self.configureAudioSession()
if webRTCCallbacks.getJsep() != nil{
handleRemoteJsep(webrtcCallbacks: webRTCCallbacks)
}
}
mediaSenders;
private func createMediaSenders() {
let streamId = "stream"
// Audio
let audioTrack = self.createAudioTrack()
self.pc.add(audioTrack, streamIds: [streamId])
// Video
/* let videoTrack = self.createVideoTrack()
self.localVideoTrack = videoTrack
self.peerConnection.add(videoTrack, streamIds: [streamId])
self.remoteVideoTrack = self.peerConnection.transceivers.first { $0.mediaType == .video }?.receiver.track as? RTCVideoTrack
// Data
if let dataChannel = createDataChannel() {
dataChannel.delegate = self
self.localDataChannel = dataChannel
}*/
}
private func createAudioTrack() -> RTCAudioTrack {
let audioConstrains = RTCMediaConstraints(mandatoryConstraints: nil, optionalConstraints: nil)
let audioSource = sessionFactory.audioSource(with: audioConstrains)
let audioTrack = sessionFactory.audioTrack(with: audioSource, trackId: "audio0")
return audioTrack
}
audioSession ;
private func configureAudioSession() {
self.rtcAudioSession.lockForConfiguration()
do {
try self.rtcAudioSession.setCategory(AVAudioSession.Category.playAndRecord.rawValue)
try self.rtcAudioSession.setMode(AVAudioSession.Mode.voiceChat.rawValue)
} catch let error {
debugPrint("Error changeing AVAudioSession category: \(error)")
}
self.rtcAudioSession.unlockForConfiguration()
}
Please consider that because I worked with callbacks and delegates code includes delegates and callback chunks. you can ignore them accordingly!!
FOR REFERENCE You can also check the example at this link

AVAudioEngine not posting notification in SpriteKit

I have a problem with AVAudioEngine in SpriteKit game - when plugging/unplugging of headphones the engine stops and when the next sound play the app crashes. It is a known bug (or feature?) - with a suggestion to fix it - to use notification center, AVAudioEngine should post notification when it is changing it's states. I have made this code:
let notificationName = Notification.Name("AVAudioEngineConfigurationChange")
NotificationCenter.default.addObserver(self, selector: #selector(self.restartEngine(notification:)), name: notificationName, object: nil)
When I do this:
NotificationCenter.default.post(name: notificationName, object: nil)
My selector gets called. However when I plug/unplug headphones - nothing happens. Swift 3, xcode 8, iOS 9.3
Any suggestions on how to fix it?
The original Cocoa name of the notification is AVAudioEngineConfigurationChangeNotification, but the Swift constant is called AVAudioEngineConfigurationChange. Therefore you can handle the notification by using either:
let notificationName = Notification.Name("AVAudioEngineConfigurationChangeNotification")
or
let notificationName = Notification.Name.AVAudioEngineConfigurationChange
Your selector should then be called when plugging-in/unplugging headphones.
This appears to only be a partial solution for SpriteKit audio flakiness. I tried the following inside the notification handler:
try? self.audioEngine.start()
I was able to stop the crash, but my SKAudioNode wouldn't make any sound after the plug/unplug event, until the process was restarted.

Post NSUserDefaults changes between iOS Keyboard Extension and Containing app?

I thought I have tried everything and read every possible articles in this forum. But nothing seems to work. Here is some code snippet and some settings:
On Extension side:
let thisDefaults = NSUserDefaults(suiteName: "group.request.com")
thisDefaults?.setObject("Hello", forKey: "prevWord")
thisDefaults?.setObject("World", forKey: "word")
let success = thisDefaults?.Synchronize()
NSNotificationCenter.defaultCenter().postNotificationName("ExtensionRequestChanges", object: nil)
On Containingg app side:
NSNotificationCenter.defaultCenter().addObserver(self, selector: Selector("DidReceiveRequest:", name: "ExtensionRequestChanges",
object: nil)
func DidReceiveRequest(notification: NSNotification) {
// Read defaults
let thisDefaults = NSUserDefaults(suiteName: "group.request.com")
let word = thisDefaults?.stringForKey("word")
let prevWord = thisDefaults?.stringForKey("prevWord")
...
}
On the project settings:
. registered "group.request.com" app group to both the extension and containing app
. set "RequestOpenAccess" in NSExtensionAttributes to YES
But my DidReceiveRequest function never seemed to get called! Why???
Please help.
NSNotificationCenter only works in the process it's running in. Your keyboard extension and container app are separate processes. To post interprocess notifications, you should investigate the CFNotificationCenterGetDarwinNotifyCenterAPI call, which returns a Core Foundation notification center that can post notifications between processes, or other forms of interprocess communication.

Timer not working on real iPhone

I'm trying to use local notification but something is not working.
I have a class notification that handle all the code related to the notifications.
It's apparently working. What is not working is the way I try to trigger my notification.
When the user clicks on the home button, I call my notification class that starts a NSTimer. It repeats every second, and each 10 seconds I call a webservice.
Everything works great on my simulator, but it doesn't work on my real iPhone.
Here the code:
//as a class variable
let notif = Notification()
func applicationDidEnterBackground(application: UIApplication) {
notif.triggerTimer()
}
The notification class
class Notification: NSObject, WsOrderStatusProtocol, WsPinRequestProtocol {
var timer = NSTimer()
var time = 0
var sendNotification:Bool = true
var wsos = WsOrderStatus()
var wsoc = PinRequest()
override init() {
super.init()
self.wsos.delegate = self
self.wsoc.delegate = self
}
func triggerTimer() {
print("log INFO : class Notification, methode : triggerTimer")
NSNotificationCenter.defaultCenter().addObserver(self, selector:"orderCoupon:", name: "actionOrderCouponPressed", object: nil)
NSNotificationCenter.defaultCenter().addObserver(self, selector:"cancelTimer:", name: "actionCancelTimerPressed", object: nil)
timer = NSTimer.scheduledTimerWithTimeInterval(1.0, target: self, selector: Selector("launchNotification"), userInfo: nil, repeats: true)
}
func launchNotification() {
print("log INFO : class Notification, methode : launchNotification")
time += 1
print("time \(time)")
if time % 10 == 0 {
print("modulo 10")
wsos.getOrderStatus()
}
}
}
In the simulator, I see the logs et the logs that counts to 10 etc, but with my real iphone, I only see the first log "print("log INFO : class Notification, methode : triggerTimer")" then nothing...
Do you know why ?
As Paul says in his comment, your app only spends a very brief time in the background before being suspended. Suspended means that your code doesn't run at all any more, so timers stop.
The simulator doesn't always follow the same rules. When its behavior is different than that of a device then ignore it. It lies.
If you want to have more time to do work in the background, you can ask for it using the method beginBackgroundTaskWithExpirationHandler. Make that call in your applicationDidEnterBackground method.
From testing I've found that that gives you 3 minutes of extra time. After that your expiration handler block gets executed and then you get suspended.
Apple does not want your app running indefinitely from the background. It drains the battery.
I've found that it is possible to lie and tell the system that you are an app that plays sounds from the background, and write your expiration handler to play a short "sound of silence" and then ask for another background task using beginBackgroundTaskWithExpirationHandler. However, doing that will get you rejected from the app store.

In Swift, how to ivalidate NSTimer in AppDelegate when application going background?

I need to translate my iOS application from obj-c to swift. I have a NStimer in ViewController that loads metadata from shoutcast every 30 seconds, but when application resign active it stops, when enter foreground it runs again.
Edit: OK. Problem solved! I added two observers in viewDidLoad with name UIApplicationWillResignActiveNotification and UIApplicationWillEnterForegroundNotification, like below:
override func viewDidLoad() {
NSLog("System Version is \(UIDevice.currentDevice().systemVersion)");
super.viewDidLoad()
self.runTimer()
NSNotificationCenter.defaultCenter().addObserver(self, selector: "invalidateTimer", name: UIApplicationWillResignActiveNotification, object: nil)
NSNotificationCenter.defaultCenter().addObserver(self, selector: "runTimer", name: UIApplicationWillEnterForegroundNotification, object: nil)
}
and I made two functions. First one for run timer:
func runTimer(){
loadMetadata()
myTimer.invalidate()
NSLog("timer run");
myTimer = NSTimer.scheduledTimerWithTimeInterval(30.0, target: self, selector: "loadMetadata", userInfo: nil, repeats: true)
let mainLoop = NSRunLoop.mainRunLoop()
mainLoop.addTimer(myTimer, forMode: NSDefaultRunLoopMode)
}
and second to stop it:
func invalidateTimer(){
myTimer.invalidate()
NSLog("timer invalidated %u", myTimer);
}
I hope this can help someone. :)
I suggest you use the appropriate system for your task: https://developer.apple.com/library/ios/documentation/iphone/conceptual/iPhoneOSProgrammingGuide/BackgroundExecution/BackgroundExecution.html#//apple_ref/doc/uid/TP40007072-CH4-SW56
Apps that need to check for new content periodically can ask the
system to wake them up so that they can initiate a fetch operation for
that content. To support this mode, enable the Background fetch option
from the Background modes section of the Capabilities tab in your
Xcode project. (You can also enable this support by including the
UIBackgroundModes key with the fetch value in your app’s Info.plist
file.)...
When a good opportunity arises, the system wakes or launches your app
into the background and calls the app delegate’s
application:performFetchWithCompletionHandler: method. Use that method
to check for new content and initiate a download operation if content
is available.

Resources