I am using AVFoundation / AudioKit in order to record the internal microphone of the iPhone / iPad. It should be possible to continue using the app after switching the output between BluetoothA2DP and the internal speaker. The microphone should continue to take the input from the internal microphone of the device. And it does. Everything is working fine but only until I want to change the output device.
func basicAudioSetup(){
// microphone
self.microphone = AKMicrophone()
// select input of device
if let input = AudioKit.inputDevice {
try! self.microphone?.setDevice(input)
}
AKSettings.sampleRate = 44100
AKSettings.channelCount = 2
AKSettings.playbackWhileMuted = true
AKSettings.enableRouteChangeHandling = false
AKSettings.useBluetooth = true
AKSettings.allowAirPlay = true
AKSettings.defaultToSpeaker = true
AKSettings.audioInputEnabled = true
// init DSP
self.dsp = AKClock(amountSamples: Int32(self.amountSamples), amountGroups: Int32(self.amountGroups), bpm: self.bpm, iPad:self.iPad)
self.masterbusTracker = AKAmplitudeTracker(self.dsp)
self.mixer.connect(input: self.masterbusTracker)
self.player = AKPlayer()
self.mixer.connect(input: self.player)
self.microphone?.stop()
self.microphoneTracker = AKAmplitudeTracker(self.microphone)
self.microphoneTracker?.stop()
self.microphoneRecorder = try! AKNodeRecorder(node: self.microphone)
self.microphoneMonitoring = AKBooster(self.microphoneTracker)
self.microphoneMonitoring?.gain = 0
self.mixer.connect(input: self.microphoneMonitoring)
AudioKit.output = self.mixer
// the following line is actually happening inside a customized AudioKit.start() function to make sure that only BluetoothA2DP is used for better sound quality:
try AKSettings.setSession(category: .playAndRecord, with: [.defaultToSpeaker, .allowBluetoothA2DP, .allowAirPlay, .mixWithOthers])
do {
try AudioKit.start()
}catch{
print("AudioKit did not start")
}
// adding Notifications to manually restart the engine.
NotificationCenter.default.addObserver(self, selector: #selector(self.audioRouteChangeListener(notification:)), name: NSNotification.Name.AVAudioSessionRouteChange, object: nil)
}
#objc func audioRouteChangeListener(notification:NSNotification) {
let audioRouteChangeReason = notification.userInfo![AVAudioSessionRouteChangeReasonKey] as! UInt
let checkRestart = {
print("ROUTE CHANGE")
do{
try AudioKit.engine.start()
}catch{
print("error rebooting engine")
}
}
if audioRouteChangeReason == AVAudioSessionRouteChangeReason.newDeviceAvailable.rawValue ||
audioRouteChangeReason == AVAudioSessionRouteChangeReason.oldDeviceUnavailable.rawValue{
if Thread.isMainThread {
checkRestart()
} else {
DispatchQueue.main.async(execute: checkRestart)
}
}
}
I noticed, that when the microphone is connected, AVAudioSessionRouteChange is never called when switching from the internal speaker to Bluetooth. I do receive messages when starting with and switching from Bluetooth to the internal speaker:
[AVAudioEngineGraph.mm:1481:Start: (err = PerformCommand(*ioNode,
kAUStartIO, NULL, 0))
What does this message exactly mean? I tried everything, from manually disconnecting all inputs of the engine / de- and reactivating the session to rebuilding the whole chain. Nothing works.
Theoretically the input source is not changing, because it is staying on the input of the phone. Any help highly appreciated.
FYI: I am using a customized version of AudioKit library where I removed its internal Notifications of AVAudioSessionRouteChange to avoid unwanted Doppelgaenger. This customized library also sets the session category and options internally for the same reason and to ensure, that only BluetoothA2DP is used.
Related
I am trying to migrate an app from AudioKit v4 to v5 and I am having a hard time finding documentation on the migration, and I can't find these in the Cookbook. Previously we could set defaultToSpeaker and audioInputEnabled through AKSettings. Now, these properties are gone and I can't find how can I replace them.
v4:
AKSettings.audioInputEnabled = true
AKSettings.defaultToSpeaker = true
Does anyone know how these parameters can be set with the new version? Any feedback is highly appreciated!
Nazarii,
In AudioKit 5, here's how I set up my audio input parameters:
import AudioKit
import AVFoundation
class Conductor {
static let sharedInstance = Conductor()
// Instantiate the audio engine and Mic Input node objects
let engine = AudioEngine()
var mic: AudioEngine.InputNode!
// Add effects for the Mic Input.
var delay: Delay!
var reverb: Reverb!
let mixer = Mixer()
// MARK: Initialize the audio engine settings.
init() {
// AVAudioSession requires the AVFoundation framework to be imported in the header.
do {
Settings.bufferLength = .medium
try AVAudioSession.sharedInstance().setPreferredIOBufferDuration(Settings.bufferLength.duration)
try AVAudioSession.sharedInstance().setCategory(.playAndRecord,
options: [.defaultToSpeaker, .mixWithOthers, .allowBluetoothA2DP])
try AVAudioSession.sharedInstance().setActive(true)
} catch let err {
print(err)
}
// The audio signal path with be:
// input > mic > delay > reverb > mixer > output
// Mic is connected to the audio engine's input...
mic = engine.input
// Mic goes into the delay...
delay = Delay(mic)
delay.time = AUValue(0.5)
delay.feedback = AUValue(30.0)
delay.dryWetMix = AUValue(15.0)
// Delay output goes into the reverb...
reverb = Reverb(delay)
reverb.loadFactoryPreset(.largeHall2)
reverb.dryWetMix = AUValue(0.4)
// Reverb output goes into the mixer...
mixer.addInput(reverb)
// Engine output is connected to the mixer.
engine.output = mixer
// Uncomment the following method, if you don't want to Start and stop the audio engine via the SceneDelegate.
// startAudioEngine()
}
// MARK: Start and stop the audio engine via the SceneDelegate
func startAudioEngine() {
do {
print("Audio engine was started.")
try engine.start()
} catch {
Log("AudioKit did not start! \(error)")
}
}
func stopAudioEngine() {
engine.stop()
print("Audio engine was stopped.")
}
}
Please let me know if this works for you.
Take care,
Mark
I am developing an iOS 14 app that plays fragments of an audio file for the user to imitate. If the user wants to, the app can record the user's responses and play these back immediately. The user can also export an audio recording that comprises the original fragments, plus the user's responses.
I am using AudioKit 4.11
Because it is possible the user may never wish to take advantage of the app's recording abilities, the app initially adopts the audio session category of .playback. If the user wants to use the recording feature, the app triggers the standard Apple dialog for requesting microphone access, and if this is granted, switches the session category to .playAndRecord.
I have found that when the session category is .playback and the user has not yet granted microphone permission, I am able to listen to the app's output on a Bluetooth speaker, or on my Jabra Elite 65t Bluetooth earbuds when the app is running on a real iPhone. In the example below, this is the case when the app first runs and the user has only ever tapped "Play sound" or "Stop".
However, as soon as I tap "Play sound and record response" and grant microphone access, I am unable to listen to the app's output on a Bluetooth device, regardless of whether the session category applicable at the time is .playback (after tapping "Play sound and record response") or .playAndRecord (after tapping "Play sound") - unless I subsequently go to my phone's Privacy settings and toggle microphone access to off. Playback is available only through the phone's speaker, or through plugged in headphones.
When setting the session category of .playAndRecord I have tried invoking the .allowBluetoothA2DP option.
Apple's advice implies this should allow me to listen to my app's sound over Bluetooth in the circumstances I have described above (see https://developer.apple.com/documentation/avfoundation/avaudiosession/categoryoptions/1771735-allowbluetootha2dp). However I've not found this to be the case.
The code below represents a runnable app (albeit one requiring the presence of AudioKit 4.11) that illustrates the problem in a simplified form. The only elements not shown here are an NSMicrophoneUsageDescription that I added to info.plist, and the file "blues.mp3" which I imported into the project.
ContentView:
import SwiftUI
import AudioKit
import AVFoundation
struct ContentView: View {
private var pr = PlayerRecorder()
var body: some View {
VStack{
Text("Play sound").onTapGesture{
pr.setupforPlay()
pr.playSound()
}
.padding()
Text("Play sound and record response").onTapGesture{
if recordingIsAllowed() {
pr.activatePlayAndRecord()
pr.startSoundAndResponseRecording()
}
}
.padding()
Text("Stop").onTapGesture{
pr.stop()
}
.padding()
}
}
func recordingIsAllowed() -> Bool {
var retval = false
AVAudioSession.sharedInstance().requestRecordPermission { granted in
retval = granted
}
return retval
}
}
PlayerRecorder:
import Foundation
import AudioKit
class PlayerRecorder {
private var mic: AKMicrophone!
private var micBooster: AKBooster!
private var mixer: AKMixer!
private var outputBooster: AKBooster!
private var player: AKPlayer!
private var playerBooster: AKBooster!
private var recorder: AKNodeRecorder!
private var soundFile: AKAudioFile!
private var twentySecondTimer = Timer()
init() {
AKSettings.defaultToSpeaker = true
AKSettings.disableAudioSessionDeactivationOnStop = true
AKSettings.notificationsEnabled = true
}
func activatePlayAndRecord() {
do {
try AKManager.shutdown()
} catch {
print("Shutdown failed")
}
setupForPlayAndRecord()
}
func playSound() {
do {
soundFile = try AKAudioFile(readFileName: "blues.mp3")
} catch {
print("Failed to open sound file")
}
do {
try player.load(audioFile: soundFile!)
} catch {
print("Player failed to load sound file")
}
if micBooster != nil{
micBooster.gain = 0.0
}
player.play()
}
func setupforPlay() {
do {
try AKSettings.setSession(category: .playback)
} catch {
print("Failed to set session category to .playback")
}
mixer = AKMixer()
outputBooster = AKBooster(mixer)
player = AKPlayer()
playerBooster = AKBooster(player)
playerBooster >>> mixer
AKManager.output = outputBooster
if !AKManager.engine.isRunning {
try? AKManager.start()
}
}
func setupForPlayAndRecord() {
AKSettings.audioInputEnabled = true
do {
try AKSettings.setSession(category: .playAndRecord)
/* Have tried the following instead of the line above, but without success
let options: AVAudioSession.CategoryOptions = [.allowBluetoothA2DP]
try AKSettings.setSession(category: .playAndRecord, options: options.rawValue)
Have also tried:
try AKSettings.setSession(category: .multiRoute)
*/
} catch {
print("Failed to set session category to .playAndRecord")
}
mic = AKMicrophone()
micBooster = AKBooster(mic)
mixer = AKMixer()
outputBooster = AKBooster(mixer)
player = AKPlayer()
playerBooster = AKBooster(player)
mic >>> micBooster
micBooster >>> mixer
playerBooster >>> mixer
AKManager.output = outputBooster
micBooster.gain = 0.0
outputBooster.gain = 1.0
if !AKManager.engine.isRunning {
try? AKManager.start()
}
}
func startSoundAndResponseRecording() {
// Start player and recorder. After 20 seconds, call a function that stops the player
// (while allowing recording to continue until user taps Stop button).
activatePlayAndRecord()
playSound()
// Force removal of any tap not previously removed with stop() call for recorder
var mixerNode: AKNode?
mixerNode = mixer
for i in 0..<8 {
mixerNode?.avAudioUnitOrNode.removeTap(onBus: i)
}
do {
recorder = try? AKNodeRecorder(node: mixer)
try recorder.record()
} catch {
print("Failed to start recorder")
}
twentySecondTimer = Timer.scheduledTimer(timeInterval: 20.0, target: self, selector: #selector(stopPlayerOnly), userInfo: nil, repeats: false)
}
func stop(){
twentySecondTimer.invalidate()
if player.isPlaying {
player.stop()
}
if recorder != nil {
if recorder.isRecording {
recorder.stop()
}
}
if AKManager.engine.isRunning {
do {
try AKManager.stop()
} catch {
print("Error occurred while stopping engine.")
}
}
print("Stopped")
}
#objc func stopPlayerOnly () {
player.stop()
if !mic.isStarted {
mic.start()
}
if !micBooster.isStarted {
micBooster.start()
}
mic.volume = 1.0
micBooster.gain = 1.0
outputBooster.gain = 0.0
}
}
Three additional lines of code near the beginning of setupForPlayAndRecord() solve the problem:
func setupForPlayAndRecord() {
AKSettings.audioInputEnabled = true
// Adding the following three lines solves the problem
AKSettings.useBluetooth = true
let categoryOptions: AVAudioSession.CategoryOptions = [.allowBluetoothA2DP]
AKSettings.bluetoothOptions = categoryOptions
do {
try AKSettings.setSession(category: .playAndRecord)
} catch {
print("Failed to set session category to .playAndRecord")
}
mic = AKMicrophone()
micBooster = AKBooster(mic)
mixer = AKMixer()
outputBooster = AKBooster(mixer)
player = AKPlayer()
playerBooster = AKBooster(player)
mic >>> micBooster
micBooster >>> mixer
playerBooster >>> mixer
AKManager.output = outputBooster
micBooster.gain = 0.0
outputBooster.gain = 1.0
if !AKManager.engine.isRunning {
try? AKManager.start()
}
}
My iOS app receives notification to refresh its state from our main service. Right now, we are fetching the latest state and we play a continues sound when there is updated data. Our devices are locked in guided mode to stop them from turning off.
We are making changes such that the device can go to sleep after xx minutes of inactivity. However, we are noticing that the sound doesn't play when the app is in background and it always seems to fail on AVAudioSession.sharedInstance().setActive(true). Interestingly, if put breakpoints and run it through the debugger, it works fine but when running normally, it fails with this error:
Error Domain=NSOSStatusErrorDomain Code=561015905
We have the "Audio, AirPlay, and PnP" enabled under background modes under capabilities. Here is the code for playing the sound:
func playSound(shouldPlay: Bool) {
guard let url = Bundle.main.url(forResource: "sms_alert_circles", withExtension: "caf") else { return }
let audioSession = AVAudioSession.sharedInstance()
do {
try self.audioSession.setCategory(.playback, mode: .default, options: .mixWithOthers)
try self.audioSession.setActive(true)
/* The following line is required for the player to work on iOS 11. Change the file type accordingly*/
self.player = try AVAudioPlayer(contentsOf: url, fileTypeHint: AVFileType.caf.rawValue)
guard let player = self.player else { return }
if shouldPlay == true {
player.volume = 1.0
player.numberOfLoops = 1
player.play()
} else {
player.stop()
try self.audioSession.setActive(false)
}
} catch let error {
print("Error playing sounds")
print(error.localizedDescription)
}
}
I am hoping someone can point out the issue.
It seems that my app is not working with AirPods. Right now I'm using this code for the playback and record:
let audioSession = AVAudioSession.sharedInstance()
do { try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: AVAudioSessionCategoryOptions.defaultToSpeaker)
}catch {
print("audioSession properties weren't set because of an error.")
}
Will it be enough if I change defaultToSpeaker to allowBluetooth?
P.S. I know it's quite a stupid question because it'd be much simpler to just change this line and check, but I don't have AirPods with me right now, so the only option for me is to upload the new build to Testflight (and I want to do this with minimum iterations).
update: (quite naive approach — but all I need is to use bluetooth headphones if they are available):
func selectDevice(audioSession: AVAudioSession) {
var headphonesExist = false
var bluetoothExist = false
var speakerExist = false
let currentRoute = AVAudioSession.sharedInstance().currentRoute
for output in audioSession.currentRoute.outputs {
print(output)
if output.portType == AVAudioSessionPortHeadphones || output.portType == AVAudioSessionPortHeadsetMic {
headphonesExist = true
}
if output.portType == AVAudioSessionPortBluetoothA2DP || output.portType == AVAudioSessionPortBluetoothHFP {
bluetoothExist = true
}
if output.portType == AVAudioSessionPortBuiltInSpeaker {
speakerExist = true
}
}
if bluetoothExist == true {
do { try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: AVAudioSessionCategoryOptions.allowBluetooth) } catch {
print("error with audiosession: bluetooth")
}
}
}
You need to add support for bluetooth to your options parameter like so:
[AVAudioSessionCategoryOptions.defaultToSpeaker, .allowBluetoothA2DP]
.allowBluetoothA2DP will allow for high quality audio output to the bluetooth device and restrict microphone input on said device, while .allowBluetooth will default HFP compatible (input/output) bluetooth devices to the lower quality HFP, which supports microphone input.
Here is a full source to request proper permission.
All you have to do is to add a mode with '.allowBluetoothA2DP'
I've applied on Swift 5
func requestPermissionForAudioRecording(){
recordingSession = AVAudioSession.sharedInstance()
do {
// only with this without options, will not capable with your Airpod, the Bluetooth device.
// try recordingSession.setCategory(.playAndRecord, mode: .default)
try recordingSession.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowAirPlay, .allowBluetoothA2DP])
try recordingSession.setActive(true)
recordingSession.requestRecordPermission() { allowed in
DispatchQueue.main.async {
if allowed {
// Recording permission has been allowed
self.recordingPermissionGranted = true
} else {
// failed to record!
}
}
}
} catch let err {
// failed to record!
print("AudioSession couldn't be set!", err)
}
}
I am using AVAudioSession to listen to voice input. it works fine for wired headphones but it is not working for connected bluetooth device. Following is the code I am using to set input to bluetooth mic
func setupSessionForRecording() {
let audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: [.allowBluetooth])
} catch let error as NSError {
debugPrint("Error in listening "+error.localizedDescription)
}
var inputsPriority: [(type: String, input: AVAudioSessionPortDescription?)] = [
(AVAudioSessionPortLineIn, nil),
(AVAudioSessionPortHeadsetMic, nil),
(AVAudioSessionPortBluetoothHFP, nil),
(AVAudioSessionPortUSBAudio, nil),
(AVAudioSessionPortCarAudio, nil),
(AVAudioSessionPortBuiltInMic, nil),
]
for availableInput in audioSession.availableInputs! {
guard let index = inputsPriority.index(where: { $0.type == availableInput.portType }) else { continue }
inputsPriority[index].input = availableInput
}
guard let input = inputsPriority.filter({ $0.input != nil }).first?.input else {
fatalError("No Available Ports For Recording")
}
do {
try audioSession.setPreferredInput(input)
try audioSession.setMode(AVAudioSessionModeMeasurement)
try audioSession.setActive(true, with: .notifyOthersOnDeactivation)
try audioSession.setPreferredIOBufferDuration(10)
} catch {
fatalError("Error Setting Up Audio Session")
}
}
This code stops taking input from device mic and I also get sound in bluetooth headset that it is ready to listen but it doesn't pick any input from device.
Also,
When I am trying to play any audio into bluetooth headset It doesn't work. Here is the code to play audio
do {
let output = AVAudioSession.sharedInstance().currentRoute.outputs[0].portType
if output == "Receiver" || output == "Speaker"{
try AVAudioSession.sharedInstance().overrideOutputAudioPort(.speaker)
}
else{
try AVAudioSession.sharedInstance().overrideOutputAudioPort(.none)
}
print("Voice Out \(output)" )
} catch let error as NSError {
print("audioSession error: \(error.localizedDescription)")
os_log("Error during changing the current audio route: %#" , log: PollyVoiceViewController.log, type: .error, error)
} catch {
os_log("Unknown error during changing the current audio route", log: PollyVoiceViewController.log, type: .error)
}
do {
let soundData = try Data(contentsOf: url as URL)
self.audioPlayer = try AVAudioPlayer(data: soundData)
self.audioPlayer?.prepareToPlay()
self.audioPlayer?.volume = 3.0
self.audioPlayer?.delegate = self
self.audioPlayer?.play()
} catch let error as NSError {
print("Error getting the audio file"+error.description)
}
The reason is: BluetoothHFP is not available in AVAudioSessionModeMeasurement mode
AVAudioSessionModeMeasurement
After you set try audioSession.setMode(AVAudioSessionModeMeasurement), the audioSession.availableInputs is not contain the BluetoothHFP.
This mode is intended for apps that need to minimize the amount of system-supplied signal processing to input and output signals. If recording on devices with more than one built-in microphone, the primary microphone is used.
And in the document of setPreferredInput(_:)
The AVAudioSessionPortDescription must be in the availableInputs array.
The value of the inPort parameter must be one of the AVAudioSessionPortDescription objects in the availableInputs array. If this parameter specifies a port that is not already part of the current audio route and the app’s session controls audio routing, this method initiates a route change to use the preferred port.
And it must set up after setting the mode.
You must set a preferred input port only after setting the audio session’s category and mode and activating the session.