The main issue here is, when the microphone input is added, the device loses any haptic system sounds.
I setup the audio session here:
let audioSession = AVAudioSession.sharedInstance()
do {
try self.audioSession.setCategory(.playAndRecord, options: .mixWithOthers)
try self.audioSession.setAllowHapticsAndSystemSoundsDuringRecording(true)
try self.audioSession.setActive(true)
} catch { }
I make sure I am using setAllowHapticsAndSystemSoundsDuringRecording.
Though out the app, I am adding the microphone and removing it on-demand:
do {
let microphonePermission = AVCaptureDevice.authorizationStatus(for: AVMediaType.audio)
if microphonePermission != .denied && microphonePermission != .restricted {
let audioDevice = AVCaptureDevice.default(for: AVMediaType.audio)
let audioDeviceInput = try AVCaptureDeviceInput(device: audioDevice!)
if self.session.canAddInput(audioDeviceInput) {
self.session.addInput(audioDeviceInput)
}
else { print("Could not add audio device input to the session.") }
} else {
}
}
catch { print("Could not create audio device input: \(error).") }
As soon as the microphone is added, it loses haptic feedback and system sounds.
Related
I'm currently trying to debug my video camera module that records video and audio with av foundation while mixing with background audio. It only adds the audio input when the user starts recording and should remove the audio input after the user stops recording. It successfully works the first time on app launch when the user starts recording but fails to remove the audio input after showing me the recorded preview with the following error:
2023-02-10 12:44:01.834776-0800 Camera[14061:986111] [avas] AVAudioSession_iOS.mm:1271 Deactivating an audio session that has running I/O. All I/O should be stopped or paused prior to deactivating the audio session.
Error occurred while removing audio device input: Error Domain=NSOSStatusErrorDomain Code=560030580 "(null)"
Here is my initial capture session code:
func setUp(){
do{
self.session.beginConfiguration()
let cameraDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .front)
if cameraDevice != nil {
/* only add video in setup */
if self.session.canAddInput(videoInput) {
self.session.addInput(videoInput)
self.videoDeviceInput = videoInput
}
if self.session.canAddOutput(self.output){
self.session.addOutput(self.output)
}
if self.session.canAddOutput(self.photoOutput){
self.session.addOutput(self.photoOutput)
}
//for audio mixing, make sure this is default set to true
self.session.automaticallyConfiguresApplicationAudioSession = true
self.session.commitConfiguration()
}
}
catch{
print(error.localizedDescription)
}
}
Here is my start recording code that gets triggered when the users uses the camera module to start recording a video with audio that mixes with the background audio:
func startRecording() {
/* add audio input only when user starts recording */
do
{
try AVAudioSession.sharedInstance().setActive(false)
try AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.ambient)
try AVAudioSession.sharedInstance().setCategory(.playAndRecord, mode: .default, options: AVAudioSession.CategoryOptions.mixWithOthers)
try AVAudioSession.sharedInstance().setMode(AVAudioSession.Mode.videoRecording)
try AVAudioSession.sharedInstance().setActive(true)
let audioDevice = AVCaptureDevice.default(for: .audio)
let audioInput = try AVCaptureDeviceInput(device: audioDevice!)
if self.session.canAddInput(audioInput){
self.session.automaticallyConfiguresApplicationAudioSession = false
self.session.addInput(audioInput)
}
} catch {
print("Can't Set Audio Session Category: \(error)")
}
// MARK: Temporary URL for recording Video
let tempURL = NSTemporaryDirectory() + "\(Date()).mov"
//Need to correct image orientation before moving further
if let videoOutputConnection = output.connection(with: .video) {
//For frontCamera settings to capture mirror image
if self.videoDeviceInput.device.position == .front {
videoOutputConnection.automaticallyAdjustsVideoMirroring = false
videoOutputConnection.isVideoMirrored = true
} else {
videoOutputConnection.automaticallyAdjustsVideoMirroring = true
}
}
output.startRecording(to: URL(fileURLWithPath: tempURL), recordingDelegate: self)
isRecording = true
}
Here is my stop recording method:
func stopRecording(){
output.stopRecording()
isRecording = false
self.flashOn = false
/* attempts to remove Audio input */
do{
try AVAudioSession.sharedInstance().setActive(false)
let audioDevice = AVCaptureDevice.default(for: .audio)
let audioInput = try AVCaptureDeviceInput(device: audioDevice!)
self.session.removeInput(audioInput)
let audioSession = AVAudioSession.sharedInstance()
try audioSession.setCategory(.ambient, mode: .default, options: [.mixWithOthers])
try AVAudioSession.sharedInstance().setActive(true)
} catch {
print("Error occurred while removing audio device input: \(error)")
}
}
The stopRecording method outputs the error which makes the camera no longer attach the audio for future recordings but successfully works only for the first recording.
Basically, I feel like I'm really close but I've been splitting my hair trying to debug this for the past week with limited documents and solutions. It works for the first recording session with the audio and future recording sessions no longer have any audio attached to the video that the camera has recorded upon calling start recording again.
I am trying to build an audio app for apple watch. But the problem is whenever I keep my hands down , audio will stop playing.
I have turned background mode on as well.
Can anyone please help me with this? I am stuck at this part.
Here is the Code I have used for playing audio.
func play(url : URL) {
do {
if #available(watchOSApplicationExtension 4.0, *) {
WKExtension.shared().isFrontmostTimeoutExtended = true
} else {
// Fallback on earlier versions
}
self.player = try AVAudioPlayer(contentsOf: url)
player!.prepareToPlay()
player?.delegate = self
player?.play()
print("-----------------")
print("Playing Audio")
print("*****************\nCurrent Time \(String(describing: self.player?.currentTime))")
} catch let error as NSError {
self.player = nil
print(error.localizedDescription)
} catch {
print("*************************")
print("AVAudioPlayer init failed")
}
}
Make sure you are trying to play with Audio Data, not Audio URL and have added policy: .longFormAudio in your category setup. As per Apple documentation, these two settings have to be set for audio to play in background mode.
// Set up the session.
let session = AVAudioSession.sharedInstance()
do {
try session.setCategory(
.playback,
mode: .default,
policy: .longFormAudio
)
} catch let error {
fatalError("*** Unable to set up the audio session: \(error.localizedDescription) ***")
}
// Set up the player.
let player: AVAudioPlayer
do {
player = try AVAudioPlayer(data: audioData)
} catch let error {
print("*** Unable to set up the audio player: \(error.localizedDescription) ***")
// Handle the error here.
return
}
// Activate and request the route.
session.activate(options: []) { (success, error) in
guard error == nil else {
print("*** An error occurred: \(error!.localizedDescription) ***")
// Handle the error here.
return
}
// Play the audio file.
player.play()
}
I have tested this code and its working with only Bluetooth connectivity in Watch application not in watch speaker.
Simply turning on background mode is not enough. You also need to activate the AVAudioSession.
It's all well documented by Apple here: Playing Background Audio.
Configure and Activate the Audio Session
Before you can play audio, you need to set up and activate the audio session.
session.setCategory(AVAudioSession.Category.playback,
mode: .default,
policy: .longForm,
options: [])
Next, activate the session, by calling the activate(options:completionHandler:) method.
session.activate(options: []) { (success, error) in
// Check for an error and play audio.
}
Ref: https://developer.apple.com/documentation/watchkit/playing_background_audio
Example:
var player: AVAudioPlayer?
let session: AVAudioSession = .sharedInstance()
func prepareSession() {
do {
try session.setCategory(AVAudioSession.Category.playback,
mode: .default,
policy: .longForm,
options: [])
}
catch {
print(error)
}
}
func play(url: URL) {
do {
player = try AVAudioPlayer(contentsOf: url)
}
catch {
print(error)
return
}
session.activate(options: []) { (success, error) in
guard error == nil else {
print(error!)
return
}
// Play the audio file
self.player?.play()
}
}
Simple Test:
prepareSession()
if let url = Bundle.main.url(forResource: "test", withExtension: "mp3") {
play(url: url)
}
else {
print("test.mp3 not found in project: put any mp3 file in and name it so")
}
It seems that my app is not working with AirPods. Right now I'm using this code for the playback and record:
let audioSession = AVAudioSession.sharedInstance()
do { try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: AVAudioSessionCategoryOptions.defaultToSpeaker)
}catch {
print("audioSession properties weren't set because of an error.")
}
Will it be enough if I change defaultToSpeaker to allowBluetooth?
P.S. I know it's quite a stupid question because it'd be much simpler to just change this line and check, but I don't have AirPods with me right now, so the only option for me is to upload the new build to Testflight (and I want to do this with minimum iterations).
update: (quite naive approach — but all I need is to use bluetooth headphones if they are available):
func selectDevice(audioSession: AVAudioSession) {
var headphonesExist = false
var bluetoothExist = false
var speakerExist = false
let currentRoute = AVAudioSession.sharedInstance().currentRoute
for output in audioSession.currentRoute.outputs {
print(output)
if output.portType == AVAudioSessionPortHeadphones || output.portType == AVAudioSessionPortHeadsetMic {
headphonesExist = true
}
if output.portType == AVAudioSessionPortBluetoothA2DP || output.portType == AVAudioSessionPortBluetoothHFP {
bluetoothExist = true
}
if output.portType == AVAudioSessionPortBuiltInSpeaker {
speakerExist = true
}
}
if bluetoothExist == true {
do { try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: AVAudioSessionCategoryOptions.allowBluetooth) } catch {
print("error with audiosession: bluetooth")
}
}
}
You need to add support for bluetooth to your options parameter like so:
[AVAudioSessionCategoryOptions.defaultToSpeaker, .allowBluetoothA2DP]
.allowBluetoothA2DP will allow for high quality audio output to the bluetooth device and restrict microphone input on said device, while .allowBluetooth will default HFP compatible (input/output) bluetooth devices to the lower quality HFP, which supports microphone input.
Here is a full source to request proper permission.
All you have to do is to add a mode with '.allowBluetoothA2DP'
I've applied on Swift 5
func requestPermissionForAudioRecording(){
recordingSession = AVAudioSession.sharedInstance()
do {
// only with this without options, will not capable with your Airpod, the Bluetooth device.
// try recordingSession.setCategory(.playAndRecord, mode: .default)
try recordingSession.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowAirPlay, .allowBluetoothA2DP])
try recordingSession.setActive(true)
recordingSession.requestRecordPermission() { allowed in
DispatchQueue.main.async {
if allowed {
// Recording permission has been allowed
self.recordingPermissionGranted = true
} else {
// failed to record!
}
}
}
} catch let err {
// failed to record!
print("AudioSession couldn't be set!", err)
}
}
I am using AVAudioSession to listen to voice input. it works fine for wired headphones but it is not working for connected bluetooth device. Following is the code I am using to set input to bluetooth mic
func setupSessionForRecording() {
let audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: [.allowBluetooth])
} catch let error as NSError {
debugPrint("Error in listening "+error.localizedDescription)
}
var inputsPriority: [(type: String, input: AVAudioSessionPortDescription?)] = [
(AVAudioSessionPortLineIn, nil),
(AVAudioSessionPortHeadsetMic, nil),
(AVAudioSessionPortBluetoothHFP, nil),
(AVAudioSessionPortUSBAudio, nil),
(AVAudioSessionPortCarAudio, nil),
(AVAudioSessionPortBuiltInMic, nil),
]
for availableInput in audioSession.availableInputs! {
guard let index = inputsPriority.index(where: { $0.type == availableInput.portType }) else { continue }
inputsPriority[index].input = availableInput
}
guard let input = inputsPriority.filter({ $0.input != nil }).first?.input else {
fatalError("No Available Ports For Recording")
}
do {
try audioSession.setPreferredInput(input)
try audioSession.setMode(AVAudioSessionModeMeasurement)
try audioSession.setActive(true, with: .notifyOthersOnDeactivation)
try audioSession.setPreferredIOBufferDuration(10)
} catch {
fatalError("Error Setting Up Audio Session")
}
}
This code stops taking input from device mic and I also get sound in bluetooth headset that it is ready to listen but it doesn't pick any input from device.
Also,
When I am trying to play any audio into bluetooth headset It doesn't work. Here is the code to play audio
do {
let output = AVAudioSession.sharedInstance().currentRoute.outputs[0].portType
if output == "Receiver" || output == "Speaker"{
try AVAudioSession.sharedInstance().overrideOutputAudioPort(.speaker)
}
else{
try AVAudioSession.sharedInstance().overrideOutputAudioPort(.none)
}
print("Voice Out \(output)" )
} catch let error as NSError {
print("audioSession error: \(error.localizedDescription)")
os_log("Error during changing the current audio route: %#" , log: PollyVoiceViewController.log, type: .error, error)
} catch {
os_log("Unknown error during changing the current audio route", log: PollyVoiceViewController.log, type: .error)
}
do {
let soundData = try Data(contentsOf: url as URL)
self.audioPlayer = try AVAudioPlayer(data: soundData)
self.audioPlayer?.prepareToPlay()
self.audioPlayer?.volume = 3.0
self.audioPlayer?.delegate = self
self.audioPlayer?.play()
} catch let error as NSError {
print("Error getting the audio file"+error.description)
}
The reason is: BluetoothHFP is not available in AVAudioSessionModeMeasurement mode
AVAudioSessionModeMeasurement
After you set try audioSession.setMode(AVAudioSessionModeMeasurement), the audioSession.availableInputs is not contain the BluetoothHFP.
This mode is intended for apps that need to minimize the amount of system-supplied signal processing to input and output signals. If recording on devices with more than one built-in microphone, the primary microphone is used.
And in the document of setPreferredInput(_:)
The AVAudioSessionPortDescription must be in the availableInputs array.
The value of the inPort parameter must be one of the AVAudioSessionPortDescription objects in the availableInputs array. If this parameter specifies a port that is not already part of the current audio route and the app’s session controls audio routing, this method initiates a route change to use the preferred port.
And it must set up after setting the mode.
You must set a preferred input port only after setting the audio session’s category and mode and activating the session.
i have an app which should record audio when a button i pressed.
In my ViewDidLoad i preapare the recorder, the problem is that streaming audio interrupts when the line 'self.audioRecorder.prepareToRecord()' is called.
My setup :
do {
recordingSession = AVAudioSession.sharedInstance()
try recordingSession.setCategory(AVAudioSessionCategoryRecord, withOptions: [.DuckOthers, .AllowBluetooth, .MixWithOthers])
recordingSession.requestRecordPermission() { [unowned self] (allowed: Bool) -> Void in
dispatch_async(dispatch_get_main_queue()) {
do {
if allowed {
self.audioRecorder = try AVAudioRecorder(URL: self.tempAudioPath, settings: self.settings)
self.audioRecorder.delegate = self
self.audioRecorder.prepareToRecord()
//self.audioRecorder.record()
} else {
// failed to record!
print("No Access to Micro")
}
}catch{}
}
}
} catch {
print (error)
}
is there a way to preapare the audio recorder for record, and continue to play audio in background ? (duck it when recording the audio)
Per Apple's documentation for AVAudioSessionCategoryRecord, "this category silences playback audio". Have you tried setting the category to AVAudioSessionCategoryPlayAndRecord?