So after a lot of searching I was able to find the code block that allows background audio to play while at the same time recording video.
I have pasted said code block below.
fileprivate func setBackgroundAudioPreference() {
guard allowBackgroundAudio == true else {
return
}
guard audioEnabled == true else {
return
}
do{
if #available(iOS 10.0, *) {
try AVAudioSession.sharedInstance().setCategory(.playAndRecord, mode: .default, options: [.mixWithOthers, .allowBluetooth, .allowAirPlay, .allowBluetoothA2DP])
} else {
let options: [AVAudioSession.CategoryOptions] = [.mixWithOthers, .allowBluetooth]
let category = AVAudioSession.Category.playAndRecord
let selector = NSSelectorFromString("setCategory:withOptions:error:")
AVAudioSession.sharedInstance().perform(selector, with: category, with: options)
}
try AVAudioSession.sharedInstance().setActive(true)
session.automaticallyConfiguresApplicationAudioSession = false
}
catch {
print("[SwiftyCam]: Failed to set background audio preference")
}
}
However, I have one small issue. For some reason when the camera loads the background Audio volume is reduced. When I record a video with instagram the audio doesn't get reduced and it still records is there any way I can change my current code block to not lower the volume while recoding with video?
I read the documentation and apparently .duckOthers option should be the only option that reduces the volume. But this one does as well
Okay so I found the answer after diving further into some of the documentation.
Updated code posted below. All you have to do is set the .defaultToSpeaker option
fileprivate func setBackgroundAudioPreference() {
guard allowBackgroundAudio == true else {
return
}
guard audioEnabled == true else {
return
}
do{
if #available(iOS 10.0, *) {
try AVAudioSession.sharedInstance().setCategory(.playAndRecord, mode: .default, options: [.mixWithOthers, .allowBluetooth, .allowAirPlay, .allowBluetoothA2DP,.defaultToSpeaker])
} else {
let options: [AVAudioSession.CategoryOptions] = [.mixWithOthers, .allowBluetooth]
let category = AVAudioSession.Category.playAndRecord
let selector = NSSelectorFromString("setCategory:withOptions:error:")
AVAudioSession.sharedInstance().perform(selector, with: category, with: options)
}
try AVAudioSession.sharedInstance().setActive(true)
session.automaticallyConfiguresApplicationAudioSession = false
}
catch {
print("[SwiftyCam]: Failed to set background audio preference")
}
}
Related
I am trying to build an audio app for apple watch. But the problem is whenever I keep my hands down , audio will stop playing.
I have turned background mode on as well.
Can anyone please help me with this? I am stuck at this part.
Here is the Code I have used for playing audio.
func play(url : URL) {
do {
if #available(watchOSApplicationExtension 4.0, *) {
WKExtension.shared().isFrontmostTimeoutExtended = true
} else {
// Fallback on earlier versions
}
self.player = try AVAudioPlayer(contentsOf: url)
player!.prepareToPlay()
player?.delegate = self
player?.play()
print("-----------------")
print("Playing Audio")
print("*****************\nCurrent Time \(String(describing: self.player?.currentTime))")
} catch let error as NSError {
self.player = nil
print(error.localizedDescription)
} catch {
print("*************************")
print("AVAudioPlayer init failed")
}
}
Make sure you are trying to play with Audio Data, not Audio URL and have added policy: .longFormAudio in your category setup. As per Apple documentation, these two settings have to be set for audio to play in background mode.
// Set up the session.
let session = AVAudioSession.sharedInstance()
do {
try session.setCategory(
.playback,
mode: .default,
policy: .longFormAudio
)
} catch let error {
fatalError("*** Unable to set up the audio session: \(error.localizedDescription) ***")
}
// Set up the player.
let player: AVAudioPlayer
do {
player = try AVAudioPlayer(data: audioData)
} catch let error {
print("*** Unable to set up the audio player: \(error.localizedDescription) ***")
// Handle the error here.
return
}
// Activate and request the route.
session.activate(options: []) { (success, error) in
guard error == nil else {
print("*** An error occurred: \(error!.localizedDescription) ***")
// Handle the error here.
return
}
// Play the audio file.
player.play()
}
I have tested this code and its working with only Bluetooth connectivity in Watch application not in watch speaker.
Simply turning on background mode is not enough. You also need to activate the AVAudioSession.
It's all well documented by Apple here: Playing Background Audio.
Configure and Activate the Audio Session
Before you can play audio, you need to set up and activate the audio session.
session.setCategory(AVAudioSession.Category.playback,
mode: .default,
policy: .longForm,
options: [])
Next, activate the session, by calling the activate(options:completionHandler:) method.
session.activate(options: []) { (success, error) in
// Check for an error and play audio.
}
Ref: https://developer.apple.com/documentation/watchkit/playing_background_audio
Example:
var player: AVAudioPlayer?
let session: AVAudioSession = .sharedInstance()
func prepareSession() {
do {
try session.setCategory(AVAudioSession.Category.playback,
mode: .default,
policy: .longForm,
options: [])
}
catch {
print(error)
}
}
func play(url: URL) {
do {
player = try AVAudioPlayer(contentsOf: url)
}
catch {
print(error)
return
}
session.activate(options: []) { (success, error) in
guard error == nil else {
print(error!)
return
}
// Play the audio file
self.player?.play()
}
}
Simple Test:
prepareSession()
if let url = Bundle.main.url(forResource: "test", withExtension: "mp3") {
play(url: url)
}
else {
print("test.mp3 not found in project: put any mp3 file in and name it so")
}
I'm implementing video chat using webrtc. In that I want use the main speaker when the other participant joins the session. For that, I wrote this code, but I'm getting a low voice volume (means voice coming from ear speaker)
func audioSetting() {
RTCAudioSession.sharedInstance().isAudioEnabled = true
let session = AVAudioSession.sharedInstance()
do {
try session.setCategory(AVAudioSession.Category.playAndRecord, mode: .videoChat, options: [])
if self.speakerOn {
try session.overrideOutputAudioPort(.none)
}
else {
try session.overrideOutputAudioPort(.speaker)
}
try session.setActive(true)
self.speakerOn = !self.speakerOn
}
catch let error {
print("Couldn't set audio to speaker: \(error)")
}
}
I am working on webRTC with socket.IO,
func setSpeakerStates(enabled: Bool)
{
let session = AVAudioSession.sharedInstance()
var _: Error?
try? session.setCategory(AVAudioSession.Category.playAndRecord)
try? session.setMode(AVAudioSession.Mode.voiceChat)
if enabled {
try? session.overrideOutputAudioPort(AVAudioSession.PortOverride.speaker)
} else {
try? session.overrideOutputAudioPort(AVAudioSession.PortOverride.none)
}
try? session.setActive(true)
}
Please try this method at the end of the viewdidload after adding audio streaming and video streaming.
I am trying to create an app that leverages both STT (Speech to Text) and TTS (Text to Speech) at the same time. However, I am running into a couple of foggy issues and would appreciate your kind expertise.
The app consists of a button at the center of the screen which, upon clicking, starts the required speech recognition functionality using the code below.
// MARK: - Constant Properties
let audioEngine = AVAudioEngine()
// MARK: - Optional Properties
var recognitionRequest: SFSpeechAudioBufferRecognitionRequest?
var recognitionTask: SFSpeechRecognitionTask?
var speechRecognizer: SFSpeechRecognizer?
// MARK: - Functions
internal func startSpeechRecognition() {
// Instantiate the recognitionRequest property.
self.recognitionRequest = SFSpeechAudioBufferRecognitionRequest()
// Set up the audio session.
let audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(.record, mode: .measurement, options: [.defaultToSpeaker, .duckOthers])
try audioSession.setActive(true, options: .notifyOthersOnDeactivation)
} catch {
print("An error has occurred while setting the AVAudioSession.")
}
// Set up the audio input tap.
let inputNode = self.audioEngine.inputNode
let inputNodeFormat = inputNode.outputFormat(forBus: 0)
self.audioEngine.inputNode.installTap(onBus: 0, bufferSize: 512, format: inputNodeFormat, block: { [unowned self] buffer, time in
self.recognitionRequest?.append(buffer)
})
// Start the recognition task.
guard
let speechRecognizer = self.speechRecognizer,
let recognitionRequest = self.recognitionRequest else {
fatalError("One or more properties could not be instantiated.")
}
self.recognitionTask = speechRecognizer.recognitionTask(with: recognitionRequest, resultHandler: { [unowned self] result, error in
if error != nil {
// Stop the audio engine and recognition task.
self.stopSpeechRecognition()
} else if let result = result {
let bestTranscriptionString = result.bestTranscription.formattedString
self.command = bestTranscriptionString
print(bestTranscriptionString)
}
})
// Start the audioEngine.
do {
try self.audioEngine.start()
} catch {
print("Could not start the audioEngine property.")
}
}
internal func stopSpeechRecognition() {
// Stop the audio engine.
self.audioEngine.stop()
self.audioEngine.inputNode.removeTap(onBus: 0)
// End and deallocate the recognition request.
self.recognitionRequest?.endAudio()
self.recognitionRequest = nil
// Cancel and deallocate the recognition task.
self.recognitionTask?.cancel()
self.recognitionTask = nil
}
When used alone, this code works like a charm. However, when I want to read that transcribed text using an AVSpeechSynthesizer object, nothing seems to be clear.
I went through the suggestions of multiple Stack Overflow posts, which suggested modifying
audioSession.setCategory(.record, mode: .measurement, options: [.defaultToSpeaker, .duckOthers])
To the following
audioSession.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .duckOthers])
Yet in vain. The app was still crashing after running STT then TTS, respectively.
The solution was for me to use this rather than the aforementioned
audioSession.setCategory(.multiRoute, mode: .default, options: [.defaultToSpeaker, .duckOthers])
This got me completely overwhelmed as I really have no clue what was intricately going on. I would highly appreciate any relevant explanation!
I am developing an app with both SFSpeechRecognizer and AVSpeechSythesizer too, and for me the .setCategory(.playAndRecord, mode: .default) works fine and it is the best category for our needs, according to Apple. Even, I am able to .speak() every transcription of the SFSpeechRecognitionTask while the audio engine is running without any problem. My opinion is somewhere in your programm's logic causes the crash. It would be good if you can update your question with the corresponding error.
And about why the .multiRoute category works: I guess there is a problem with the AVAudioInputNode. If you see in the console and error like this
Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: IsFormatSampleRateAndChannelCountValid(hwFormat)
or like this
Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: nullptr == Tap()
you only needs to reorder some parts of the code like moving the setup of the audio session somewhere where it only gets called once, or ensure that the tap of the input node is always removed before installing a new one even if the recognition task finish successfully or not. And maybe (I have never worked with it) the .multiRoute is able to reuse the same input node by its nature of working with different audio streams and routes.
I leave below the logic I use with my programm following Apple's WWDC session:
Setting category
override func viewDidLoad() { //or init() or when necessarily
super.viewDidLoad()
try? AVAudioSession.sharedInstance().setCategory(.playAndRecord, mode: .default)
}
Validations/permissions
func shouldProcessSpeechRecognition() {
guard AVAudioSession.sharedInstance().recordPermission == .granted,
speechRecognizerAuthorizationStatus == .authorized,
let speechRecognizer = speechRecognizer, speechRecognizer.isAvailable else { return }
//Continue only if we have authorization and recognizer is available
startSpeechRecognition()
}
Starting STT
func startSpeechRecognition() {
let format = audioEngine.inputNode.outputFormat(forBus: 0)
audioEngine.inputNode.installTap(onBus: 0, bufferSize: 1024, format: format) { [unowned self] (buffer, _) in
self.recognitionRequest.append(buffer)
}
audioEngine.prepare()
do {
try audioEngine.start()
recognitionTask = speechRecognizer!.recognitionTask(with: recognitionRequest, resultHandler: {...}
} catch {...}
}
Ending STT
func endSpeechRecognition() {
recognitionTask?.finish()
stopAudioEngine()
}
Canceling STT
func cancelSpeechRecognition() {
recognitionTask?.cancel()
stopAudioEngine()
}
Stoping audio engine
func stopAudioEngine() {
audioEngine.stop()
audioEngine.inputNode.removeTap(onBus: 0)
recognitionRequest.endAudio()
}
And with that, anywhere in my code I can call an AVSpeechSynthesizer instance and speak an utterance.
It seems that my app is not working with AirPods. Right now I'm using this code for the playback and record:
let audioSession = AVAudioSession.sharedInstance()
do { try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: AVAudioSessionCategoryOptions.defaultToSpeaker)
}catch {
print("audioSession properties weren't set because of an error.")
}
Will it be enough if I change defaultToSpeaker to allowBluetooth?
P.S. I know it's quite a stupid question because it'd be much simpler to just change this line and check, but I don't have AirPods with me right now, so the only option for me is to upload the new build to Testflight (and I want to do this with minimum iterations).
update: (quite naive approach — but all I need is to use bluetooth headphones if they are available):
func selectDevice(audioSession: AVAudioSession) {
var headphonesExist = false
var bluetoothExist = false
var speakerExist = false
let currentRoute = AVAudioSession.sharedInstance().currentRoute
for output in audioSession.currentRoute.outputs {
print(output)
if output.portType == AVAudioSessionPortHeadphones || output.portType == AVAudioSessionPortHeadsetMic {
headphonesExist = true
}
if output.portType == AVAudioSessionPortBluetoothA2DP || output.portType == AVAudioSessionPortBluetoothHFP {
bluetoothExist = true
}
if output.portType == AVAudioSessionPortBuiltInSpeaker {
speakerExist = true
}
}
if bluetoothExist == true {
do { try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: AVAudioSessionCategoryOptions.allowBluetooth) } catch {
print("error with audiosession: bluetooth")
}
}
}
You need to add support for bluetooth to your options parameter like so:
[AVAudioSessionCategoryOptions.defaultToSpeaker, .allowBluetoothA2DP]
.allowBluetoothA2DP will allow for high quality audio output to the bluetooth device and restrict microphone input on said device, while .allowBluetooth will default HFP compatible (input/output) bluetooth devices to the lower quality HFP, which supports microphone input.
Here is a full source to request proper permission.
All you have to do is to add a mode with '.allowBluetoothA2DP'
I've applied on Swift 5
func requestPermissionForAudioRecording(){
recordingSession = AVAudioSession.sharedInstance()
do {
// only with this without options, will not capable with your Airpod, the Bluetooth device.
// try recordingSession.setCategory(.playAndRecord, mode: .default)
try recordingSession.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowAirPlay, .allowBluetoothA2DP])
try recordingSession.setActive(true)
recordingSession.requestRecordPermission() { allowed in
DispatchQueue.main.async {
if allowed {
// Recording permission has been allowed
self.recordingPermissionGranted = true
} else {
// failed to record!
}
}
}
} catch let err {
// failed to record!
print("AudioSession couldn't be set!", err)
}
}
i have an app which should record audio when a button i pressed.
In my ViewDidLoad i preapare the recorder, the problem is that streaming audio interrupts when the line 'self.audioRecorder.prepareToRecord()' is called.
My setup :
do {
recordingSession = AVAudioSession.sharedInstance()
try recordingSession.setCategory(AVAudioSessionCategoryRecord, withOptions: [.DuckOthers, .AllowBluetooth, .MixWithOthers])
recordingSession.requestRecordPermission() { [unowned self] (allowed: Bool) -> Void in
dispatch_async(dispatch_get_main_queue()) {
do {
if allowed {
self.audioRecorder = try AVAudioRecorder(URL: self.tempAudioPath, settings: self.settings)
self.audioRecorder.delegate = self
self.audioRecorder.prepareToRecord()
//self.audioRecorder.record()
} else {
// failed to record!
print("No Access to Micro")
}
}catch{}
}
}
} catch {
print (error)
}
is there a way to preapare the audio recorder for record, and continue to play audio in background ? (duck it when recording the audio)
Per Apple's documentation for AVAudioSessionCategoryRecord, "this category silences playback audio". Have you tried setting the category to AVAudioSessionCategoryPlayAndRecord?