I'd like to know how to properly let my iPhone speak one sentence while my app is in background but then return to whatever was playing before.
My question is quite similar to AVSpeechSynthesizer in background mode but again with the difference that I want to be able to "say something" while in background without having to stop Music that is playing. So while my AVSpeechSynthesizer is speaking, music should pause (or be a bit less loud) but then it should resume. Even when my app is currently in background.
What I am trying to archive is a spoken summary of tracking-stats while GPS-Tracking in my fitness app. And chances are that you are listening to music is quite high, and I don't want to disturb the user...
I found the answer myself...
The important part ist to configure the AVAudioSession with the .duckOthers option:
let audioSession = AVAudioSession.sharedInstance()
try audioSession.setCategory(AVAudioSessionCategoryPlayback, with: .duckOthers)
This will make playback of f.e. music less loud but this would make it stay less loud even when speech is done. This is why you need to set a delegate for the AVSpeechSynthesizer and then handle it like so:
func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance: AVSpeechUtterance) {
guard !synthesizer.isSpeaking else { return }
let audioSession = AVAudioSession.sharedInstance()
try? audioSession.setActive(false)
}
That way, music will continue with normal volume after speech is done.
Also, right before speaking, I activate my audioSession just to make sure (not sure if that would really be necessary, but since I do so, I have no more problems...)
For swift 3, import AVKit then add
try? AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
for swift 4.2 / Xcode 10
unfortunately .duckOthers is no longer available; I managed to make it work like that:
let audioSession = AVAudioSession.sharedInstance()
try! audioSession.setCategory(AVAudioSession.Category.ambient, mode: .default)
Swift 5.0
let synthesizer = AVSpeechSynthesizer()
let synthesizerVoice = AVSpeechSynthesisVoice(language: "en-US")
let str = "String"
let utterance = AVSpeechUtterance(string: str)
let audioSession = AVAudioSession.sharedInstance()
try! audioSession.setCategory(
AVAudioSession.Category.playback,
options: AVAudioSession.CategoryOptions.mixWithOthers
)
utterance.rate = 0.5
utterance.voice = synthesizerVoice
synthesizer.speak(utterance)
Background music don't stop and your word sound play even phone in silent mode by this two line of code before usual text to speech codes :
let audioSession = AVAudioSession.sharedInstance()
try!audioSession.setCategory(AVAudioSessionCategoryPlayback, with: AVAudioSessionCategoryOptions.mixWithOthers)
Related
I'm trying to create an app with Swift.
I integrated correctly speech-to-text and text-to-speech: my app works perfectly. You can find my project here.
After speech-to-text, the app makes an http request to a server (sending the text recognized) and the response (It is a string, i.e.: "Ok, I'll show you something") is reproduced vocally from text-to-speech. But, there is a big issue and I can't solve it.
When the app is reproducing the text vocally, the voice is too slow, as if it were in the background, as if there was something to be reproduced more important than the voice (actually nothing).
Debugging, I discovered that the issue starts using audioEngine (AVAudioEngine) inside the function recordAndRecognizeSpeech(). Running the app without using this function and playing a random text it works like a charm.
So, in my opinion when the app is reproducing the text vocally, it thinks there is still active audioengine, so the volume is very slow.
But, before reproducing the text, I called these functions (look inside ac function, line 96):
audioEngine.stop()
audioEngine.reset()
How can I solve this issue?
EDIT:
I found a partial solution. Now before the app plays the text vocally my code is:
audioEngine.inputNode.removeTap(onBus: 0)
audioEngine.stop()
audioEngine.reset()
recognitionTask?.cancel()
isRecording = false
microphoneButton.setTitle("Avvia..", for: UIControl.State.normal);
do {
let audioSession = AVAudioSession.sharedInstance()
try audioSession.setCategory(AVAudioSession.Category.ambient)
try audioSession.setActive(false, options: .notifyOthersOnDeactivation)
} catch {
print(error)
}
make_request(msg: self.speech_result.text!)
The function .setCategory works and the volume is like the default one. When I try to recall recordAndRecognizeSpeech() function, the app gives me this exception:
VAEInternal.h:70:_AVAE_Check: required condition is false: [AVAudioIONodeImpl.mm:910:SetOutputFormat: (IsFormatSampleRateAndChannelCountValid(hwFormat))]
This exception is caused by .setCategory(AVAudioSession.Category.ambient), it should be .playAndRecord, but with this value the volume returns to be low.
try this one.
let speaker = AVSpeechSynthesizer()
func say(text: String, language: String) {
// Start audio session
let audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(AVAudioSession.Category.playAndRecord)
try audioSession.setMode(AVAudioSession.Mode.default)
try audioSession.setActive(true)
try AVAudioSession.sharedInstance().overrideOutputAudioPort(AVAudioSession.PortOverride.speaker)
} catch {
return
}
if speaker.isSpeaking {
speaker.stopSpeaking(at: .immediate)
} else {
myUtterance = AVSpeechUtterance(string: text)
myUtterance.rate = AVSpeechUtteranceDefaultSpeechRate
myUtterance.voice = AVSpeechSynthesisVoice(language: language)
myUtterance.pitchMultiplier = 1
myUtterance.volume = 2
DispatchQueue.main.async {
self.speaker.speak(myUtterance)
}
}
}
Try This .
set rate for play speedly
var speedd = AVSpeechSynthesizer()
var voicert = AVSpeechUtterance()
voicert = AVSpeechUtterance(string: "Your post appears to contain code that is not properly formatted as code. Please indent all code by 4 spaces using the code toolbar button or the CTRL+K keyboard shortcut. For more editing help, click the [?] toolbar icon")
voicert.voice = AVSpeechSynthesisVoice(language: "en-US")
voicert.rate = 0.5
speedd.speak(voicert)
I am successfully able to use Speech (speech recognition) and I can use AVFoundation to play wav files in Xcode 8/IOS 10. I just can't use them both together. I have working speech recognition code where I import Speech. When I import AVFoundation into the same app and use the following code, there is no sound and no errors are generated:
var audioPlayer: AVAudioPlayer!
func playAudio() {
let path = Bundle.main.path(forResource: "file.wav", ofType: nil)!
let url = URL(fileURLWithPath: path)
do {
let sound = try AVAudioPlayer(contentsOf: url)
audioPlayer = sound
sound.play()
} catch {
//handle error
}
}
I assume it is because both use audio. Can anyone suggest how to use both in the same app? I also find that I cannot use speech recognition and text-to-speech together in the same app.
I just bumped into the same problem, and here is how I solved,
add the following line when Speech Recognition is done. What it does is basically setting the audio session back to AVAudioSessionCategoryPlayback category.
let audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(AVAudioSessionCategoryPlayback)
try audioSession.setActive(false, with: .notifyOthersOnDeactivation)
} catch {
// handle errors
}
hope it helps.
You should change this line:
try audioSession.setCategory(AVAudioSessionCategoryPlayback)
on to:
try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord)
This should work ;-)
It seems that AVAudioPlayer stops playing the sample if you're using AVAudioSession to record the microphone as in Apple's speech recognition example.
However, I've managed to circumvent this by using AVCaptureSession to capture audio as described in this answer.
My app has a "Click" sound functionality. I used the
import AVFoundation
then the following function to run the "Click" sound:
var audioPlayer = AVAudioPlayer()
func playSound() {
var soundPath = NSBundle.mainBundle().pathForResource("tick", ofType: "wav")
var soundURL = NSURL.fileURLWithPath(soundPath!)
self.audioPlayer = AVAudioPlayer(contentsOfURL: soundURL, error: nil)
self.audioPlayer.play()
}
Now if the user is running a music player, my app causes the music player to stop. I read about the Audio Session Default Behavior in the documentation, but I don't know how to apply it.
Can you please help?
Thank you!
If you are wondering the syntax for swift 2, here it is:
let audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, withOptions: .DuckOthers)
} catch {
print("AVAudioSession cannot be set: \(error)")
}
Depending on what you want the app to behave, i.e, how your app's sound effect or music should interact with other app's background audio session, you might need to tweak both the audio session category and categoryOption.
If you just want to play the sound effect, like "tick" sound, then, AVAudioSessionCategoryAmbient and DuckOthers should be used respectively, for example:
let audioSession = AVAudioSession.sharedInstance()
var error: NSErrorPointer = nil
audioSession.setCategory(AVAudioSessionCategoryAmbient, withOptions: .DuckOthers, error: error)
However, I suppose you are actually trying to play a sound effect, in this case, the AudioServices API is a more suitable choice. You can check func AudioServicesPlaySystemSound(inSystemSoundID: SystemSoundID) in AudioToolbox framework for more details.
Another common scenario. If you want to have your app to play audio exclusively, even if there're other app's playing the music in the background, you need to set the category to AVAudioSessionCategorySoloAmbient, for example:
let audioSession = AVAudioSession.sharedInstance()
var error: NSErrorPointer = nil
audioSession.setCategory(AVAudioSessionCategorySoloAmbient, error: error)
I hope you've got what you're looking for.
I'm trying to add an audio input to my AVCaptureSession() and it works great. However I would also like to support users who wish to play music in the background from other apps such as Spotify and maintain this audioInput for my recording. How is this possible?
let captureSession = AVCaptureSession()
let audioDevice = AVCaptureDevice.defaultDeviceWithMediaType(AVMediaTypeAudio)
audioInput = AVCaptureDeviceInput.deviceInputWithDevice(audioDevice, error:&err) as? AVCaptureDeviceInput
if captureSession.canAddInput(videoCapture) {
captureSession.addInput(videoCapture)
// This line Kills spotify playing in the background
captureSession.addInput(audioInput as AVCaptureInput)
}
The category AVAudioSessionCategoryPlayback is one of the few categories that allow for backgrounding. The option AVAudioSessionCategoryOptionMixWithOthers will make sure your audio won’t stop any currently playing background audio and also make sure that when the user plays music in the future, it won’t kick off your background task.
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback, withOptions: .MixWithOthers)
Maybe this will be helpful
AVSpeechSynthesizer has a fairly simple API, which doesn't have support for saving to an audio file built-in.
I'm wondering if there's a way around this - perhaps recording the output as it's played silently, for playback later? Or something more efficient.
This is finally possible, in iOS 13 AVSpeechSynthesizer now has write(_:toBufferCallback:):
let synthesizer = AVSpeechSynthesizer()
let utterance = AVSpeechUtterance(string: "test 123")
utterance.voice = AVSpeechSynthesisVoice(language: "en")
var output: AVAudioFile?
synthesizer.write(utterance) { (buffer: AVAudioBuffer) in
guard let pcmBuffer = buffer as? AVAudioPCMBuffer else {
fatalError("unknown buffer type: \(buffer)")
}
if pcmBuffer.frameLength == 0 {
// done
} else {
// append buffer to file
if output == nil {
output = AVAudioFile(
forWriting: URL(fileURLWithPath: "test.caf"),
settings: pcmBuffer.format.settings,
commonFormat: .pcmFormatInt16,
interleaved: false)
}
output?.write(from: pcmBuffer)
}
}
As of now AVSpeechSynthesizer does not support this . There in no way get the audio file using AVSpeechSynthesizer . I tried this few weeks ago for one of my apps and found out that it is not possible , Also nothing has changed for AVSpeechSynthesizer in iOS 8.
I too thought of recording the sound as it is being played , but there are so many flaws with that approach like user might be using headphones, the system sound might be low or mute , it might catch other external sound, so its not advisable to go with that approach.
You can use OSX to prepare AIFF files (or, maybe, some OSX-based service) via NSSpeechSynthesizer method
startSpeakingString:toURL: