This is stupid simple but I cannot get it to work.
I want to stop recording before the phone speaks something. No data is being passed.
let words = "Hello world"
let utt = AVSpeechUtterance(string:words)
stopRecordingWithCompletion() {
voice.speak(utt)
}
func stopRecordinWithCompletion(closure: () -> Void) {
recognitionRequest?.endAudio()
recognitionRequest = nil
recognitionTask?.cancel()
recognitionTask = nil
let inputNode = audioEngine.inputNode
let bus = 0
inputNode?.removeTap(onBus: bus)
self.audioEngine.stop()
closure()
}
What am I doing wrong?
Your current approach is not really ideal for this.
To begin with, AVSpeechSynthesizer provides a delegate you can monitor for changes, including when it is about to speak.
speechSynthesizer(_:willSpeakRangeOfSpeechString:utterance:)
Just observe this, and call your stop function. No closure is required since it is a synchronous function call.
In summary:
Conform to AVSpeechSynthesizerDelegate
Implement speechSynthesizer(_:willSpeakRangeOfSpeechString:utterance:)
When the function above is called, have it call your stopRecording() function
An example of the delegate setup:
extension YourClassHere: AVSpeechSynthesizerDelegate {
func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer,
willSpeakRangeOfSpeechString characterRange: NSRange,
utterance: AVSpeechUtterance) {
stopRecording()
}
}
Related
I have a list of strings that I need to display on the screen and simultaneously convert to speech using speechSynthesizer.speak
The issue I am facing is that I am not able how to figure out how to wait for one utterance to finish and them move to the next one. What happens is that the text is displayed one after the other without syncing with the audio.
func speechAndText(text: String){
let speechUtterance = AVSpeechUtterance(string: text)
speechUtterance.rate = AVSpeechUtteranceMaximumSpeechRate/2.0
speechUtterance.pitchMultiplier = 1.2
speechUtterance.volume = 1
speechUtterance.voice = AVSpeechSynthesisVoice(language: "en-GB")
DispatchQueue.main.async {
self.textView.text = text
}
self.speechSynthesizer.speak(speechUtterance)
}
func speakPara(_ phrases: [String]) {
if let phrase = phrases.first {
let group = DispatchGroup()
group.enter()
DispatchQueue.global(qos: .default).async {
self.speechAndText(text: phrase)
group.leave()
}
group.wait()
let rest = Array(phrases.dropFirst())
if !rest.isEmpty {
self.speakPara(rest)
}
}
}
The speakParafunction takes the list of strings to be uttered and sends one string at a time to speechAndText to convert to speech and write on the display.
Any help would be appreciated. Thank you.
No need for GCD, you can use the delegate methods of AVSpeechSynthesizer.
Set self as delegate:
self.speechSynthesizer.delegate = self
and implement didFinish by speaking the next text.
extension YourClass : AVSpeechSynthesizerDelegate {
func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer,
didFinish utterance: AVSpeechUtterance) {
currentIndex += 1
if currentIndex < phrases.lastIndex {
self.speechAndText(text: phrases[currentIndex])
}
}
}
currentIndex and phrases are two instance properties. phrases stores the array of strings that you want to speak, and currentIndex stores the index of the text that is currently being spoken. speakPara can be implemented like so:
func speakPara(_ phrases: [String]) {
// reset both helper properties
self.phrases = phrases
self.currentIndex = 0
// stop speaking whatever it was speaking and start speaking the first item (if any)
self.speechSynthesizer.stopSpeaking(at: .immediate)
phrases.map { self.speechAndText(text: $0) }
}
I want to pause speech utterance and it has to complete the current sentence it is speaking out then it has to pause but the API provides only two pause types immediate and word not current sentence. I tried this,
myutterance = AVSpeechUtterance(string:readTextView.text)
synth .speak(myutterance)
synth .pauseSpeaking(at: AVSpeechBoundary.immediate)
But it pauses immediately after text is completed.
I just tried whatever you did and it read the whole sentence without giving enough pause,
let someText = "Some Sample text. That will read and pause after every sentence"
let speechSynthesizer = AVSpeechSynthesizer()
let myutterance = AVSpeechUtterance(string:someText)
speechSynthesizer .speak(myutterance)
//Normal reading without pause
speechSynthesizer .pauseSpeaking(at: AVSpeechBoundary.immediate)
To pause after every sentence you can break the whole text to simple components and read them individually in a loop like below by using the postUtteranceDelay property.
//To pause after every sentence
let components = someText.components(separatedBy: ".")
for str in components{
let myutterance = AVSpeechUtterance(string:str)
speechSynthesizer .speak(myutterance)
myutterance.postUtteranceDelay = 1 //Set it to whatever value you would want.
}
To pause after completing the currently speaking sentence we need to do a tiny hack,
var isPaused:Bool = false
let someText = "Some Sample text. That will read and pause after every sentence. The next sentence should'nt be heard if you had pressed the pause button. Let us try this."
let speechSynthesizer = AVSpeechSynthesizer()
var currentUtterance:AVSpeechUtterance?
override func viewDidLoad() {
super.viewDidLoad()
speechSynthesizer.delegate = self
}
#IBAction func startSpeaking(_ sender: Any) {
//Pause after every sentence
let components = someText.components(separatedBy: ".")
for str in components{
let myutterance = AVSpeechUtterance(string:str)
speechSynthesizer .speak(myutterance)
myutterance.postUtteranceDelay = 1
}
}
#IBAction func pauseSpeaking(_ sender: Any) {
isPaused = !isPaused
}
func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, willSpeakRangeOfSpeechString characterRange: NSRange, utterance: AVSpeechUtterance){
print("location\(characterRange.location)")
print("Range + Length\(characterRange.location + characterRange.length)")
print("currentUtterance!.speechString\(currentUtterance!.speechString)")
print("currentUtterance!.count\(currentUtterance!.speechString.characters.count)")
if isPaused && (characterRange.location + characterRange.length) == currentUtterance!.speechString.characters.count{
speechSynthesizer.stopSpeaking(at:.word)
}
}
func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didStart utterance: AVSpeechUtterance){
currentUtterance = utterance
}
I'm trying to set my AVAudioSession to inactive to get back to normal state.
My utterance function:
class SSpeech : NSObject, AVSpeechSynthesizerDelegate {
var group = DispatchGroup();
var queue = DispatchQueue(label: "co.xxxx.speech", attributes: [])
class var sharedInstance: SSpeech {
struct Static {
static var instance: SSpeech?
}
if !(Static.instance != nil) {
Static.instance = SSpeech()
}
return Static.instance!
}
required override init() {
super.init();
self.speechsynt.delegate = self;
}
deinit {
print("deinit SSpeech")
}
let audioSession = AVAudioSession.sharedInstance();
var speechsynt: AVSpeechSynthesizer = AVSpeechSynthesizer()
var queueTalks = SQueue<String>();
func pause() {
speechsynt.pauseSpeaking(at: .word)
}
func talk(_ sentence: String, languageCode code:String = SUtils.selectedLanguage.code, withEndPausing: Bool = false) {
if SUser.sharedInstance.currentUser.value!.speechOn != 1 {
return
}
queue.async{
self.queueTalks.enQueue(sentence)
do {
let category = AVAudioSessionCategoryPlayback;
var categoryOptions = AVAudioSessionCategoryOptions.duckOthers
if #available(iOS 9.0, *) {
categoryOptions.formUnion(AVAudioSessionCategoryOptions.interruptSpokenAudioAndMixWithOthers)
}
try self.audioSession.setCategory(category, with: categoryOptions)
try self.audioSession.setActive(true);
} catch _ {
return;
}
self.utteranceTalk(sentence, initSentence: false, speechsynt: self.speechsynt, languageCode:code, withEndPausing: withEndPausing)
do {
try self.audioSession.setCategory(AVAudioSessionCategoryPlayback, with: AVAudioSessionCategoryOptions.mixWithOthers)
} catch _ {
return;
}
}
}
func utteranceTalk(_ sentence: String, initSentence: Bool, speechsynt: AVSpeechSynthesizer, languageCode:String = "en-US", withEndPausing: Bool = false){
if SUser.sharedInstance.currentUser.value!.speechOn != 1 {
return
}
let nextSpeech:AVSpeechUtterance = AVSpeechUtterance(string: sentence)
nextSpeech.voice = AVSpeechSynthesisVoice(language: languageCode)
if !initSentence {
nextSpeech.rate = 0.4;
}
if(withEndPausing){
nextSpeech.postUtteranceDelay = 0.2;
}
speechsynt.speak(nextSpeech)
}
func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance:AVSpeechUtterance) {
print("Speaker has finished to talk")
queue.async {
do {
try self.audioSession.setActive(false, with: AVAudioSessionSetActiveOptions.notifyOthersOnDeactivation)
}
catch {}
}
}
}
}
My method is correctly called, but my audioSession still active when the utterance is finished. i've tried lot of thing but nothing work :(.
I would suggest using an AvAudioPlayer. They have very easy start and stop commands.
first declare the audio player as a variable
var SoundEffect: AVAudioPlayer!
then select the file you need
let path = Bundle.main.path(forResource: "Untitled2.wav", ofType:nil)!
let url = URL(fileURLWithPath: path)
let sound = try AVAudioPlayer(contentsOf: url)
SoundEffect = sound
sound.numberOfLoops = -1
sound.play()
and to stop the audio player
if SoundEffect != nil {
SoundEffect.stop()
SoundEffect = nil
}
You cannot stop or deactive AudioSession, your app gets it upon launching. Documentation:
An audio session is the intermediary between your app and iOS used to configure your app’s audio behavior. Upon launch, your app automatically gets a singleton audio session.
So method -setActive: does not make your AudioSession "active", it just puts its category and mode configuration into action. For getting back to the "normal state", you could set default settings or just call setActive(false, with:.notifyOthersOnDeactivation), that will be enough.
A part from documentation of AVAudioSession:
Discussion
If another active audio session has higher priority than yours (for
example, a phone call), and neither audio session allows mixing,
attempting to activate your audio session fails. Deactivating your
session will fail if any associated audio objects (such as queues,
converters, players, or recorders) are currently running.
My guess is that the failure to deactivate the session is the running process(es) of your queue as I highlighted in the document quote.
Probably you should make the deactivation process synchronous instead of asynchronous OR make sure that all the running actions under your queue has been processed.
Give this a try:
func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance:AVSpeechUtterance) {
print("Speaker has finished to talk")
queue.sync { // <---- `async` changed to `sync`
do {
try self.audioSession.setActive(false, with: AVAudioSessionSetActiveOptions.notifyOthersOnDeactivation)
}
catch {}
}
}
}
I am working on a camera that is wrapped up in a base viewController, with a delegate style interface for callbacks, so all I have to do as a client is subclass the camera view controller, implement the delegate methods, and add the UI buttons.
My question is about recording video. Video recording is started on a unique background task to ensure that the recording can be written to a temporary file. This is done on my
private let sessionQueue = DispatchQueue(label: "com.andrewferrarone.sessionQueue") session queue:
public func startRecording()
{
guard let movieFileOutput = self.movieFileOutput else { return }
//get video preview layer's video orientation on main queue
guard let videoPreviewLayerOrientation = self.previewView.videoPreviewLayer?.connection.videoOrientation else {
print("handleRecord: videoPreviewLayer is nil")
return
}
self.sessionQueue.async {
if !movieFileOutput.isRecording {
if UIDevice.current.isMultitaskingSupported {
self.backgroundRecordingID = UIApplication.shared.beginBackgroundTask(expirationHandler: nil)
}
//update orientation on the movie file output video connection before recording
let movieFileOutputConnection = self.movieFileOutput?.connection(withMediaType: AVMediaTypeVideo)
movieFileOutputConnection?.videoOrientation = videoPreviewLayerOrientation
//start recording to a temporary file:
let outputFileName = UUID().uuidString
let outputFilePath = (NSTemporaryDirectory() as NSString).appendingPathComponent((outputFileName as NSString).appendingPathExtension("mov")!)
movieFileOutput.startRecording(toOutputFileURL: URL(fileURLWithPath: outputFilePath), recordingDelegate: self)
}
}
}
so the recording is setup as a background task dispatched to self.sessionQueue. When I stop recording I receive an AVCaptureFileOutputRecordingDelegate method. In this method, I want to callback my delegate with the filepath, and then cleanup. How do I ensure that the delegate can persist the recording from the temporary file path before cleanup happens and the temporary file is removed?
public func capture(_ captureOutput: AVCaptureFileOutput!, didFinishRecordingToOutputFileAt outputFileURL: URL!, fromConnections connections: [Any]!, error: Error!)
{
//cleanup func for later
func cleanup()
{
let path = outputFileURL.path
if FileManager.default.fileExists(atPath: path) {
do {
try FileManager.default.removeItem(atPath: path)
}
catch let error {
print("Could not remove file at url: \(outputFileURL), error: \(error.localizedDescription)")
}
}
if let currentBackgroundRecordingID = self.backgroundRecordingID {
self.backgroundRecordingID = UIBackgroundTaskInvalid
if currentBackgroundRecordingID != UIBackgroundTaskInvalid {
UIApplication.shared.endBackgroundTask(currentBackgroundRecordingID)
}
}
}
var success = true
if error != nil {
print("Movie file finishing error: \(error)")
success = ((error as NSError).userInfo[AVErrorRecordingSuccessfullyFinishedKey] as AnyObject).boolValue
}
DispatchQueue.main.async {
self.delegate?.camera(self, didFinishRecordingToOutputFileAt: outputFileURL, success: success)
}
cleanup()
}
So I called my delegate back on the main queue with the results, but then I need to call cleanup() should I do this on the main queue right after calling back my delegate? is this safe? or if I leave it the way it is now, then we are on self.sessionQueue, and I am unsure if cleanup() will happen before the delegate method implementation has time to persist the recording. If anyone can give me some insight into what is going on and what would be the safest thing to do here, that would be great. According to apple, the documentation says do not assume that AVCaptureFileOutputRecordingDelegate didFinishRecordingToOutputFileAt method is called on a specific thread. Thanks for your time and help!
How about:
DispatchQueue.main.async {
self.delegate?.camera(self, didFinishRecordingToOutputFileAt: outputFileURL, success: success)
self.sessionQueue.async {
cleanup()
}
}
I think that would be the standard way of handling this situation. When the delegate method finishes, you assume that the delegate is done with the file (or copied it somewhere else).
It has been a while since I tried to properly mix the SKTTSPlayer with the AVAudioSession and I have many problems that I can't resolve.
I want the SKTTSPlayer to duck the other audio when it is playing and unduck the other audio when it has finished playing.
Normally, with the AVSpeechSynthesizer, it is pretty easy to duck and unduck the other audio, here is a block of code that does this work pretty easily :
override func viewDidLoad() {
super.viewDidLoad()
var audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(AVAudioSessionCategoryPlayback, withOptions: .DuckOthers)
} catch {
// TODO: Handle error
}
synth.delegate = self
speak("I really want to unduck this session")
}
func setAudioSessionActive(beActive: Bool) {
do {
try AVAudioSession.sharedInstance().setActive(beActive)
print("Setting AVAudiosession active: ", beActive)
} catch let error as NSError {
print("Setting AVAudiosession state failed: \(error.description)")
}
}
func speak(string: String) {
let utterance = AVSpeechUtterance(string: string)
utterance.rate = AVSpeechUtteranceDefaultSpeechRate / 2
utterance.voice = AVSpeechSynthesisVoice(language: language)
synth.speakUtterance(utterance)
}
func speechSynthesizer(synthesizer: AVSpeechSynthesizer, didFinishSpeechUtterance utterance: AVSpeechUtterance) {
if (synthesizer.speaking) {
return
}
setAudioSessionActive(false)
}
As you can see, this can do the work pretty easily, you activate the session once and then everytime an utterance is finished, you desactivate the seesion.
This is not so simple with the SKTTSPlayer. Even if it seems that the SKTTSPlayer is an object of AVSpeechSynthesizer modified by Skobbler, you cannot unduck a session by just desactivate it. If you does that, the session will unDuck and after Duck again.
Here is a block of code that can reproduce the error :
override func viewDidLoad() {
super.viewDidLoad()
var audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(AVAudioSessionCategoryPlayback, withOptions: .DuckOthers)
} catch {
// TODO: Handle error
}
let advisorSettings = SKAdvisorSettings()
advisorSettings.language = SKAdvisorLanguage.EN_US
advisorSettings.advisorType = SKAdvisorType.TextToSpeech
SKRoutingService.sharedInstance().advisorConfigurationSettings = advisorSettings
let ttsSettings = SKAdvisorTTSSettings()
ttsSettings.rate = 0.08
ttsSettings.pitchMultiplier = 0.8
ttsSettings.volume = 1
ttsSettings.preUtteranceDelay = 0.1
ttsSettings.postUtteranceDelay = 0.1
SKTTSPlayer.sharedInstance().textToSpeechConfig = ttsSettings
SKTTSPlayer.sharedInstance().delegate = self
speak("I really want to unduck this session")
}
func setAudioSessionActive(beActive: Bool) {
do {
try AVAudioSession.sharedInstance().setActive(beActive)
print("Setting AVAudiosession active: ", beActive)
} catch let error as NSError {
print("Setting AVAudiosession state failed: \(error.description)")
}
}
func speak(string: String) {
SKTTSPlayer.sharedInstance().playString(string, forLanguage: SKAdvisorLanguage.EN_US)
}
func speechSynthesizer(synthesizer: AVSpeechSynthesizer, didFinishSpeechUtterance utterance: AVSpeechUtterance) {
setAudioSessionActive(false)
}
func TTSPlayer(TTSPlayer: SKTTSPlayer!, willPlayUtterance utterance: AVSpeechUtterance!) {
NSLog("will play")
}
The only workaround I found for this problem is to create a blanksound with an AVAudioPlayer. This sound is played everytime an utterance is finished and the session is unDuck when the blanksound is finished.
override func viewDidLoad() {
super.viewDidLoad()
var audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(AVAudioSessionCategoryPlayback, withOptions: .DuckOthers)
} catch {
// TODO: Handle error
}
let advisorSettings = SKAdvisorSettings()
advisorSettings.language = SKAdvisorLanguage.EN_US
advisorSettings.advisorType = SKAdvisorType.TextToSpeech
SKRoutingService.sharedInstance().advisorConfigurationSettings = advisorSettings
let ttsSettings = SKAdvisorTTSSettings()
ttsSettings.rate = 0.08
ttsSettings.pitchMultiplier = 0.8
ttsSettings.volume = 1
ttsSettings.preUtteranceDelay = 0.1
ttsSettings.postUtteranceDelay = 0.1
SKTTSPlayer.sharedInstance().textToSpeechConfig = ttsSettings
SKTTSPlayer.sharedInstance().delegate = self
speak("I really want to unduck this session")
}
func setAudioSessionActive(beActive: Bool) {
do {
try AVAudioSession.sharedInstance().setActive(beActive)
print("Setting AVAudiosession active: ", beActive)
} catch let error as NSError {
print("Setting AVAudiosession state failed: \(error.description)")
}
}
func speak(string: String) {
SKTTSPlayer.sharedInstance().playString(string, forLanguage: SKAdvisorLanguage.EN_US)
}
func speechSynthesizer(synthesizer: AVSpeechSynthesizer, didFinishSpeechUtterance utterance: AVSpeechUtterance) {
initializeAudioPlayer()
}
func TTSPlayer(TTSPlayer: SKTTSPlayer!, willPlayUtterance utterance: AVSpeechUtterance!) {
NSLog("will play")
}
func initializeAudioPlayer(){
let blankSoundURL = NSBundle.mainBundle().pathForResource("blankSound", ofType: "mp3")
if !NSFileManager.defaultManager().fileExistsAtPath(blankSoundURL!)
{
return
}
else
{
do {
audioPlayer = try? AVAudioPlayer(contentsOfURL: NSURL(fileURLWithPath: blankSoundURL!), fileTypeHint: nil)
audioPlayer.delegate = self
audioPlayer.prepareToPlay()
audioPlayer.play()
}
}
}
func audioPlayerDidFinishPlaying(player: AVAudioPlayer, successfully flag: Bool) {
setAudioSessionActive(false)
}
This is, I think, pretty difficult for nothing and it makes the AVAudioSession hard to work with.
Do you guys have any ideas why I can't unDuck a session with the SKTTSPlayer in the normal way ?
Thank you for reading, I know this is a pretty long post, but I think it might help someone.
Using the TTS option
Set our audio engine to TTS (this way you'll receive the text that needs
to be played)
Implement the didUpdateFilteredAudioInstruction callback here is where
the text instruction is received.
At this time you will have the play the instruction using your custom
TTS engine this is implementation specific to your 3'rd party engine
Return false (this will stop the default tts engine from playing the
instruction again)
The end code will look something like this:
(BOOL)routingService:(SKRoutingService )routingService
didUpdateFilteredAudioInstruction:(NSString)instruction
forLanguage:(SKAdvisorLanguage)language {
//your code goes here - you need to play the "instruction" in a
certain "language"
return false;
}
Using your own AVSpeechSynthesizer:
If it's a TTS engine issue then the only way of addressing this would be
to change the TTS engine you are using (we have limited control over it
see
http://developer.skobbler.com/docs/ios/2.5.1/Classes/SKAdvisorTTSSettings.h
tml ). See also the iOS API documentation:
https://developer.apple.com/library/prerelease/ios/documentation/AVFoundati
on/Reference/AVSpeechSynthesizer_Ref/index.html