Recording Audio on iOS Without Stopping Music Playback - ios

When recording audio with an AVAudioSession audio playback from the music app or others is stopped. This is described in the audio session documentation:
AVAudioSessionCategoryRecord
For recording audio; this category silences playback audio.
Is there any way to change this behavior and do one of the following?
1) Continue playing other apps' audio and allow the phone to record the audio that is being played.
2) Resume other apps' audio once a brief recording is finished. There are notifications apps can respond to when audio interruptions begin or end. Is there something specific that must be done for other apps to receive these notifications?

from 24 hours of diligent searching and piecemealing things together, this is surprisingly simple:
let session = AVAudioSession.sharedInstance()
session.setCategory(AVAudioSessionCategoryPlayAndRecord, with: [.mixWithOthers, .allowBluetoothA2DP])
And that's it for anyone still searching how to do this.

You can use
Obj-C
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayAndRecord
withOptions:AVAudioSessionCategoryOptionMixWithOthers error:nil];
or
Swift
let audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord,
with: AVAudioSessionCategoryOptions.mixWithOthers)
} catch let error as NSError {
print("audioSession error: \(error.localizedDescription)")
}

1) Not possible for AVAudioSession. All you can do is check [[AVAudioSession sharedInstance] isOtherAudioPlaying] and ask user would he like to stop music streaming or not.
2) App can't receive any notifications when iPod state changed, we can't even observe isOtherAudioPlaying property... Solution I use on my projects: add a NSTimer and once in sec check [[AVAudioSession sharedInstance] isOtherAudioPlaying] BOOL value.

Related

Speech Recognition with AVAudioEngine Blocks Sound After Recording

I am trying to run the Apple SpeakToMe: Using Speech Recognition with AVAudioEngine sample from their website here. My problem is that when you stop the AVAudioEngine and SpeechRecognizer you can no longer use system sounds.
How do you release the AVAudioEngine and SpeechRecognizer so that sounds will work again after recording stops?
To duplicate this:
download the sample code
add a UITextField to the storyboard.
run the project and type into the text field (you'll notice you can hear your typing event sounds).
Then start recording and stop recording
Type into the text field again (Now there will be no sound)
UPDATE
This only happens on a real device - not on the simulator.
After hours of debugging I came across the un-released object causing issues. In their sample code they do not release the AVAudioSession. This causes the sound channels to be blocked.
The fix is to make the AVAudioSession a property:
private var audioSession : AVAudioSession?
And then set audioSession.active to false when stopping the recording:
if let audioSession = audioSession {
do {
try audioSession.setActive(false, with: .notifyOthersOnDeactivation)
} catch {
// handle error
}
}

Performing vibration in recording app

I am trying to perform a vibration in an app similar to Snapchat, that uses both audio output and input as well as supports audio mixing from other apps, but this seems to be a harder task that I initially thought it would be. Important to know is that I am not trying to vibrate during playback or recording. From reading all the documentation I could find on the subject, this is what I have come to understand:
In order to support both playback and recording (output and input), I need to use AVAudioSessionCategoryPlayAndRecord
Making the phone vibrate through AudioServicesPlaySystemSound (kSystemSoundID_Vibrate) is not supported in any of the recording categories, including AVAudioSessionCategoryPlayAndRecord.
Enabling other apps to play audio can be done by adding the option AVAudioSessionCategoryOptionMixWithOthers.
Therefore, I do this in my app delegate:
NSError *error = nil;
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayAndRecord withOptions:AVAudioSessionCategoryOptionDefaultToSpeaker | AVAudioSessionCategoryOptionMixWithOthers error:&error];
The possible solutions to doing the vibration that I have tried but failed at are:
Deactivating the shared AVAudioSession before vibrating, and then activate it straight after.
[[AVAudioSession sharedInstance] setActive:NO error:nil];
AudioServicesPlaySystemSound (kSystemSoundID_Vibrate);
[[AVAudioSession sharedInstance] setActive:YES error:nil];
This successfully performs the vibration, but afterwards, when I try to record a movie, the audio is ducked (or something else is causing it to be very silent). It also gives me an error saying that I am not allowed to deactivate a session without removing its I/O devices first.
Changing category before vibrating, and then changing it back.
[[AVAudioSession sharedInstance] setActive:NO error:nil];
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryAmbient error:nil];
[[AVAudioSession sharedInstance] setActive:YES error:nil];
AudioServicesPlaySystemSound(kSystemSoundID_Vibrate);
[[AVAudioSession sharedInstance] setActive:NO error:nil];
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayAndRecord withOptions:AVAudioSessionCategoryOptionDefaultToSpeaker | AVAudioSessionCategoryOptionMixWithOthers error:nil];
[[AVAudioSession sharedInstance] setActive:YES error:nil];
This solution comes up every now and then, but does not seem to work for me. No vibration occurs, even though the categories seems to be set. This might still be a valid solution if I set usesApplicationAudioSession = YES on my AVCaptureSession, but I haven't made it work yet.
Sources:
https://developer.apple.com/library/ios/documentation/Audio/Conceptual/AudioSessionProgrammingGuide/ConfiguringanAudioSession/ConfiguringanAudioSession.html
https://developer.apple.com/library/ios/documentation/Audio/Conceptual/AudioSessionProgrammingGuide/AudioSessionBasics/AudioSessionBasics.html
https://developer.apple.com/library/ios/documentation/Audio/Conceptual/AudioSessionProgrammingGuide/AudioGuidelinesByAppType/AudioGuidelinesByAppType.html#//apple_ref/doc/uid/TP40007875-CH11-SW1
So, I've been recently trying to play a simple beep in my app and one of the methods of doing so I stumbled upon is:
AudioServicesPlaySystemSound(SystemSoundID inSystemSoundID);
It's a function that plays a system sound from /System/Library/Audio/UISounds/ it's supported on all iOS versions.
The sound 1350 equals to RingerVibeChanged (vibration). So..
AudioServicesPlaySystemSound(1350);
...vibrates for about 0.5 seconds.
Incase you're interested in more sounds here is a link to all the playable sounds and what iOS versions they've been added in:
http://iphonedevwiki.net/index.php/AudioServices
You didn't say what your supported device requirements are, but for newer devices, you can use the new taptic engine APIs, such as:
UIImpactFeedbackGenerator
UISelectionFeedbackGenerator
UINotificationFeedbackGenerator
https://developer.apple.com/reference/uikit/uifeedbackgenerator#2555399
For reference, I'm not using AVCaptureSession but an AVAudioEngine to record something, and for my case I pinpointed it down to having to pause the input in order to play the vibration properly with AudioServicesPlaySystemSound during a record session.
So in my case, I did something like this
audioEngine?.pause()
AudioServicesPlaySystemSound(1519)
do {
try audioEngine.start()
} catch let error {
print("An error occurred starting audio engine. \(error.localizedDescription)")
}
I'm not sure if it would also work with doing something similar for AVCaptureSession (i.e., stopRunning() and startRunning()), but leaving this here in case someone wants to give it a try.
You may try setting the allowHapticsAndSystemSoundsDuringRecording flag on AVAudioSession to true (the flag defaults to false) by calling
sessionInstance.setAllowHapticsAndSystemSoundsDuringRecording(true)
It worked for me.
Note the system prevents haptic and sound feedback (such as those from a UIPickerView or UIDatePicker) from playing, to avoid unexpected system noise while recording.

App volume goes quiet on iPad when using GKVoiceChat

In my iOS game, I support push-to-talk using Game Center's GKVoiceChat.
When two iPhones are connected in a multiplayer match, this works as expected: the game's sounds are heard at roughly the same volume as the other player's voice (voice may be a tiny bit louder), and the game's volume is consistent whether or not the other player is using the push-to-talk function.
However, on an iPad, the volume of the game's sounds is drastically reduced; game sounds are played at roughly one quarter the volume of the voice sounds, so quiet that unless you put your ear to the speaker, you're hard pressed to tell that any game sounds are being played at all. (Voice sounds are at full volume.) In comparison, the iPhone's volume is deafening.
Here's how I'm setting up audio:
AVAudioSession* avSession = [AVAudioSession sharedInstance];
NSError *myError = nil;
[avSession setActive:YES error:&myError];
if(myError)
NSLog(#"Error initializing audio session: %#", myError);
[avSession setCategory:AVAudioSessionCategoryPlayAndRecord
error: &myError];
if(myError)
NSLog(#"Error setting audio session category: %#", myError);
[avSession setMode:AVAudioSessionModeVoiceChat
error:&myError];
if(myError)
NSLog(#"Error setting audio session mode: %#", myError);
// By default, AVAudioSessionCategoryPlayAndRecord sends audio output to the phone's earpiece; instead, we want to force it to the speakers
[avSession overrideOutputAudioPort:AVAudioSessionPortOverrideSpeaker
error:&myError];
if(myError)
NSLog(#"Error changing audio port to speakers: %#", myError);
Then, later, when a multiplayer match is set up, we set up the voice chat like this:
self.myVoiceChannel = [[self myMatch] voiceChatWithName:#"allPlayers"];
[[self myVoiceChannel] start];
[[self myVoiceChannel] setActive:NO];
self.myVoiceChannel.volume = 1.0;
I've confirmed that commenting out the [[self myVoiceChannel] start] statement is sufficient to restore the iPad volume to the expected levels.
What's surprising is that [[AVAudioSession sharedInstance] mode] never gets set to AVAudioSessionModeGameChat---no matter when I expect it, it's always AVAudioSessionModeVoiceChat. From the AVAudioSession documentation, it seemed like when I initiated a GKVoiceChat, this would be changed automatically.
Any ideas why the iPad's audio would be mixed so differently from the iPhone?
It looks like this is a known issue since iOS 6.
The only options for working around it is to crank up your game's sounds during voice chats (on the iPad only, obviously).

When is AVAudioSessionSilenceSecondaryAudioHintNotification triggered?

Quoted from Xcode doc,
AVAudioSessionSilenceSecondaryAudioHintNotification Posted on the main
thread when the primary audio from other applications starts and
stops.
Subscribe to this notification to ensure that your app is notified
when optional secondary audio muting should begin or end.
However, when my app's audio is playing, and I press the remote-control to start playing music from the Music app. This notification is not triggered in my observer callback. I believe the registration was successful.
Am I having the wrong expectation? Is it supposed to be triggered in a different scenario? Any examples?
You need to set your AVAudioSession Category to AVAudioSessionCategoryAmbient, then your app will allow background apps (such as music or podcasts) to play. If you set it to AVAudioSessionCategorySoloAmbient then it will not accept background music
You can do it like this:
NSError *categoryError = nil;
if ([[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryAmbient error:&categoryError]) {
printf("Setting AVAudioSession CategoryAmbient Succeeded\n");
} else {
printf("Setting AVAudioSession CategoryAmbient Failed\n");
}
Check out Audio Session Categories for more details

Changing AVAudioSession modes in app

Is it possible to change the session category and options when a specific behavior is needed in app? For example setting:
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayAndRecord error:&sessionError];
according to apple docs will not honor the silent switch.
The app records and plays audio. On audio playback, the app needs to honor the silent switch. Question is: Can I set the category like so:
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryAmbient error:&sessionError];
when playback begins so that the app honors the silent switch. And if I do so, how will this affect routing? Apple Docs state:
The session's category and mode together define how the application
intends to use audio. Typically, you should set the category and mode
before activating the session. You may also set the category or mode
while the session is active, but this will result in an immediate
route change.
It is possible to change, quoting the docs:
Each app running in iOS has a single audio session, which in turn has
a single category. You can change your audio session’s category while
your app is running.
https://developer.apple.com/library/ios/documentation/AVFoundation/Reference/AVAudioSession_ClassReference/#//apple_ref/doc/constant_group/Audio_Session_Categories
So it is just a matter of calling the setCategory: method when you want the app to change mode.
For example, you start your app while allowing sound to play from other apps:
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryAmbient error:nil]
(...)
}
And when a user presses a play button on your UI, change to Playback mode:
- (void)playAudio {
if ([AVAudioSession sharedInstance].otherAudioPlaying) {
// you can check and play only if there is no other audio playing
// maybe use another category, or enable mixing or duck option
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryOptionDuckOthers error:nil];
} else {
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:nil];
}
[[AVAudioSession sharedInstance] setActive:YES error:nil];
}
If you want to resume the other app audio after your playback you can do a notification, or just close the session to have the other app audio to continue stopped:
- (void)stopAudio {
if (self.otherAudioShouldResume) {
[[AVAudioSession sharedInstance] setActive:NO withOptions:AVAudioSessionSetActiveOptionNotifyOthersOnDeactivation error:nil];
} else {
[[AVAudioSession sharedInstance] setActive:NO error:nil];
}
This code is an overview only, you might need to perform more functions to achieve a working example, also remember to check the return of these functions (BOOL) and log the errors for debugging.
Based on the Apple document, about AVAudioSession -setMode:error:.
Discussion
The session's category and mode together define how the
application intends to use audio. Typically, you set the category and
mode before activating the session. You can also set the category or
mode while the session is active, but this results in an immediate
route change.
It sounds like the setCategory:error: and '-setMode:error:' are called before setActive:error: generally. But if those methods are called when the session is active, the audio route will change immediately.
In my case,
- (void)playAudio
{
/// call `-setMode:error:` for playing audio
}
- (void)stopAudio
{
/// call `-setMode:error:` back to original configuration, or set to another mode. It will result in route change immediately.
}

Resources