How to tell if IOS device only supports 48kHz in hardware - ios

Newer IOS devices like the 6S only support native 48kHz playback. Not really much of a problem since standard CoreAudio graphs resample just fine. Problem is, if you're doing a VOIP type of app with the voice processing unit, you can't set the phone to 44.1kHz; it creates a nice Darth-Vader like experience!
Formerly, I used to check the model of the device and simply say 'If it's a 6S or later, then I have to resample 44.1 to 48kHz), and this worked fine. I didn't like this fix, so I tried the following code:
session = [AVAudioSession sharedInstance];
[session setActive:YES error:&nsError];
if (systemSampleRate == 44100) // We may need to resample if it's a phone that only supports 48kHz like the 6S or 6SPlus
{
[session setCategory:AVAudioSessionCategoryPlayback
withOptions:0
error:&nsError];
result = [session setPreferredSampleRate:systemSampleRate error:&nsError];
hardwareSampleRate = [session sampleRate];
NSLog (#"Phone reports sample rate of %f", hardwareSampleRate);
if (hardwareSampleRate != (double)systemSampleRate) // We can't set it!!!!
needsResampling = YES;
else
{
[session setCategory:AVAudioSessionCategoryRecord
withOptions:AVAudioSessionCategoryOptionAllowBluetooth
error:&nsError];
result = [session setPreferredSampleRate:systemSampleRate error:&nsError];
hardwareSampleRate = [session sampleRate];
if (hardwareSampleRate != (double)systemSampleRate) // We can't set it!!!!
needsResampling = YES;
else
needsResampling = NO;
}
}
MOST of the time, this works. The 6S devices would report 48kHz, and all others would report 44.1kHz. BUT, if it had been tied to a bluetooth headset type of system that only supports 8kHz mic audio and 44.1kHz playback, the first hardwareSample Rate value reports 44.1!!!! So I go ahead thinking the device natively supports 44.1 and everything screws up.
SO the question is: how do I find out if the native playback device on IOS physically only supports 48kHz, or can support both 44.1 and 48kHz? Apple's public document on this is worthless, it simply chastises people for assuming a device supports both without telling you how to figure it out.

You really do just have to assume that the sample rate can change. If systemSampleRate is an external requirement, try to set the sample rate to that, and then work with what you get. The catch is that you have to do this check every time your audio render chain starts or is interrupted in case the sample rate changes.
I use two different ways to handle this, both involve tearing down and reinitializing my audio unit chain if the sample rate changes.
One simple way is to make all of my audio unit's sample rates the system sample rate (provided by the sample rate property on an active audio session). I assume that this is the highest quality method as there is no sample rate conversion.
If I have a sample rate requirement I will create my chain with my required sample rate. then check if the system sample rate is different from my requirement. If it is different, I will put converter units between the system unit (remote io) and the ends of my chain.
The bottom line is that the most important information is whether or not the system sample rate is different from your requirement, not whether or not it can change. It's a total pain, and a bunch of audio apps broke when the 6S came out, but it's the right way to handle it moving forward.

Related

How to patch portaudio for low-latency iOS usage?

I am currently working on a patch that should make portaudio work for iOS. For now I successfully applied portaudio member Hans Petter's patch for iOS usage
https://www.dropbox.com/s/6hf9bjqpa6b6uv3/0001-Add-basic-support-for-iOS-to-portaudio.patch?dl=0
and at least I call tell that the audio process does work, however, it is currently stuck at 1024 samples. When I try decreasing it to lower values the callback function terminates immediately. When using 48 kHz it terminates without any warning or error message but when using 44.1 kHz it crashes with t the following error:
Assertion failed: (*streamCallbackResult == paContinue || *streamCallbackResult == paComplete || *streamCallbackResult == paAbort), function PaUtil_EndBufferProcessing, file pa_process.c, line 1499.
dyld4 config: DYLD_LIBRARY_PATH=/usr/lib/system/introspection DYLD_IMAGE_SUFFIX=_debug DYLD_INSERT_LIBRARIES=/Developer/usr/lib/libBacktraceRecording.dylib:/Developer/Library/PrivateFrameworks/DTDDISupport.framework/libViewDebuggerSupport.dylib
referring to line 1497 in pa_process.c -->
assert( *streamCallbackResult == paContinue
|| *streamCallbackResult == paComplete
|| streamCallbackResult == paAbort ); / don't forget to pass in a valid callback result value */
I'd like to get as low as possible in terms of buffer size (ideally 128 or even 64), however, since I am not familiar with audio on iOS am now seeking for first inspirational comments. On OSX, Win or Linux this problem does not exist. Maybe iOS expects special low-latency flags ?
Meantime I figured that iOS is able to handle audio frame sizes < 1024 (down to 128 and possibly less) and that the portaudio patch requires additional modifications and calls in order to make it work. Once completed I will provide the patch and further details here.
Integrating Objective C code is the secret and changing the file's suffix to mm:
auto session = [AVAudioSession sharedInstance];
[session setActive:TRUE error: nil];
[session setPreferredSampleRate: sampleRate error: nil];
[session setPreferredIOBufferDuration: bufferDuration error:nil];

AVAudioSession properties after Initializing AUGraph

To start a call, our VOIP app sets up an AVAudioSession, then builds, initializes and runs an AUGraph.
During the call, we allow the user to switch back and forth between a speakerphone mode using code such as:
avSession = [AVAudioSession sharedInstance];
AVAudioSessionCategoryOptions categoryOptions = [avSession categoryOptions];
categoryOptions |= AVAudioSessionCategoryOptionDefaultToSpeaker;
NSLog(#"AudioService:setSpeaker:setProperty:DefaultToSpeaker=1 categoryOptions = %lx", (unsigned long)categoryOptions);
BOOL success = [avSession setCategory:AVAudioSessionCategoryPlayAndRecord withOptions:categoryOptions error:&error];
Which works just fine. But if we try to certain AVAudioSession queries after the AUGraph has been initialized, for example:
AVAudioSessionDataSourceDescription *myInputDataSource = [avSession inputDataSource];
the result is null. Running the same line of code BEFORE we execute the AUGraphInitialize gives the correct non-null result. Can anyone explain what is going on here and how to properly access AVAudioSession properties/methods while using AUGraph?
This is expected behavior per the developer documentation, inputDataSource should return nil if it is not possible to switch sources. So Apple is really not letting anything bad happen via a mis-config, but a nil source can also give the wrong idea. Hope this helps.
Discussion
The value of this property is nil if switching between multiple input
sources is not currently possible. This feature is supported only on
certain devices and peripherals–for example, on an iPhone equipped with
both front- and rear-facing microphones.

App volume goes quiet on iPad when using GKVoiceChat

In my iOS game, I support push-to-talk using Game Center's GKVoiceChat.
When two iPhones are connected in a multiplayer match, this works as expected: the game's sounds are heard at roughly the same volume as the other player's voice (voice may be a tiny bit louder), and the game's volume is consistent whether or not the other player is using the push-to-talk function.
However, on an iPad, the volume of the game's sounds is drastically reduced; game sounds are played at roughly one quarter the volume of the voice sounds, so quiet that unless you put your ear to the speaker, you're hard pressed to tell that any game sounds are being played at all. (Voice sounds are at full volume.) In comparison, the iPhone's volume is deafening.
Here's how I'm setting up audio:
AVAudioSession* avSession = [AVAudioSession sharedInstance];
NSError *myError = nil;
[avSession setActive:YES error:&myError];
if(myError)
NSLog(#"Error initializing audio session: %#", myError);
[avSession setCategory:AVAudioSessionCategoryPlayAndRecord
error: &myError];
if(myError)
NSLog(#"Error setting audio session category: %#", myError);
[avSession setMode:AVAudioSessionModeVoiceChat
error:&myError];
if(myError)
NSLog(#"Error setting audio session mode: %#", myError);
// By default, AVAudioSessionCategoryPlayAndRecord sends audio output to the phone's earpiece; instead, we want to force it to the speakers
[avSession overrideOutputAudioPort:AVAudioSessionPortOverrideSpeaker
error:&myError];
if(myError)
NSLog(#"Error changing audio port to speakers: %#", myError);
Then, later, when a multiplayer match is set up, we set up the voice chat like this:
self.myVoiceChannel = [[self myMatch] voiceChatWithName:#"allPlayers"];
[[self myVoiceChannel] start];
[[self myVoiceChannel] setActive:NO];
self.myVoiceChannel.volume = 1.0;
I've confirmed that commenting out the [[self myVoiceChannel] start] statement is sufficient to restore the iPad volume to the expected levels.
What's surprising is that [[AVAudioSession sharedInstance] mode] never gets set to AVAudioSessionModeGameChat---no matter when I expect it, it's always AVAudioSessionModeVoiceChat. From the AVAudioSession documentation, it seemed like when I initiated a GKVoiceChat, this would be changed automatically.
Any ideas why the iPad's audio would be mixed so differently from the iPhone?
It looks like this is a known issue since iOS 6.
The only options for working around it is to crank up your game's sounds during voice chats (on the iPad only, obviously).

`[AVCaptureSession canAddOutput:output]` returns NO intermittently. Can I find out why?

I am using canAddOutput: to determine if I can add a AVCaptureMovieFileOutput to a AVCaptureSession and I'm finding that canAddOutput: is sometimes returning NO, and mostly returning YES. Is there a way to find out why a NO was returned? Or a way to eliminate the situation that is causing the NO to be returned? Or anything else I can do that will prevent the user from just seeing an intermittent failure?
Some further notes: This happens approximately once in 30 calls. As my app is not launched, it has only been tested on one device: an iPhone 5 running 7.1.2
Here is quote from documentation (discussion of canAddOutput:)
You cannot add an output that reads from a track of an asset other than the asset used to initialize the receiver.
Explanation that will help you (Please check if your code is matching to this guide, if you're doing all right, it should not trigger error, because basically canAddOuput: checks the compatibility).
AVCaptureSession
Used for the connection between the organizations Device Input and output, similar to the connection of the DShow the filter. If you can connect the input and output, after the start, the data will be read from input to the output.
Several main points:
a) AVCaptureDevice, the definition of equipment, both camera Device.
b) AVCaptureInput
c) AVCaptureOutput
Input and output are not one-to-one, such as the video output while video + audio Input.
Before and after switching the camera:
AVCaptureSession * session = <# A capture session #>;
[session beginConfiguration];
[session removeInput: frontFacingCameraDeviceInput];
[session addInput: backFacingCameraDeviceInput];
[session commitConfiguration];
Add the capture INPUT:
To add a capture device to a capture session, you use an instance of AVCaptureDeviceInput (a concrete
subclass of the abstract AVCaptureInput class). The capture device input manages the device's ports.
NSError * error = nil;
AVCaptureDeviceInput * input =
[AVCaptureDeviceInput deviceInputWithDevice: device error: & error];
if (input) {
// Handle the error appropriately.
}
Add output, output classification:
To get output from a capture session, you add one or more outputs. An output is an instance of a concrete
subclass of AVCaptureOutput;
you use:
AVCaptureMovieFileOutput to output to a movie file
AVCaptureVideoDataOutput if you want to process frames from the video being captured
AVCaptureAudioDataOutput if you want to process the audio data being captured
AVCaptureStillImageOutput if you want to capture still images with accompanying metadata
You add outputs to a capture session using addOutput:.
You check whether a capture output is compatible
with an existing session using canAddOutput:.
You can add and remove outputs as you want while the
session is running.
AVCaptureSession * captureSession = <# Get a capture session #>;
AVCaptureMovieFileOutput * movieInput = <# Create and configure a movie output #>;
if ([captureSession canAddOutput: movieInput]) {
[captureSession addOutput: movieInput];
}
else {
// Handle the failure.
}
Save a video file, add the video file output:
You save movie data to a file using an AVCaptureMovieFileOutput object. (AVCaptureMovieFileOutput
is a concrete subclass of AVCaptureFileOutput, which defines much of the basic behavior.) You can configure
various aspects of the movie file output, such as the maximum duration of the recording, or the maximum file
size. You can also prohibit recording if there is less than a given amount of disk space left.
AVCaptureMovieFileOutput * aMovieFileOutput = [[AVCaptureMovieFileOutput alloc]
init];
CMTime maxDuration = <# Create a CMTime to represent the maximum duration #>;
aMovieFileOutput.maxRecordedDuration = maxDuration;
aMovieFileOutput.minFreeDiskSpaceLimit = <# An appropriate minimum given the quality
of the movie format and the duration #>;
Processing preview video frame data, each frame view finder data can be used for subsequent high-level processing, such as face detection, and so on.
An AVCaptureVideoDataOutput object uses delegation to vend video frames.
You set the delegate using
setSampleBufferDelegate: queue:.
In addition to the delegate, you specify a serial queue on which they
delegate methods are invoked. You must use a serial queue to ensure that frames are delivered to the delegate
in the proper order.
You should not pass the queue returned by dispatch_get_current_queue since there
is no guarantee as to which thread the current queue is running on. You can use the queue to modify the
priority given to delivering and processing the video frames.
Data processing for the frame, there must be restrictions on the size (image size) and the processing time limit, if the processing time is too long, the underlying sensor will not send data to the layouter and the callback.
You should set the session output to the lowest practical resolution for your application.
Setting the output
to a higher resolution than necessary wastes processing cycles and needlessly consumes power.
You must ensure that your implementation of
captureOutput: didOutputSampleBuffer: fromConnection: is able to process a sample buffer within
the amount of time allotted to a frame. If it takes too long, and you hold onto the video frames, AVFoundation
will stop delivering frames, not only to your delegate but also other outputs such as a preview layer.
Deal with the capture process:
AVCaptureStillImageOutput * stillImageOutput = [[AVCaptureStillImageOutput alloc]
init];
NSDictionary * outputSettings = [[NSDictionary alloc] initWithObjectsAndKeys: AVVideoCodecJPEG,
AVVideoCodecKey, nil];
[StillImageOutput setOutputSettings: outputSettings];
Able to support different format also supports directly generate jpg stream.
If you want to capture a JPEG image, you should typically not specify your own compression format. Instead,
you should let the still image output do the compression for you, since its compression is hardware-accelerated.
If you need a data representation of the image, you can use jpegStillImageNSDataRepresentation: to
get an NSData object without re-compressing the data, even if you modify the image's metadata.
Camera preview display:
You can provide the user with a preview of what's being recorded using an AVCaptureVideoPreviewLayer
object. AVCaptureVideoPreviewLayer is a subclass of CALayer (see Core Animation Programming Guide. You don't need any outputs to show the preview.
AVCaptureSession * captureSession = <# Get a capture session #>;
CALayer * viewLayer = <# Get a layer from the view in which you want to present the
The preview #>;
AVCaptureVideoPreviewLayer * captureVideoPreviewLayer = [[AVCaptureVideoPreviewLayer
alloc] initWithSession: captureSession];
[viewLayer addSublayer: captureVideoPreviewLayer];
In general, the preview layer behaves like any other CALayer object in the render tree (see Core Animation
Programming Guide). You can scale the image and perform transformations, rotations and so on just as you
would any layer. One difference is that you may need to set the layer's orientation property to specify how
it should rotate images coming from the camera. In addition, on iPhone 4 the preview layer supports mirroring
(This is the default when previewing the front-facing camera).
Referring from this answer, there might be a possibility that this delegate method may be running in the background, which causes the previous AVCaptureSession not disconnected properly sometimes resulting in canAddOutput: to be NO sometimes.
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputMetadataObjects:(NSArray *)metadataObjects fromConnection:(AVCaptureConnection *)connection
The solution might be to use stopRunning in the above delegate(Of course after doing necessary actions and condition checks, you need to finish off your previous sessions properly right?).
Adding on to that, It would be better if you provide some code of what you are trying to do.
It's can be one from this 2 cases
1) Session is running
2) You already added output
You can't add 2 output or 2 input, and also you can't create 2 different sessions
It may be a combination of:
Calling this method when the camera is busy.
Not properly removing your previously connected AVCaptureSession.
You should try to only add it once (where I guess canAddOutput: will always be YES) and just pause/resume your session as needed:
// Stop session if possible
if (_captureSession.running && !_captureInProgress)
{
[_captureSession stopRunning];
NBULogVerbose(#"Capture session: {\n%#} stopped running", _captureSession);
}
You can take a look here.
I think this will help you
canAddOutput:
Returns a Boolean value that indicates whether a given output can be added to the session.
- (BOOL)canAddOutput:(AVCaptureOutput *)output
Parameters
output
An output that you want to add to the session.
Return Value
YES if output can be added to the session, otherwise NO.
Availability
Available in OS X v10.7 and later.
Here is the link for apple doc Click here

Erratic behavior when setting the microphone gain with AVAudioSession

I am trying to set the microphone gain with "setInputGain" in AVAudioSession to handle very weak sounds, but I am only partly successful. I am checking if "isInputGainSettable" and then I try to change the gain with a slider. I am checking if the gain actually changes, both by reading back the value and checking an actual recorded sound. The result is as follows:
The code I am using
-(void)viewDidLoad
{
self.audioSession = [AVAudioSession sharedInstance];
if(self.audioSession.isInputGainSettable){
[self.audioSession setActive:YES error:nil];
}
}
-(IBAction)setGain:(id)sender
{
float gain = self.gainSlider.value;
NSError* error;
BOOL gainset = [self.audioSession setInputGain:gain error:&error];
if (!gainset) NSLog(#"failed %#", error);
NSLog(#"audiosession gain: %.2f ",self.audioSession.inputGain);
}
I am not getting any error messages. I have been searching SO and elsewhere and people are both reporting problems, but also that they are able to set the gain on iPads and older iPhones. The only "trick" that I have seen reported is to "wait a while" before setting the gain, something I have tried without success.
So the question is if there something I have missed, and if I should be able to set the gain on iPads and older iPhones?
I was exhausted as well and searched for almost three hours.
The only thing which worked for me was calling the setGain method in viewDidAppear.

Resources