Remote IO audio is very noisy - ios

I am new to core audio and remote io. I need data of size 320 bytes which I encode and send. Also a minimum of 50 frames per second. Here is what I have done:
AudioComponentDescription desc;
desc.componentType = kAudioUnitType_Output;
desc.componentSubType = kAudioUnitSubType_RemoteIO;
desc.componentFlags = 0;
desc.componentFlagsMask = 0;
desc.componentManufacturer = 0;
// Get component
AudioComponent inputComponent = AudioComponentFindNext(NULL, &desc);
// Get audio units
 AudioComponentInstanceNew(inputComponent, &audioUnit);
// Enable IO for recording
UInt32 flag = 1; AudioUnitSetProperty(audioUnit,  kAudioOutputUnitProperty_EnableIO,  kAudioUnitScope_Input,   kInputBus,  &flag,   sizeof(flag));
// Enable IO for playback AudioUnitSetProperty(audioUnit,  kAudioOutputUnitProperty_EnableIO,   kAudioUnitScope_Output,   kOutputBus, &flag,   sizeof(flag));
UInt32 shouldAllocateBuffer = 1;
AudioUnitSetProperty(audioUnit, kAudioUnitProperty_ShouldAllocateBuffer, kAudioUnitScope_Global, 1, &shouldAllocateBuffer, sizeof(shouldAllocateBuffer));
// Describe format
audioFormat.mSampleRate = 8000.00;
audioFormat.mFormatID = kAudioFormatLinearPCM;
audioFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger|kAudioFormatFlagIsPacked;
audioFormat.mFramesPerPacket = 1;
audioFormat.mChannelsPerFrame = 1;
audioFormat.mBitsPerChannel = 16;
audioFormat.mBytesPerPacket = 2;
audioFormat.mBytesPerFrame = 2;
// Apply format AudioUnitSetProperty(audioUnit,  kAudioUnitProperty_StreamFormat,  kAudioUnitScope_Output, 1,   &audioFormat,  sizeof(audioFormat));
AudioUnitSetProperty(audioUnit,  kAudioUnitProperty_StreamFormat,  kAudioUnitScope_Input,  0,  &audioFormat,  sizeof(audioFormat));
// Set input callback
AURenderCallbackStruct callbackStruct;
callbackStruct.inputProc = recordingCallback;
callbackStruct.inputProcRefCon = self;
AudioUnitSetProperty(audioUnit,   kAudioOutputUnitProperty_SetInputCallback,   kAudioUnitScope_Global,   1,   &callbackStruct,   sizeof(callbackStruct));
// Set output callback
callbackStruct.inputProc = playbackCallback;
callbackStruct.inputProcRefCon = self;
 AudioUnitSetProperty(audioUnit, kAudioUnitProperty_SetRenderCallback,  kAudioUnitScope_Global,   0, &callbackStruct,  sizeof(callbackStruct));
// Initialise
AudioUnitInitialize(audioUnit);
AudioOutputUnitStart(audioUnit);
With this settings, i get 186 frames in the callback method when tried with device.
I have allocated by buffer:
bufferList = (AudioBufferList*) malloc(sizeof(AudioBufferList));
bufferList->mNumberBuffers = 1; //mono input
for(UInt32 i=0;i<bufferList->mNumberBuffers;i++)
{
bufferList->mBuffers[i].mNumberChannels = 1;
bufferList->mBuffers[i].mDataByteSize = 2*186;
bufferList->mBuffers[i].mData = malloc(bufferList->mBuffers[i].mDataByteSize);
} 
From this 372(2 x 186)  bytes in the callback, i took 320 byte data and used as per my requirement. It is working, but very noisy.
Someone please help me. I am in big trouble.

A couple of suggestions -
Sample rate and buffer size get set using the AVAudioSession class.
386 is an unusual number of frames. Your callback is probably asking for 512 or 1024. You might try using a ring buffer to allow varying varying buffer size/ frame rates to suit your needs.
Here are some examples:
MixerHost
TimeCode

Related

Using AudioQueueNewInput to record stereo

I would like to use AudioQueueNewInput to create a stereo recording. I configured it as follows:
audioFormat.mFormatID = kAudioFormatLinearPCM;
hardwareChannels = 2;
audioFormat.mChannelsPerFrame = hardwareChannels;
audioFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked | kAudioFormatFlagIsBigEndian;
audioFormat.mFramesPerPacket = 1;
audioFormat.mBitsPerChannel = 16;
audioFormat.mBytesPerPacket = (audioFormat.mBitsPerChannel / 8) * hardwareChannels;
audioFormat.mBytesPerFrame = audioFormat.mBytesPerPacket;
OSStatus result = AudioQueueNewInput(
&audioFormat,
recordingCallback,
(__bridge void *)(self), // userData
NULL, // run loop
NULL, // run loop mode
0, // flags
&queueObject
);
AudioQueueStart (
queueObject,
NULL // start time. NULL means as soon as possible.
);
I tested this code on an iPhone 6s plus with an external stereo microphone. It does not seem to record stereo. Both the left and right channels get identical streams of data. What else do I need to do to record stereo?

AudioUnit: Error reported by mediaserverd - "Error 'what' getting client format for physical format"

I have a VOIP application that uses AudioUnit. On iOS 9, I see an error message that says the following. I do not see this in iOS 8:
mediaserverd[43] : 13:16:38.880 ERROR: [0x1f72d000] >va>
500: Error 'what' getting client format for physical format [
16/44100/2; flags: 0xc; bytes/packet: 4; frames/packet: 1;
bytes/frame: 4; ]
Does anyone know what this means? The audio seems to work fine, but I'm still a bit irked by it and would like to address it if I can. My AudioUnit setup code is below. Help appreciated. Thanks.
CheckError(NewAUGraph(&audioState.graph), "NewAUGraph failed");
AUNode rioNode;
AudioComponentDescription desc;
desc.componentType = kAudioUnitType_Output;
desc.componentSubType = kAudioUnitSubType_VoiceProcessingIO;
desc.componentFlags = 0; // Documentation states "Set this value to zero"
desc.componentFlagsMask = 0; // Documentation states "Set this value to zero"
desc.componentManufacturer = kAudioUnitManufacturer_Apple;
CheckError(AUGraphAddNode(audioState.graph, &desc, &rioNode), "AUGraphAddNode failed for RIO");
CheckError(AUGraphOpen(audioState.graph),"Failed to open graph");
CheckError(AUGraphNodeInfo(audioState.graph, rioNode, NULL, &audioState.rio),"Failed to get rio node info");
UInt32 flag = 1;
CheckError(AudioUnitSetProperty(audioState.rio,
kAudioOutputUnitProperty_EnableIO,
kAudioUnitScope_Input,
1,
&flag,
sizeof(flag)), "Enable IO Failed");
AudioStreamBasicDescription audioFormat;
size_t bytesPerSample = sizeof(SInt16);
audioFormat.mSampleRate = 8000;
audioFormat.mFormatID = kAudioFormatLinearPCM;
audioFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
audioFormat.mFramesPerPacket = 1;
audioFormat.mChannelsPerFrame = 1;
audioFormat.mBitsPerChannel = 8 * (UInt32)bytesPerSample;
audioFormat.mBytesPerPacket = (UInt32)bytesPerSample;
audioFormat.mBytesPerFrame = (UInt32)bytesPerSample;
CheckError(AudioUnitSetProperty(audioState.rio,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Output,
INPUT_BUS,
&audioFormat,
sizeof(audioFormat)), "Could not set RIO input stream format");
CheckError(AudioUnitSetProperty(audioState.rio,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Input,
OUTPUT_BUS,
&audioFormat,
sizeof(audioFormat)), "Could not set RIO output stream format");
AURenderCallbackStruct callbackStruct;
// Set input callback
callbackStruct.inputProc = &recordingCallback;
callbackStruct.inputProcRefCon = &audioState;
CheckError(AudioUnitSetProperty(audioState.rio,
kAudioOutputUnitProperty_SetInputCallback,
kAudioUnitScope_Global,
INPUT_BUS,
&callbackStruct,
sizeof(callbackStruct)), "Could not set callback for recording");
// Set output callback
callbackStruct.inputProc = &playbackCallback;
callbackStruct.inputProcRefCon = &audioState;
CheckError(AudioUnitSetProperty(audioState.rio,
kAudioUnitProperty_SetRenderCallback,
kAudioUnitScope_Global,
OUTPUT_BUS,
&callbackStruct,
sizeof(callbackStruct)), "Could not set callback for playback");

Only play audio from array once without looping

I'm completely a beginner when it comes to audio programming and right now I'm playing around with AudioUnit. I'm following http://www.cocoawithlove.com/2010/10/ios-tone-generator-introduction-to.html and I've ported over the code to work with iOS7. The problem is that I only want it to play the generated sine wave once and not keep on playing the sound wave. I am not sure how to accomplish this though.
Generating audio samples:
OSStatus RenderTone(
void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData)
{
// Fixed amplitude is good enough for our purposes
const double amplitude = 0.25;
// Get the tone parameters out of the view controller
ToneGeneratorViewController *viewController =
(ToneGeneratorViewController *)inRefCon;
double theta = viewController->theta;
double theta_increment =
2.0 * M_PI * viewController->frequency / viewController->sampleRate;
// This is a mono tone generator so we only need the first buffer
const int channel = 0;
Float32 *buffer = (Float32 *)ioData->mBuffers[channel].mData;
// Generate the samples
for (UInt32 frame = 0; frame < inNumberFrames; frame++)
{
buffer[frame] = sin(theta) * amplitude;
theta += theta_increment;
if (theta > 2.0 * M_PI)
{
theta -= 2.0 * M_PI;
}
}
// Store the updated theta back in the view controller
viewController->theta = theta;
return noErr;
}
Creating AudioUnit:
// Configure the search parameters to find the default playback output unit
// (called the kAudioUnitSubType_RemoteIO on iOS but
// kAudioUnitSubType_DefaultOutput on Mac OS X)
AudioComponentDescription defaultOutputDescription;
defaultOutputDescription.componentType = kAudioUnitType_Output;
defaultOutputDescription.componentSubType = kAudioUnitSubType_RemoteIO;
defaultOutputDescription.componentManufacturer = kAudioUnitManufacturer_Apple;
defaultOutputDescription.componentFlags = 0;
defaultOutputDescription.componentFlagsMask = 0;
// Get the default playback output unit
AudioComponent defaultOutput = AudioComponentFindNext(NULL, &defaultOutputDescription);
NSAssert(defaultOutput, #"Can't find default output");
// Create a new unit based on this that we'll use for output
OSErr err = AudioComponentInstanceNew(defaultOutput, &toneUnit);
NSAssert1(toneUnit, #"Error creating unit: %ld", err);
// Set our tone rendering function on the unit
AURenderCallbackStruct input;
input.inputProc = RenderTone;
input.inputProcRefCon = self;
err = AudioUnitSetProperty(toneUnit,
kAudioUnitProperty_SetRenderCallback,
kAudioUnitScope_Input,
0,
&input,
sizeof(input));
NSAssert1(err == noErr, #"Error setting callback: %ld", err);
// Set the format to 32 bit, single channel, floating point, linear PCM
const int four_bytes_per_float = 4;
const int eight_bits_per_byte = 8;
AudioStreamBasicDescription streamFormat;
streamFormat.mSampleRate = sampleRate;
streamFormat.mFormatID = kAudioFormatLinearPCM;
streamFormat.mFormatFlags =
kAudioFormatFlagsNativeFloatPacked | kAudioFormatFlagIsNonInterleaved;
streamFormat.mBytesPerPacket = four_bytes_per_float;
streamFormat.mFramesPerPacket = 1;
streamFormat.mBytesPerFrame = four_bytes_per_float;
streamFormat.mChannelsPerFrame = 1;
streamFormat.mBitsPerChannel = four_bytes_per_float * eight_bits_per_byte;
err = AudioUnitSetProperty (toneUnit,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Input,
0,
&streamFormat,
sizeof(AudioStreamBasicDescription));
NSAssert1(err == noErr, #"Error setting stream format: %ld", err);
Thanks!
The problem is that I only want it to play the generated sine wave once
What you should do is stopping the audio unit after a certain time.
You could, e.g., set an NSTimer when you call AudioOutputUnitStart and then when the timer fires, you call AudioOutputUnitStop (actually, your audio unit disposal code). Even simpler, you could use performSelector:withObject:afterDelay: and call your audio unit disposal method.
Hope this helps.

consuming audio data from circular buffer in a render callback attached to the input scope of a remoteio audio unit

The title pretty much sums up what I'm trying to achieve. I am trying to use Michael Tyson's TPCircularBuffer inside of a render callback while the circular buffer is getting filled with incoming audio data. I want to send the audio from the render callback to the output element of the RemoteIO audio unit so I can hear it through the device speakers.
The audio is interleaved stereo 16 bit coming in as packets of 2048 frames. Here's how I've set up my audio session:
#define kInputBus 1
#define kOutputBus 0
NSError *err = nil;
NSTimeInterval ioBufferDuration = 46;
AVAudioSession *session = [AVAudioSession sharedInstance];
[session setCategory:AVAudioSessionCategoryPlayback withOptions:AVAudioSessionCategoryOptionMixWithOthers error:&err];
[session setPreferredIOBufferDuration:ioBufferDuration error:&err];
[session setActive:YES error:&err];
AudioComponentDescription defaultOutputDescription;
defaultOutputDescription.componentType = kAudioUnitType_Output;
defaultOutputDescription.componentSubType = kAudioUnitSubType_RemoteIO;
defaultOutputDescription.componentManufacturer = kAudioUnitManufacturer_Apple;
defaultOutputDescription.componentFlags = 0;
defaultOutputDescription.componentFlagsMask = 0;
AudioComponent defaultOutput = AudioComponentFindNext(NULL, &defaultOutputDescription);
NSAssert(defaultOutput, #"Can't find default output.");
AudioComponentInstanceNew(defaultOutput, &remoteIOUnit);
UInt32 flag = 0;
OSStatus status = AudioUnitSetProperty(remoteIOUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Input, kOutputBus, &flag, sizeof(flag));
size_t bytesPerSample = sizeof(AudioUnitSampleType);
AudioStreamBasicDescription streamFormat = {0};
streamFormat.mSampleRate = 44100.00;
streamFormat.mFormatID = kAudioFormatLinearPCM;
streamFormat.mFormatFlags = kAudioFormatFlagsCanonical;
streamFormat.mBytesPerPacket = bytesPerSample;
streamFormat.mFramesPerPacket = 1;
streamFormat.mBytesPerFrame = bytesPerSample;
streamFormat.mChannelsPerFrame = 2;
streamFormat.mBitsPerChannel = bytesPerSample * 8;
streamFormat.mReserved = 0;
status = AudioUnitSetProperty(remoteIOUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, kInputBus, &streamFormat, sizeof(streamFormat));
AURenderCallbackStruct callbackStruct;
callbackStruct.inputProc = render;
callbackStruct.inputProcRefCon = self;
status = AudioUnitSetProperty(remoteIOUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Global, kOutputBus, &callbackStruct, sizeof(callbackStruct));
And here's where the audio data gets loaded into the circular buffer and used in the render callback:
#define kBufferLength 2048
-(void)loadBytes:(Byte *)byteArrPtr{
TPCircularBufferProduceBytes(&buffer, byteArrPtr, kBufferLength);
}
OSStatus render(
void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData)
{
AUDIOIO *audio = (__bridge AUDIOIO *)inRefCon;
AudioSampleType *outSample = (AudioSampleType *)ioData->mBuffers[0].mData;
//Zero outSample
memset(outSample, 0, kBufferLength);
int bytesToCopy = ioData->mBuffers[0].mDataByteSize;
SInt16 *targetBuffer = (SInt16 *)ioData->mBuffers[0].mData;
//Pull audio
int32_t availableBytes;
SInt16 *buffer = TPCircularBufferTail(&audio->buffer, &availableBytes);
memcpy(targetBuffer, buffer, MIN(bytesToCopy, availableBytes));
TPCircularBufferConsume(&audio->buffer, MIN(bytesToCopy, availableBytes));
return noErr;
}
There is something wrong with this setup because I am not getting any audio through the speakers, but I'm also not getting any errors when I test on my device. As far as I can tell the TPCircularBuffer is being filled and read from correctly. I've followed the Apple documentation for setting up the audio session. I am considering trying to set up an AUGraph next but I want to see if anyone could suggest a solution for what I'm trying to do here. Thanks!
For stereo (2 channels per frame), your bytes per frame and bytes per packet have to be twice your sample size in bytes. Same with bits per channel in terms of bits.
Added: If availableBytes/yourFrameSize isn't almost always as large or larger than inNumberFrames, you won't get much continuous sound.
At a glance, it looks like you've got everything set up correctly. You're missing a call to AudioOutputUnitStart() though:
...
// returns an OSStatus indicating success / fail
AudioOutputUnitStart(remoteIOUnit);
// now your callback should be being called
...
I believe one your problem is with using streamFormat.mBitsPerChannel = bytesPerSample * 8;
You assign bytesPerSample to be sizeof(AudioUnitSampleType) which is essentially 4 bytes.
So streamFormat.mBytesPerPacket = bytesPerSample; is ok.
But the assignment streamFormat.mBitsPerChannel = bytesPerSample * 8; is saying that you want 32 bits per sample instead of 16 bits per sample.
I would not create your audio format based on AudioUnitSampleType because this has nothing to do with your personal format that you want to utilize. I would create defines and do something like this:
#define BITS_PER_CHANNEL 16
#define SAMPLE_RATE 44100.0
#define CHANNELS_PER_FRAME 2
#define BYTES_PER_FRAME CHANNELS_PER_FRAME * (BITS_PER_CHANNEL / 8) //ie 4
#define FRAMES_PER_PACKET 1
#define BYTES_PER_PACKET FRAMES_PER_PACKET * BYTES_PER_FRAME
streamFormat.mSampleRate = SAMPLE_RATE; // 44100.0
streamFormat.mBitsPerChannel = BITS_PER_CHANNEL; //16
streamFormat.mChannelsPerFrame = CHANNELS_PER_FRAME; // 2
streamFormat.mFramesPerPacket = FRAMES_PER_PACKET; //1
streamFormat.mBytesPerFrame = BYTES_PER_FRAME; // 4 total, 2 for left ch, 2 for right ch
streamFormat.mBytesPerPacket = BYTES_PER_PACKET;
streamFormat.mReserved = 0;
streamFormat.mFormatID = kAudioFormatLinearPCM; // double check this also
streamFormat.mFormatFlags = kAudioFormatFlagsCanonical;`
You also need to look at the return values set to err and status immediately after each are run. You still need to add error checking at some of the calls as well such as
checkMyReturnValueToo = AudioComponentInstanceNew(defaultOutput, &remoteIOUnit);
You also have an extremely high value for your buffer duration. You have 46 and I am not sure where that came from. That means you want 46 seconds worth of audio during each audio callback. Usually you want something less than one second depending on your latency requirements. Most likely iOS will not use anything that high but you should try setting it to say 0.025 or so (25ms). You can try to lower it if you need faster latency.

What stream format should iOS5 Effect Units use

I'm trying to use a Low Pass Filter AU. I keep getting a kAudioUnitErr_FormatNotSupported (-10868) error when setting the stream format to the filter unit, but if I just use the Remote IO unit there's no error.
The stream format I'm using is (Updated):
myASBD.mSampleRate = hardwareSampleRate;
myASBD.mFormatID = kAudioFormatLinearPCM;
myASBD.mFormatFlags = kAudioFormatFlagIsSignedInteger;
myASBD.mBitsPerChannel = 8 * sizeof(float);
myASBD.mFramesPerPacket = 1;
myASBD.mChannelsPerFrame = 1;
myASBD.mBytesPerPacket = sizeof(float) * myASBD.mFramesPerPacket;
myASBD.mBytesPerFrame = sizeof(float) * myASBD.mChannelsPerFrame;
And I'm setting the filter stream like this:
// Sets input stream type to ASBD
setupErr = AudioUnitSetProperty(filterUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &myASBD, sizeof(myASBD));
NSLog(#"Filter in: %i", setupErr);
//NSAssert(setupErr == noErr, #"No ASBD on Finput");
//Sets output stream type to ASBD
setupErr = AudioUnitSetProperty(filterUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 0, &myASBD, sizeof(myASBD));
NSLog(#"Filter out: %i", setupErr);
NSAssert(setupErr == noErr, #"No ASBD on Foutput");
The canonical format for iOS filter audio units is 8.24 fixed-point (linear PCM), which is 32 bits per channel, not 16.
what format is working wit the reverb unit??? I'm getting weird errors tryn to record a buffer....any news on this topic?
Try this for the canonical format.
size_t bytesPerSample = sizeof (AudioUnitSampleType); //Default is 4 bytes
myASBD.mSampleRate = hardwareSampleRate;
myASBD.mFormatID = kAudioFormatLinearPCM;
myASBD.mFormatFlags = kAudioFormatFlagsAudioUnitCanonical; //Canonical AU format
myASBD.mBytesPerPacket = bytesPerSample;
myASBD.mFramesPerPacket = 1;
myASBD.mBytesPerFrame = bytesPerSample;
myASBD.mChannelsPerFrame = 2; //Stereo
myASBD.mBitsPerChannel = 8 * bytesPerSample; //32bit integer
You will need to make sure all your AudioUnits ASBDs are configured uniformly.
If you are doing heavy audio processing, floats (supported in iOS5) is not a bad idea too.
size_t bytesPerSample = sizeof (float); //float is 4 bytes
myASBD.mSampleRate = hardwareSampleRate;
myASBD.mFormatID = kAudioFormatLinearPCM;
myASBD.mFormatFlags = kAudioFormatFlagIsFloat;
myASBD.mBytesPerPacket = bytesPerSample;
myASBD.mFramesPerPacket = 1;
myASBD.mBytesPerFrame = bytesPerSample;
myASBD.mChannelsPerFrame = 2;
myASBD.mBitsPerChannel = 8 * bytesPerSample; //32bit float

Resources