I have AudioBuffer as shown below. It can play through the speaker. I would like to know a way to amplify those buffer before I play. How shall I modify?
/**
This callback is called when the audioUnit needs new data to play through the
speakers. If you don't have any, just don't write anything in the buffers
*/
static OSStatus playbackCallback(void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData) {
// Notes: ioData contains buffers (may be more than one!)
// Fill them up as much as you can. Remember to set the size value in each buffer to match how
// much data is in the buffer.
for (int i=0; i < ioData->mNumberBuffers; i++) { // in practice we will only ever have 1 buffer, since audio format is mono
AudioBuffer buffer = ioData->mBuffers[i];
// NSLog(#" Buffer %d has %d channels and wants %d bytes of data.", i, buffer.mNumberChannels, buffer.mDataByteSize);
// copy temporary buffer data to output buffer
UInt32 size = min(buffer.mDataByteSize, [iosAudio tempBuffer].mDataByteSize); // dont copy more data then we have, or then fits
memcpy(buffer.mData, [iosAudio tempBuffer].mData, size);
buffer.mDataByteSize = size; // indicate how much data we wrote in the buffer
// uncomment to hear random noise
/*
UInt16 *frameBuffer = buffer.mData;
for (int j = 0; j < inNumberFrames; j++) {
frameBuffer[j] = rand();
}
*/
}
return noErr;
}
Related
I use Audiokit (in Objective-C) for realtime audio processing. I feed a C++ algorithm through a tap or lazy tap where the buffer is being modified.
I thought that would be obvious but...how can I playback the modified buffer in the output? Are taps only for analysis?
[self->microphoneGain.avAudioNode installTapOnBus:0 bufferSize:1024 format:format block:^(AVAudioPCMBuffer * _Nonnull buffer, AVAudioTime * _Nonnull when) {
if (buffer.frameLength == 0) {
return;
}
// Process data -> return modified buffer
processData(buffer.floatChannelData[0], buffer.floatChannelData[1], buffer.frameLength);
// -> How to play back buffer?
}];
Furthermore, I can't get taps buffer size lower than 4800 samples. What would be my best option to get a better latency? I read about AUAudioUnit subclassing, render callback or realtime mode for AudioEngine, but I'm quite lost when trying to implement one of these with AudioKit. Thanks!
EDIT:
I managed to set a render callback which has apparently solved both of my problems.
AURenderCallbackStruct processingCallback;
processingCallback.inputProc = processingCalbackProc;
processingCallback.inputProcRefCon = (__bridge void *)(self);
OSStatus status = AudioUnitSetProperty(AudioKit.engine.outputNode.audioUnit,
kAudioUnitProperty_SetRenderCallback,
kAudioUnitScope_Input,
0,
&processingCallback,
sizeof(processingCallback));
if(status != noErr) {
return false;
}
OSStatus processingCalbackProc (void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData)
{
__unsafe_unretained MyClass *self = (__bridge MyClass *)inRefCon;
printf("%u, ", (unsigned int)inNumberFrames); // -> low latency!
if (!ioData) ioData = self->audioBufferList;
OSStatus status = AudioUnitRender(AudioKit.engine.outputNode.audioUnit,
ioActionFlags,
inTimeStamp,
1,
inNumberFrames,
ioData);
if(status != noErr) { return status; }
// Get buffers
unsigned int inputChannels = 2;
float *buffer[inputChannels];
for (int i = 0; i < inputChannels; i++) {
buffer[i] = (float *)ioData->mBuffers[i].mData;
}
// Process data
processData(buffer[0], buffer[1], inNumberFrames);
return noErr;
}
Now I can easily get buffers as low as 256samples (probably even less but not needed in my case) and when buffer[n]are modified, it outputs the modified buffers.
Everything seems to be fine, I just hope this is the right approach.
I am trying to implement playing pcm audio received from remote server via socket. Here was my previous question link. This works fine as I use circular buffer to always feed in the incoming buffer.
However I have a problem that there is a huge noise sound that is being produced if I have no buffer supplied to my output. This happens when I begin to use AudioOutputUnitStart(_audioUnit) and when there is no buffer to play.
I suspect I have to fix this in my OutputRenderCallback function below or may be there is something else I need to do :
static OSStatus OutputRenderCallback(void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData){
Test *output = (__bridge Test*)inRefCon;
TPCircularBuffer *circularBuffer = [output outputShouldUseCircularBuffer];
if( !circularBuffer ){
SInt32 *left = (SInt32*)ioData->mBuffers[0].mData;
for(int i = 0; i < inNumberFrames; i++ ){
left[ i ] = 0.0f;
}
return noErr;
};
int32_t bytesToCopy = ioData->mBuffers[0].mDataByteSize;
SInt16* outputBuffer = ioData->mBuffers[0].mData;
uint32_t availableBytes;
SInt16 *sourceBuffer = TPCircularBufferTail(circularBuffer, &availableBytes);
int32_t amount = MIN(bytesToCopy,availableBytes);
memcpy(outputBuffer, sourceBuffer, amount);
TPCircularBufferConsume(circularBuffer,amount);
return noErr;
}
I highly appreciate you help.Thanks.
An audio unit callback requires that you always put the requested amount of samples in the AudioBufferList buffers. Your code does not do that if the amount (from that available circular buffer) is less.
So put something in the output buffer always, as your code does if there is no circular buffer.
BTW: calling a method:
[output outputShouldUseCircularBuffer]
inside a callback is a violation of Apple's rules for real-time audio.
I am posting my answer incase someone else stumbles at the same point as I was. I am new to objective c so incase someone has a better solution. I do welcome any suggestions.
As #hotpaw2 suggested the AudioBufferList needs to be feed with samples and in my case when my circularBuffer had nothing inside of it. I had to feed the AudioBufferList with frames being set to 0.0f
static OSStatus OutputRenderCallback(void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData){
Test *output = (__bridge Test*)inRefCon;
TPCircularBuffer *circularBuffer = [output outputShouldUseCircularBuffer];
int32_t bytesToCopy = ioData->mBuffers[0].mDataByteSize;
SInt16* outputBuffer = ioData->mBuffers[0].mData;
uint32_t availableBytes;
SInt16 *sourceBuffer = TPCircularBufferTail(circularBuffer, &availableBytes);
int32_t amount = MIN(bytesToCopy,availableBytes);
if (amount>0) {
memcpy(outputBuffer, sourceBuffer, amount);
TPCircularBufferConsume(circularBuffer,amount);
}
else{
SInt32 *left = (SInt32*)ioData->mBuffers[0].mData;
for(int i = 0; i < inNumberFrames; i++ ){
left[ i ] = 0.0f;
}
return noErr;
}
return noErr; }
I'm developing an app for transfer audio data between 2 iOS devices.
At the moment I'm using NSInputStream for receive the incoming audio and the data into a queue (AudioQueueNewInput).
The callback of the AudioQueue is
void TDAudioQueueInputCallback(
void* inUserData,
AudioQueueRef inAudioQueue,
AudioQueueBufferRef inBuffer,
const AudioTimeStamp* inStartTime,
UInt32 inNumPackets,
const AudioStreamPacketDescription* inPacketDesc)
{
NSLog(#"recordCallback %u", (unsigned int)inBuffer->mAudioDataByteSize);
TDAudioQueue *audioQueue = (__bridge TDAudioQueue *)inUserData;
AudioBufferList *audioBufferList = [audioQueue getBufferListFromQueueBuffer:inBuffer];
[audioQueue sendAudioBufferToRecord:audioBufferList];
[audioQueue didFreeAudioQueueBuffer:inBuffer];
}
getBufferListFromQueueBuffer method
-(AudioBufferList *) getBufferListFromQueueBuffer: (AudioQueueBufferRef ) data
{
if (data->mAudioDataByteSize > 0)
{
NSUInteger len = data->mAudioDataByteSize;
//I guess you can use Byte*, void* or Float32*. I am not sure if that makes any difference.
Byte * byteData = (Byte*) malloc (len);
memcpy (byteData, data->mAudioData, len);
if (byteData)
{
AudioBufferList * theDataBuffer =(AudioBufferList*)malloc(sizeof(AudioBufferList) * 1);
theDataBuffer->mNumberBuffers = 1;
theDataBuffer->mBuffers[0].mDataByteSize = len;
theDataBuffer->mBuffers[0].mNumberChannels = 1;
theDataBuffer->mBuffers[0].mData = byteData;
// Read the data into an AudioBufferList
return theDataBuffer;
}
}
return nil;
}
After this I'm saving the audio into a file.
The problem is I can hear the first 2 or 3 seconds but then I start to get this warning and my audio file gets silence.
The warning is:
[AQConverterThread] >aq> 995: Input ring buffer returning 0x400 frames of silence
I am using AudioUnit to play input from the microphone to the earphones.
It's working great. Now I need to increase the volume of weak sounds and decrease strong ones.
I found a way to increase the sound:
static OSStatus performRender (void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData)
{
OSStatus err = noErr;
if (*cd.audioChainIsBeingReconstructed == NO)
{
// we are calling AudioUnitRender on the input bus of AURemoteIO
// this will store the audio data captured by the microphone in ioData
err = AudioUnitRender(cd.rioUnit, ioActionFlags, inTimeStamp, 1, inNumberFrames, ioData);
// filter out the DC component of the signal
cd.dcRejectionFilter->ProcessInplace((Float32*) ioData->mBuffers[0].mData, inNumberFrames);
//Add Volume
float desiredGain = 2.0f;
for(UInt32 bufferIndex = 0; bufferIndex < ioData->mNumberBuffers; ++bufferIndex) {
float *rawBuffer = (float *)ioData->mBuffers[bufferIndex].mData;
vDSP_vsmul(rawBuffer, 1, &desiredGain, rawBuffer, 1, inNumberFrames);
}
// mute audio if needed
if (*cd.muteAudio)
{
for (UInt32 i=0; i<ioData->mNumberBuffers; ++i)
memset(ioData->mBuffers[i].mData, 0, ioData->mBuffers[i].mDataByteSize);
}
}
return err;
}
My question is how to I get what is the current volume so I would know how much to gain it and vice versa
Thanks!
Getting the "volume" depends on the type of AudioUnit. Some audio units have input levels, output levels, and "global" volume levels.
// MatrixMixer
Float32 volume = 0;
OSStatus result = AudioUnitGetParameter(mxmx_unit, kMatrixMixerParam_Volume, kAudioUnitScope_Global, 0, &volume);
// MultiChannelMixer
Float32 volume = 0;
OSStatus result = AudioUnitGetParameter(mcmx_unit, kMultiChannelMixerParam_Volume, kAudioUnitScope_Global, 0, &volume);
I've successfully recorded audio from the microphone into an audio file using Audio Units with the help of openframeworks and this website http://atastypixel.com/blog/using-remoteio-audio-unit.
I want to be able to stream the file back to audio units and play the audio. According to Play an audio file using RemoteIO and Audio Unit I can use ExtAudioFileOpenURL and ExtAudioFileRead. However, how do I play audio data in my buffer?
This is what I currently have:
static OSStatus setupAudioFileRead() {
//construct the file destination URL
CFURLRef destinationURL = audioSystemFileURL();
OSStatus status = ExtAudioFileOpenURL(destinationURL, &audioFileRef);
CFRelease(destinationURL);
if (checkStatus(status)) { ofLog(OF_LOG_ERROR, "ofxiPhoneSoundStream: Couldn't open file to read"); return status; }
while( TRUE ) {
// Try to fill the buffer to capacity.
UInt32 framesRead = 8000;
status = ExtAudioFileRead( audioFileRef, &framesRead, &inputBufferList );
// error check
if( checkStatus(status) ) { break; }
// 0 frames read means EOF.
if( framesRead == 0 ) { break; }
//play audio???
}
return noErr;
}
From this author: http://atastypixel.com/blog/using-remoteio-audio-unit/, if you scroll down to the PLAYBACK section, try something like this:
static OSStatus playbackCallback(void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData) {
// Notes: ioData contains buffers (may be more than one!)
// Fill them up as much as you can. Remember to set the size value in each buffer to match how
// much data is in the buffer.
for (int i=0; i < ioData->mNumberBuffers; i++)
{
AudioBuffer buffer = ioData->mBuffers[i];
// copy from your whatever buffer data to output buffer
UInt32 size = min(buffer.mDataByteSize, your buffer.size);
memcpy(buffer.mData, your buffer, size);
buffer.mDataByteSize = size; // indicate how much data we wrote in the buffer
// To test if your Audio Unit setup is working - comment out the three
// lines above and uncomment the for loop below to hear random noise
/*
UInt16 *frameBuffer = buffer.mData;
for (int j = 0; j < inNumberFrames; j++) {
frameBuffer[j] = rand();
}
*/
}
return noErr;
}
If you are only looking for recording from MIC to a file and play it back, the Apple's Speakhere sample is probably much more ready to use.
Basically,
1. Create a RemoteIO unit (See references about how to create RemoteIO);
Create a FilePlayer audio unit which is a dedicated audio unit to read an audio file and provide audio data in the file to output units, for example, the RemoteIO unit created in step 1. To actually use the FilePlayer, a lot of settings (specify which file to play, which part of the file to play, etc.) are needed to be done on the it;
Set kAudioUnitProperty_SetRenderCallback and kAudioUnitProperty_StreamFormat properties of the RemoteIO unit. The first property is essentially a callback function from which the RemoteIO unit pulls audio data and play it. The second property must be set in accordance to StreamFormat that supported by the FilePlayer. It can be derived from a get-property function invoked on the FilePlayer.
Define the callback set in step 3 where the most important thing to do is asking the FilePlayer to render into the buffer provided by the callback for which you will need to invoke AudioUnitRender() on the FilePlayer.
Finally start the RemoteIO unit to play the file.
Above is just a preliminary outline of basic things to do to play files using audio units on iOS. You can refer to Chris Adamson and Kevin Avila's Learning Core Audio for details.
It's a relatively simple approach that utilizes the audio unit mentioned in the Tasty Pixel blog. In the recording callback, instead of filling the buffer with data from the microphone, you could fill it with data from the file using ExtAudioFileRead. I'll try and paste an example below. Mind you this will just work for .caf files.
In the start method call an readAudio or initAudioFile function, something that just gets all the info about the file.
- (void) start {
readAudio();
OSStatus status = AudioOutputUnitStart(audioUnit);
checkStatus(status);
}
Now in the readAudio method you initialize the audio file reference as such.
ExtAudioFileRef fileRef;
void readAudio() {
NSString * name = #"AudioFile";
NSString * source = [[NSBundle mainBundle] pathForResource:name ofType:#"caf"];
const char * cString = [source cStringUsingEncoding:NSASCIIStringEncoding];
CFStringRef str = CFStringCreateWithCString(NULL, cString, kCFStringEncodingMacRoman);
CFURLRef inputFileURL = CFURLCreateWithFileSystemPath(kCFAllocatorDefault, str, kCFURLPOSIXPathStyle, false);
AudioFileID fileID;
OSStatus err = AudioFileOpenURL(inputFileURL, kAudioFileReadPermission, 0, &fileID);
CheckError(err, "AudioFileOpenURL");
err = ExtAudioFileOpenURL(inputFileURL, &fileRef);
CheckError(err, "ExtAudioFileOpenURL");
err = ExtAudioFileSetProperty(fileRef, kExtAudioFileProperty_ClientDataFormat, sizeof(AudioStreamBasicDescription), &audioFormat);
CheckError(err, "ExtAudioFileSetProperty");
}
Now that you have the Audio Data at hand, next step is pretty easy. In the recordingCallback read the data from the file instead of the mic.
static OSStatus recordingCallback(void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData) {
// Because of the way our audio format (setup below) is chosen:
// we only need 1 buffer, since it is mono
// Samples are 16 bits = 2 bytes.
// 1 frame includes only 1 sample
AudioBuffer buffer;
buffer.mNumberChannels = 1;
buffer.mDataByteSize = inNumberFrames * 2;
buffer.mData = malloc( inNumberFrames * 2 );
// Put buffer in a AudioBufferList
AudioBufferList bufferList;
bufferList.mNumberBuffers = 1;
bufferList.mBuffers[0] = buffer;
// Then:
// Obtain recorded samples
OSStatus err = ExtAudioFileRead(fileRef, &inNumberFrames, &bufferList);
// Now, we have the samples we just read sitting in buffers in bufferList
// Process the new data
[iosAudio processAudio:&bufferList];
// release the malloc'ed data in the buffer we created earlier
free(bufferList.mBuffers[0].mData);
return noErr;
}
This worked for me.