I am using AudioUnit class for recording and playback. During recording i can listen sound. Problem is that when i use sample rate 44100 then it working fine but if i use sample rate 8000 then it generating noise. After recording with 8000 sample rate when i play then there is no noise, there is actual sound.
Means only the time of recording it generate noise with actual sound.
My AudioStreamBasicDescription setting is-
audioStreamDescription.Format = AudioFormatType.LinearPCM;
audioStreamDescription.FormatFlags = AudioFormatFlags.LinearPCMIsSignedInteger |
AudioFormatFlags.LinearPCMIsPacked;
audioStreamDescription.SampleRate = 8000; // 44100;
audioStreamDescription.BitsPerChannel = 16;
audioStreamDescription.ChannelsPerFrame = 1;
audioStreamDescription.BytesPerFrame = (16 / 8);
audioStreamDescription.FramesPerPacket = 1;
audioStreamDescription.BytesPerPacket = audioStreamDescription.BytesPerFrame * audioStreamDescription.FramesPerPacket;
audioStreamDescription.Reserved = 0;
AudioUnit setting is-
public void prepareAudioUnit()
{
// Getting AudioComponent Remote output
_audioComponent = AudioComponent.FindComponent(AudioTypeOutput.Remote);
// creating an audio unit instance
audioUnit = new AudioUnit.AudioUnit(_audioComponent);
// turning on microphone
audioUnit.SetEnableIO(true, AudioUnitScopeType.Input, 1 );
audioUnit.SetEnableIO(true, AudioUnitScopeType.Output, 0 );
// setting audio format
var austat = audioUnit.SetFormat(audioStreamDescription, AudioUnitScopeType.Output, 1);
var austatInput = audioUnit.SetFormat(audioStreamDescription, AudioUnitScopeType.Input, 0);
//audioUnit.SetSampleRate(8000.0f, AudioUnitScopeType.Output, 0);
//audioUnit.SetSampleRate(8000.0f, AudioUnitScopeType.Input, 1);
// setting callback method
audioUnit.SetRenderCallback(render_CallBack, AudioUnitScopeType.Input, 0);
audioUnit.Initialize();
}
Now, my main question is how i can remove that noise which is comming with actual sound?
If i am not able to explain properly then please let me know.
I am totally new into this bluetooth side, if anyone can help me or provide over links for further information could be really helpful.
I am trying to create a seamless high quality bit rate transfer of audio from iPhone to a third party bluetooth device (i.e a headphone).
Generally, from iPhone the audio will be compressed encoded and transfer for fast connectivity to a bluetooth device, but the audio quality will decrease.
I am finding ways to change the codec used and to have a better bit rate so the audio listening on the bluetooth is of good quality for both listening and recording.
You can use AudioUnit in iOS and change the sample rate using AudioStreamBasicDescription
AudioStreamBasicDescription audioDescription = {0};
audioDescription.mFormatID = kAudioFormatLinearPCM;
audioDescription.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked | kAudioFormatFlagsNativeEndian;
audioDescription.mChannelsPerFrame = 1;
audioDescription.mBytesPerPacket = sizeof(SInt16)*audioDescription.mChannelsPerFrame;
audioDescription.mFramesPerPacket = 1;
audioDescription.mBytesPerFrame = sizeof(SInt16)*audioDescription.mChannelsPerFrame;
audioDescription.mBitsPerChannel = 8 * sizeof(SInt16);
audioDescription.mSampleRate = 8000.0;
And you can change the preferred sample rate of your app:
let session = AVAudioSession.sharedInstance()
try session.setCategory(AVAudioSessionCategoryPlayAndRecord)
let bufferDuration :NSTimeInterval = 0.005
try session.setPreferredIOBufferDuration(bufferDuration)
try session.setPreferredSampleRate(8000.0)
try session.setActive(true)
Playback through my AudioUnit works fine until I start getting gyroscope updates from a CMMotionManager. I assumed this was due to a performance hit, but when I measured the runtime of my callback during said gyroscope updates it isn't as high as other CMMotionManager-less trials with smooth playback, yet the playback is choppy.
Some visual explanation (The red is the time between consecutive callbacks. The green--it's hard to see but there's bits of green right underneath all the red--is the runtime of the callback, which is consistently just a few milliseconds less):
Sorry if the graph is a bit messy, hopefully I'm still getting my point across.
In sum, rather than the runtime of the callback, the quality of the playback seems more tied to the "steadiness" of the frequency at which the callback is, erm, called back. What could be going on here? Could my callback runtimes just be off? That seems unlikely. I'm timing my callback via calls to clock() at the beginning and end. Is my AudioUnit setup wrong? It is admittedly a bit hacked together, and I'm not using an AUGraph or anything.
AudioUnit initialization code:
AudioComponentDescription desc;
desc.componentType = kAudioUnitType_Output;
desc.componentSubType = kAudioUnitSubType_RemoteIO; // Remote I/O is for talking with the hardware
desc.componentFlags = 0;
desc.componentFlagsMask = 0;
desc.componentManufacturer = kAudioUnitManufacturer_Apple;
AudioComponent component = AudioComponentFindNext(NULL, &desc);
AudioComponentInstanceNew(component, &myAudioUnit);
UInt32 enableIO;
AudioUnitElement inputBus = 1;
AudioUnitElement outputBus = 0;
//Disabling IO for recording
enableIO = 0;
AudioUnitSetProperty(myAudioUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Input, inputBus, &enableIO, sizeof(enableIO));
//Enabling IO for playback
enableIO = 1;
AudioUnitSetProperty(myAudioUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Output, outputBus, &enableIO, sizeof(enableIO));
UInt32 bytesPerSample = BIT_DEPTH/8.0;
AudioStreamBasicDescription stereoStreamFormat = {0};
stereoStreamFormat.mBitsPerChannel = 8 * bytesPerSample;
stereoStreamFormat.mBytesPerFrame = bytesPerSample;
stereoStreamFormat.mBytesPerPacket = bytesPerSample;
stereoStreamFormat.mChannelsPerFrame = 2; // 2 indicates stereo
stereoStreamFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger |
kAudioFormatFlagsNativeEndian |
kAudioFormatFlagIsPacked |
kAudioFormatFlagIsNonInterleaved;
stereoStreamFormat.mFormatID = kAudioFormatLinearPCM;
stereoStreamFormat.mFramesPerPacket = 1;
stereoStreamFormat.mReserved = 0;
stereoStreamFormat.mSampleRate = SAMPLE_RATE;
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, inputBus, &stereoStreamFormat, sizeof(AudioStreamBasicDescription));
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, outputBus, &stereoStreamFormat, sizeof(AudioStreamBasicDescription));
//Setting input callback
AURenderCallbackStruct callbackStruct;
callbackStruct.inputProc = &recordingCallback; //TODO: Should there be an ampersand?
callbackStruct.inputProcRefCon = myAudioUnit;
AudioUnitSetProperty(myAudioUnit, kAudioOutputUnitProperty_SetInputCallback, kAudioUnitScope_Output, inputBus, &callbackStruct, sizeof(callbackStruct)); //TODO: Not sure of scope and bus/element
//Setting output callback
callbackStruct.inputProc = &playbackCallback;
callbackStruct.inputProcRefCon = myAudioUnit;
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Input, outputBus, &callbackStruct, sizeof(callbackStruct));
AudioUnitInitialize(myAudioUnit);
RemoteIO Playback Callback:
static OSStatus playbackCallback (void *inRefCon, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inBusNumber, UInt32 inNumberFrames, AudioBufferList *ioData) {
timeOfCallback = clock();
if (timeOfPrevCallback != 0) {
callbackInterval = (double)timeOfCallback - timeOfPrevCallback;
}
clock_t t1, t2;
t1 = clock();
FooBarClass::myCallbackFunction((short *)ioData->mBuffers[0].mData, (short *)ioData->mBuffers[1].mData);
t2 = clock();
cout << "Callback duration: " << ((double)(t2-t1))/CLOCKS_PER_SEC << endl;
cout << "Callback interval: " << callbackInterval/CLOCKS_PER_SEC << endl;
timeOfPrevCallback = timeOfCallback;
//prevCallbackInterval = callbackInterval;
return noErr;
}
In myCallbackFunction I'm reading from a handful of .wav files, filtering each one and mixing them together, and copying the output to the buffers passed to it. In the graph where I mention "incrementally adding computation" I'm referring to the number of input files I'm mixing together. Also, if it matters, gyroscope updates occur via an NSTimer that goes off every 1/25 of a second:
[self.getMotionManager startDeviceMotionUpdates];
gyroUpdateTimer = [NSTimer scheduledTimerWithTimeInterval:GYRO_UPDATE_INTERVAL target:self selector:#selector(doGyroUpdate) userInfo:nil repeats:YES];
...
+(void)doGyroUpdate {
double currentYaw = motionManager.deviceMotion.attitude.yaw;
// a couple more lines of not very expensive code
}
I should also be more clear about what I mean by choppiness in this sense: The audio isn't skipping, it just sounds really bad, as if an additional, crackly track was getting mixed in while the other tracks play smoothly. I'm not talking about clipping either, which it isn't because the only difference between good and bad playback is the gyroscope updates.
Thanks in advance for any tips.
----- Update 1 -----
My runtimes were a bit off because I was using clock(), which apparently doesn't work right for multithreaded applications. Apparently get_clock_time() is the proper way to measure runtimes across multiple threads but it's not implemented for Darwin. Though it's not an ideal solution, I'm using gettimeofday() now to measure the callback run time and intervals. Aside from the now steady intervals between successive callbacks (which were previously pretty erratic), things are more or less the same:
----- Update 2 -----
Interestingly enough, when I start and then stop CMMotionManager updates altogether via stopDeviceMotionUpdates, the crackliness persists...
----- Update 3 -----
'Crackliness' doesn't start until the first CMMotionManager is received, i.e. when the deviceMotion property is first checked after the NSTimer is first triggered. After that, crackliness persists regardless of the update frequency and even after updates are stopped.
You are trying to call Objective C methods, do synchronous file reads, and/or do significant computation (your C++ function) inside a real-time audio render callback. Also, logging to cout from inside a real-time thread is most likely not going to work reliably. These operations can take too long to meet the latency critical real-time requirements of Audio Unit callbacks.
Instead, for any data that does not have a tightly bounded maximum latency to generate, you might just copy that data from a lock free circular fifo or queue inside your render callback, and fill that audio data fifo slightly ahead of time in another thread (perhaps running on an NSTimer or CADisplayLink).
I had a similar issue when activating the location services. The issue was only present on slower devices like the iPod touch 16gb and not present on other hardware. I saw that you have in your graph title BUF_SIZE: 1024
Is this your call back time?
I fixed my problem by increasing the callback time (buffer size).
If you can handle more latency, try increasing the callback time using
NSTimeInterval _preferredDuration = (2048) / 44100.0 ; // Try bigger values here
NSError* err;
[[AVAudioSession sharedInstance]setPreferredIOBufferDuration:_preferredDuration error:&err];
When play the lpcm data decoded from ffmpeg with audioqueue, the elapsed time got by AudioQueueGetCurrentTime exceeds the duration of media. But when decode the same media with AVFoundation framework, the elapsed time equals duration of the media, and so when read media by ffmpeg with no decoded, then send the compressed media data to audioqueue, the elapsed time also equals duration of the media. The AudioStreamBasicDescription set as following:
asbd.mSampleRate = 44100;
asbd.mFormatID = kAudioFormatLinearPCM;
asbd.mFormatFlags = kAudioFormatFlagsCanonical;
asbd.mBytesPerPacket = 4;
asbd.mFramesPerPacket = 1;
asbd.mBytesPerFrame = 4;
asbd.mChannelsPerFrame = 2;
asbd.mBitsPerChannel = 16;
asbd.mReserved = 0;
When playing with data decoded from AVFoundation, the setting of AudioStreamBasicDescription is the same as above. By my test found that AudioTimeStamp.mSampleTime got by AudioQueueGetCurrentTime is different between ffmpeg and AVFoundation, the value of ffmpeg is greater than AVFoundation. So I want to know how this happen, and how to fix it?
Thanks!
Here the mistake is asbd.mSampleRate = 44100 is not always right, so sometimes the result is right, but others is wrong. Then you should set the asbd.mSampleRate = audioCodecCtx->sample_rate, this always work well!
I am working on a iOS project that needs acoustic echo cancelation so the kAudioUnitSubType_VoiceProcessingIO subtype seems to be a good choice.
Below is my audio unit description
//io unit description
AudioComponentDescription ioUnitDescription;
ioUnitDescription.componentType = kAudioUnitType_Output;
ioUnitDescription.componentSubType = kAudioUnitSubType_VoiceProcessingIO;
ioUnitDescription.componentManufacturer = kAudioUnitManufacturer_Apple;
ioUnitDescription.componentFlags = 0;
ioUnitDescription.componentFlagsMask = 0;
And based on my experience with RemoteIO subtype, I enabled the input element:
UInt32 enable = 1;
AudioUnitSetProperty(ioUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Input, 1, &enable, sizeof(enable));
However, I got error when initializing the audio graph. The same audio graph works well if the VoiceProcessingIO is replaced by RemoteIO.
Is there any difference between RemoteIO and VoiceProcessingIO that needs special attention?
Thanks,
Chuankai
In my experience the VoiceProcessingIO audio unit is much more finicky regarding buffer size and sample rate. Try a sample rate below 32000 Hz (perhaps start with 8000 Hz and make your way upward) and a fairly large buffer size (say 2048 samples or so). This is not documented. rdar number to come once I have a chance to file one.
I use the following format during set-up:
size_t bytesPerSample = sizeof(AudioSampleType);
AudioStreamBasicDescription canonicalFormat;
canonicalFormat.mSampleRate = self.samplerate;
canonicalFormat.mFormatID = kAudioFormatLinearPCM;
canonicalFormat.mFormatFlags = kAudioFormatFlagsCanonical;
canonicalFormat.mFramesPerPacket = 1;
canonicalFormat.mChannelsPerFrame = 1;
canonicalFormat.mBitsPerChannel = 8 * bytesPerSample;
canonicalFormat.mBytesPerPacket = bytesPerSample;
canonicalFormat.mBytesPerFrame = bytesPerSample;