I am working on a project that requires almost live (we can live with some core audio latency) audio monitoring from a bluetooth microphone to the iPhone's default
I've tried both:
UInt32 allowBluetoothInput = 1;
OSStatus stat = AudioSessionSetProperty (kAudioSessionProperty_OverrideCategoryEnableBluetoothInput, sizeof (allowBluetoothInput), &allowBluetoothInput);
UInt32 audioRouteOverride = kAudioSessionOverrideAudioRoute_Speaker;
stat= AudioSessionSetProperty (kAudioSessionProperty_OverrideAudioRoute, sizeof (audioRouteOverride), &audioRouteOverride);
and
UInt32 overrideAudioRoute = kAudioSessionOverrideAudioRoute_None;
AudioSessionSetProperty (kAudioSessionProperty_OverrideAudioRoute, sizeof (overrideAudioRoute), &overrideAudioRoute);
UInt32 doChangeDefaultRoute = 1;
AudioSessionSetProperty(kAudioSessionProperty_OverrideCategoryDefaultToSpeaker, sizeof(doChangeDefaultRoute), &doChangeDefaultRoute);
but neither yields the correct behavior
Is there any property I can set to get this to work?
You are talking about a speak through application? I did some R&D on this awhile back and ran into the issues you are talking about. At the present it seems that this is not possible on iOS. You cannot configure your iPhone to record from the microphone and route that audio recording out to the speaker for instant playback. The best you can do is record it, stop it and send it. Recording and playing through the speaker cannot be configured in an audio session. Spending time researching this (like I did) will be upsetting. It has nothing to do with core audio hacking it has to do with what is allowed by the system
Related
I am using novocaine to get mic audio data and PyAudio on another computer to play the audio.
I am using WAMP to send over the data, so far data sends just fine.
But I am having trouble trying to get the data received from the mic on iOS device to play on the Py device.
Novocaine returns an array of floats when getting audio so I figured I do this in order to transfer the data properly.
NSMutableData *outBuffer = [NSMutableData dataWithCapacity:0];
[outBuffer appendBytes:&newAudio length:sizeof(float)];
When playing with the data on python device I get a length of 4 however, when I try to print it out there is always a problem is UTF formatting issue.
Basic of playing audio with PyAudio
I am unsure how I should be translating this float audio data.
Edit:
Novocaine *audioManager = [Novocaine audioManager];
[audioManager setInputBlock:^(float *newAudio, UInt32 numSamples, UInt32 numChannels) {
// Now you're getting audio from the microphone every 20 milliseconds or so. How's that for easy?
// Audio comes in interleaved, so,
// if numChannels = 2, newAudio[0] is channel 1, newAudio[1] is channel 2, newAudio[2] is channel 1, etc.
}];
[audioManager play];
This is what I would like to do:
Get audio from the microphone
Encode it in AAC, G.711 or G.726
Write the encoded frames to a socket.
And this is how I'm trying to get there:
I'm getting audio (PCM) from the microphone using TheAmazingAudioEngine and putting it in a buffer;
Using TPAACAudioConverter I'm reading audio from my buffer and writing to a temp file (AAC);
In the processing thread of TPAACAudioConverter I replaced this:
OSStatus status = ExtAudioFileWrite(destinationFile, numFrames, &fillBufList);
with this:
OSStatus status = ExtAudioFileWrite(destinationFile, numFrames, &fillBufList);
UInt32 framesWritten = numFrames;
totalFramesWritten += framesWritten;
AudioBufferList readData;
readData.mNumberBuffers = 1;
ExtAudioFileSeek(destinationFile, totalFramesWritten - framesWritten);
OSStatus readStatus = ExtAudioFileRead(destinationFile, &numFrames, &readData);
ExtAudioFileSeek(destinationFile, totalFramesWritten);
NSLog(#"Bytes read=%d", numFrames);
but what I get is 0 numFrames read from file.
Any idea on what I may be doing wrong or any suggestion on alternative paths to achieve what I need?
The issue is that whatever ExtAudioFile does under the hood doesn't allow for seeking on a file that is open for writing. If you look at the documentation for ExtAudioFileSeek it says "This function's behavior with files open for writing is currently undefined".
You can solve this by using the more extensible (and difficult) Audio File Services and the Audio Converter Services directly instead of the convenient Extended audio file services.
I abandoned this approach and reused the AQRecorder class from the SpeakHere example by Apple.
The project is available here https://github.com/robovm/apple-ios-samples/tree/master/SpeakHere.
I am using FFMPEG in combination with FFmpegAudioPlayer to do live streaming. The issue I am having is that, while the audio can be decoded and played, there's a constant clicking/screeching noise in the audio that isn't present when streaming the same source by other applications. So I am guess the issue arise due to how I process the FFMPEG AV_CODEC_ID_PCM_S16BE audio data before handing it to AudioQueue:
audioFormat.mFormatID = kAudioFormatLinearPCM;
audioFormat.mFormatFlags = kAudioFormatFlagsCanonical;//kAudioFormatFlagIsBigEndian|kAudioFormatFlagIsAlignedHigh;
audioFormat.mSampleRate = pAudioCodecCtx->sample_rate;
audioFormat.mBitsPerChannel = 8*av_get_bytes_per_sample(AV_SAMPLE_FMT_S16);
audioFormat.mChannelsPerFrame = pAudioCodecCtx->channels;
audioFormat.mBytesPerFrame = pAudioCodecCtx->channels * av_get_bytes_per_sample(AV_SAMPLE_FMT_S16);
audioFormat.mBytesPerPacket= pAudioCodecCtx->channels * av_get_bytes_per_sample(AV_SAMPLE_FMT_S16);
audioFormat.mFramesPerPacket = 1;
audioFormat.mReserved = 0;
pSwrCtx = swr_alloc_set_opts(pSwrCtx,
1,//pAudioCodecCtx->channel_layout,
AV_SAMPLE_FMT_S16,
pAudioCodecCtx->sample_rate,
1,//pAudioCodecCtx->channel_layout,
AV_SAMPLE_FMT_S16,
pAudioCodecCtx->sample_rate,
0,
0);
outCount = swr_convert(pSwrCtx,
(uint8_t **)(&pOut),
in_samples,
(const uint8_t **)pAVFrame1->extended_data,
in_samples);
Please also note that I've tried many different parameters for swr_alloc_set_opts, but either the audio became unrecognizable or the noise persisted.
Here's a sample of the audio with clicking sound, if it helps.
I don't know exactly, but s16be is integer (16bit) whereas kAudioFormatLinearPCM is float (32bit).
If I were in your shoes, I'll just use s16be and kAudioFormatLinearPCM format which means fixing AudioCodecCtx->channels * av_get_bytes_per_sample(AV_SAMPLE_FMT_S16) and others.
Then insert PCM format conversion step between ffmpeg -> iOS data flow.
This post looks like very helpful: iOS Core Audio : Converting between kAudioFormatFlagsCanonical and kAudioFormatFlagsAudioUnitCanonical
It turned out the noise isn't a problem in decoding the audio stream, but a problem in the camera device that feed the audio stream to our app.
The noise is quite inaudible when connected to the Android app, caused us to miss it when testing Android app and thought it's a problem with how our iOS app handles audio rather than something that's wrong with the device.
I have followed method that takes care about my speaker state:
AudioSession.h
enum {
kAudioSessionOverrideAudioRoute_None = 0,
kAudioSessionOverrideAudioRoute_Speaker = 'spkr'
};
MyClass
#synthesize speakerEnabled;
...
- (void)setSpeakerEnabled:(BOOL)enable {
speakerEnabled = enable;
if(enable) {
UInt32 audioRouteOverride = kAudioSessionOverrideAudioRoute_Speaker;
AudioSessionSetProperty (kAudioSessionProperty_OverrideAudioRoute
, sizeof (audioRouteOverride)
, &audioRouteOverride);
} else {
UInt32 audioRouteOverride = kAudioSessionOverrideAudioRoute_None;
AudioSessionSetProperty (kAudioSessionProperty_OverrideAudioRoute
, sizeof (audioRouteOverride)
, &audioRouteOverride);
}
}
However it works on iPhone only, for iPad - nothing happens.
When I press on button: Speaker On, I enter to if(enable) and AudioSessionSetProperty receives kAudioSessionOverrideAudioRoute_Speaker;
When I press on button: Speaker Off, I enter to else and AudioSessionSetProperty receives kAudioSessionOverrideAudioRoute_None;
I started to debug and din't find difference between to devices.
I have iPad2 iOS 6.1.
Did I miss something?
please, help me
Edit
As LombaX says I added on launch application AVAudioSession category:
NSError *err = nil;
BOOL success = [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:&err];
if(!success){
[MyLogger logc:MyLoggerLog format:"%#",[err localizedDescription]];
}
success = YES
Still doesn't work.
I think you misunderstand the use of this property. You are hoping to enable/disable your speaker. That is not quite the intent of kAudioSessionProperty_OverrideAudioRoute. Rather, it affects the output sound routing, in a fairly limited manner.
These are the various possible output routes avaiable to the iPhone
extern const CFStringRef kAudioSessionOutputRoute_LineOut
extern const CFStringRef kAudioSessionOutputRoute_Headphones
extern const CFStringRef kAudioSessionOutputRoute_BluetoothHFP
extern const CFStringRef kAudioSessionOutputRoute_BluetoothA2DP
extern const CFStringRef kAudioSessionOutputRoute_BuiltInReceiver
extern const CFStringRef kAudioSessionOutputRoute_BuiltInSpeaker
extern const CFStringRef kAudioSessionOutputRoute_USBAudio
extern const CFStringRef kAudioSessionOutputRoute_HDMI
extern const CFStringRef kAudioSessionOutputRoute_AirPlay
These are only possible routes - what is actually available is context-dependent. Apple severely restricts your ability to determine these routes in-app as it is something that the user needs to control in a device-consistent manner. Most of them are determined implicitly by the user plugging/unplugging hardware (headphone, USB, HDMI, line out), and Apple doesn't want your app to mess with user expectations here.
Airplay can be selected using MPVolumeView's routeButton if the media context is correct (and airplay is available). Bluetooth can be guided by OverrideCategoryEnableBluetoothInput (which controls both input and output)
IN particular, note that kAudioSessionOutputRoute_BuiltInReceiver is the low-level speaker on the iPhone you hold to your ear when making a phone call. This is the default audio output route for the iPhone if an external device (eg headphones) is not plugged in. kAudioSessionOutputRoute_BuiltInSpeaker is the 'handsfree' louder speaker at the bottom of the phone.
You can reroute from whatever the current default is to this BuiltInSpeaker by setting one of these override properties:
key: kAudioSessionProperty_OverrideAudioRoute
values: kAudioSessionOverrideAudioRoute_Speaker
: kAudioSessionOverrideAudioRoute_None
Specifies whether or not to override the audio session category’s normal audio route.
key: kAudioSessionProperty_OverrideCategoryDefaultToSpeaker
values: TRUE
: FALSE
Specifies whether or not to route audio to the speaker (instead of to the receiver) when no other audio route, such as a headset, is connected.
Both of these are only designed to be used with the kAudioSessionCategory_PlayAndRecord Audio Session category.
Notice that in both cases you are not choosing amongst any output route, you are only overriding the "default route" in favour of the built-in (loud)speaker.
The iPad, lacking a phone, does not have a BuiltInReceiver type of speaker. It's default route, in the absence of connected gadgets or airplay, is that very same BuiltInSpeaker. Therefore, overriding doesn't have any effect.
Assuming you are really trying to mute the audio in your app, how you achieve that depends on many other aspects of your app design. If you want to mute the device, Apple would rather the user controls this via the ring/silent switch. It seems they wouldn't have it any other way:
I've had a response from Apple on this.
They've said they don't and never have provided a method for detecting hardware mute switch and don't intend to do so. https://stackoverflow.com/a/8009073/1375695
"The speaker setting is an overide for the device, not for a given sound"
http://lists.apple.com/archives/coreaudio-api/2009/Mar/msg00300.html
Have you tried to set the audio category before playing?
I remember I had the same problem a while ago, and found this solution (adjust the category)
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:nil];
I didn't found anything on the documentation, I think that on iPad you must set the category because iPhone and iPad differs about a detail: on iPhone, the left hardware switch can mute the speaker. On iPad, the same switch is used to lock the rotation.
Try and let me know
EDIT:
useful link:
http://developer.apple.com/library/ios/#documentation/Audio/Conceptual/AudioSessionProgrammingGuide/Cookbook/Cookbook.html
I really need help with this issue. I'm developing an iOS application with audio units, the recorded audio needs to at 8bit / 8000 hertz sample rate using alaw format. How ever I'm getting a distorted voice coming out the speaker.
I came across this sample online:
http://www.stefanpopp.de/2011/capture-iphone-microphone/comment-page-1/
while trying to debug my app I used my audioFormat in his application and I am getting the same distorted sound. I guessing I either have incorrect settings or I need to do something else to enable this to work. Given the application in the link and the below audioFormat can anyone tell me if I'm doing something wrong or missing something ? I don't know a lot about this stuff, thanks.
Audio Format:
AudioStreamBasicDescription audioFormat;
audioFormat.mSampleRate = 8000;
audioFormat.mFormatID = kAudioFormatALaw;
audioFormat.mFormatFlags = kAudioFormatFlagIsPacked | kAudioFormatFlagIsSignedInteger;
audioFormat.mFramesPerPacket = 1;
audioFormat.mChannelsPerFrame = 1;
audioFormat.mBitsPerChannel = 8;
audioFormat.mBytesPerPacket = 1;
audioFormat.mBytesPerFrame = 1;
Eventually got it to play correctly. I'm posting here to help out anyone else facing similar issues.
Main issue I was facing is that there is a huge difference between the simulator and an actual device. Running the app on the device the sound quality was better but it kept skipping every second or 2, I found a setting that seemed to fix this and a setting to change the buffer size / duration. (The duration setting does not work on the simulator, some of my issues were needing it to run at a certain rate to sync with something else, this was causing distorted sounds)
status = AudioSessionInitialize(NULL, kCFRunLoopDefaultMode, NULL, audioUnit);
UInt32 audioCategory = kAudioSessionCategory_PlayAndRecord;
status = AudioSessionSetProperty(kAudioSessionProperty_AudioCategory, sizeof(audioCategory), &audioCategory);
[self hasError:status:__FILE__:__LINE__];
Float32 preferredBufferSize = 0.005805; // in seconds
status = AudioSessionSetProperty(kAudioSessionProperty_PreferredHardwareIOBufferDuration, sizeof(preferredBufferSize), &preferredBufferSize);
[self hasError:status:__FILE__:__LINE__];
status = AudioSessionSetActive(true);
The first audio session property is what stopped the skipping making it play much more smoothly. The second adjusts the buffer duration, this is in seconds how often the callbacks are fired and will give you a different buffer size. Its best effort meaning it will get as close as it can to the value you provide but it seems to have a list of available sizes and picks the closest.
See the post I link to in my question for a very good tutorial / sample program to get started with this stuff.