GNAudioSourceMic raw audio location - ios

Question

I'm currently developing an app which uses Gracenote Mobile Client to create a fingerprint as well as identify which music I'm listening to. I've successfully implemented it on my project but now due to a business requirement I've to use the audio recorded by Gracenote for a different processing.
The point is: As GNAudioSourceMic encapsulates the whole microphone recording operations such as startRecording/stopRecording so I've no access to Microphone raw audio.
This is the code I'm using:
- (void)viewDidLoad
{
[super viewDidLoad];
[self setNeedsStatusBarAppearanceUpdate];
[self setupUI];
#try {
self.config = [GNConfig init:GRACENOTE_CLIENTID];
}
#catch (NSException * e) {
NSLog(#"%s clientId can't be nil or the empty string",__PRETTY_FUNCTION__);
[self.view setUserInteractionEnabled:FALSE];
return;
}
// Debug is disabled in the GUI by default
#ifdef DEBUG
[self.config setProperty:#"debugEnabled" value:#"1"];
#else
[self.config setProperty:#"debugEnabled" value:#"0"];
#endif
[self.config setProperty:#"lookupmodelocalonly" value:#"0"];
// -------------------------------------------------------------------------------
//Init AudioSource to Start Recording.
// -------------------------------------------------------------------------------
self.recognizeFromPCM = [GNRecognizeStream gNRecognizeStream:self.config];
self.audioConfig = [GNAudioConfig gNAudioConfigWithSampleRate:44100 bytesPerSample:2 numChannels:1];
self.objAudioSource = [GNAudioSourceMic gNAudioSourceMic:self.audioConfig];
self.objAudioSource.delegate=self;
NSError *err;
RecognizeStreamOperation *op = [RecognizeStreamOperation recognizeStreamOperation:self.config];
op.viewControllerDelegate = self;
err = [self.recognizeFromPCM startRecognizeSession:op audioConfig:self.audioConfig];
if (err) {
NSLog(#"ERROR: %#",[err localizedDescription]);
}
[self.objAudioSource startRecording];
[self performSelectorInBackground:#selector(setUpRecognizePCMSession) withObject:nil];
}
-(void) startRecordMicrophone{
#ifdef DEBUG
NSLog(#"%s startRecording",__PRETTY_FUNCTION__);
#endif
NSError *error;
error = [self.recognizeFromPCM idNow];
if (error) {
NSLog(#"ERROR: %#",[error localizedDescription]);
}
}
Does someone have been exposed to the same need as explained above ?
Thanks in advance

Answer 1

After much googling yesterday I came up with a solution which isn't what I was previously expecting but it works as good as I want to. I've decided to record the iOS microphone myself and then call a method on Grancenote SDK to recognise what I've just recorded.
Here's what has worked for me.
MicrophoneInput.h
#import <Foundation/Foundation.h>
#import <UIKit/UIKit.h>
#import <AVFoundation/AVFoundation.h>
#interface MicrophoneInput : UIViewController {
AVAudioPlayer *audioPlayer;
AVAudioRecorder *audioRecorder;
int recordEncoding;
enum
{
ENC_AAC = 1,
ENC_ALAC = 2,
ENC_IMA4 = 3,
ENC_ILBC = 4,
ENC_ULAW = 5,
ENC_PCM = 6,
} encodingTypes;
}
-(IBAction) startRecording;
-(IBAction) stopRecording;
#end
MicrophoneInput.m
#import "MicrophoneInput.h"
#implementation MicrophoneInput
- (void)viewDidLoad
{
[super viewDidLoad];
recordEncoding = ENC_PCM;
}
-(IBAction) startRecording
{
NSLog(#"startRecording");
[audioRecorder release];
audioRecorder = nil;
// Init audio with record capability
AVAudioSession *audioSession = [AVAudioSession sharedInstance];
[audioSession setCategory:AVAudioSessionCategoryRecord error:nil];
NSMutableDictionary *recordSettings = [[NSMutableDictionary alloc] initWithCapacity:10];
recordSettings[AVFormatIDKey] = #(kAudioFormatLinearPCM);
recordSettings[AVSampleRateKey] = #8000.0f;
recordSettings[AVNumberOfChannelsKey] = #1;
recordSettings[AVLinearPCMBitDepthKey] = #16;
recordSettings[AVLinearPCMIsBigEndianKey] = #NO;
recordSettings[AVLinearPCMIsFloatKey] = #NO;
//set the export session's outputURL to <Documents>/output.caf
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = paths[0];
NSURL* outURL = [NSURL fileURLWithPath:[documentsDirectory stringByAppendingPathComponent:#"output.caf"]];
[[NSFileManager defaultManager] removeItemAtURL:outURL error:nil];
NSLog(#"url loc is %#", outURL);
NSError *error = nil;
audioRecorder = [[ AVAudioRecorder alloc] initWithURL:outURL settings:recordSettings error:&error];
if ([audioRecorder prepareToRecord] == YES){
[audioRecorder record];
}else {
int errorCode = CFSwapInt32HostToBig ([error code]);
NSLog(#"Error: %# [%4.4s])" , [error localizedDescription], (char*)&errorCode);
}
NSLog(#"recording");
}
-(IBAction) stopRecording
{
NSLog(#"stopRecording");
[audioRecorder stop];
NSLog(#"stopped");
}
- (void)dealloc
{
[audioPlayer release];
[audioRecorder release];
[super dealloc];
}
#end
Obs.: If you're using ARC don't forget to add -fno-objc-arc compiler flag on Compiling BuildPhase as shown below.
YourViewController.h
//Libraries
#import <AVFoundation/AVFoundation.h>
#import <AudioToolbox/AudioToolbox.h>
//Echonest Codegen
#import "MicrophoneInput.h"
//GracenoteMusic
#import <GracenoteMusicID/GNRecognizeStream.h>
#import <GracenoteMusicID/GNAudioSourceMic.h>
#import <GracenoteMusicID/GNAudioConfig.h>
#import <GracenoteMusicID/GNCacheStatus.h>
#import <GracenoteMusicID/GNConfig.h>
#import <GracenoteMusicID/GNSampleBuffer.h>
#import <GracenoteMusicID/GNOperations.h>
#import <GracenoteMusicID/GNSearchResponse.h>
#interface YourViewController : UIViewController<GNSearchResultReady>
#end
YourViewController.m
#import "YourViewController.h"
#interface YourViewController ()
//Record
#property(strong,nonatomic) MicrophoneInput* recorder;
#property (strong,nonatomic) GNConfig *config;
#end
#implementation YourViewController
#pragma mark - UIViewController lifecycle
- (void)viewDidLoad
{
[super viewDidLoad];
self.recorder = [[MicrophoneInput alloc] init];
#try {
self.config = [GNConfig init:GRACENOTE_CLIENTID];
}
#catch (NSException * e) {
NSLog(#"%s clientId can't be nil or the empty string",__PRETTY_FUNCTION__);
[self.view setUserInteractionEnabled:FALSE];
return;
}
// Debug is disabled in the GUI by default
#ifdef DEBUG
[self.config setProperty:#"debugEnabled" value:#"1"];
#else
[self.config setProperty:#"debugEnabled" value:#"0"];
#endif
[self.config setProperty:#"lookupmodelocalonly" value:#"0"];
}
-(void)viewDidAppear:(BOOL)animated{
[self performSelectorInBackground:#selector(startRecordMicrophone) withObject:nil];
}
-(void) startRecordMicrophone{
#ifdef DEBUG
NSLog(#"%s startRecording",__PRETTY_FUNCTION__);
#endif
[self.recorder startRecording];
[self performSelectorOnMainThread:#selector(makeMyProgressBarMoving) withObject:nil waitUntilDone:NO];
}
-(void) stopRecordMicrophone{
#ifdef DEBUG
NSLog(#"%s stopRecording",__PRETTY_FUNCTION__);
#endif
[self.recorder stopRecording];
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = paths[0];
NSString *filePath =[documentsDirectory stringByAppendingPathComponent:#"output.caf"];
NSData* sampleData = [[NSData alloc] initWithContentsOfFile:filePath];
GNSampleBuffer *sampleBuffer = [GNSampleBuffer gNSampleBuffer:sampleData numChannels:1 sampleRate:8000];
[GNOperations recognizeMIDStreamFromPcm:self config:self.config sampleBuffer:sampleBuffer];
}
#pragma mark - UI methods
-(void)makeMyProgressBarMoving {
float actual = [self.progressBar progress];
if (actual < 1) {
[self.loadingAnimationView showNextLevel];
self.progressBar.progress = actual + 0.0125;
[NSTimer scheduledTimerWithTimeInterval:0.25f target:self selector:#selector(makeMyProgressBarMoving) userInfo:nil repeats:NO];
}
else{
self.progressBar.hidden = YES;
[self stopRecordMicrophone];
}
}
#pragma mark - GNSearchResultReady methods
- (void) GNResultReady:(GNSearchResult*)result{
NSLog(#"%s",__PRETTY_FUNCTION__);
}
#end
Credits go to Brian Whitman and Echo Nest Library for the MicrophoneInput solution.
Hope it helps someone out who is facing the same situation.
Cheers

Answer 2

The Gracenote SDK does provide access to the audio data even when using the included GnMic class. The GnMic class defines a GnMicDelegate protocol, which you can use to get notified when a new audio buffer is available. You must:
GnViewController.h
Add the GnMicDelegate protocol to your class definition
#interface GnViewController : UIViewController<CLLocationManagerDelegate, UITableViewDataSource, UITableViewDelegate, UINavigationBarDelegate, UIActionSheetDelegate, GnMicDelegate>
GnViewController.m
Assign your class as the delegate for the GnMic instance
self.gnMic = [[GnMic alloc] initWithSampleRate: 44100 bitsPerChannel: 16 numberOfChannels: 1 delegate:nil];
self.gnMic.gnMicDelegate = self;
Implement the protocol method. This will get called each time a new audio buffer is available for processing
- (void) audioBufferDidBecomeReady:(NSData *)samples {
// Do something with the audio samples
}