Recording Data from Callbacks using ExtAudioFileWrite - ios

I'm currently recording stereo audio from the microphone of the iPhone and I have to record the data from the callbacks for analysis.
Currently my AudioStreamBasicDescription format is
AudioStreamBasicDescription format;
format.mSampleRate = 0;
format.mFormatID = kAudioFormatLinearPCM;
format.mFormatFlags = kAudioFormatFlagIsFloat | kAudioFormatFlagsNativeEndian | kAudioFormatFlagIsPacked | kAudioFormatFlagIsNonInterleaved;
format.mFramesPerPacket = 1;
format.mChannelsPerFrame = 2;
format.mBitsPerChannel = 32;
format.mBytesPerPacket = 4;
format.mBytesPerFrame = 4;
and the buffer list I render data into is
inputBufferList->mNumberBuffers = NUMCHANNELS;
for (size_t n = 0; n < NUMCHANNELS; n++) {
inputBufferList->mBuffers[n].mDataByteSize = inNumberFrames * sizeof(float);
inputBufferList->mBuffers[n].mNumberChannels = 1;
inputBufferList->mBuffers[n].mData = malloc(inputBufferList->mBuffers[n].mDataByteSize);
}
When I try to write this data into the ExtAudioFileWrite, it gives an error and it was said that the format is wrong. Is there any tutorial on how to write stereo audio using ExtAudioFileWrite?
Edit:
Here is how I'm setting it up
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString* destinationFilePath = [[NSString alloc] initWithFormat: #"%#/testrecording.wav", documentsDirectory];
CFURLRef destinationURL = CFURLCreateWithFileSystemPath(kCFAllocatorDefault, (CFStringRef)destinationFilePath, kCFURLPOSIXPathStyle, false);
OSStatus status;
ExtAudioFileRef cfref;
status = ExtAudioFileCreateWithURL(destinationURL, kAudioFileWAVEType,
&format, NULL, kAudioFileFlags_EraseFile,
&cfref);
The status shows an exception in this

1718449215 is kAudioConverterErr_FormatNotSupported ('fmt?'), so I'm guessing that WAVE might not support float LPCM. You could try changing to kAudioFormatFlagIsSignedInteger or switching file format, e.g. kAudioFileM4AType, kAudioFileCAFType, or (maybe?) kAudioFileAIFFType.
Don't forget to update format sizes for the former, and filename extension for the latter.

Related

Play aac audio from RTMP stream ios

I'm trying to play audio I'm receiving from an RTMP stream (I have managed to play the video part). The audio comes in .aac format. I have the NSData coming. Then I'm putting it into a CMAudiSampleBuffer and enqueing it into a AVSampleBufferAudioRenderer. (Basically I'm doing the same thing that I have done for the video packets).
Everything is going fine except that I get no sound. Now I'm pretty new to objective-c and iOS programming so the issue ight come from somewhere else, all ideas are welcome.
Here is the code I use to make the format description
-(void)createFormatDescription:(NSData*)payload
{
OSStatus status;
NSData* data = [NSData dataWithData:[payload subdataWithRange:NSMakeRange(2, [payload length]-2)]];
const uint8_t* bytesBuffer = [data bytes];
_type = bytesBuffer[0]>>3;
_frequency = [self getSampleRate:(bytesBuffer[0] & 0b00000111) << 1 | (bytesBuffer[1] >> 7)];
_channel = (bytesBuffer[1] & 0b01111000) >> 3;
AudioStreamBasicDescription audioFormat;
audioFormat.mFormatID = kAudioFormatMPEG4AAC;
audioFormat.mSampleRate = _frequency;
audioFormat.mFormatFlags = _type;
audioFormat.mBytesPerPacket = 0;
audioFormat.mFramesPerPacket = 1024;
audioFormat.mBytesPerFrame = 0;
audioFormat.mChannelsPerFrame = _channel;
audioFormat.mBitsPerChannel = 0;
audioFormat.mReserved = 0;
status = CMAudioFormatDescriptionCreate(kCFAllocatorDefault, &audioFormat, 0, nil, 0, nil, nil, &_formatDesc);
}
Here is the code that I use the add the adts data in front of the packets and create the buffers :
- (NSData*) adts:(int)length
{
int size = 7;
int fullSize =length + size;
uint8_t adts[size];
adts[0] = 0xFF;
adts[1] = 0xF9;
adts[2] = (_type - 1) << 6 | (_frequency << 2) | (_channel >> 2);
adts[3] = (_channel & 3) << 6 | (fullSize >> 11);
adts[4] = (fullSize & 0x7FF) >> 3;
adts[5] = ((fullSize & 7) << 5) + 0x1F;
adts[6] = 0xFC;
NSData* result = [NSData dataWithBytes:adts length:size];
return result;
}
-(void)enqueueBuffer:(RTMPMessage*)message {
OSStatus status;
NSData* payloadData = [NSData dataWithData:[message.payloadData
subdataWithRange:NSMakeRange(2, [message.payloadData length]-2)]];
NSData* adts = [NSData dataWithData:[self adts:(int)[payloadData length]]];
NSMutableData* data = [NSMutableData dataWithData:adts];
[data appendData:payloadData];
uint8_t* bytesBuffer[[data length]];
[data getBytes:bytesBuffer length:[data length]];
const size_t sampleSize = [data length];
AudioStreamPacketDescription packetDescription;
packetDescription.mDataByteSize = (int)sampleSize;
packetDescription.mStartOffset = 0;
packetDescription.mVariableFramesInPacket = 0;
CMBlockBufferRef blockBuffer = NULL;
CMSampleBufferRef sampleBuffer = NULL;
CMTime time = CMTimeMake(5, _frequency);
status = CMBlockBufferCreateWithMemoryBlock(NULL, bytesBuffer, [data length], kCFAllocatorNull, NULL, 0, [data length], 0, &blockBuffer);
status = CMAudioSampleBufferCreateWithPacketDescriptions(kCFAllocatorDefault, blockBuffer, true, NULL, NULL, _formatDesc, 1, time, &packetDescription, &sampleBuffer);
CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, YES);
CFMutableDictionaryRef dict = (CFMutableDictionaryRef)CFArrayGetValueAtIndex(attachments, 0);
CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);
[_audioRenderer enqueueSampleBuffer:sampleBuffer];
}
Thanks in advance for any help
ADTS header is not required. AVAudioSampleRenderer just need naked aac compressed packet for playing. But the precondition is that you set the correct formatDescription, and correct parameters for samplebuffer creation.
You need aware that, HE-AAC(LC+SBR) packed like a AAC-LC, but has 22050 sample rate. HE-V2(LC+SBR+PS) packed like a AAC-LC, but has 22050 sample rate, and one channel per sample.
And all HE-AAC(v1,v2), samplesPerFrame always 2048, not like LC's 1024.
That's all I know how to play aac stream with AVAudioSampleRenderer correctly. It's a long way succeed..

Is there a way to read .wav files as float/int arrays in swift/objective-c and after processing the samples, output it to a speaker or store as pcm?

I'm trying to build an iOS app and I need to read a .wav file or microphone input as a float or int array to feed to already existing audio signal processing algorithms in C. Is there an easy way like how wavread is in Matlab?
void readAudio() {
NSString * name = #"Test";
NSString * source = [[NSBundle mainBundle] pathForResource:name ofType:#"caf"];
const char * cString = [source cStringUsingEncoding:NSASCIIStringEncoding];
CFStringRef str = CFStringCreateWithCString(NULL, cString, kCFStringEncodingMacRoman);
CFURLRef inputFileURL = CFURLCreateWithFileSystemPath(kCFAllocatorDefault, str, kCFURLPOSIXPathStyle, false);
AudioFileID fileID;
OSStatus err = AudioFileOpenURL(inputFileURL, kAudioFileReadPermission, 0, &fileID);
CheckError(err, "AudioFileOpenURL");
ExtAudioFileRef fileRef;
err = ExtAudioFileOpenURL(inputFileURL, &fileRef);
CheckError(err, "ExtAudioFileOpenURL");
AudioStreamBasicDescription clientFormat;
memset(&clientFormat, 0, sizeof(clientFormat));
clientFormat.mFormatID = kAudioFormatLinearPCM;
clientFormat.mFramesPerPacket = 1;
clientFormat.mChannelsPerFrame = 1;
clientFormat.mBitsPerChannel = 16;
clientFormat.mBytesPerPacket = clientFormat.mChannelsPerFrame * sizeof(SInt16);
clientFormat.mBytesPerFrame = clientFormat.mChannelsPerFrame * sizeof(SInt16);
clientFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
clientFormat.mSampleRate = 8000;
err = ExtAudioFileSetProperty(fileRef, kExtAudioFileProperty_ClientDataFormat, sizeof(AudioStreamBasicDescription), &clientFormat);
CheckError(err, "ExtAudioFileSetProperty");
int numSamples = 64;
UInt32 sizePerPacket = clientFormat.mBytesPerPacket;
UInt32 packetsPerBuffer = numSamples;
UInt32 outputBufferSize = packetsPerBuffer * sizePerPacket;
UInt8 *outputBuffer = (UInt8 *)malloc(sizeof(UInt8 *) * outputBufferSize);
AudioBufferList convertedData;
convertedData.mNumberBuffers = 1;
convertedData.mBuffers[0].mNumberChannels = clientFormat.mChannelsPerFrame;
convertedData.mBuffers[0].mDataByteSize = outputBufferSize;
convertedData.mBuffers[0].mData = outputBuffer;
UInt32 frameCount = numSamples;
short *samplesAsCArray, *output = (short *)malloc(sizeof(UInt8 *) * numSamples);
while (frameCount > 0) {
err = ExtAudioFileRead(fileRef, &frameCount, &convertedData);
if(frameCount > 0) {
uint64_t startTime = mach_absolute_time() ;
AudioBuffer audioBuffer = convertedData.mBuffers[0];
samplesAsCArray = (short *)audioBuffer.mData;
FIRFilter(samplesAsCArray, audioBuffer.mDataByteSize/(sizeof(short)), output);
memcpy([iosAudio tempBuffer].mData, output, audioBuffer.mDataByteSize);
uint64_t duration = mach_absolute_time() - startTime;
NSLog(#"%f milliseconds", (float)duration/1e6);
/*for (int i =0; i< frameCount; i++) {
printf("%d\n", output[i]);
}*/
}
}
free(output);
}
I've implemented this much until now. I need to be able to save the edited file as a linear PCM (CAF, as apple works better with that), as text and be able to play the edited audio in real time also.

Recording from RemoteIO: resulting .caf is pitch shifted slower + distorted

So I've cobbled together some routines for recording audio based on some posts here. The posts I've referenced are here and here, along with reading the sites they reference.
My setup: I have an existing AUGraph: (several AUSamplers -> Mixer -> RemoteIO). The AUSamplers are connected to tracks in a MusicPlayer instance. That all works fine but
I want to add recording to it.
Recording is working but the resulting .caf is pitch/tempo shifted slower + has bad sound quality. Must be something wrong with the format I am specifying?
Can someone eyeball this and tell me where I am setting the format incorrectly?
EDIT: could this be a stereo/mono issue? I mean to recording in mono.
I set the stream format on the RemoteIO instance to this:
AudioStreamBasicDescription audioFormat;
audioFormat.mSampleRate = 44100.00;
audioFormat.mFormatID = kAudioFormatLinearPCM;
audioFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
audioFormat.mFramesPerPacket = 1;
audioFormat.mChannelsPerFrame = 1;
audioFormat.mBitsPerChannel = 16;
audioFormat.mBytesPerPacket = 2;
audioFormat.mBytesPerFrame = 2;
// Apply format
result = AudioUnitSetProperty(ioUnit,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Output,
kInputBus,
&audioFormat,
sizeof(audioFormat));
Then from a button action I create a fileRef and attach a renderCallback to the RemoteIO instance:
- (void)startRecording
{
OSStatus result;
AudioStreamBasicDescription audioFormat;
audioFormat.mSampleRate = 44100.00;
audioFormat.mFormatID = kAudioFormatLinearPCM;
audioFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
audioFormat.mFramesPerPacket = 1;
audioFormat.mChannelsPerFrame = 1;
audioFormat.mBitsPerChannel = 16;
audioFormat.mBytesPerPacket = 2;
audioFormat.mBytesPerFrame = 2;
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *destinationFilePath = [[NSString alloc] initWithFormat: #"%#/output.caf", documentsDirectory];
CFURLRef destinationURL = CFURLCreateWithFileSystemPath(kCFAllocatorDefault,
(__bridge CFStringRef)destinationFilePath,
kCFURLPOSIXPathStyle,
false);
result = ExtAudioFileCreateWithURL(destinationURL,
kAudioFileWAVEType,
&audioFormat,
NULL,
kAudioFileFlags_EraseFile,
&extAudioFileRef);
CFRelease(destinationURL);
NSAssert(result == noErr, #"Couldn't create file for writing");
result = ExtAudioFileSetProperty(extAudioFileRef,
kExtAudioFileProperty_ClientDataFormat,
sizeof(AudioStreamBasicDescription),
&audioFormat);
NSAssert(result == noErr, #"Couldn't create file for format");
result = ExtAudioFileWriteAsync(extAudioFileRef, 0, NULL);
NSAssert(result == noErr, #"Couldn't initialize write buffers for audio file");
printf("Adding render to remoteIO \n");
result = AudioUnitAddRenderNotify(ioUnit, renderCallback, (__bridge void*)self);
if (result) {[self printErrorMessage: #"AudioUnitAddRenderNotify" withStatus: result]; return;}
}
Finally in my rendercallback I write out the data in the postRender phase:
static OSStatus renderCallback (void * inRefCon,
AudioUnitRenderActionFlags * ioActionFlags,
const AudioTimeStamp * inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList * ioData)
{
OSStatus result;
if (*ioActionFlags == kAudioUnitRenderAction_PostRender){
double timeInSeconds = inTimeStamp->mSampleTime / kSampleRate;
printf("%fs inBusNumber: %lu inNumberFrames: %lu \n", timeInSeconds, inBusNumber, inNumberFrames);
MusicPlayerController* THIS = (__bridge MusicPlayerController *)inRefCon;
result = ExtAudioFileWriteAsync(THIS->extAudioFileRef, inNumberFrames, ioData);
if(result) printf("ExtAudioFileWriteAsync %ld \n", result);
}
return noErr;
}
Ok - found some code that solves this - though I don't fully understand why.
I had been setting the mBitsPerChannel to 16 for both the RemoteIO output stream and the ExtFileRef. The result was slowed down & scratchy audio. Setting the ExtFileRef mBitsPerChannel to 32 plus adding the kAudioFormatFlagsNativeEndian flag solves the problem: the .caf audio is perfect (while leaving the RemoteIO output stream settings to what they were).
But then also setting the RemoteIO output stream settings to match my new settings also works. So I'm confused. Shouldn't this work so long as the AudioStreamBasicDescription settings are symmetrical for the RemoteIO instance and the ExtFileRef?
Anyway... the working setting is below.
size_t bytesPerSample = sizeof (AudioUnitSampleType);
AudioStreamBasicDescription audioFormat;
audioFormat.mSampleRate= graphSampleRate;
audioFormat.mFormatID=kAudioFormatLinearPCM;
audioFormat.mFormatFlags=kAudioFormatFlagsNativeEndian|kAudioFormatFlagIsSignedInteger|kAudioFormatFlagIsPacked;
audioFormat.mBytesPerPacket=bytesPerSample;
audioFormat.mBytesPerFrame=bytesPerSample;
audioFormat.mFramesPerPacket=1;
audioFormat.mChannelsPerFrame=1;
audioFormat.mBitsPerChannel= 8 * bytesPerSample;
audioFormat.mReserved=0;

How to trim audio data?

I wanted to remove last 5 second audio data and save it to different location as new audio.I'm trying to get it done by following code using ExtAudioFile service but here my audio output size is increasing from 2.5 MB to 26.5 MB..where am i wrong.
UInt32 size; NSString *docsDir;
NSArray *dirPaths;
dirPaths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
docsDir = [dirPaths objectAtIndex:0];
NSString *destinationURL = [docsDir
stringByAppendingPathComponent:#"Audio3.m4a"];
NSURL * soundFilePath = [NSURL fileURLWithPath:[[NSBundle mainBundle]
pathForResource:#"Audio1"
ofType:#"m4a"]];
ExtAudioFileRef inputFile= NULL;
ExtAudioFileRef outputFile= NULL;
ExtAudioFileOpenURL((CFURLRef)soundFilePath, &inputFile);
AudioStreamBasicDescription destFormat;
destFormat.mFormatID = kAudioFormatMPEG4AAC;
destFormat.mFormatFlags = kAudioFormatFlagsCanonical;
destFormat.mSampleRate = 441000;
destFormat.mBytesPerPacket = 2;
destFormat.mFramesPerPacket = 1;
destFormat.mBytesPerFrame = 2;
destFormat.mChannelsPerFrame = 2;
destFormat.mBitsPerChannel = 16;
destFormat.mReserved = 0;
OSStatus createStatus =ExtAudioFileCreateWithURL((CFURLRef)[NSURL fileURLWithPath:destinationURL],kAudioFileM4AType,&destFormat,NULL,kAudioFileFlags_EraseFile,&outputFile);
//ExtAudioFileDispose(outputFile);
NSLog(#"createStatus: %i", createStatus);
//this is not needed as file url is already opened.
ExtAudioFileOpenURL((CFURLRef)soundFilePath, &inputFile);
//ExtAudioFileOpenURL((CFURLRef)[NSURL fileURLWithPath:destinationURL], &outputFile);
//find out how many frames long this file is 
SInt64 length = 0;
UInt32 dataSize2 = (UInt32)sizeof(length);
ExtAudioFileGetProperty(inputFile, kExtAudioFileProperty_FileLengthFrames, &dataSize2, &length);
AudioStreamBasicDescription clientFormat;
clientFormat.mFormatID = kAudioFormatMPEG4AAC;
clientFormat.mSampleRate = 441000;
clientFormat.mFormatFlags = kAudioFormatFlagsCanonical;
clientFormat.mBitsPerChannel = 16;
clientFormat.mChannelsPerFrame = 2;
clientFormat.mFramesPerPacket = 1;
clientFormat.mBytesPerPacket = 2;
clientFormat.mBytesPerFrame = 2;
destFormat.mReserved = 0;
size = sizeof(clientFormat);
//set the intermediate format to canonical on the source file for conversion (?) 
ExtAudioFileSetProperty(inputFile, kExtAudioFileProperty_ClientDataFormat, size, &clientFormat);
ExtAudioFileSetProperty(outputFile, kExtAudioFileProperty_ClientDataFormat, size, &clientFormat);
OSStatus seekStatus = ExtAudioFileSeek(outputFile, 0);  
NSLog(#"seekstatus %i", seekStatus);
SInt64 newLength = length - (5); //shorten by 5 seconds worth of frames 
NSLog(#"length: %i frames", length);
UInt8 *buffer = malloc(64*1024); //64K 
UInt32 totalFramecount = 0;
while(true) {
AudioBufferList bufferList;
bufferList.mNumberBuffers = 1;
bufferList.mBuffers[0].mNumberChannels = 2;
bufferList.mBuffers[0].mData = buffer; //pointer to buffer of audio data 
bufferList.mBuffers[0].mDataByteSize =64*1024; //number of bytes in the buffer 
UInt32 frameCount = 64*1024 / 2; //2 bytes per frame 
// Read a chunk of input 
SInt64 outFrameOffset;
ExtAudioFileTell(inputFile, &outFrameOffset) ;
NSLog(#"head status %i", outFrameOffset);
OSStatus status = ExtAudioFileRead(inputFile, &frameCount, &bufferList);
totalFramecount += frameCount;
NSLog(#"read status %i", status);
NSLog(#"loaded %i frames and stopping at %i", totalFramecount, newLength);
if (!frameCount ||(totalFramecount >= newLength)) {
//termination condition 
break;
}
OSStatus writeStatus = ExtAudioFileWrite(outputFile, frameCount, &bufferList);
NSLog(#"ws: %i", writeStatus);
}
free(buffer);
ExtAudioFileDispose(inputFile);
ExtAudioFileDispose(outputFile);
You are taking compressed audio (M4A) and uncompressing it, which is what you need to do to trim down the audio content. If you want to get back to the 2.5 MB range, you will need to recompress your audio when done.
Keep in mind that repeated lossy uncompress-edit-recompress cycles will degrade the quality of your audio. If you are going to be performing a lot of audio edit operations, you should transform your audio from compressed to uncompressed, then run your transforms, and finally recompress at the end.

AudioFileWriteBytes is taking way too long to do its job

This problem may be a bit too vast and nebulous for this space, but I'll give it a go.
I have an array of samples that I'm trying to write to a .wav file on my iOS and it is taking up to a minute and a half to do. Here is the loop where the drag is occurring:
for (int i=0; i< 1430529; i++) // 1430529 is the length of the array of samples
{
SInt16 sample;
sample = sample_array[i];
audioErr = AudioFileWriteBytes(audioFile, false, sampleCount*2, &bytesToWrite, &sample);
sampleCount++;
}
Any ideas?
EDIT 1
If it helps, this is the code that precedes it:
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
// THIS IS MY BIT TO CONVERT THE LOCATION TO NSSTRING
NSString *filePath = [[NSString alloc]init];
filePath = [NSString stringWithUTF8String:location];
// HERE I WANT TO REMOVE THE FILE NAME FROM THE LOCATION.
NSString *truncatedFilePath = filePath;
truncatedFilePath = [truncatedFilePath stringByReplacingOccurrencesOfString:#"/recordedFile.wav"
// withString:#"/newFile.caf"];
withString:#"/recordedFile.wav"];
NSLog(truncatedFilePath);
NSURL *fileURL = [NSURL fileURLWithPath:truncatedFilePath];
AudioStreamBasicDescription asbd;
memset(&asbd,0, sizeof(asbd));
asbd.mSampleRate = SAMPLE_RATE;
asbd.mFormatID = kAudioFormatLinearPCM;
asbd.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked;
asbd.mBitsPerChannel = 16;
asbd.mChannelsPerFrame = 1;
asbd.mFramesPerPacket = 1;
asbd.mBytesPerFrame = 2;
asbd.mBytesPerPacket = 2;
AudioFileID audioFile;
OSStatus audioErr = noErr;
audioErr = AudioFileCreateWithURL((CFURLRef)fileURL, kAudioFileWAVEType, &asbd, kAudioFileFlags_EraseFile, &audioFile);
assert (audioErr == noErr);
long sampleCount = 0;
UInt32 bytesToWrite = 2;
Why do you need the loop ? Can't you write all the samples in one go, e.g.
numSamples = 1430529;
bytesToWrite = numSamples * 2;
audioErr = AudioFileWriteBytes(audioFile, false, 0, &bytesToWrite, sample_array);
?
Perhaps the number of bytes you are writing at each call to AudioFileWriteBytes is too small. How large is bytesToWrite?

Resources