I'm working on an audio recorder that renders the waveform of each recording. Whenever a recording is made, the NSURL of the audio file is converted to an AVAsset. With the AVAsset I'm able to extract the samples of the audio track. This works fine for audio recordings that are short ( <40seconds), but the process takes 15-20 seconds on a 2.5 min track and only gets worse the longer the track is. Anyone have any tips or recommendations on how to get around this problem?
AVAssetReader *reader = [[AVAssetReader alloc] initWithAsset:self.audioAsset error:&error];
AVAssetReaderTrackOutput *output = [[AVAssetReaderTrackOutput alloc] initWithTrack:songTrack outputSettings:outputSettingsDict];
[reader addOutput:output];
NSMutableData * data = [NSMutableData dataWithLength:32768];
NSMutableArray *allSamples = [NSMutableArray array];
while (reader.status == AVAssetReaderStatusReading) {
CMSampleBufferRef sampleBufferRef = [output copyNextSampleBuffer];
if (sampleBufferRef) {
CMBlockBufferRef blockBufferRef = CMSampleBufferGetDataBuffer(sampleBufferRef);
size_t bufferLength = CMBlockBufferGetDataLength(blockBufferRef);
if (data.length < bufferLength) {
[data setLength:bufferLength];
}
CMBlockBufferCopyDataBytes(blockBufferRef, 0, bufferLength, data.mutableBytes);
Float32 *samples = (Float32 *)data.mutableBytes;
int sampleCount = (int)(bufferLength / bytesPerInputSample);
for (int i = 0; i < sampleCount; i++) {
[allSamples addObject:#(samples[i*channelCount])];
}
CMSampleBufferInvalidate(sampleBufferRef);
CFRelease(sampleBufferRef);
}
}
Related
I have an array of floats that is raw audio data from a 3rd party source. I would like to pass this through to a Speech Recognition request via appendAudioPCMBuffer but that accepts an AVAudioPCMBuffer. How could I convert my NSMutableArray to AVAudioPCMBuffer?
For reference, this is how the buffer variable gets created before its passed to this function. It is written in C.
void CallNativePlugin( const float buffer[], int size ) {
NSMutableArray *myArray = [[NSMutableArray alloc] init];
for (int i = 0; i < size; i++) {
NSNumber *number = [[NSNumber alloc] initWithFloat:buffer[i]];
[myArray addObject:number];
[delegateObject recognizeSpeechFromBuffer:myArray ];
}
}
Then the current code I have to take that buffer and pass is to the speech recognizer (objective-c):
-(void) recognizeSpeechFromBuffer: (NSMutableArray*) buffer {
NSLog( #"Array length: %lu#", (unsigned long) buffer.count );
recognitionRequest = [[SFSpeechAudioBufferRecognitionRequest alloc] init];
recognitionRequest.shouldReportPartialResults = YES;
recognitionTask = [speechRecognizer recognitionTaskWithRequest:recognitionRequest.resultHandler:^(SFSpeechRecognitionResult * _Nullable result, NSError * _Nullable error) {
BOOL isFinal = NO;
if (result) {
NSLog(#"RESULT:%#",result.bestTranscription.formattedString);
isFinal = !result.isFinal;
}
if (error) {
recognitionRequest = nil;
recognitionTask = nil;
}
}];
// Do something like [recognitionRequest appendAudioPCMBuffer:buffer];
}
Since Objective-C is available in CallNativePlugin, you could rework it to create the AVAudioPCMBuffer there, assuming you know the audio sample rate and can adjust the delegate protocol.
// Assuming mono, sample rate ????
void CallNativePlugin( const float buffer[], int size ) {
AVAudioFormat *fmt = [[AVAudioFormat alloc] initWithCommonFormat:AVAudioPCMFormatFloat32 sampleRate:/* ?? */ channels:1 interleaved:YES];
AVAudioPCMBuffer *buf = [[AVAudioPCMBuffer alloc] initWithPCMFormat:fmt frameCapacity:size];
memcpy(buf.floatChannelData[0], buffer, sizeof(float) * size);
buf.frameLength = size;
[delegateObject recognizeSpeechFromPCMBuffer:buf];
}
If that isn't possible you can create an AVAudioPCMBuffer in -recognizeSpeechFromPCMBuffer: similarly and assign the floats individually.
I'm trying to play audio I'm receiving from an RTMP stream (I have managed to play the video part). The audio comes in .aac format. I have the NSData coming. Then I'm putting it into a CMAudiSampleBuffer and enqueing it into a AVSampleBufferAudioRenderer. (Basically I'm doing the same thing that I have done for the video packets).
Everything is going fine except that I get no sound. Now I'm pretty new to objective-c and iOS programming so the issue ight come from somewhere else, all ideas are welcome.
Here is the code I use to make the format description
-(void)createFormatDescription:(NSData*)payload
{
OSStatus status;
NSData* data = [NSData dataWithData:[payload subdataWithRange:NSMakeRange(2, [payload length]-2)]];
const uint8_t* bytesBuffer = [data bytes];
_type = bytesBuffer[0]>>3;
_frequency = [self getSampleRate:(bytesBuffer[0] & 0b00000111) << 1 | (bytesBuffer[1] >> 7)];
_channel = (bytesBuffer[1] & 0b01111000) >> 3;
AudioStreamBasicDescription audioFormat;
audioFormat.mFormatID = kAudioFormatMPEG4AAC;
audioFormat.mSampleRate = _frequency;
audioFormat.mFormatFlags = _type;
audioFormat.mBytesPerPacket = 0;
audioFormat.mFramesPerPacket = 1024;
audioFormat.mBytesPerFrame = 0;
audioFormat.mChannelsPerFrame = _channel;
audioFormat.mBitsPerChannel = 0;
audioFormat.mReserved = 0;
status = CMAudioFormatDescriptionCreate(kCFAllocatorDefault, &audioFormat, 0, nil, 0, nil, nil, &_formatDesc);
}
Here is the code that I use the add the adts data in front of the packets and create the buffers :
- (NSData*) adts:(int)length
{
int size = 7;
int fullSize =length + size;
uint8_t adts[size];
adts[0] = 0xFF;
adts[1] = 0xF9;
adts[2] = (_type - 1) << 6 | (_frequency << 2) | (_channel >> 2);
adts[3] = (_channel & 3) << 6 | (fullSize >> 11);
adts[4] = (fullSize & 0x7FF) >> 3;
adts[5] = ((fullSize & 7) << 5) + 0x1F;
adts[6] = 0xFC;
NSData* result = [NSData dataWithBytes:adts length:size];
return result;
}
-(void)enqueueBuffer:(RTMPMessage*)message {
OSStatus status;
NSData* payloadData = [NSData dataWithData:[message.payloadData
subdataWithRange:NSMakeRange(2, [message.payloadData length]-2)]];
NSData* adts = [NSData dataWithData:[self adts:(int)[payloadData length]]];
NSMutableData* data = [NSMutableData dataWithData:adts];
[data appendData:payloadData];
uint8_t* bytesBuffer[[data length]];
[data getBytes:bytesBuffer length:[data length]];
const size_t sampleSize = [data length];
AudioStreamPacketDescription packetDescription;
packetDescription.mDataByteSize = (int)sampleSize;
packetDescription.mStartOffset = 0;
packetDescription.mVariableFramesInPacket = 0;
CMBlockBufferRef blockBuffer = NULL;
CMSampleBufferRef sampleBuffer = NULL;
CMTime time = CMTimeMake(5, _frequency);
status = CMBlockBufferCreateWithMemoryBlock(NULL, bytesBuffer, [data length], kCFAllocatorNull, NULL, 0, [data length], 0, &blockBuffer);
status = CMAudioSampleBufferCreateWithPacketDescriptions(kCFAllocatorDefault, blockBuffer, true, NULL, NULL, _formatDesc, 1, time, &packetDescription, &sampleBuffer);
CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, YES);
CFMutableDictionaryRef dict = (CFMutableDictionaryRef)CFArrayGetValueAtIndex(attachments, 0);
CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);
[_audioRenderer enqueueSampleBuffer:sampleBuffer];
}
Thanks in advance for any help
ADTS header is not required. AVAudioSampleRenderer just need naked aac compressed packet for playing. But the precondition is that you set the correct formatDescription, and correct parameters for samplebuffer creation.
You need aware that, HE-AAC(LC+SBR) packed like a AAC-LC, but has 22050 sample rate. HE-V2(LC+SBR+PS) packed like a AAC-LC, but has 22050 sample rate, and one channel per sample.
And all HE-AAC(v1,v2), samplesPerFrame always 2048, not like LC's 1024.
That's all I know how to play aac stream with AVAudioSampleRenderer correctly. It's a long way succeed..
I'm trying to reverse an AVAsset audio and save it to a file. To make things clear, I've made simple application with the issue https://github.com/ksenia-lyagusha/AudioReverse.git
The application takes mp4 video file from bundle, exports it to Temporary folder in the sandbox as single m4a file, then tries to read it from there, reverse and save result file back.
Temporary m4a file is OK.
The only result of my reverse part is Audio file in the Sandbox with white noise.
There is the part of code below, that is in charge of reversing AVAsset. It is based on related questions
How to reverse an audio file?
iOS audio manipulation - play local .caf file backwards
However, it doesn't work for me.
OSStatus theErr = noErr;
UInt64 fileDataSize = 0;
AudioFileID inputAudioFile;
AudioStreamBasicDescription theFileFormat;
UInt32 thePropertySize = sizeof(theFileFormat);
theErr = AudioFileOpenURL((__bridge CFURLRef)[NSURL URLWithString:inputPath], kAudioFileReadPermission, 0, &inputAudioFile);
thePropertySize = sizeof(fileDataSize);
theErr = AudioFileGetProperty(inputAudioFile, kAudioFilePropertyAudioDataByteCount, &thePropertySize, &fileDataSize);
UInt32 ps = sizeof(AudioStreamBasicDescription) ;
AudioFileGetProperty(inputAudioFile, kAudioFilePropertyDataFormat, &ps, &theFileFormat);
UInt64 dataSize = fileDataSize;
void *theData = malloc(dataSize);
// set up output file
AudioFileID outputAudioFile;
AudioStreamBasicDescription myPCMFormat;
myPCMFormat.mSampleRate = 44100;
myPCMFormat.mFormatID = kAudioFormatLinearPCM;
// kAudioFormatFlagsCanonical is deprecated
myPCMFormat.mFormatFlags = kAudioFormatFlagIsFloat | kAudioFormatFlagIsNonInterleaved;
myPCMFormat.mChannelsPerFrame = 1;
myPCMFormat.mFramesPerPacket = 1;
myPCMFormat.mBitsPerChannel = 32;
myPCMFormat.mBytesPerPacket = (myPCMFormat.mBitsPerChannel / 8) * myPCMFormat.mChannelsPerFrame;
myPCMFormat.mBytesPerFrame = myPCMFormat.mBytesPerPacket;
NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:#"ReverseAudio.caf"];
NSURL *outputURL = [NSURL fileURLWithPath:exportPath];
theErr = AudioFileCreateWithURL((__bridge CFURLRef)outputURL,
kAudioFileCAFType,
&myPCMFormat,
kAudioFileFlags_EraseFile,
&outputAudioFile);
//Read data into buffer
//if readPoint = dataSize, then bytesToRead = 0 in while loop and
//it is endless
SInt64 readPoint = dataSize-1;
UInt64 writePoint = 0;
while(readPoint > 0)
{
UInt32 bytesToRead = 2;
AudioFileReadBytes(inputAudioFile, false, readPoint, &bytesToRead, theData);
// bytesToRead is now the amount of data actually read
UInt32 bytesToWrite = bytesToRead;
AudioFileWriteBytes(outputAudioFile, false, writePoint, &bytesToWrite, theData);
// bytesToWrite is now the amount of data actually written
writePoint += bytesToWrite;
readPoint -= bytesToRead;
}
free(theData);
AudioFileClose(inputAudioFile);
AudioFileClose(outputAudioFile);
If I change file type in AudioFileCreateWithURL from kAudioFileCAFType to another the result file is not created in the Sandbox at all.
Thanks for any help.
You get white noise because your in and out file formats are incompatible. You have different sample rates and channels and probably other differences. To make this work you need to have a common (PCM) format mediating between reads and writes. This is a reasonable job for the new(ish) AVAudio frameworks. We read from file to PCM, shuffle the buffers, then write from PCM to file. This approach is not optimised for large files, as all data is read into the buffers in one go, but is enough to get you started.
You can call this method from your getAudioFromVideo completion block. Error handling ignored for clarity.
- (void)readAudioFromURL:(NSURL*)inURL reverseToURL:(NSURL*)outURL {
//prepare the in and outfiles
AVAudioFile* inFile =
[[AVAudioFile alloc] initForReading:inURL error:nil];
AVAudioFormat* format = inFile.processingFormat;
AVAudioFrameCount frameCount =(UInt32)inFile.length;
NSDictionary* outSettings = #{
AVNumberOfChannelsKey:#(format.channelCount)
,AVSampleRateKey:#(format.sampleRate)};
AVAudioFile* outFile =
[[AVAudioFile alloc] initForWriting:outURL
settings:outSettings
error:nil];
//prepare the forward and reverse buffers
self.forwaredBuffer =
[[AVAudioPCMBuffer alloc] initWithPCMFormat:format
frameCapacity:frameCount];
self.reverseBuffer =
[[AVAudioPCMBuffer alloc] initWithPCMFormat:format
frameCapacity:frameCount];
//read file into forwardBuffer
[inFile readIntoBuffer:self.forwaredBuffer error:&error];
//set frameLength of reverseBuffer to forwardBuffer framelength
AVAudioFrameCount frameLength = self.forwaredBuffer.frameLength;
self.reverseBuffer.frameLength = frameLength;
//iterate over channels
//stride is 1 or 2 depending on interleave format
NSInteger stride = self.forwaredBuffer.stride;
for (AVAudioChannelCount channelIdx = 0;
channelIdx < self.forwaredBuffer.format.channelCount;
channelIdx++) {
float* forwaredChannelData =
self.forwaredBuffer.floatChannelData[channelIdx];
float* reverseChannelData =
self.reverseBuffer.floatChannelData[channelIdx];
int32_t reverseIdx = 0;
//iterate over samples, allocate to reverseBuffer in reverse order
for (AVAudioFrameCount frameIdx = frameLength;
frameIdx >0;
frameIdx--) {
float sample = forwaredChannelData[frameIdx*stride];
reverseChannelData[reverseIdx*stride] = sample;
reverseIdx++;
}
}
//write reverseBuffer to outFile
[outFile writeFromBuffer:self.reverseBuffer error:nil];
}
I wasn't able to find the problem in your code, however I suggest you reversing AVAsset using AVAssetWriter. Following code is based on iOS reverse audio through AVAssetWritet. I've added additional method there to make it work. Finally I've got reversed file.
static NSMutableArray *samples;
static OSStatus sampler(CMSampleBufferRef sampleBuffer, CMItemCount index, void *refcon)
{
[samples addObject:(__bridge id _Nonnull)(sampleBuffer)];
return noErr;
}
- (void)reversePlayAudio:(NSURL *)inputURL
{
AVAsset *asset = [AVAsset assetWithURL:inputURL];
AVAssetReader* reader = [[AVAssetReader alloc] initWithAsset:asset error:nil];
AVAssetTrack* audioTrack = [[asset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
NSMutableDictionary* audioReadSettings = [NSMutableDictionary dictionary];
[audioReadSettings setValue:[NSNumber numberWithInt:kAudioFormatLinearPCM]
forKey:AVFormatIDKey];
AVAssetReaderTrackOutput* readerOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:audioTrack outputSettings:audioReadSettings];
[reader addOutput:readerOutput];
[reader startReading];
NSDictionary *outputSettings = #{AVFormatIDKey : #(kAudioFormatMPEG4AAC),
AVSampleRateKey : #(44100.0),
AVNumberOfChannelsKey : #(1),
AVEncoderBitRateKey : #(128000),
AVChannelLayoutKey : [NSData data]};
AVAssetWriterInput *writerInput = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeAudio
outputSettings:outputSettings];
NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:#"reverseAudio.m4a"];
NSURL *exportURL = [NSURL fileURLWithPath:exportPath];
NSError *writerError = nil;
AVAssetWriter *writer = [[AVAssetWriter alloc] initWithURL:exportURL
fileType:AVFileTypeAppleM4A
error:&writerError];
[writerInput setExpectsMediaDataInRealTime:NO];
writer.shouldOptimizeForNetworkUse = NO;
[writer addInput:writerInput];
[writer startWriting];
[writer startSessionAtSourceTime:kCMTimeZero];
CMSampleBufferRef sample;// = [readerOutput copyNextSampleBuffer];
samples = [[NSMutableArray alloc] init];
while (sample != NULL) {
sample = [readerOutput copyNextSampleBuffer];
if (sample == NULL)
continue;
CMSampleBufferCallForEachSample(sample, &sampler, NULL);
CFRelease(sample);
}
NSArray* reversedSamples = [[samples reverseObjectEnumerator] allObjects];
for (id reversedSample in reversedSamples) {
if (writerInput.readyForMoreMediaData) {
[writerInput appendSampleBuffer:(__bridge CMSampleBufferRef)(reversedSample)];
}
else {
[NSThread sleepForTimeInterval:0.05];
}
}
[samples removeAllObjects];
[writerInput markAsFinished];
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0);
dispatch_async(queue, ^{
[writer finishWritingWithCompletionHandler:^{
// writing is finished
// reversed audio file in TemporaryDirectory in the Sandbox
}];
});
}
Known issues of the code.
There might be some problems with the memory, if the audio is long.
The audio file's duration is longer than original's. (As a quick fix you might cut it down as usual AVAsset).
I am trying to grab the audio track from a video as raw pcm data. The plan is pass this data into a float* table array in libpd. I have managed to get some sample data but the number of samples reported is way too low. For example for a 29sec clip I am getting a reported 3968 samples. What I am looking forward is the amplitudes at each sample. For a track of audio length of 29 secs I would expect to have an array of 1278900 in size (at 44.1khz).
Here is what I have put together based on other examples:
- (void)audioCapture {
NSLog(#"capturing");
NSError *error;
AVAssetReader* reader = [[AVAssetReader alloc] initWithAsset:self.videoAsset error:&error];
NSArray *audioTracks = [self.videoAsset tracksWithMediaType:AVMediaTypeAudio];
AVAssetTrack *audioTrack = nil;
if ([audioTracks count] > 0)
audioTrack = [audioTracks objectAtIndex:0];
// Decompress to Linear PCM with the asset reader
NSDictionary *decompressionAudioSettings = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithUnsignedInt:kAudioFormatLinearPCM], AVFormatIDKey,
nil];
AVAssetReaderOutput *output = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:audioTrack outputSettings:decompressionAudioSettings];
[reader addOutput:output];
[reader startReading];
CMSampleBufferRef sample = [output copyNextSampleBuffer];
//array for our sample amp values
SInt16* samples = NULL;
//sample count;
CMItemCount numSamplesInBuffer = 0;
//sample buffer
CMBlockBufferRef buffer;
while( sample != NULL )
{
sample = [output copyNextSampleBuffer];
if( sample == NULL )
continue;
buffer = CMSampleBufferGetDataBuffer( sample );
size_t lengthAtOffset;
size_t totalLength;
char* data;
if( CMBlockBufferGetDataPointer( buffer, 0, &lengthAtOffset, &totalLength, &data ) != noErr )
{
NSLog( #"error!" );
break;
}
numSamplesInBuffer = CMSampleBufferGetNumSamples(sample);
AudioBufferList audioBufferList;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
sample,
NULL,
&audioBufferList,
sizeof(audioBufferList),
NULL,
NULL,
kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
&buffer
);
for (int bufferCount=0; bufferCount < audioBufferList.mNumberBuffers; bufferCount++) {
samples = (SInt16 *)audioBufferList.mBuffers[bufferCount].mData;
NSLog(#"idx %f",(double)samples[bufferCount]);
}
}
NSLog(#"num samps in buf %ld",numSamplesInBuffer);
//[PdBase copyArray:(float *)samples
//toArrayNamed:#"DestTable" withOffset:0
//count:pdTableSize];
}
I've managed to get the raw data from a MPMediaItem using an AVAssetReader after combining the answers of a couple of SO questions like this one and this one and a nice blog post. I'm also able to play this raw data using FMOD, but then a problem arises.
It appears the resulting audio is of lower quality than the original track. Though AVAssetTrack formatDescription tells me there are 2 channels in the data, the result sounds mono. It also sounds a bit dampened (less crispy) like the bitrate is lowered.
Am I doing something wrong or is the quality of the MPMediaItem data lowered on purpose by the AVAssetReader (because of piracy)?
#define OUTPUTRATE 44100
Initializing the AVAssetReader and AVAssetReaderTrackOutput
// prepare AVAsset and AVAssetReaderOutput etc
MPMediaItem* mediaItem = ...;
NSURL* ipodAudioUrl = [mediaItem valueForProperty:MPMediaItemPropertyAssetURL];
AVURLAsset * asset = [[AVURLAsset alloc] initWithURL:ipodAudioUrl options:nil];
NSError * error = nil;
assetReader = [[AVAssetReader alloc] initWithAsset:asset error:&error];
if(error)
NSLog(#"error creating reader: %#", [error debugDescription]);
AVAssetTrack* songTrack = [asset.tracks objectAtIndex:0];
NSArray* trackDescriptions = songTrack.formatDescriptions;
numChannels = 2;
for(unsigned int i = 0; i < [trackDescriptions count]; ++i)
{
CMAudioFormatDescriptionRef item = (CMAudioFormatDescriptionRef)[trackDescriptions objectAtIndex:i];
const AudioStreamBasicDescription* bobTheDesc = CMAudioFormatDescriptionGetStreamBasicDescription (item);
if(bobTheDesc && bobTheDesc->mChannelsPerFrame == 1) {
numChannels = 1;
}
}
NSDictionary* outputSettingsDict = [[[NSDictionary alloc] initWithObjectsAndKeys:
[NSNumber numberWithInt:kAudioFormatLinearPCM],AVFormatIDKey,
[NSNumber numberWithInt:OUTPUTRATE],AVSampleRateKey,
[NSNumber numberWithInt:16],AVLinearPCMBitDepthKey,
[NSNumber numberWithBool:NO],AVLinearPCMIsBigEndianKey,
[NSNumber numberWithBool:NO],AVLinearPCMIsFloatKey,
[NSNumber numberWithBool:NO],AVLinearPCMIsNonInterleaved,
nil] autorelease];
AVAssetReaderTrackOutput * output = [[[AVAssetReaderTrackOutput alloc] initWithTrack:songTrack outputSettings:outputSettingsDict] autorelease];
[assetReader addOutput:output];
[assetReader startReading];
Initializing FMOD and the FMOD sound
// Init FMOD
FMOD_RESULT result = FMOD_OK;
unsigned int version = 0;
/*
Create a System object and initialize
*/
result = FMOD::System_Create(&system);
ERRCHECK(result);
result = system->getVersion(&version);
ERRCHECK(result);
if (version < FMOD_VERSION)
{
fprintf(stderr, "You are using an old version of FMOD %08x. This program requires %08x\n", version, FMOD_VERSION);
exit(-1);
}
result = system->setSoftwareFormat(OUTPUTRATE, FMOD_SOUND_FORMAT_PCM16, 1, 0, FMOD_DSP_RESAMPLER_LINEAR);
ERRCHECK(result);
result = system->init(32, FMOD_INIT_NORMAL | FMOD_INIT_ENABLE_PROFILE, NULL);
ERRCHECK(result);
// Init FMOD sound stream
CMTimeRange timeRange = [songTrack timeRange];
float durationInSeconds = timeRange.duration.value / timeRange.duration.timescale;
FMOD_CREATESOUNDEXINFO exinfo = {0};
memset(&exinfo, 0, sizeof(FMOD_CREATESOUNDEXINFO));
exinfo.cbsize = sizeof(FMOD_CREATESOUNDEXINFO); /* required. */
exinfo.decodebuffersize = OUTPUTRATE; /* Chunk size of stream update in samples. This will be the amount of data passed to the user callback. */
exinfo.length = OUTPUTRATE * numChannels * sizeof(signed short) * durationInSeconds; /* Length of PCM data in bytes of whole song (for Sound::getLength) */
exinfo.numchannels = numChannels; /* Number of channels in the sound. */
exinfo.defaultfrequency = OUTPUTRATE; /* Default playback rate of sound. */
exinfo.format = FMOD_SOUND_FORMAT_PCM16; /* Data format of sound. */
exinfo.pcmreadcallback = pcmreadcallback; /* User callback for reading. */
exinfo.pcmsetposcallback = pcmsetposcallback; /* User callback for seeking. */
result = system->createStream(NULL, FMOD_OPENUSER, &exinfo, &sound);
ERRCHECK(result);
result = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel);
ERRCHECK(result);
Reading from the AVAssetReaderTrackOutput into a ring buffer
AVAssetReaderTrackOutput * trackOutput = (AVAssetReaderTrackOutput *)[assetReader.outputs objectAtIndex:0];
CMSampleBufferRef sampleBufferRef = [trackOutput copyNextSampleBuffer];
if (sampleBufferRef)
{
AudioBufferList audioBufferList;
CMBlockBufferRef blockBuffer;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBufferRef, NULL, &audioBufferList, sizeof(audioBufferList), NULL, NULL, 0, &blockBuffer);
if(blockBuffer == NULL)
{
stopLoading = YES;
continue;
}
if(&audioBufferList == NULL)
{
stopLoading = YES;
continue;
}
if(audioBufferList.mNumberBuffers != 1)
NSLog(#"numBuffers = %lu", audioBufferList.mNumberBuffers);
for( int y=0; y<audioBufferList.mNumberBuffers; y++ )
{
AudioBuffer audioBuffer = audioBufferList.mBuffers[y];
SInt8 *frame = (SInt8*)audioBuffer.mData;
for(int i=0; i<audioBufferList.mBuffers[y].mDataByteSize; i++)
{
ringBuffer->push_back(frame[i]);
}
}
CMSampleBufferInvalidate(sampleBufferRef);
CFRelease(sampleBufferRef);
}
I'm not familiar with FMOD, so I can't comment there. AVAssetReader doesn't do any "copy protection" stuff, so that's not a worry. (If you can get the AVAssetURL, the track is DRM free)
Since you are using non-interleaved buffers, there will only be one buffer, so I guess your last bit of code might be wrong
Here's an example of some code that's working well for me. Btw, your for loop is probably not going to be very performant. You may consider using memcpy or something...
If you are not restricted to your existing ring buffer, try TPCircularBuffer (https://github.com/michaeltyson/TPCircularBuffer) it is amazing.
CMSampleBufferRef nextBuffer = NULL;
if(_reader.status == AVAssetReaderStatusReading)
{
nextBuffer = [_readerOutput copyNextSampleBuffer];
}
if (nextBuffer)
{
AudioBufferList abl;
CMBlockBufferRef blockBuffer;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
nextBuffer,
NULL,
&abl,
sizeof(abl),
NULL,
NULL,
kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
&blockBuffer);
// the correct way to get the number of bytes in the buffer
size_t size = CMSampleBufferGetTotalSampleSize(nextBuffer);
memcpy(ringBufferTail, abl.mBuffers[0].mData, size);
CFRelease(nextBuffer);
CFRelease(blockBuffer);
}
Hope this helps
You're initialiazing FMOD to output mono audio. Try
result = system->setSoftwareFormat(OUTPUTRATE, FMOD_SOUND_FORMAT_PCM16, 2, 0, FMOD_DSP_RESAMPLER_LINEAR);