I am trying to grab the audio track from a video as raw pcm data. The plan is pass this data into a float* table array in libpd. I have managed to get some sample data but the number of samples reported is way too low. For example for a 29sec clip I am getting a reported 3968 samples. What I am looking forward is the amplitudes at each sample. For a track of audio length of 29 secs I would expect to have an array of 1278900 in size (at 44.1khz).
Here is what I have put together based on other examples:
- (void)audioCapture {
NSLog(#"capturing");
NSError *error;
AVAssetReader* reader = [[AVAssetReader alloc] initWithAsset:self.videoAsset error:&error];
NSArray *audioTracks = [self.videoAsset tracksWithMediaType:AVMediaTypeAudio];
AVAssetTrack *audioTrack = nil;
if ([audioTracks count] > 0)
audioTrack = [audioTracks objectAtIndex:0];
// Decompress to Linear PCM with the asset reader
NSDictionary *decompressionAudioSettings = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithUnsignedInt:kAudioFormatLinearPCM], AVFormatIDKey,
nil];
AVAssetReaderOutput *output = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:audioTrack outputSettings:decompressionAudioSettings];
[reader addOutput:output];
[reader startReading];
CMSampleBufferRef sample = [output copyNextSampleBuffer];
//array for our sample amp values
SInt16* samples = NULL;
//sample count;
CMItemCount numSamplesInBuffer = 0;
//sample buffer
CMBlockBufferRef buffer;
while( sample != NULL )
{
sample = [output copyNextSampleBuffer];
if( sample == NULL )
continue;
buffer = CMSampleBufferGetDataBuffer( sample );
size_t lengthAtOffset;
size_t totalLength;
char* data;
if( CMBlockBufferGetDataPointer( buffer, 0, &lengthAtOffset, &totalLength, &data ) != noErr )
{
NSLog( #"error!" );
break;
}
numSamplesInBuffer = CMSampleBufferGetNumSamples(sample);
AudioBufferList audioBufferList;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
sample,
NULL,
&audioBufferList,
sizeof(audioBufferList),
NULL,
NULL,
kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
&buffer
);
for (int bufferCount=0; bufferCount < audioBufferList.mNumberBuffers; bufferCount++) {
samples = (SInt16 *)audioBufferList.mBuffers[bufferCount].mData;
NSLog(#"idx %f",(double)samples[bufferCount]);
}
}
NSLog(#"num samps in buf %ld",numSamplesInBuffer);
//[PdBase copyArray:(float *)samples
//toArrayNamed:#"DestTable" withOffset:0
//count:pdTableSize];
}
Related
I'm trying to reverse an AVAsset audio and save it to a file. To make things clear, I've made simple application with the issue https://github.com/ksenia-lyagusha/AudioReverse.git
The application takes mp4 video file from bundle, exports it to Temporary folder in the sandbox as single m4a file, then tries to read it from there, reverse and save result file back.
Temporary m4a file is OK.
The only result of my reverse part is Audio file in the Sandbox with white noise.
There is the part of code below, that is in charge of reversing AVAsset. It is based on related questions
How to reverse an audio file?
iOS audio manipulation - play local .caf file backwards
However, it doesn't work for me.
OSStatus theErr = noErr;
UInt64 fileDataSize = 0;
AudioFileID inputAudioFile;
AudioStreamBasicDescription theFileFormat;
UInt32 thePropertySize = sizeof(theFileFormat);
theErr = AudioFileOpenURL((__bridge CFURLRef)[NSURL URLWithString:inputPath], kAudioFileReadPermission, 0, &inputAudioFile);
thePropertySize = sizeof(fileDataSize);
theErr = AudioFileGetProperty(inputAudioFile, kAudioFilePropertyAudioDataByteCount, &thePropertySize, &fileDataSize);
UInt32 ps = sizeof(AudioStreamBasicDescription) ;
AudioFileGetProperty(inputAudioFile, kAudioFilePropertyDataFormat, &ps, &theFileFormat);
UInt64 dataSize = fileDataSize;
void *theData = malloc(dataSize);
// set up output file
AudioFileID outputAudioFile;
AudioStreamBasicDescription myPCMFormat;
myPCMFormat.mSampleRate = 44100;
myPCMFormat.mFormatID = kAudioFormatLinearPCM;
// kAudioFormatFlagsCanonical is deprecated
myPCMFormat.mFormatFlags = kAudioFormatFlagIsFloat | kAudioFormatFlagIsNonInterleaved;
myPCMFormat.mChannelsPerFrame = 1;
myPCMFormat.mFramesPerPacket = 1;
myPCMFormat.mBitsPerChannel = 32;
myPCMFormat.mBytesPerPacket = (myPCMFormat.mBitsPerChannel / 8) * myPCMFormat.mChannelsPerFrame;
myPCMFormat.mBytesPerFrame = myPCMFormat.mBytesPerPacket;
NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:#"ReverseAudio.caf"];
NSURL *outputURL = [NSURL fileURLWithPath:exportPath];
theErr = AudioFileCreateWithURL((__bridge CFURLRef)outputURL,
kAudioFileCAFType,
&myPCMFormat,
kAudioFileFlags_EraseFile,
&outputAudioFile);
//Read data into buffer
//if readPoint = dataSize, then bytesToRead = 0 in while loop and
//it is endless
SInt64 readPoint = dataSize-1;
UInt64 writePoint = 0;
while(readPoint > 0)
{
UInt32 bytesToRead = 2;
AudioFileReadBytes(inputAudioFile, false, readPoint, &bytesToRead, theData);
// bytesToRead is now the amount of data actually read
UInt32 bytesToWrite = bytesToRead;
AudioFileWriteBytes(outputAudioFile, false, writePoint, &bytesToWrite, theData);
// bytesToWrite is now the amount of data actually written
writePoint += bytesToWrite;
readPoint -= bytesToRead;
}
free(theData);
AudioFileClose(inputAudioFile);
AudioFileClose(outputAudioFile);
If I change file type in AudioFileCreateWithURL from kAudioFileCAFType to another the result file is not created in the Sandbox at all.
Thanks for any help.
You get white noise because your in and out file formats are incompatible. You have different sample rates and channels and probably other differences. To make this work you need to have a common (PCM) format mediating between reads and writes. This is a reasonable job for the new(ish) AVAudio frameworks. We read from file to PCM, shuffle the buffers, then write from PCM to file. This approach is not optimised for large files, as all data is read into the buffers in one go, but is enough to get you started.
You can call this method from your getAudioFromVideo completion block. Error handling ignored for clarity.
- (void)readAudioFromURL:(NSURL*)inURL reverseToURL:(NSURL*)outURL {
//prepare the in and outfiles
AVAudioFile* inFile =
[[AVAudioFile alloc] initForReading:inURL error:nil];
AVAudioFormat* format = inFile.processingFormat;
AVAudioFrameCount frameCount =(UInt32)inFile.length;
NSDictionary* outSettings = #{
AVNumberOfChannelsKey:#(format.channelCount)
,AVSampleRateKey:#(format.sampleRate)};
AVAudioFile* outFile =
[[AVAudioFile alloc] initForWriting:outURL
settings:outSettings
error:nil];
//prepare the forward and reverse buffers
self.forwaredBuffer =
[[AVAudioPCMBuffer alloc] initWithPCMFormat:format
frameCapacity:frameCount];
self.reverseBuffer =
[[AVAudioPCMBuffer alloc] initWithPCMFormat:format
frameCapacity:frameCount];
//read file into forwardBuffer
[inFile readIntoBuffer:self.forwaredBuffer error:&error];
//set frameLength of reverseBuffer to forwardBuffer framelength
AVAudioFrameCount frameLength = self.forwaredBuffer.frameLength;
self.reverseBuffer.frameLength = frameLength;
//iterate over channels
//stride is 1 or 2 depending on interleave format
NSInteger stride = self.forwaredBuffer.stride;
for (AVAudioChannelCount channelIdx = 0;
channelIdx < self.forwaredBuffer.format.channelCount;
channelIdx++) {
float* forwaredChannelData =
self.forwaredBuffer.floatChannelData[channelIdx];
float* reverseChannelData =
self.reverseBuffer.floatChannelData[channelIdx];
int32_t reverseIdx = 0;
//iterate over samples, allocate to reverseBuffer in reverse order
for (AVAudioFrameCount frameIdx = frameLength;
frameIdx >0;
frameIdx--) {
float sample = forwaredChannelData[frameIdx*stride];
reverseChannelData[reverseIdx*stride] = sample;
reverseIdx++;
}
}
//write reverseBuffer to outFile
[outFile writeFromBuffer:self.reverseBuffer error:nil];
}
I wasn't able to find the problem in your code, however I suggest you reversing AVAsset using AVAssetWriter. Following code is based on iOS reverse audio through AVAssetWritet. I've added additional method there to make it work. Finally I've got reversed file.
static NSMutableArray *samples;
static OSStatus sampler(CMSampleBufferRef sampleBuffer, CMItemCount index, void *refcon)
{
[samples addObject:(__bridge id _Nonnull)(sampleBuffer)];
return noErr;
}
- (void)reversePlayAudio:(NSURL *)inputURL
{
AVAsset *asset = [AVAsset assetWithURL:inputURL];
AVAssetReader* reader = [[AVAssetReader alloc] initWithAsset:asset error:nil];
AVAssetTrack* audioTrack = [[asset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];
NSMutableDictionary* audioReadSettings = [NSMutableDictionary dictionary];
[audioReadSettings setValue:[NSNumber numberWithInt:kAudioFormatLinearPCM]
forKey:AVFormatIDKey];
AVAssetReaderTrackOutput* readerOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:audioTrack outputSettings:audioReadSettings];
[reader addOutput:readerOutput];
[reader startReading];
NSDictionary *outputSettings = #{AVFormatIDKey : #(kAudioFormatMPEG4AAC),
AVSampleRateKey : #(44100.0),
AVNumberOfChannelsKey : #(1),
AVEncoderBitRateKey : #(128000),
AVChannelLayoutKey : [NSData data]};
AVAssetWriterInput *writerInput = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeAudio
outputSettings:outputSettings];
NSString *exportPath = [NSTemporaryDirectory() stringByAppendingPathComponent:#"reverseAudio.m4a"];
NSURL *exportURL = [NSURL fileURLWithPath:exportPath];
NSError *writerError = nil;
AVAssetWriter *writer = [[AVAssetWriter alloc] initWithURL:exportURL
fileType:AVFileTypeAppleM4A
error:&writerError];
[writerInput setExpectsMediaDataInRealTime:NO];
writer.shouldOptimizeForNetworkUse = NO;
[writer addInput:writerInput];
[writer startWriting];
[writer startSessionAtSourceTime:kCMTimeZero];
CMSampleBufferRef sample;// = [readerOutput copyNextSampleBuffer];
samples = [[NSMutableArray alloc] init];
while (sample != NULL) {
sample = [readerOutput copyNextSampleBuffer];
if (sample == NULL)
continue;
CMSampleBufferCallForEachSample(sample, &sampler, NULL);
CFRelease(sample);
}
NSArray* reversedSamples = [[samples reverseObjectEnumerator] allObjects];
for (id reversedSample in reversedSamples) {
if (writerInput.readyForMoreMediaData) {
[writerInput appendSampleBuffer:(__bridge CMSampleBufferRef)(reversedSample)];
}
else {
[NSThread sleepForTimeInterval:0.05];
}
}
[samples removeAllObjects];
[writerInput markAsFinished];
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0);
dispatch_async(queue, ^{
[writer finishWritingWithCompletionHandler:^{
// writing is finished
// reversed audio file in TemporaryDirectory in the Sandbox
}];
});
}
Known issues of the code.
There might be some problems with the memory, if the audio is long.
The audio file's duration is longer than original's. (As a quick fix you might cut it down as usual AVAsset).
I need to be able to assemble audio from several files into a single buffer (stereo). My code is working as expected if I load each file into its own buffer. Looping through several files and losing into one larger buffer only plays back the segment from the last file.
Its possible that the header info is getting copied each time, or that the same area of the buffer is just being over-written with each new file.
Any suggestions would be appreciated.
Some code is listed below. I'm reading from encrypted files, so I'm using NSData and AudioFileOpenWithCallbacks.
// Assign the frame count to the soundStructArray instance variable
UInt64 totalFrames = [[inputNotes.stopTimes lastObject] intValue];
self.soundStructArray[0]->frameCount = (UInt32)totalFrames;
self.soundStructArray[0]->audioDataLeft =
(AudioUnitSampleType *) calloc (totalFrames, sizeof (AudioUnitSampleType));
AudioStreamBasicDescription importFormat = {0};
// if (2 == channelCount) {
self.soundStructArray[0]->isStereo = YES;
self.soundStructArray[0]->audioDataRight =
(AudioUnitSampleType *) calloc (totalFrames, sizeof (AudioUnitSampleType));
// Allocate memory for the buffer list struct according to the number of
// channels it represents.
AudioBufferList *bufferList;
UInt32 channelCount = 2;
bufferList = (AudioBufferList *) malloc (
sizeof (AudioBufferList) + sizeof (AudioBuffer) * (channelCount - 1)
);
if (NULL == bufferList) {DLog (#"*** malloc failure for allocating bufferList memory"); return;}
// initialize the mNumberBuffers member
bufferList->mNumberBuffers = channelCount;
// initialize the mBuffers member to 0
AudioBuffer emptyBuffer = {0};
size_t arrayIndex;
for (arrayIndex = 0; arrayIndex < channelCount; arrayIndex++) {
bufferList->mBuffers[arrayIndex] = emptyBuffer;
}
// set up the AudioBuffer structs in the buffer list
bufferList->mBuffers[0].mNumberChannels = 1;
bufferList->mBuffers[0].mDataByteSize = (UInt32)totalFrames * sizeof (AudioUnitSampleType);
bufferList->mBuffers[0].mData = self.soundStructArray[0]->audioDataLeft;
if (2 == channelCount) {
bufferList->mBuffers[1].mNumberChannels = 1;
bufferList->mBuffers[1].mDataByteSize = (UInt32)totalFrames * sizeof (AudioUnitSampleType);
bufferList->mBuffers[1].mData = self.soundStructArray[0]->audioDataRight;
}
NSString *fileType = #"m4a";
for (int audioFile = 0; audioFile < inputVoicesCount; ++audioFile) {
#autoreleasepool {
NSData *encData;
NSData *audioData;
AudioFileID refAudioFileID;
DLog (#"readAudioFilesIntoMemory - file %i", audioFile);
NSString *source = [[NSBundle mainBundle] pathForResource:[inputNotes.notes objectAtIndex:audioFile] ofType:fileType];
// NSURL *url = [NSURL encryptedFileURLWithPath:source];
if ([[NSFileManager defaultManager] fileExistsAtPath:source])
{
//File exists
encData = [[NSData alloc] initWithContentsOfFile:source];
if (encData)
{
NSError *error;
audioData = [RNDecryptor decryptData:encData
withPassword:key
error:&error];
}
}
else
{
DLog(#"File does not exist");
}
OSStatus result = AudioFileOpenWithCallbacks((__bridge void *)(audioData), readProc, 0, getSizeProc, NULL, kAudioFileMPEG4Type, &refAudioFileID);
if(result != noErr){
DLog(#"problem in theAudioFileReaderWithData function: result code %i \n", result);
}
// Instantiate an extended audio file object.
ExtAudioFileRef audioFileObject = 0;
result = ExtAudioFileWrapAudioFileID(refAudioFileID, NO, &audioFileObject);
if (result != noErr){
DLog(#"problem in theAudioFileReaderWithData function Wraping the audio FileID: result code %i \n", result);
}
// Get the audio file's number of channels.
AudioStreamBasicDescription fileAudioFormat = {0};
UInt32 formatPropertySize = sizeof (fileAudioFormat);
result = ExtAudioFileGetProperty (
audioFileObject,
kExtAudioFileProperty_FileDataFormat,
&formatPropertySize,
&fileAudioFormat
);
if (noErr != result) {[self printErrorMessage: #"ExtAudioFileGetProperty (file audio format)" withStatus: result]; return;}
importFormat = stereoStreamFormat;
result = ExtAudioFileSetProperty (
audioFileObject,
kExtAudioFileProperty_ClientDataFormat,
sizeof (importFormat),
&importFormat
);
if (noErr != result) {[self printErrorMessage: #"ExtAudioFileSetProperty (client data format)" withStatus: result]; return;}
// Assign the frame count to the soundStructArray instance variable
UInt64 desiredFrames = (UInt64) ([[inputNotes.stopTimes objectAtIndex:audioFile] intValue] - [[inputNotes.startTimes objectAtIndex:audioFile] intValue]);
// Perform a synchronous, sequential read of the audio data out of the file and
// into the soundStructArray[audioFile].audioDataLeft and (if stereo) .audioDataRight members.
UInt32 numberOfPacketsToRead = (UInt32) desiredFrames;
result = ExtAudioFileRead (
audioFileObject,
&numberOfPacketsToRead,
bufferList
);
if (noErr != result) {
[self printErrorMessage: #"ExtAudioFileRead failure - " withStatus: result];
// If reading from the file failed, then free the memory for the sound buffer.
// free (soundStructArray[audioFile].audioDataLeft);
// soundStructArray[audioFile].audioDataLeft = 0;
free (self.soundStructArray[0]->audioDataLeft);
self.soundStructArray[0]->audioDataLeft = 0;
free (self.soundStructArray[0]->audioDataRight);
self.soundStructArray[0]->audioDataRight = 0;
ExtAudioFileDispose (audioFileObject);
return;
}
ExtAudioFileDispose (audioFileObject);
AudioFileClose(refAudioFileID);
}
}//end of #autoreleasepool
free (bufferList);
// Set the sample index to zero, so that playback starts at the
// beginning of the sound.
self.soundStructArray[0]->sampleNumber = 0;
DLog (#"Finished reading all files into memory");
readingFiles = NO;
}
I'm working on an audio recorder that renders the waveform of each recording. Whenever a recording is made, the NSURL of the audio file is converted to an AVAsset. With the AVAsset I'm able to extract the samples of the audio track. This works fine for audio recordings that are short ( <40seconds), but the process takes 15-20 seconds on a 2.5 min track and only gets worse the longer the track is. Anyone have any tips or recommendations on how to get around this problem?
AVAssetReader *reader = [[AVAssetReader alloc] initWithAsset:self.audioAsset error:&error];
AVAssetReaderTrackOutput *output = [[AVAssetReaderTrackOutput alloc] initWithTrack:songTrack outputSettings:outputSettingsDict];
[reader addOutput:output];
NSMutableData * data = [NSMutableData dataWithLength:32768];
NSMutableArray *allSamples = [NSMutableArray array];
while (reader.status == AVAssetReaderStatusReading) {
CMSampleBufferRef sampleBufferRef = [output copyNextSampleBuffer];
if (sampleBufferRef) {
CMBlockBufferRef blockBufferRef = CMSampleBufferGetDataBuffer(sampleBufferRef);
size_t bufferLength = CMBlockBufferGetDataLength(blockBufferRef);
if (data.length < bufferLength) {
[data setLength:bufferLength];
}
CMBlockBufferCopyDataBytes(blockBufferRef, 0, bufferLength, data.mutableBytes);
Float32 *samples = (Float32 *)data.mutableBytes;
int sampleCount = (int)(bufferLength / bytesPerInputSample);
for (int i = 0; i < sampleCount; i++) {
[allSamples addObject:#(samples[i*channelCount])];
}
CMSampleBufferInvalidate(sampleBufferRef);
CFRelease(sampleBufferRef);
}
}
I need to convert PCM audio data with format:
Data format: 1 ch, 16000 Hz, 'lpcm' (0x0000000C) 16-bit little-endian signed integer
no channel layout.
estimated duration: 1.101063 sec
audio bytes: 35234
audio packets: 17617
bit rate: 256000 bits per second
packet size upper bound: 2
maximum packet size: 2
audio data file offset: 44
optimized
source bit depth: I16
to 16bit 2ch (stereo) 44100Hz PCM.
My input file comes as NSData and ideally would be if I could end up with NSData instead of saving output to file. I've seen many tutorials and examples of converting different audio formats but they seems very complicated and I'm wondering if there is any simple solution to do that. This is code I've tried so far:
-(void)convertAudioToRequiredFormat:(NSData *)data {
AudioFileID refAudioFileID;
ExtAudioFileRef inputFileID;
ExtAudioFileRef outputFileID;
OSStatus result = AudioFileOpenWithCallbacks((__bridge void *)(data), readProc, 0, getSizeProc, 0, kAudioFormatLinearPCM, &refAudioFileID);
if (result != noErr) {
DLog(#"error reading input audio file");
}
result = ExtAudioFileWrapAudioFileID(refAudioFileID, false, &inputFileID);
if (result != noErr){
DLog(#"problem in theAudioFileReaderWithData function Wraping the audio FileID: result code %i \n", (int)result);
}
AudioStreamBasicDescription clientFormat;
memset(&clientFormat, 0, sizeof(clientFormat));
clientFormat.mFormatID = kAudioFormatLinearPCM;
clientFormat.mSampleRate = 16000;
clientFormat.mFramesPerPacket = 1;
clientFormat.mBytesPerPacket = 2; //16 bits * 1 channel
clientFormat.mBytesPerFrame = 2;
clientFormat.mChannelsPerFrame = 1; //1 channel
clientFormat.mBitsPerChannel = 16;
clientFormat.mFormatFlags = kCAFLinearPCMFormatFlagIsLittleEndian;
clientFormat.mReserved = 0;
AudioStreamBasicDescription outputFormat;
memset(&outputFormat, 0, sizeof(outputFormat));
outputFormat.mFormatID = kAudioFormatLinearPCM;
outputFormat.mSampleRate = 44100;
outputFormat.mFramesPerPacket = 1; //it is always 1 for PCM
outputFormat.mBytesPerPacket = 4; //4 Bytes = 2 * 16 bits
outputFormat.mBytesPerFrame = 4;
outputFormat.mChannelsPerFrame = 2; //2 channels = stereo
outputFormat.mBitsPerChannel = 16; //16 bits per channel
outputFormat.mFormatFlags = kCAFLinearPCMFormatFlagIsLittleEndian;
clientFormat.mReserved = 0;
UInt32 outputFormatSize = sizeof(outputFormat);
result = 0;
result = AudioFormatGetProperty(kAudioFormatProperty_FormatInfo, 0, NULL, &outputFormatSize, &outputFormat);
if(result != noErr)
NSLog(#"could not set the output format with status code %i \n",(int)result);
NSArray *docPaths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *docPath = [docPaths objectAtIndex:0];
NSString *path = [docPath stringByAppendingPathComponent:#"newFormat.wav"];
CFURLRef sourceURL = (__bridge CFURLRef)[[NSURL alloc] initFileURLWithPath:path];
NSFileManager *fm = [NSFileManager defaultManager];
if (![fm fileExistsAtPath:path]) {
NSData *content = [NSData dataWithBytes:NULL length:0];
[fm createFileAtPath:path contents:content attributes:nil];
}
result = 0;
result = ExtAudioFileCreateWithURL(sourceURL, kAudioFileM4AType, &outputFormat, NULL, kAudioFileFlags_EraseFile, &outputFileID);
if(result != noErr){
NSLog(#"ExtAudioFileCreateWithURL failed for outputFileID with status %i \n", (int)result);
}
int size = sizeof(clientFormat);
result = 0;
result = ExtAudioFileSetProperty(inputFileID, kExtAudioFileProperty_ClientDataFormat, size, &clientFormat);
if(result != noErr)
NSLog(#"error on ExtAudioFileSetProperty for input File with result code %i \n", (int)result);
size = sizeof(clientFormat);
result = 0;
result = ExtAudioFileSetProperty(outputFileID, kExtAudioFileProperty_ClientDataFormat, size, &clientFormat);
if(result != noErr)
NSLog(#"error on ExtAudioFileSetProperty for output File with result code %i \n", (int)result);
int totalFrames = 0;
UInt32 outputFilePacketPosition = 0; //in bytes
UInt32 encodedBytes = 0;
while (1) {
UInt32 bufferByteSize = 22050 * 4 * 2;
char srcBuffer[bufferByteSize];
UInt32 numFrames = (bufferByteSize/clientFormat.mBytesPerFrame);
AudioBufferList fillBufList;
fillBufList.mNumberBuffers = 1;
fillBufList.mBuffers[0].mNumberChannels = clientFormat.mChannelsPerFrame;
fillBufList.mBuffers[0].mDataByteSize = bufferByteSize;
fillBufList.mBuffers[0].mData = srcBuffer;
result = 0;
result = ExtAudioFileRead(inputFileID, &numFrames, &fillBufList);
if (result != noErr) {
NSLog(#"Error on ExtAudioFileRead with result code %i \n", (int)result);
totalFrames = 0;
break;
}
if (!numFrames)
break;
totalFrames = totalFrames + numFrames;
result = 0;
result = ExtAudioFileWrite(outputFileID,
numFrames,
&fillBufList);
if(result!= noErr){
NSLog(#"ExtAudioFileWrite failed with code %i \n", (int)result);
}
encodedBytes += numFrames * clientFormat.mBytesPerFrame;
}
//Clean up
ExtAudioFileDispose(inputFileID);
ExtAudioFileDispose(outputFileID);
AudioFileClose(refAudioFileID);
}
static OSStatus readProc(void* clientData, SInt64 position, UInt32 requestCount, void* buffer, UInt32* actualCount)
{
NSData *inAudioData = (__bridge NSData *) clientData;
size_t dataSize = inAudioData.length;
size_t bytesToRead = 0;
if(position < dataSize) {
size_t bytesAvailable = dataSize - position;
bytesToRead = requestCount <= bytesAvailable ? requestCount : bytesAvailable;
[inAudioData getBytes: buffer range:NSMakeRange(position, bytesToRead)];
} else {
NSLog(#"data was not read \n");
bytesToRead = 0;
}
if(actualCount)
*actualCount = bytesToRead;
return noErr;
}
static SInt64 getSizeProc(void* clientData) {
NSData *inAudioData = (__bridge NSData *) clientData;
size_t dataSize = inAudioData.length;
return dataSize;
}
Unfortunately it doesn't work and I'm ending up with EXC_BAD_ACCESS in that line:
result = ExtAudioFileCreateWithURL(sourceURL, kAudioFileM4AType, &outputFormat, NULL, kAudioFileFlags_EraseFile, &outputFileID);
I have no idea what's causing that error (wrong AudioStreamBasicDescription?). Can someone help me to fix it? Or maybe there is easier way to convert that audio data to desired PCM format?
Some tipps.
1.)
AudioFileID refAudioFileID;
ExtAudioFileRef inputFileID;
ExtAudioFileRef outputFileID;
^ init the pointers with NULL.
2.)
CFURLRef sourceURL = (__bridge CFURLRef)[[NSURL alloc] initFileURLWithPath:path];
^ do:
NSURL *sourceURL = [NSURL fileURLWithPath:path];
3.)
NSData *content = [NSData dataWithBytes:NULL length:0];
^ use:
NSData *content = [NSData data];
4.)
result = ExtAudioFileCreateWithURL(sourceURL, kAudioFileM4AType, &outputFormat, NULL, kAudioFileFlags_EraseFile, &outputFileID);
^ it looks as if you are using the wrong kAudioFileType: try kAudioFileAIFFType or kAudioFileWAVEType
My two cents on this. Have luck!
I've managed to get the raw data from a MPMediaItem using an AVAssetReader after combining the answers of a couple of SO questions like this one and this one and a nice blog post. I'm also able to play this raw data using FMOD, but then a problem arises.
It appears the resulting audio is of lower quality than the original track. Though AVAssetTrack formatDescription tells me there are 2 channels in the data, the result sounds mono. It also sounds a bit dampened (less crispy) like the bitrate is lowered.
Am I doing something wrong or is the quality of the MPMediaItem data lowered on purpose by the AVAssetReader (because of piracy)?
#define OUTPUTRATE 44100
Initializing the AVAssetReader and AVAssetReaderTrackOutput
// prepare AVAsset and AVAssetReaderOutput etc
MPMediaItem* mediaItem = ...;
NSURL* ipodAudioUrl = [mediaItem valueForProperty:MPMediaItemPropertyAssetURL];
AVURLAsset * asset = [[AVURLAsset alloc] initWithURL:ipodAudioUrl options:nil];
NSError * error = nil;
assetReader = [[AVAssetReader alloc] initWithAsset:asset error:&error];
if(error)
NSLog(#"error creating reader: %#", [error debugDescription]);
AVAssetTrack* songTrack = [asset.tracks objectAtIndex:0];
NSArray* trackDescriptions = songTrack.formatDescriptions;
numChannels = 2;
for(unsigned int i = 0; i < [trackDescriptions count]; ++i)
{
CMAudioFormatDescriptionRef item = (CMAudioFormatDescriptionRef)[trackDescriptions objectAtIndex:i];
const AudioStreamBasicDescription* bobTheDesc = CMAudioFormatDescriptionGetStreamBasicDescription (item);
if(bobTheDesc && bobTheDesc->mChannelsPerFrame == 1) {
numChannels = 1;
}
}
NSDictionary* outputSettingsDict = [[[NSDictionary alloc] initWithObjectsAndKeys:
[NSNumber numberWithInt:kAudioFormatLinearPCM],AVFormatIDKey,
[NSNumber numberWithInt:OUTPUTRATE],AVSampleRateKey,
[NSNumber numberWithInt:16],AVLinearPCMBitDepthKey,
[NSNumber numberWithBool:NO],AVLinearPCMIsBigEndianKey,
[NSNumber numberWithBool:NO],AVLinearPCMIsFloatKey,
[NSNumber numberWithBool:NO],AVLinearPCMIsNonInterleaved,
nil] autorelease];
AVAssetReaderTrackOutput * output = [[[AVAssetReaderTrackOutput alloc] initWithTrack:songTrack outputSettings:outputSettingsDict] autorelease];
[assetReader addOutput:output];
[assetReader startReading];
Initializing FMOD and the FMOD sound
// Init FMOD
FMOD_RESULT result = FMOD_OK;
unsigned int version = 0;
/*
Create a System object and initialize
*/
result = FMOD::System_Create(&system);
ERRCHECK(result);
result = system->getVersion(&version);
ERRCHECK(result);
if (version < FMOD_VERSION)
{
fprintf(stderr, "You are using an old version of FMOD %08x. This program requires %08x\n", version, FMOD_VERSION);
exit(-1);
}
result = system->setSoftwareFormat(OUTPUTRATE, FMOD_SOUND_FORMAT_PCM16, 1, 0, FMOD_DSP_RESAMPLER_LINEAR);
ERRCHECK(result);
result = system->init(32, FMOD_INIT_NORMAL | FMOD_INIT_ENABLE_PROFILE, NULL);
ERRCHECK(result);
// Init FMOD sound stream
CMTimeRange timeRange = [songTrack timeRange];
float durationInSeconds = timeRange.duration.value / timeRange.duration.timescale;
FMOD_CREATESOUNDEXINFO exinfo = {0};
memset(&exinfo, 0, sizeof(FMOD_CREATESOUNDEXINFO));
exinfo.cbsize = sizeof(FMOD_CREATESOUNDEXINFO); /* required. */
exinfo.decodebuffersize = OUTPUTRATE; /* Chunk size of stream update in samples. This will be the amount of data passed to the user callback. */
exinfo.length = OUTPUTRATE * numChannels * sizeof(signed short) * durationInSeconds; /* Length of PCM data in bytes of whole song (for Sound::getLength) */
exinfo.numchannels = numChannels; /* Number of channels in the sound. */
exinfo.defaultfrequency = OUTPUTRATE; /* Default playback rate of sound. */
exinfo.format = FMOD_SOUND_FORMAT_PCM16; /* Data format of sound. */
exinfo.pcmreadcallback = pcmreadcallback; /* User callback for reading. */
exinfo.pcmsetposcallback = pcmsetposcallback; /* User callback for seeking. */
result = system->createStream(NULL, FMOD_OPENUSER, &exinfo, &sound);
ERRCHECK(result);
result = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel);
ERRCHECK(result);
Reading from the AVAssetReaderTrackOutput into a ring buffer
AVAssetReaderTrackOutput * trackOutput = (AVAssetReaderTrackOutput *)[assetReader.outputs objectAtIndex:0];
CMSampleBufferRef sampleBufferRef = [trackOutput copyNextSampleBuffer];
if (sampleBufferRef)
{
AudioBufferList audioBufferList;
CMBlockBufferRef blockBuffer;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBufferRef, NULL, &audioBufferList, sizeof(audioBufferList), NULL, NULL, 0, &blockBuffer);
if(blockBuffer == NULL)
{
stopLoading = YES;
continue;
}
if(&audioBufferList == NULL)
{
stopLoading = YES;
continue;
}
if(audioBufferList.mNumberBuffers != 1)
NSLog(#"numBuffers = %lu", audioBufferList.mNumberBuffers);
for( int y=0; y<audioBufferList.mNumberBuffers; y++ )
{
AudioBuffer audioBuffer = audioBufferList.mBuffers[y];
SInt8 *frame = (SInt8*)audioBuffer.mData;
for(int i=0; i<audioBufferList.mBuffers[y].mDataByteSize; i++)
{
ringBuffer->push_back(frame[i]);
}
}
CMSampleBufferInvalidate(sampleBufferRef);
CFRelease(sampleBufferRef);
}
I'm not familiar with FMOD, so I can't comment there. AVAssetReader doesn't do any "copy protection" stuff, so that's not a worry. (If you can get the AVAssetURL, the track is DRM free)
Since you are using non-interleaved buffers, there will only be one buffer, so I guess your last bit of code might be wrong
Here's an example of some code that's working well for me. Btw, your for loop is probably not going to be very performant. You may consider using memcpy or something...
If you are not restricted to your existing ring buffer, try TPCircularBuffer (https://github.com/michaeltyson/TPCircularBuffer) it is amazing.
CMSampleBufferRef nextBuffer = NULL;
if(_reader.status == AVAssetReaderStatusReading)
{
nextBuffer = [_readerOutput copyNextSampleBuffer];
}
if (nextBuffer)
{
AudioBufferList abl;
CMBlockBufferRef blockBuffer;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
nextBuffer,
NULL,
&abl,
sizeof(abl),
NULL,
NULL,
kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
&blockBuffer);
// the correct way to get the number of bytes in the buffer
size_t size = CMSampleBufferGetTotalSampleSize(nextBuffer);
memcpy(ringBufferTail, abl.mBuffers[0].mData, size);
CFRelease(nextBuffer);
CFRelease(blockBuffer);
}
Hope this helps
You're initialiazing FMOD to output mono audio. Try
result = system->setSoftwareFormat(OUTPUTRATE, FMOD_SOUND_FORMAT_PCM16, 2, 0, FMOD_DSP_RESAMPLER_LINEAR);