I am working on online radio app I managed to play the streamed mp3 packets from the Icecast server using AudioQueueServices, what I am struggling with is implementing a recording feature.
Since the streaming is in mp3 format I can not write the Audio packets directly to file using AudioFileWritePackets.
To leverage The automatic conversion of Extended Audio I am using ExtAudioWriteFile to write to a wav file. I have setup the AudioStreamBasicDescription of the incoming packets using the FileStreamOpen call back function AudioFileStream_PropertyListenerProc and the destination format I populated manually.The code successfully creates the file and writes the packet to it but on playback what I hear is a white noise;
Here is my code
// when the recording button is pressed this function creates the file and setup the asbd
-(void)startRecording{
recording = true;
OSStatus status;
NSURL *baseUrl=[self applicationDocumentsDirectory];//returns the document direcotry of the app
NSURL *audioUrl = [NSURL URLWithString:#"Recorded.wav" relativeToURL:baseUrl];
//asbd setup for the destination file/wav file
AudioStreamBasicDescription dstFormat;
dstFormat.mSampleRate=44100.0;
dstFormat.mFormatID=kAudioFormatLinearPCM; dstFormat.mFormatFlags=kAudioFormatFlagsNativeEndian|kAudioFormatFlagIsSignedInteger|kAudioFormatFlagIsPacked;
dstFormat.mBytesPerPacket=4;
dstFormat.mBytesPerFrame=4;
dstFormat.mFramesPerPacket=1;
dstFormat.mChannelsPerFrame=2;
dstFormat.mBitsPerChannel=16;
dstFormat.mReserved=0;
//creating the file
status = ExtAudioFileCreateWithURL(CFBridgingRetain(audioUrl), kAudioFileWAVEType, &(dstFormat), NULL, kAudioFileFlags_EraseFile, &recordingFilRef);
// tell the EXtAudio File ApI what format we will be sending samples
//recordasbd is the asbd of incoming packets populated in AudioFileStream_PropertyListenerProc
status = ExtAudioFileSetProperty(recordingFilRef, kExtAudioFileProperty_ClientDataFormat, sizeof(recordasbd), &recordasbd);
}
// a handler called by packetproc call back function in AudiofileStreamOpen
- (void)handlePacketsProc:(const void *)inInputData numberBytes:(UInt32)inNumberBytes numberPackets:(UInt32)inNumberPackets packetDescriptions:(AudioStreamPacketDescription *)inPacketDescriptions {
if(recording){
// wrap the destination buffer in an audiobuffer list
convertedData.mNumberBuffers= 1;
convertedData.mBuffers[0].mNumberChannels = recordasbd.mChannelsPerFrame;
convertedData.mBuffers[0].mDataByteSize = inNumberBytes;
convertedData.mBuffers[0].mData = inInputData;
ExtAudioFileWrite(recordingFilRef,recordasbd.mFramesPerPacket * inNumberPackets, &convertedData);
}
}
My questions are:
Is my approach right can I write mp3 packets to wav file this way If so what am I missing ??
If my approach is wrong please tell me any other way you think is right.A nudge in the right direction is more than enough for me
I am so grateful for any help I have read every SO question I could get my hands on this topic, I also looked closely at apples Convertfile example but I could not figure out what I am missng
Thanks in advance for any help
Why not write the raw mp3 packets directly to a file? Without using ExtAudioFile at all.
They will form a valid mp3 file and will be much smaller than the equivalent wav file.
Related
I'm trying to convert a MIDI file to an Audio File (.m4a) in Swift.
Right now I'm using MIKMIDI as a tool to sequence and playback MIDI files, however it does not include the ability to save the playback into a file. MIKMID's creator outlines the process to do this here. In an attempt to capture and save the output to an audio file, I've followed this example to try and replace the MIKMIDI Graph's RemoteIO node with a GeneralIO node in Swift. When I try to save the output to a file using AudioUnitRender and ExtAudioFileWrite, they both return error -50 (kAudio_ParamError).
var channels = 2
var buffFrames = 512
var bufferList = AudioBufferList.allocate(maximumBuffers: 1)
for i in 0...bufferList.count-1{
var buffer = AudioBuffer()
buffer.mNumberChannels = 2
buffer.mDataByteSize = UInt32(buffFrames*sizeofValue(AudioUnitSampleType))
buffer.mData = calloc(buffFrames, sizeofValue(AudioUnitSampleType))
bufferList[i] = buffer
result = AudioUnitRender(generalIOAudioUnit, &flags, &inTimeStamp, busNum, UInt32(buffFrames), bufferList.unsafeMutablePointer)
inTimeStamp.mSampleTime += 1
result = ExtAudioFileWrite(extAudioFile, UInt32(buffFrames), bufferList.unsafeMutablePointer)
}
What is causing error -50, and how can I resolve it to render the MIDI (offline) to .m4a files?
UPDATE: I have resolved the ExtAudioFileWrite error -50 by changing mNumberChannels and channels to = 1. Now I get a one second audio file with noise. AudioUnitRender still returns error -50.
There are a couple of problems with your code:
your AudioBufferList doesn't agree with the client format, try
let bufferList = AudioBufferList.allocate(maximumBuffers: Int(clientFormat.mChannelsPerFrame))
you're replacing the wrong node from the AUGraph, and connecting the remaining node to itself, resulting in an infinite loop on AudioUnitRender.
But the main problem is that you are not implementing the solution that the author suggested. You wish that you could call AudioUnitRender with sample timestamps, faster than realtime, but the author said no, you'll have to manually convert sample time to hosttime and implement the better part of a midi player if you want that.
So you could do that (sounds hard), or file a feature request, or maybe record to file in realtime as you listen to the music by adding a render notification to the graph's remote IO audio unit with AudioUnitAddRenderNotify and writing the samples during the kAudioUnitRenderAction_PostRender phase.
This is what I would like to do:
Get audio from the microphone
Encode it in AAC, G.711 or G.726
Write the encoded frames to a socket.
And this is how I'm trying to get there:
I'm getting audio (PCM) from the microphone using TheAmazingAudioEngine and putting it in a buffer;
Using TPAACAudioConverter I'm reading audio from my buffer and writing to a temp file (AAC);
In the processing thread of TPAACAudioConverter I replaced this:
OSStatus status = ExtAudioFileWrite(destinationFile, numFrames, &fillBufList);
with this:
OSStatus status = ExtAudioFileWrite(destinationFile, numFrames, &fillBufList);
UInt32 framesWritten = numFrames;
totalFramesWritten += framesWritten;
AudioBufferList readData;
readData.mNumberBuffers = 1;
ExtAudioFileSeek(destinationFile, totalFramesWritten - framesWritten);
OSStatus readStatus = ExtAudioFileRead(destinationFile, &numFrames, &readData);
ExtAudioFileSeek(destinationFile, totalFramesWritten);
NSLog(#"Bytes read=%d", numFrames);
but what I get is 0 numFrames read from file.
Any idea on what I may be doing wrong or any suggestion on alternative paths to achieve what I need?
The issue is that whatever ExtAudioFile does under the hood doesn't allow for seeking on a file that is open for writing. If you look at the documentation for ExtAudioFileSeek it says "This function's behavior with files open for writing is currently undefined".
You can solve this by using the more extensible (and difficult) Audio File Services and the Audio Converter Services directly instead of the convenient Extended audio file services.
I abandoned this approach and reused the AQRecorder class from the SpeakHere example by Apple.
The project is available here https://github.com/robovm/apple-ios-samples/tree/master/SpeakHere.
consider these two scenarios for reading/writing data from Audio files (for the purpose of sending over a network):
Scenario 1: Audio File Services:
Using AudioFileReadPackets from Audio File Services. This generates audio packets that you can easily send over the network. On the receiving side you use AudioFileStreamOpen and AudioFileStreamParseBytes to parse the data.
AudioFileStreamParseBytes then has two callback functions: AudioFileStream_PropertyListenerProc and AudioFileStream_PacketsProc. These guys are called when a new property is discovered in the stream and when packets are received from the stream, respectively. Once you receive the packets, you can feed it to an audio queue using Audio Queue Service which plays the file just fine.
Note: This method does NOT work with music files stored in the iPod library, which brings us to the 2nd scenario:
Scenario 2: AVAssetReader:
With AVAssetReader you can read from the iPod music library and send packets over the network. Typically you would load the packets directly on an Audio Queue similar to above. However, in this scenario you will have to create a thread to ensure that you block receiving packets when the queue is full, and unblock when queue buffers are available (see this example).
Question:
Is it possible to use AVAssetReader to send packets over, only to have it read by AudioFileStreamParseBytes? (the motive would be that the AudioFileStreamParseBytes's callbacks will handle the threading/blocking business and save you that pain). I tried doing it like so:
1. first read the audio file using AVAssetReader
//NSURL *assetURL = [NSURL URLWithString:#"ipod-library://item/item.m4a?id=1053020204400037178"];
AVURLAsset *songAsset = [AVURLAsset URLAssetWithURL:assetURL options:nil];
NSError * error = nil;
AVAssetReader* reader = [[AVAssetReader alloc] initWithAsset:songAsset error:&error];
AVAssetTrack* track = [songAsset.tracks objectAtIndex:0];
// Note: I don't supply an audio format description here, rather I pass on nil to keep the original
// file format. In another piece of code (see here: http://stackoverflow.com/questions/12264799/why-is-audio-coming-up-garbled-when-using-avassetreader-with-audio-queue?answertab=active#tab-top) I can extract the audio format from the track, let's say it's an AAC format.
AVAssetReaderTrackOutput* readerOutput = [AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:track
outputSettings:nil];
[reader addOutput:readerOutput];
[reader startReading];
2. set up the streamer
// notice how i manually add the audio file type (for the file hint parameter)
// using the info from step one.. If i leave it as 0, this call fails and returns
// the typ? error, which is :The specified file type is not supported.
streamer->err = AudioFileStreamOpen((__bridge void*)streamer,
ASPropertyListenerProc, ASPacketsProc,
kAudioFileAAC_ADTSType, &(streamer->audioFileStream));
3. once I receive the data, I parse the bytes:
streamer->err = AudioFileStreamParseBytes(streamer->audioFileStream, inDataByteSize, inData, 0);
problem: When I do it this way.. I send the bytes and the AudioFileStreamParseBytes does not fail. However, the callbacks *AudioFileStream_PropertyListenerProc* and *AudioFileStream_PacketsProc* are never called. Which makes me think that the parser has failed to parse the bytes and extract any useful information out of them.. in the documentation for AudioStreamParseBytes it states:* You should provide at least more than a single packet’s worth of audio file data, but it is better to provide a few packets to a few seconds data at a time.* I'm sending over 900 bytes, which is just below GKSession's data limit. I'm pretty sure 900 bytes is enough (when testing this under scenario 1, the total bytes was 417 each time and it worked fine).
Any ideas?
The short answer is that it simply doesn't make sense to have packets of audio data be parsed by AudioFileStreamParseBytes.. in the docs AudioFileStreamParseBytes is a function dependent on the existence of an audio file (thus the parameter inAudioFileStream.. which is defined as the ID of the parser to which you wish to pass data. The parser ID is returned by the AudioFileStreamOpen function.)
so lesson learned: don't try to pigeon hole iOS functions to fit your situation.. it should be the other way around.
What I ended up doing was feeding the data directly to an Audio Queue.. without going through all these unnecessary intermediary functions.. a more in depth way would be feeding the data to audio units.. but my application didn't need that level of control
I'm wanting to load a local .caf audio file and reverse the audio (play it backwards). I've gathered that I basically need to flip an array of buffer data from posts like this
However, I'm not sure how to access this buffer data from a given audio file. I have a little experience playing sounds back with AVaudioPlayer and ObjectAL(an obj-c openAL library), but I don't know how to access something lower level like this buffer data array.
Could I please get an example of how I would go about getting access to that array?
Your problem reduces to the same problem described here, which was linked by P-i in the comment under your question. Kiran answered that question and re-posted his answer for you here. Kiran's answer is accurate, but you may need a few more details to be able to decide how to proceed because you're starting with a CAF file.
The simplest audio file format, linear pulse-code modulation (LPCM), is the easiest to read byte-for-byte or sample-for-sample. This means it's the easiest to reverse. Kiran's solution does just that.
The CAF format is a container/wrapper format, however. While your CAF file could contain a WAV file, it could also contain a compressed file format that cannot be manipulated in the same fashion.
You should consider first converting the CAF file to WAV, then reversing it as shown in the other solution. There are various libraries that will do this conversion for you, but a good place to start might be with the AudioToolbox framework, which includes Audio Converter Services. Alternately, if you can use the WAV file format from the start, you can prevent the need to convert to WAV.
You may need to know more if you find Kiran's sample code gives you an error (Core Audio is finicky). A great place to start is with the Core Audio 'Bible', "Learning Core Audio", written by Chris Adamson and Kevin Avila. That book builds your knowledge of digital sound using great samples. You should also check out anything written by Michael Tyson, who started the Amazing Audio Engine project on github, and wrote AudioBus.
I have worked on a sample app, which records what user says and plays them backwards. I have used CoreAudio to achieve this. Link to app code.
As each sample is 16-bits in size(2 bytes)(mono channel). You can load each sample at a time by copying it into a different buffer by starting at the end of the recording and reading backwards. When you get to the start of the data you have reversed the data and playing will be reversed.
// set up output file
AudioFileID outputAudioFile;
AudioStreamBasicDescription myPCMFormat;
myPCMFormat.mSampleRate = 16000.00;
myPCMFormat.mFormatID = kAudioFormatLinearPCM ;
myPCMFormat.mFormatFlags = kAudioFormatFlagsCanonical;
myPCMFormat.mChannelsPerFrame = 1;
myPCMFormat.mFramesPerPacket = 1;
myPCMFormat.mBitsPerChannel = 16;
myPCMFormat.mBytesPerPacket = 2;
myPCMFormat.mBytesPerFrame = 2;
AudioFileCreateWithURL((__bridge CFURLRef)self.flippedAudioUrl,
kAudioFileCAFType,
&myPCMFormat,
kAudioFileFlags_EraseFile,
&outputAudioFile);
// set up input file
AudioFileID inputAudioFile;
OSStatus theErr = noErr;
UInt64 fileDataSize = 0;
AudioStreamBasicDescription theFileFormat;
UInt32 thePropertySize = sizeof(theFileFormat);
theErr = AudioFileOpenURL((__bridge CFURLRef)self.recordedAudioUrl,kAudioFileReadPermission, 0, &inputAudioFile);
thePropertySize = sizeof(fileDataSize);
theErr = AudioFileGetProperty(inputAudioFile, kAudioFilePropertyAudioDataByteCount, &thePropertySize, &fileDataSize);
UInt32 dataSize = fileDataSize;
void* theData = malloc(dataSize);
//Read data into buffer
UInt32 readPoint = dataSize;
UInt32 writePoint = 0;
while( readPoint > 0 )
{
UInt32 bytesToRead = 2;
AudioFileReadBytes( inputAudioFile, false, readPoint, &bytesToRead, theData );
AudioFileWriteBytes( outputAudioFile, false, writePoint, &bytesToRead, theData );
writePoint += 2;
readPoint -= 2;
}
free(theData);
AudioFileClose(inputAudioFile);
AudioFileClose(outputAudioFile);
I think this sample code could help you.
Mixer Host Sample Code
It will load two caf files from the bundle and play it. It contains a function call readAudioFilesIntoMemory which is loading a caf file to a data array as you said.
The whole program is an example of core audio, I hope it can help you :)
Why not let CoreAudio's AudioConverter do it for you? See this post about "Getting PCM from MP3/AAC/ALAC File", and Apple's Core Audio Essentials
you can use libsox for iphone framework to apply audio effects easily.
it includes a sample project that shows how to do it.
libsox ios
I am trying to convert caf file to m4a file using AudioUnit. I have implemented the code to convert. When I tried to run the application, I am getting following error message;
couldn't set destination client format (-66672)
I got the sample code from following link;
http://developer.apple.com/library/ios/#samplecode/iPhoneExtAudioFileConvertTest/Introduction/Intro.html
CODE:size = sizeof(clientFormat);
XThrowIfError(ExtAudioFileSetProperty(sourceFile, kExtAudioFileProperty_ClientDataFormat, size, &clientFormat), "couldn't set source client format");
//UInt32 encoderSpecifier = kAudioFormatMPEG4AAC;
//XThrowIfError(AudioFormatGetPropertyInfo(kAudioFormatProperty_Encoders, sizeof(encoderSpecifier), &encoderSpecifier, &size), "AudioFormatGetPropertyInfo: couldn't get property info");
size = sizeof(clientFormat);
XThrowIfError(ExtAudioFileSetProperty(destinationFile, kExtAudioFileProperty_ClientDataFormat, size, &clientFormat), "couldn't set destination client format");
AudioConverterRef audioConverter;
size = sizeof(audioConverter);
XThrowIfError(ExtAudioFileGetProperty(destinationFile, kExtAudioFileProperty_AudioConverter, &size, &audioConverter), "Couldn't get Audio Converter!");
I am not getting the solution for it. I have tried like setting the properties to the output file. But I am getting the same issue.
Please help me to resolve it.
I encountered this one too - It's not well-documented, but the reason is probably that you have to set the audio category to one compatible with hardware encoding.
In particular, any audio session that provides mixing with other sounds on the device will stop the encoder from working.
I know that AVAudioSessionCategoryPlayAndRecord, AVAudioSessionCategorySoloAmbient and AVAudioSessionCategoryAudioProcessing work for sure (as long as you're not overriding the kAudioSessionProperty_OverrideCategoryMixWithOthers property).
I've actually assembled everything you need to encode any audio file to AAC into an asynchronous class: TPAACAudioConverter