I am using PortAudio to record multichannel audio data. I want to save the data from each channel as a separate file. Therefore, I have to do some File IO work somewhere in the program. Since the document recommend not doing any IO work in the callback function since they are called in a high frequency.
Then, what is the right way to perform the IO work ? Should I start another thread?
Thanks in advance for your time on reading my post!
This is what I would do:
1) Have a buffer for each channel. These buffers will be your storage buffers and not the ones you have in the callback. I'd suggest declarinng these buffers in your paData structures. You'll have to decide the buffer length according to how much storage you have or time limit.
2) When you stop the recording, have it call a method which does your file IO work.
3) Use the buffers you had from step 1 and write to the file.
4) Repeat
Related
I'm developing an app of recording, but I have a demand that the size of input buffer should be 882 bytes. I know that I can modify the mDataByteSize of buffList like this picture.
But it can only be modified to power of 2. When I tried to modify it to 882, it warned me that "AudioUnitRender error:-50".
I hope somebody can help me because I have no way.
You can't demand a specific input size in an Audio Unit recording callback bufferList. In fact, the Audio Unit API is allowed to change the number of samples per audio buffer at run time. So your app has to support a differing number of frames than requested, each callback.
Instead your app should save the samples in another temporary FIFO buffer, and the later remove some of the samples in your desired block size when that temporary FIFO buffer become full enough. Typically a circular buffer is used to store the temporary amount until it gets filled to the size you need or larger. They you can pull out exactly 882 or whatever number of samples you need.
I would like to send a very large (~8GB) datastructure through the network, so I use the Marshal module to transform it into Bytes.
My problem is that the memory doubles, because we need to store both representations (initial data and Marshaled data).
Is there a simple way to Marshal into a Stream instead ? This would avoid to have the full Marshalled representation of the initial datastructure.
I thought of Marshaling to an out_channel in which I opened a pipe with a second thread and reading from the pipe in the main thread into s Stream, but I guess there might be a simpler solution.
Thanks !
Edit: Answer to a comment:
In the toplevel :
let a = Array.make (1024*1024*1024) 0. ;; (* Takes 8GB of RAM *)
let data = Marshal.to_bytes a [Marshal.Closures] ;; (* Takes an extra 8GB *)
It's not possible. You would have to modify the Marshal module to stream the data as it marshals something and to reconstruct the data in place without buffering it all first.
In the short run it might be simpler to implement your own specialized marshal function specific to your data. For an 8GiB array you might want to switch to using BigArray so you can send/recv the data without having to copy it.
Note: A 8GiB array will use 16GiB if the GC ever copies it, at least temporary.
From what I understand, MPI only allows to send data packets with a known size, not a stream of data. You could implement a custom stream type that split an incoming flow of data to packets of constant, small size (on close, you flush whatever remains in the buffer).
Also, you only can marshall arbitrary long data to a channel, because otherwise you take up too many space.
And then, you need to have a way to connect the channel to the stream, which AFAIK is not easily possible. Maybe you could start antoer ocaml process: the process would convert the flow of bytes (you can wrap a custom stream over Stream.of_channel) and send it through MPI. The main process would marshall data to the process's input channel.
My question is pretty much self explanatory. Sorry if it seems too dumb.
I am writing a iOS VoIP dialer and have checked some open-source code(iOS audio calling app). And almost all of those use Circular Buffer for storing recorded and received PCM audio data. SO i am wondering why we need to use a Circular Buffer in this case. What's the exact reason for using such audio buffer.
Thanks in advance.
Using a circular buffer lets you process the input and output data asynchronously from it's source. The audio render process takes place on a high priority thread. It asks for audio samples from your app (playback), and offers audio (recording/processing) on a timer in the form of callbacks.
A typical scenario would be for the audio callback to fire every 0.023 seconds to ask for (and/or offer) 1024 samples of audio. This thread is synchronized with system hardware so it is imperative that your callback returns before the 0.023 seconds is up. If you don't, the hardware won't wait for you, it will just skip that cycle and you will have an audible pop or silence, or miss audio you are trying to record.
A circular buffer's place is to pass data between threads. In an audio application that would be moving the samples to and from the audio thread asynchronously. One thread produces samples on to the "head" of the buffer, and the other thread consumes them from the "tail".
Here's an example, retrieving audio samples from the microphone and writing them to disk. Your app has subscribed to a callback that fires every 0.023 seconds, offering 1024 samples to be recorded. The naive approach would be to simply write the audio to disk from within that callback.
void myCallback(float *samples,int sampleCount, SampleSaver *saver){
SampleSaverSaveSamples(saver,samples,sampleCount);
}
This will work!! Most of the time...
The problem is that there is no guarantee that writing to disk will finish before 0.023 seconds, so every now and then, your recording has a pop in it because SampleSaver just plain took too long and the hardware just skips the next callback.
The right way to do this is to use a circular buffer. I personally use TPCircularBuffer because it's awesome. The way it works (externally) is that you ask the buffer for a pointer to write data to (the head) on one thread, then on another thread you ask the buffer for a pointer to read from (the tail). Here's how it would be done using TPCircularBuffer (skipping setup and using a simplified callback).
//this is on the high priority thread that can't wait for anything like a slow write to disk
void myCallback(float *samples,int sampleCount, TPCircularBuffer *buffer){
int32_t availableBytes = 0;
float *head = TPCircularBufferHead(buffer, &availableBytes);
memcpy(head,samples,sampleCount * sizeof(float));//copies samples to head
TPCircularBufferProduce(buffer,sampleCount * sizeof(float)); //moves buffer head "forward in the circle"
}
This operation is super quick and puts no extra pressure on that sensitive audio thread. You then create your own timer a separate thread to write the samples to disk.
//this is on some background thread that can take it's sweet time
void myLeisurelySavingCallback(TPCircularBuffer *buffer, SampleSaver *saver){
int32_t available;
float *tail = TPCircularBufferTail(buffer, &available);
int samplesInBuffer = available / sizeof(float); //mono
SampleSaverSaveSamples(saver, tail, samplesInBuffer);
TPCircularBufferConsume(buffer, samplesInBuffer * sizeof(float)); // moves tail forward
}
And there you have it, not only do you avoid audio glitches, but if you initialize a big enough buffer, you can set your write to disk callback to only fire every second or two (after the circular buffer has built up a good bit of audio) which is much easier on your system than writing to disk every 0.023 seconds!
The main reason to use the buffer is so the samples can be handled asynchronously. They are a great way to pass messages between threads without locks as well. Here is a good article explaining a neat memory trick for the implementation of a circular buffer.
Good question. There is another good reason for using Circular Buffer.
In iOS, if you use callbacks(Audio unit) for recording and playing audio(In-fact you need to use it if you want to create a real-time audio transferring app) then you will get a chunk of data for a specific amount of time(let's say 20 milliseconds) from the recorder callback. And in iOS, you will never get fixed length of data always(If you set the callback interval as 20ms then you will get 370 or 372 bytes of data. And you will never know when you will get 370 bytes or 372 bytes. Correct me if i am wrong). Then, to transfer the audio through UDP packets you need to use a codec for data encoding and decoding(G729 is generally used for VoIP apps). But g729 takes data by the multiplier of 8. Assume, you encode 368(8*46) bytes per 20ms. So what are you going to do with rest of the data ? You need to store it by sequence for the next chunk to process.
SO that's the reason. There are some other details thing but i kapt it simple for your better understanding. Just comment below if you have any question.
I'm a total noob when it comes to core audio so bear with me. Basically what I want to do is record audio data from a machine's default mic, record until the user decides to stop, and then do some analysis on the entire recording. I've been learning from the book "Learning Core Audio" by Chis Adamson and Kevin Avila (which is an awesome book, found it here: http://www.amazon.com/Learning-Core-Audio-Hands-On-Programming/dp/0321636848/ref=sr_1_1?ie=UTF8&qid=1388956621&sr=8-1&keywords=learning+core+audio ). I see how the AudioQueue works, but I'm not sure how to get data as it's coming from the buffers and store it in a global array.
The biggest problem is that I can't allocate an array a priori because we have no idea how long the user wants to record for. I'm guessing that a global array would have to be passed to the AudioQueue's callback where it would then append data from the latest buffer, however I'm not exactly sure how to do that, or if that's the correct place to be doing so.
If using AudioUnits I'm guessing that I would need to create two audio units, one a remote IO unit to get the microphone data and one generic output audio unit that would do the data appending in the unit's (I'm guessing here, really not sure) AudioUnitRender() function.
If you know where I need to be doing these things or know any resources that could help explain how this works, that would be awesome.
I eventually want to learn how to do this in iOS and Mac OS platforms. For the time being I'm just working in the Mac OS.
Since you know the sample rate, your app can pre-allocated a sufficient number of new buffers (for example, in a linked list) in the UI run-loop (for example, periodically, based on an NSTimer or CADisplayLink), into which you can then just copy data during the Audio Queue callbacks.
There are also a few async file write functions that are safe to call inside an audio callback. After recording you can copy the data back out of the file into a now-known-sized memory array (or just mmap the file).
Ok everyone, so I figured out the answer to my own question, and I only had to add about 10 lines of code to get it to work. Basically what I did was in the user data struct, or as Apple calls it the client data struct I added a variable to keep track of the total number of recorded samples, the sample rate (just so I would have access to the value inside the callback) and a pointer that would point to the audio data. In the callback function, I reallocated memory for the pointer and then copied the contents of the buffer into the newly allocated memory.
I'll post the code for my client recorder struct and the lines of code inside the callback function. I would like to post code for the entire program, but much of it was borrowed from the book "Learning Core Audio" by Chis Adamson and Kevin Avila and I don't want to infringe on any copyrights held by the book (can someone tell me if it's legal to post that here or not? if it is I'd be more than happy to post the whole program).
the client recorder struct code:
//user info struct for recording audio queue callbacks
typedef struct MyRecorder{
AudioFileID recordFile;
SInt64 recordPacket;
Boolean running;
UInt32 totalRecordedSamples;
Float32 * allData;
Float64 sampleRate;
}MyRecorder;
This struct needs to be initialized in the main loop of the program. It would look something like this:
MyRecorder recoder = {0};
I know I spelled "recorder" incorrectly.
Next, what I did inside the callback function:
//now let's also write the data to a buffer that can be accessed in the rest of the program
//first calculate the number of samples in the buffer
Float32 nSamples = RECORD_SECONDS * recorder->sampleRate;
nSamples = (UInt32) nSamples;
//so first we need to reallocate memory that the recorder.allData pointer is pointing to
//pretty simple, just add the amount of samples we just recorded to the number that we already had
//to get the current total number of samples, and the size of each sample, which we get from
//sizeof(Float32).
recorder->allData = realloc(recorder->allData, sizeof(Float32) * (nSamples + recorder->totalRecordedSamples));
//now we need to copy what's in the current buffer into the memory that we just allocated
//however, rememeber that we don't want to overwrite what we already just recorded
//so using pointer arith, we need to offset the recorder->allData pointer in memcpy by
//the current value of totalRecordedSamples
memcpy((recorder->allData) + (recorder->totalRecordedSamples), inBuffer->mAudioData, sizeof(Float32) * nSamples);
//update the number of total recorded samples
recorder->totalRecordedSamples += nSamples;
And of course at the end of my program I freed the memory in the recoder.allData pointer.
Also, my experience with C is very limited, so if I'm making some mistakes especially with memory management, please let me know. Some of the malloc, realloc, memcpy etc. type functions in C sort of freak me out.
EDIT: Also I'm now working on how to do the same thing using AudioUnits, I'll post the solution to that when done.
I'm working in iOS and have a simple OpenAL project running.
The difference to most openAL projects i've seen is that im not loading in a sound file. Instead I load an array of raw data into the alBufferData. Using a couple of equations I can load in data to produce white noise, sine and pulse waves. And all is working well.
My problem is that I need a way to modify this data whilst the sound is playing in real-time.
Is there a way to modify this data without having to create a new buffer (i tried the approach of creating a new buffer with new data and then use it instead but its nowhere near quick enough).
Any help or suggestions of other ways to accomplish this would be much appreciated.
Thanks
I haven't done it on iOS, but with openAL on the PC what you would do is chain a few buffers together. Each buffer would have a small time period's worth of data. Periodically, check to see if the playing buffer is done, and if so, add it to a free list for reuse When you want to change the sound, write the new waveform into a free buffer and add it to the chain. You select the buffer size to balance latency and required update rate - smaller buffers allow faster response to changes, but need to be generated more often.
This page suggests that a half second update rate is doable. Whether you can go faster depends on the complexity of your calculations as well as on the overhead of the OS.
Changing the data during playback is not supported in OpenAL.
However, you can still try it and see if you get acceptable defaults (though you'll be racing against the OpenAL playback mechanism, and any lag-outs in your app could throw it off, so do this at your own risk).
There's an Apple extension version of ALBufferData that tells OpenAL to use the data you give it directly, rather than making its own local copy. You set it up like so:
typedef ALvoid AL_APIENTRY (*alBufferDataStaticProcPtr) (const ALint bid,
ALenum format,
const ALvoid* data,
ALsizei size,
ALsizei freq);
static alBufferDataStaticProcPtr alBufferDataStatic = NULL;
alBufferDataStatic = (alBufferDataStaticProcPtr) alcGetProcAddress(NULL, (const ALCchar*) "alBufferDataStatic");
Call alBufferDataStatic() it like you would call alBufferData():
alBufferDataStatic(bufferId, format, data, size, frequency);
Since it's now using your sound data buffer rather than its own, you could conceivably modify that data and it won't be the wiser (provided you're not modifying things too close to where it's currently playing from in the buffer).
However, this approach is risky, since it depends on timing you're not fully in control of. To be 100% safe you'll need to use Audio Units.