How to ensure audio rendered within time limit on iOS? - ios

Question

I am rendering low-latency audio from my custom synth code via the iOS Audio Unit render callback. Obviously if my rendering code is too slow then it will return from the callback too late and there will be a buffer underrun. I know from experience this results in silence being output.
I want to know what time limit I have so that I can manage the level of processing to match the device limitations etc..
Obviously the length of the buffer (in samples) determines the duration of audio being rendered and this sets an overall limit. However I suspect that the Apple audio engine will have a smaller time limit between issuing the render callback and requiring the response.
How can I find out this time limit and is that something I can do within the callback function itself?
If I happen to exceed the time limit and cause a buffer underrun, is there a notification I can receive or a status object I can interrogate?
NB: In my app I am creating a single 'output' audio unit, so I don't need to worry about chaining audio units together.

Answer 1

The amount of audio rendering that can be done in Audio Unit callbacks depends on the iOS device model and OS version, and well as potential CPU clock speed throttling due to temperature or background modes. Thus, it needs to be profiled on the oldest, slowest iOS device you plan on your app supporting, with some margin.
To support iOS 9, I very conservatively profile my apps on an iPhone 4S test device (ARM Cortex A9 CPU at 800 MHz), or an even older slower device by using an earlier iOS version. When doing this profiling, one can add some percentage of "make work" to test an audio callback and see if there is any margin (For a 50% margin, generate the sample buffer twice, etc.) Other developers appear to be less conservative.
This is why it is important for an mobile audio developer to have (or have access to) to several iOS devices (the older the better). If the callback meets the time limit on an old slow text device, it will very likely be more than fast enough on any newer iOS device.
Depending on the OS version, an underrun can either result in silence, or the Audio Unit stopping or crashing (which can be detected by no more or not enough callbacks within some predictable amount of time).
But the best way to avoid underrun is to do most of the heavy audio work in another thread outside the audio unit thread, and pass samples to/from the audio unit callback using a lock-free circular fifo/queue.

Answer 2

Adding to what hotpaw2 said, the worst performing iOS device I have encountered is the iPhone touch 16G without the rear facing camera. I have done projects where every device except the ipod touch 16G plays audio smoothly. I had to bump up the buffer duration to the next size to accommodate.
I typically have done all audio prepping prior before the render callback in a separate lockless ring buffer and keep the render callback limited to copying data. I let the application "deal" with a buffer underruns.
I personally never measured the render callback variance but I would guess that it would be consistently equal to the buffer duration time and would extremely minimal jitter (eg 5ms). I doubt it would be 4.9 ms one time then 5.1 ms the next time.
To get some timing info, in mach_time.hyou can use mach_absolute_time() to get some timing.
You didn't really say what your timing requirements are. I assume you need low latency audio. Otherwise, you can just set the buffer duration to be pretty big. I assume that you want to increase latency for slow devices using this code. I usually find what works on an iPod 16G and use that as a worst case.
NSTimeInterval _preferredDuration = ...
NSError* err;
[[AVAudioSession sharedInstance]setPreferredIOBufferDuration:_preferredDuration error:&err];
And of course, you should get the actual duration used. The OS will pick some power of two based on the sample rate:
NSTimeInterval _actualBufferDuration;
_actualBufferDuration = [[AVAudioSession sharedInstance] IOBufferDuration];
As far as adjusting for device performance. You can set the buffer duration