I have a music-making app that uses AUSampler & AURemoteIO units to play back user-defined notes. I'm having an issue where after some use, the call to AudioUnitRender on the sampler never returns, hanging the audio thread and silencing the audio output. The CPU usage also shoots up at this point, as the audio thread continuously spits out error messages to the device console (not the debugger output):
May 11 11:45:12 <device name> mediaserverd(CoreAudio)[2296] <Notice>: HALS_IOContext.cpp:1496:IOWorkLoop_HandleOverload: HALS_IOContext::IOWorkLoop_HandleOverload: Overload on context 96 current time: 11788974 deadline: 11788958
This message is being logged by _os_log_impl inside the AUSampler render (specifically VoiceEnvelope::GetRunFrameCount).
Does anyone have suggestions on why this may be happening and how to avoid it?
I discovered the issue. I was passing offsets greater than the size of the buffer into MusicDeviceMIDIEvent which would cause the issue. This was happening due to occasional jumps in the timestamp provided to the render callback. I was able to fix the issue by checking for and ignoring events with offsets greater than the frame count for the current callback.
Related
I am working on an iOS MPEG-DASH player and I have an issue with seeking feature.
ExtAudioFileSeek(::) documentation said
Sets the file’s read position to the specified sample frame number. A subsequent call to the ExtAudioFileRead(::_:) function returns samples from precisely this location, even if it is located in the middle of a packet.
Unfortunately, the AudioToolbox.AudioFile_ReadProc loop does not seek straight to the right frame and goes trough all file segments, requesting 16 and 27682 bytes packages.
It takes a lot of time (especially for long tracks) and force to download all intermediate segments (that should not be required)
This also cause the app to crash when long tracks with high sound quality are being played.
Here my log trace. I converted frame index to "mega" for more readability.
[PlayerEngine] - Trying to seek at 16.870537 %
[AudioSource] - Pausing AudioOutputUnit
[AudioSource] - AudioOutputUnit successfully stopped
[AudioOutputUnit] - Trying to seek at frame 31.006805M
[AudioOutputUnit] - Successfully sought at frame 31.006805M
[AudioInputUnit] - Audio frames have been flushed
[AudioInputUnit] - Seek to frame 31.006806M pending
[AudioConverter] - Converted frame buffer has been flush
[AudioSource] - Resuming AudioOutputUnit
[AudioSource] - AudioOutputUnit successfully resumed
[PlayerEngine] - Successfully sought at 16.870537 %
[AudioInputUnit] - Seeking to frame 31.006805M
[CoreAudioDecoder] - Trying to seek at frame 31.006805M
[CoreAudioDecoder] - Seek on frame 31.006806M done successfully
[CoreAudioDecoder] - AudioToolbox.AudioFile_ReadProc : inClientData, inPosition:1.315635M, requestCount:16
(...)
[CoreAudioDecoder] - AudioToolbox.AudioFile_ReadProc : inClientData, inPosition:1.331687M, requestCount:27682
Is it a bug on AudioToolBox or is there a way to fix it ?
Thanks a lot !
Cool!
Since you mention read callbacks, I assume you're using not only the ExtAudioFile API but also the AudioFile API as well, something like ExtAudioFileWrapAudioFileID(AudioFileInitializeWithCallbacks(...))
Compressed audio formats don't always have a simple mapping between frame and file offsets and so the naive behaviour you're seeing is probably due to one of these APIs (AudioFile?) understandably not knowing this mapping.
Try either setting the kExtAudioFileProperty_PacketTable property on the ExtAudioFile or kAudioFilePropertyPacketTableInfo on the wrapped audio file. The former probably makes more sense. I don't know if the whole packet table info will be available to you from the beginning or if it will be revealed to you over time, nor how the APIs will react to you setting these properties multiple times.
Good luck!
IXAudio2SourceVoice has a GetState function which returns an XAUDIO2_VOICE_STATE structure. This structure has a SamplesPlayed member, which is:
Total number of samples processed by this voice since it last started, or since the last audio stream ended (as marked with the XAUDIO2_END_OF_STREAM flag).
What I want to be able to do it stop the source voice, flush all its buffers, and then reset the SamplesPlayed counter to zero. Neither calling Stop nor FlushSourceBuffers will by themselves reset SamplesPlayed. And while flagging the last buffer with XAUDIO2_END_OF_STREAM does correctly reset SamplesPlayed back to zero, this seemingly only works if that last buffer is played to completion; if the buffer is flushed, then SamplesPlayed does not get reset. I have also tried calling Discontinuity both before and after stopping/flushing with no effect.
My current workaround is, after stopping and flushing the source voice, to submit a tiny 1-sample silent buffer with the XAUDIO2_END_OF_STREAM flag set and then let the source voice play to process that buffer and thus reset SamplesPlayed to zero. This works fine-ish for my use case, but it seems pretty hacky/clumsy. Is there a better solution?
Looking at the XAudio2 source, there's no exposed way to do that in the API other than letting a packet play with XAUDIO2_END_OF_STREAM.
Calling Discontinuity sets up the end-of-stream flag on the currently playing buffer, or if there's none playing and a queued buffer it sets it there. You need to call Discontinuity and then let the voice play to completion before you recycle it.
I want to play the recorded audio directly to speaker when headset is plugged in an iOS device.
What I did is calling AudioUnitRender in AURenderCallback func so that the audio data is writed to AudioBuffer structure.
It works well if the "IO buffer duration" is not set or set to 0.020seconds. If the "IO buffer duration" is set to a small value (0.005 etc.) by calling setPreferredIOBufferDuration, AudioUnitRender() will return an error:
kAudioUnitErr_CannotDoInCurrentContext (-10863).
Any one can help to figure out why and how to resolve it please? Thanks
Just wanted to add that changing the output scope sample rate to match the input scope sample rate of the input to the OSx kAudioUnitSubType_HALOutput Audio Unit that I was using fixed this error for me
The buffer is full so wait until a subsequent render pass or use a larger buffer.
This same error code is used by AudioToolbox, AudioUnit and AUGraph but only documented for AUGraph.
To avoid spinning or waiting in the render thread (a bad idea!), many
of the calls to AUGraph can return:
kAUGraphErr_CannotDoInCurrentContext. This result is only generated
when you call an AUGraph API from its render callback. It means that
the lock that it required was held at that time, by another thread. If
you see this result code, you can generally attempt the action again -
typically the NEXT render cycle (so in the mean time the lock can be
cleared), or you can delegate that call to another thread in your app.
You should not spin or put-to-sleep the render thread.
https://developer.apple.com/reference/audiotoolbox/kaugrapherr_cannotdoincurrentcontext
The problem occurs when I often stop and start audio playback and seek a lot back and forth in an AAC audio file through an ExtAudioFileRef object. In few cases, this strange behaviour is shown by ExtAudioFileRead:
Sometimes it assigns these numbers to the mDataByteSize of the only AudioBuffer of the AudioBufferList:
-51604480
-51227648
-51350528
-51440640
-51240960
In hex, these numbers have the pattern 0xFC....00.
The code:
status = ExtAudioFileRead(_file, &numberFramesRead, ioData);
printf("s=%li d=%p d.nb=%li, d.b.d=%p, d.b.dbs=%li, d.b.nc=%li\n", status, ioData, ioData->mNumberBuffers, ioData->mBuffers[0].mData, ioData->mBuffers[0].mDataByteSize, ioData->mBuffers[0].mNumberChannels);
Output:
s=0 d=0x16668bd0 d.nb=1, d.b.d=0x30de000, d.b.dbs=1024, d.b.nc=2 // good (usual)
s=0 d=0x16668bd0 d.nb=1, d.b.d=0x30de000, d.b.dbs=-51240960, d.b.nc=2 // misbehaving
The problem occurs on an iPhone 4S on iOS 7. I could not reproduce the problem in the Simulator.
The problem occurs when concurrently calling ExtAudioFileRead() and ExtAudioFileSeek() for the same ExtAudioFileRef from two different threads/queues.
The read function was called directly from the AURenderCallback, so it was executed on AudioUnit's real-time thread while the seek was done on my own serial queue.
I've modified the code of the render callback to also dispatch_sync() to the same serial queue to which the seek gets dispatched. That solved the problem.
NB: The entire code base for this project is so large that posting any meaningful amount wold render this question too localised, I have tried to distil any code down to the bare-essentials. I'm not expecting anyone to solve my problems directly but I will up vote those answers I find helpful or intriguing.
This project uses a modified version of AudioStreamer to playback audio files that are saved to locally to the device (iPhone).
The stream is set up and scheduled on the current loop using this code (unaltered from the standard AudioStreamer project as far as I know):
CFStreamClientContext context = {0, self, NULL, NULL, NULL};
CFReadStreamSetClient(
stream,
kCFStreamEventHasBytesAvailable | kCFStreamEventErrorOccurred | kCFStreamEventEndEncountered,
ASReadStreamCallBack,
&context);
CFReadStreamScheduleWithRunLoop(stream, CFRunLoopGetCurrent(), kCFRunLoopCommonModes);
The ASReadStreamCallBack calls:
- (void)handleReadFromStream:(CFReadStreamRef)aStream
eventType:(CFStreamEventType)eventType
On the AudioStreamer object, this all works fine until the stream is read using this code:
BOOL hasBytes = NO; //Added for debugging
hasBytes = CFReadStreamHasBytesAvailable(stream);
length = CFReadStreamRead(stream, bytes, kAQDefaultBufSize);
hasBytes is YES but when CFReadStreamRead is called execution stops, the App does not crash it just stops exciting, any break points below the CFReadStreamRead call are not hit and ASReadStreamCallBack is not called again.
I am at a loss to what might cause this, my best guess is the thread is being terminated? But the hows and whys is why I'm asking SO.
Has anyone seen this behaviour before? How can I track it down and ideas on how I might solve it will be very much welcome!
Additional Info Requested via Comments
This is 100% repeatable
CFReadStreamHasBytesAvailable was added by me for debugging but removing it has no effect
First, I assume that CFReadStreamScheduleWithRunLoop() is running on the same thread as CFReadStreamRead()?
Is this thread processing its runloop? Failure to do this is my main suspicion. Do you have a call like CFRunLoopRun() or equivalent on this thread?
Typically there is no reason to spawn a separate thread for reading streams asynchronously, so I'm a little confused about your threading design. Is there really a background thread involved here? Also, typically CFReadStreamRead() would be in your client callback (when you receive the kCFStreamEventHasBytesAvailable event (which it appears to be in the linked code), but you're suggesting ASReadStreamCallBack is never called. How have you modified AudioStreamer?
It is possible that the stream pointer is just corrupt in some way. CFReadStreamRead should certainly not block if bytes are available (it certainly would never block for more than a few milliseconds for local files). Can you provide the code you use to create the stream?
Alternatively, CFReadStreams send messages asynchronously but it is possible (but not likely) that it's blocking because the runloop isn't being processed.
If you prefer, I've uploaded my AudioPlayer inspired by Matt's AudioStreamer hosted at https://code.google.com/p/audjustable/. It supports local files (as well as HTTP). I think it does what you wanted (stream files from more than just HTTP).