Deep Copy of Audio CMSampleBuffer - ios

Question

I am trying to create a copy of a CMSampleBuffer as returned by captureOutput in a AVCaptureAudioDataOutputSampleBufferDelegate.
The problem I am having is that my frames coming from delegate method captureOutput:didOutputSampleBuffer:fromConnection: being dropped after I retain them in CFArray for long time.
Obviously, I need to create deep copies of incoming buffers for further processing. I also know that CMSampleBufferCreateCopy only creates shallow copies.
There are few related questions were asked on SO:
Pulling data from a CMSampleBuffer in order to create a deep copy
Creating copy of CMSampleBuffer in Swift returns OSStatus -12743 (Invalid Media Format)
Deep Copy of CMImageBuffer or CVImageBuffer
But none of them helps me to use correctly CMSampleBufferCreate function with 12 parameters:
CMSampleBufferRef copyBuffer;
CMBlockBufferRef data = CMSampleBufferGetDataBuffer(sampleBuffer);
CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer);
CMItemCount itemCount = CMSampleBufferGetNumSamples(sampleBuffer);
CMTime duration = CMSampleBufferGetDuration(sampleBuffer);
CMTime presentationStamp = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
CMSampleTimingInfo timingInfo;
timingInfo.duration = duration;
timingInfo.presentationTimeStamp = presentationStamp;
timingInfo.decodeTimeStamp = CMSampleBufferGetDecodeTimeStamp(sampleBuffer);
size_t sampleSize = CMBlockBufferGetDataLength(data);
CMBlockBufferRef sampleData;
if (CMBlockBufferCopyDataBytes(data, 0, sampleSize, &sampleData) != kCMBlockBufferNoErr) {
VLog(#"error during copying sample buffer");
}
// Here I tried data and sampleData CMBlockBuffer instance, but no success
OSStatus status = CMSampleBufferCreate(kCFAllocatorDefault, data, isDataReady, nil, nil, formatDescription, itemCount, 1, &timingInfo, 1, &sampleSize, &copyBuffer);
if (!self.sampleBufferArray) {
self.sampleBufferArray = CFArrayCreateMutable(NULL, 0, &kCFTypeArrayCallBacks);
//EXC_BAD_ACCESS crash when trying to add sampleBuffer to the array
CFArrayAppendValue(self.sampleBufferArray, copyBuffer);
} else {
CFArrayAppendValue(self.sampleBufferArray, copyBuffer);
}
How do you deep copy Audio CMSampleBuffer? Feel free to use any language (swift/objective-c) in your answers.

Answer 1

Here is a working solution I finally implemented. I sent this snippet to Apple Developer Technical support and asked them to check if it is a correct way to copy incoming sample buffer. The basic idea is copy AudioBufferList and then create a CMSampleBuffer and set AudioBufferList to this sample.
AudioBufferList audioBufferList;
CMBlockBufferRef blockBuffer;
//Create an AudioBufferList containing the data from the CMSampleBuffer,
//and a CMBlockBuffer which references the data in that AudioBufferList.
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer, NULL, &audioBufferList, sizeof(audioBufferList), NULL, NULL, 0, &blockBuffer);
NSUInteger size = sizeof(audioBufferList);
char buffer[size];
memcpy(buffer, &audioBufferList, size);
//This is the Audio data.
NSData *bufferData = [NSData dataWithBytes:buffer length:size];
const void *copyBufferData = [bufferData bytes];
copyBufferData = (char *)copyBufferData;
CMSampleBufferRef copyBuffer = NULL;
OSStatus status = -1;
/* Format Description */
AudioStreamBasicDescription audioFormat = *CMAudioFormatDescriptionGetStreamBasicDescription((CMAudioFormatDescriptionRef) CMSampleBufferGetFormatDescription(sampleBuffer));
CMFormatDescriptionRef format = NULL;
status = CMAudioFormatDescriptionCreate(kCFAllocatorDefault, &audioFormat, 0, nil, 0, nil, nil, &format);
CMFormatDescriptionRef formatdes = NULL;
status = CMFormatDescriptionCreate(NULL, kCMMediaType_Audio, 'lpcm', NULL, &formatdes);
if (status != noErr)
{
NSLog(#"Error in CMAudioFormatDescriptionCreator");
CFRelease(blockBuffer);
return;
}
/* Create sample Buffer */
CMItemCount framesCount = CMSampleBufferGetNumSamples(sampleBuffer);
CMSampleTimingInfo timing = {.duration= CMTimeMake(1, 44100), .presentationTimeStamp= CMSampleBufferGetPresentationTimeStamp(sampleBuffer), .decodeTimeStamp= CMSampleBufferGetDecodeTimeStamp(sampleBuffer)};
status = CMSampleBufferCreate(kCFAllocatorDefault, nil , NO,nil,nil,format, framesCount, 1, &timing, 0, nil, &copyBuffer);
if( status != noErr) {
NSLog(#"Error in CMSampleBufferCreate");
CFRelease(blockBuffer);
return;
}
/* Copy BufferList to Sample Buffer */
AudioBufferList receivedAudioBufferList;
memcpy(&receivedAudioBufferList, copyBufferData, sizeof(receivedAudioBufferList));
//Creates a CMBlockBuffer containing a copy of the data from the
//AudioBufferList.
status = CMSampleBufferSetDataBufferFromAudioBufferList(copyBuffer, kCFAllocatorDefault , kCFAllocatorDefault, 0, &receivedAudioBufferList);
if (status != noErr) {
NSLog(#"Error in CMSampleBufferSetDataBufferFromAudioBufferList");
CFRelease(blockBuffer);
return;
}
Code-Level Support answer:
This code looks ok (though you’ll want to add some additional error
checking). I've successfully tested it in an app that implements the
AVCaptureAudioDataOutput delegate
captureOutput:didOutputSampleBuffer:fromConnection: method to
capture and record audio. The captured audio I'm getting when using
this deep copy code appears to be the same as what I get when directly
using the provided sample buffer (without the deep copy).
Apple Developer Technical Support

Answer 2

Couldn't find a decent answer doing this in Swift. Here's an extension:
extension CMSampleBuffer {
func deepCopy() -> CMSampleBuffer? {
guard let formatDesc = CMSampleBufferGetFormatDescription(self),
let data = self.data else {
return nil
}
let nFrames = CMSampleBufferGetNumSamples(self)
let pts = CMSampleBufferGetPresentationTimeStamp(self)
let dataBuffer = data.withUnsafeBytes { (buffer) -> CMBlockBuffer? in
var blockBuffer: CMBlockBuffer?
let length: Int = data.count
guard CMBlockBufferCreateWithMemoryBlock(
allocator: kCFAllocatorDefault,
memoryBlock: nil,
blockLength: length,
blockAllocator: nil,
customBlockSource: nil,
offsetToData: 0,
dataLength: length,
flags: 0,
blockBufferOut: &blockBuffer) == noErr else {
print("Failed to create block")
return nil
}
guard CMBlockBufferReplaceDataBytes(
with: buffer.baseAddress!,
blockBuffer: blockBuffer!,
offsetIntoDestination: 0,
dataLength: length) == noErr else {
print("Failed to move bytes for block")
return nil
}
return blockBuffer
}
guard let dataBuffer = dataBuffer else {
return nil
}
var newSampleBuffer: CMSampleBuffer?
CMAudioSampleBufferCreateReadyWithPacketDescriptions(
allocator: kCFAllocatorDefault,
dataBuffer: dataBuffer,
formatDescription: formatDesc,
sampleCount: nFrames,
presentationTimeStamp: pts,
packetDescriptions: nil,
sampleBufferOut: &newSampleBuffer
)
return newSampleBuffer
}
}

Answer 3

LLooggaann's solution is simpler and works well, however, in case anyone is interested, I migrated the original solution to Swift 5.6:
extension CMSampleBuffer {
func deepCopy() -> CMSampleBuffer? {
var audioBufferList : AudioBufferList = AudioBufferList()
var blockBuffer : CMBlockBuffer?
let sizeOfAudioBufferList = MemoryLayout<AudioBufferList>.size
//Create an AudioBufferList containing the data from the CMSampleBuffer.
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(self,
bufferListSizeNeededOut: nil,
bufferListOut: &audioBufferList,
bufferListSize: sizeOfAudioBufferList,
blockBufferAllocator: nil,
blockBufferMemoryAllocator: nil,
flags: 0,
blockBufferOut: &blockBuffer)
guard audioBufferList.mNumberBuffers == 1 else { return nil } //TODO: Make this generic for any number of buffers
/* Deep copy the audio buffer */
let audioBufferDataSize = Int(audioBufferList.mBuffers.mDataByteSize)
let audioBuffer = audioBufferList.mBuffers
let audioBufferDataCopyPointer = UnsafeMutableRawPointer.allocate(byteCount: audioBufferDataSize, alignment: 1)
defer {
audioBufferDataCopyPointer.deallocate()
}
memcpy(audioBufferDataCopyPointer, audioBufferList.mBuffers.mData, audioBufferDataSize)
let copiedAudioBuffer = AudioBuffer(mNumberChannels: audioBuffer.mNumberChannels,
mDataByteSize: audioBufferList.mBuffers.mDataByteSize,
mData: audioBufferDataCopyPointer)
/* Create a new audio buffer list with the deep copied audio buffer */
var copiedAudioBufferList = AudioBufferList(mNumberBuffers: 1, mBuffers: copiedAudioBuffer)
/* Copy audio format description, to be used in the new sample buffer */
guard let sampleBufferFormatDescription = CMSampleBufferGetFormatDescription(self) else { return nil }
/* Create copy of timing for new sample buffer */
var duration = CMSampleBufferGetDuration(self)
duration.value /= Int64(numSamples)
var timing = CMSampleTimingInfo(duration: duration,
presentationTimeStamp: CMSampleBufferGetPresentationTimeStamp(self),
decodeTimeStamp: CMSampleBufferGetDecodeTimeStamp(self))
/* New sample buffer preparation, using the audio format description, and the timing information. */
let sampleCount = CMSampleBufferGetNumSamples(self)
var newSampleBuffer : CMSampleBuffer?
guard CMSampleBufferCreate(allocator: kCFAllocatorDefault,
dataBuffer: nil,
dataReady: false,
makeDataReadyCallback: nil,
refcon: nil,
formatDescription: sampleBufferFormatDescription,
sampleCount: sampleCount,
sampleTimingEntryCount: 1,
sampleTimingArray: &timing,
sampleSizeEntryCount: 0,
sampleSizeArray: nil,
sampleBufferOut: &newSampleBuffer) == noErr else { return nil }
//Create a CMBlockBuffer containing a copy of the data from the AudioBufferList, add to new sample buffer.
let status = CMSampleBufferSetDataBufferFromAudioBufferList(newSampleBuffer!,
blockBufferAllocator: kCFAllocatorDefault,
blockBufferMemoryAllocator: kCFAllocatorDefault,
flags: 0,
bufferList: &copiedAudioBufferList)
guard status == noErr else { return nil }
return newSampleBuffer
}
}